Computer Vision · Cross-listed from cs.CL
Chinh Hoang and Mohammad Rashedul Hasan
Vision-
language models (VLMs) generate fluent causal explanations, but current evaluations cannot distinguish linguistic plausibility from faithful causal
reasoning. We introduce a dual-probe methodology that isolates these properties. The Text-Only Probe measures linguistic quality. The Chain-Text Probe requires models to first generate explicit causal chains. The Abstraction Gap (AG) metric quantifies the normalized performance difference. Evaluating eight VLMs on CAGE (Causal Abstraction Gap Evaluation), a
benchmark of 49,500 questions across 5,500 images spanning Pearl's causal hierarchy, we find seven models exhibit AG exceeding 0.50 with text scores of 6–8 but chain scores below 2.5. Fine-tuning on 45,000 chain-annotated examples fails to close the gap. However, one model achieves near-zero AG. The capability exists within current VLM architectures and depends on pretraining and architectural choices. CAGE provides a diagnostic tool for assessing faithful causal
reasoning in VLMs.
Read the full abstract ▾
Computer Vision
Self-Prophetic Decoding to Unlock Visual Search in LVLMs
Zhendong He, Qiyuan Dai, Guanbin Li et al. (+2 more)
Large
Vision-
Language Models (LVLMs) are rapidly evolving toward true
multimodal reasoning, with visual search representing a concrete instantiation of the thinking-with-images paradigm. However, LVLM visual search faces two key challenges: incompatibility among intrinsic capabilities after post-training, and interference in long multi-step
reasoning contexts. To address these, we identify two novel insights. First, self-regulation between pre- and post-training LVLMs leverages the intrinsic single-step capabilities of the pre-training model to mitigate capability deterioration and long-context interference. Second, probability-based prophetic sampling, replacing naive prompting, provides a probabilistic interface where the pre-training model acts as a prophet and the post-training model selectively accepts prophetic
tokens under its output distribution, preserving coherent multi-step
reasoning. Building on these insights, we introduce SeProD, a self-prophetic decoding framework that leverages intrinsic single-step capabilities to enable coherent multi-step
reasoning in a training-free, plug-and-play manner. Experiments show that SeProD consistently improves multiple visual-search LVLMs across all 12 splits of 4 visual search
benchmarks, as well as across general VQA
benchmarks, without added computational overhead, thanks to its parallel prophetic acceptance mechanism.
Read the full abstract ▾
All authors Zhendong He, Qiyuan Dai, Guanbin Li, Liang Lin, Sibei Yang
Computer Vision
SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Grouping
Hongyu Wen and Jia Deng
Transparent objects are common in daily life, and it is important to understand their multilayer depth, including the transparent surface and the objects behind it. Existing methods for multilayer depth typically extend single-layer prediction. They define layers by the front-to-back ordering of 3D points and predict the layers sequentially. However, as layered geometry can admit multiple valid groupings of 3D points into layers, a predefined grouping strategy is inherently restrictive. In this work, we propose SeeGroup, a multi-layer depth estimation method that avoids imposing a predefined grouping and allows the model itself to adaptively assign surfaces to depth maps. We formulate per-pixel multi-layer depth as a point process, treating depth layers as unordered events along each camera ray. This induces a permutation-invariant likelihood over the observed depth layers, yielding a loss that naturally supports arbitrary layer groupings. Experiments demonstrate that our method significantly advances the state of the art of multi-layer depth estimation, improving quadruplet relative depth accuracy on LayeredDepth
benchmark from 61.34% to 70.09%. Code is available at https://github.com/princeton-vl/SeeGroup.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Deep Learning Strain Estimation: Is Physics-Based Simulation the Solution?
Thierry Judge, Nicolas Duchateau, Andreas Østvik et al. (+13 more)
Speckle tracking echocardiography (STE) is the clinical standard for myocardial strain estimation. Despite good performance on global strain (GLS), its accuracy for regional strain remains limited, even though this biomarker is highly relevant for early diagnosis and the characterization of subtle abnormalities. from clinical data. Deep learning is a promising alternative, but its development is constrained by the lack of reliable motion references. Existing solutions rely either on STE-derived labels or on simulations generated by physics-based models, but these synthetic sequences still have limited realism compared with clinical data.In this paper, we propose a novel simulation strategy that incorporates speckle decorrelation measures from real
videos and uses an iterative refinement process to improve the motion realism in the simulations. We created an open-source photorealistic
dataset of 1,478
videos with reference motion, which was used to train an echocardiographic motion estimation
algorithm. The proposed method achieves unmatched performance on global and regional strain, notably reaching a GLS variability of 1.42% in an inter-expert setting compared to 1.78% for the clinical reference.
Read the full abstract ▾
All authors Thierry Judge, Nicolas Duchateau, Andreas Østvik, Khuram Faraz, Anders Austlid Taskén, Sigve Karlsen, Thor Edvardsen, Harald Brunvand, Md Abulkalam Azad, Havard Dalen, Bjørnar Grenne, Gabriel Kiss, Pierre-Yves Courand, Lasse Lovstakken, Pierre-Marc Jodoin, Olivier Bernard
Computer Vision
OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning
Yunyang Ge, Xianyi He, Zezhong Zhang et al. (+4 more)
Diffusion Transformers achieve strong
video generation quality, but the quadratic cost of full
attention limits efficiency. We introduce OSP-Next, an efficient text-to-
video generation model that integrates sparse
attention, parallelism, quantization, and reinforcement learning. OSP-Next uses a hybrid full-sparse
attention architecture, where the sparse component is implemented with Skiparse-2D
Attention. This fixed-pattern mechanism applies
token-wise and group-wise sparse
attention along spatial dimensions, leveraging locality while maintaining native compatibility with FlashAttention kernels. Based on the local equivalence of rearrangement in Skiparse-2D
Attention, we further propose Sparse Sequence Parallelism (SSP), which partitions subsequences across ranks and switches sparse patterns through a single All-to-All communication. Compared with Ulysses Sequence Parallelism (SP), SSP provides a native parallel strategy for sparse
attention and reduces communication volume by 75%. OSP-Next also incorporates HiF8 quantization to enable stable joint training with 8-bit quantization and sparse fine-tuning, and applies Mix-GRPO post-training to improve the performance of the sparse model. Experiments show that OSP-Next achieves a VBench total score of 83.73%, surpassing the Wan2.1 baseline. Under the 5-second 720P and 5-second 768P settings, OSP-Next achieves up to 1.64$\times$ single-GPU speedup and over 1.52$\times$ eight-GPU speedup on NVIDIA H200 GPUs. In addition, with only a 0.4% drop in VBench total score, OSP-Next-HiF8 achieves 1.69$\times$ and 2.27$\times$ speedups under the two settings on a single Ascend 950PR, demonstrating the efficiency and performance of OSP-Next across hardware platforms.
Read the full abstract ▾
All authors Yunyang Ge, Xianyi He, Zezhong Zhang, Bin Lin, Bin Zhu, Xinhua Cheng, Li Yuan
Computer Vision
EntroAD: Structural Entropy-Guided Prompt Adaptation for Zero-Shot Anomaly Detection
Xinyu Zhao, Qingyun Sun, Jiayi Luo et al. (+1 more)
Zero-Shot Anomaly Detection (ZSAD) aims to detect anomalies in unseen domains without target-domain adaptation. Recent CLIP-based methods have shown promising performance by leveraging
prompt learning and visual-text
alignment. However, most existing approaches rely on a single adaptation pathway, which may be insufficient for heterogeneous anomaly patterns across domains. In practice, anomalies exhibit vastly different characteristics, ranging from salient, localized structural disruptions to subtle, diffuse, and irregular variations. To address this challenge, we propose EntroAD, a structural
entropy-guided zero-shot anomaly detection framework. Unlike previous methods, EntroAD introduces a dynamic routing mechanism to process different types of anomalies with specialized adaptation strategies. Specifically, we estimate patch-level structural
entropy from self-
attention-induced patch relations and use it as a proxy for relational uncertainty to guide anomaly-aware
token routing. Based on this routing signal, we construct anomaly-aware routed
tokens to better capture anomaly cues with different structural characteristics. We further introduce a confidence-aware dual-branch
prompt adaptation module to stabilize visual-text
alignment while preserving CLIP's transferable prior. Extensive experiments on 10 industrial and medical
benchmarks show that EntroAD achieves state-of-the-art performance in challenging cross-
dataset ZSAD settings.
Read the full abstract ▾
All authors Xinyu Zhao, Qingyun Sun, Jiayi Luo, Jianxin Li
Computer Vision
A Multiscale Kinetic Framework for Image Segmentation: From Particle Systems to Continuum Models
Horacio Tettamanti, Giulia Guicciardi and Mattia Zanella
In this work, we present a multiscale kinetic framework for consensus-based image
segmentation. By interpreting an image as a system of interacting particles, each pixel is characterised by its spatial position and an internal feature encoding color information. We introduce a coupled interaction scheme governing the
evolution of particles in both position and feature spaces, from which we derive a kinetic formulation for the particle density in the space-feature domain combining transport, aggregation, and
diffusion effects. Furthermore, through a suitable scaling, we obtain a first-order macroscopic model describing the
evolution of the fraction of pixels carrying information on the fraction of pixels having a certain feature. Based on this reduced-complexity model, we present a data-oriented approach where we make use of particle-based optimisation techniques for the accurate
segmentation of images. Numerical tests show the effectiveness of the proposed framework and its robustness under different noise conditions.
Read the full abstract ▾
Computer Vision
Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization
Zhuohan Liu, Wujian Peng, Yitong Chen et al. (+1 more)
Despite the rapid progress of text-to-image (T2I) models, generating images that accurately reflect complex compositional
prompts (covering attribute bindings, object relationships, counting) still remains challenging. To address this, we propose BiDPO, a framework to enhance T2I model's capability of compositional text-to-image generation. We begin by introducing an carefully designed pipeline to construct a large-scale preference
dataset, BiComp, with strictly quality control. Then, we extend
Diffusion DPO to jointly optimize image and text preferences, which is shown to greatly effective in improving the models to follow complex text
prompt in generation. To further enhance the models for fine-grained
alignment, we employ a region-level guidance method to focus on regions relevant to compositional concepts. Experimental results demonstrate that our BiDPO substantially improves compositional fidelity, consistently outperforming prior methods across multiple
benchmarks. Our approach highlights the potential of preference-based fine-tuning for complex text-to-image tasks, offering a flexible and scalable alternative to existing techniques.
Read the full abstract ▾
All authors Zhuohan Liu, Wujian Peng, Yitong Chen, Zuxuan Wu
Computer Vision
JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models
Jiachen Qian
Forensic
vision-
language models (VLMs) have recently been developed to detect image tampering and provide natural-
language explanations. However, their robustness against
adversarial manipulation remains underexplored. Existing
adversarial attacks typically aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment. In this paper, we study judgment-explanation consistent
adversarial attacks against forensic VLMs and propose JECA^2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment. On the visual side, JECA^2 uses Grad-CAM-guided perturbations to divert attribution from tampered regions toward benign regions. On the textual side, it optimizes
prompt embeddings toward authenticity-affirming semantics under a
token-proximity constraint. Experiments on forensic VLM
benchmarks show that JECA^2 achieves higher attack success and automated judgment-explanation consistency than implemented baselines under white-box threat settings, while transfer to closed-source VLMs remains measurable but limited. Our results highlight a consistency failure mode in explanation-based forensic VLMs and motivate future robustness evaluation beyond binary detection accuracy.
Read the full abstract ▾
Computer Vision
Internally Referenced Low-Light Enhancement
Peiyuan He, Hainuo Wang, Hengxing Liu et al. (+2 more)
Self-supervised low-light image enhancement (LLIE) is highly appealing as it eliminates the reliance on external paired data. However, the lack of external references causes networks to struggle with decoupling entangled illumination, delicate textures, and amplified noise. To resolve this challenge, we propose an Internally Referenced LLIE framework that extracts reliable physical and structural references from the degraded input image itself. First, we introduce a local exposure-simulated scheme to extract a low-frequency pseudo ground-truth. This serves as an internal physical reference to guide global illumination estimation and correct color casts. Second, we propose a dual-domain preservation strategy with spatial and spectral constraints to construct internal structural references. Specifically, an Illumination-Aligned Perceptual loss preserves global structures under illumination shifts, while a Shift-Invariant Spectral Correlation loss captures fine-grained local structures and suppresses high-frequency noise. Finally, we propose a Gain-Adaptive Feature Modulation (GAFM) mechanism to address highly spatially-variant residual noise. By transforming the self-estimated illumination map into an internal spatial gain prior, GAFM dynamically guides a blind-spot network for spatially-aware denoising. Extensive experiments demonstrate that our method achieves state-of-the-art performance, delivering superior noise suppression and textural fidelity. Code will be publicly released at https://visonj.github.io/IRLE/.
Read the full abstract ▾
All authors Peiyuan He, Hainuo Wang, Hengxing Liu, Mingjia Li, Xiaojie Guo
Computer Vision
Mining Multi-Modality Spatio-Temporal Cues for Video Important Person Identification
Xiao Wang, Minglei Yang, Bin Yang et al. (+4 more)
Identifying key individuals in
video scenes is essential for applications such as automated
video editing and intelligent surveillance. Current methods primarily focus on static images and immediate visual cues, overlooking the rich spatio-temporal information in
videos. This leads to the phenomenon of Temporal Importance Shift (TIS), wherein individuals deemed significant in early frames may be demoted as the entire temporal context is considered. To address this, we introduce the
Video Important Person (VIP) identification task, aimed at automatically identifying the most influential individuals in
videos while providing textual rationales. We present Temporal-VIP, a large-scale rationale-annotated
dataset consisting of 9,249
video segments across 11 categories with aligned importance rationales. To mitigate TIS, we develop the VIP-Net framework, which includes a Social Cue Encoder (SCE) for extracting multi-modal spatio-temporal cues, a Temporal Importance Rectifier (TIR) for hierarchical cue fusion and cross-modal
alignment, and VIP Inference for ranking individuals. Experimental results show that VIP-Net achieves 67.3% accuracy, significantly outperforming state-of-the-art models (37.5%-53.9%) and yielding a mean rationale similarity of 0.63 to ground truth through feature-guided LLM refinement. The
dataset and code are available at https://huggingface.co/
datasets/yml2002/Temporal-VIP.
Read the full abstract ▾
All authors Xiao Wang, Minglei Yang, Bin Yang, Wenke Huang, Zheng Wang, Xin Xu, Mang Ye
Computer Vision
Deformable Gaussian Occupancy: Decoupling Rigid and Nonrigid Motion with Factorized Distillation
Yang Gao, Wuyang Li, Po-Chien Luan et al. (+1 more)
Understanding dynamic 3D environments is essential for safe autonomous driving, particularly when
reasoning about human-centric, nonrigid
agents. However, existing weakly supervised occupancy prediction frameworks predominantly assume rigid-body motion and rely on simple frame-to-frame offsets, limiting their ability to capture fine-grained deformations and maintain temporal coherence. To address this issue, we propose DeGO, a deformable Gaussian occupancy framework that unifies decoupled Gaussian deformation with factorized 4D foundation-model
distillation. DeGO disentangles rigid and nonrigid motion, enabling each Gaussian primitive to evolve through both deformation and offset-based updates. In parallel, a factorized 4D
distillation strategy transfers cross-camera and cross-frame knowledge from the VGGT foundation model, producing foundation-aligned features that enhance temporal consistency. Experiments on the Occ3D-NuScenes
benchmark demonstrate that our method achieves state-of-the-art performance under weak supervision, delivering 13.5% gains on human-centric instances and 10.9% overall improvements. These results highlight the effectiveness of deformation-aware and foundation-guided occupancy modeling for dynamic scene understanding. The code is publicly available: https://github.com/vita-epfl/DeGO
Read the full abstract ▾
All authors Yang Gao, Wuyang Li, Po-Chien Luan, Alexandre Alahi
Computer Vision
Resolution-free neural surrogates for geometric parameterization and mapping with spatially varying fields
Yanwen Huang, Lok Ming Lui and Gary P. T. Choi
Many imaging problems require computing spatial transformations induced by spatially varying intensity, feature, or density fields. Canonical examples include distortion correction, deformable image registration, atlas-based
segmentation, and deformation-driven image analysis. These tasks can be formulated as
geometric mapping problems in which the transformation is constrained to preserve local structure, control boundary behavior, or regulate angular distortion. Such formulations typically lead to variational models,
diffusion processes, or
elliptic partial differential equations. However, repeatedly solving high-resolution systems becomes computationally expensive when the underlying parameter fields vary across instances. In this work, we propose a resolution-free neural surrogate for
geometric parameterization and mapping problems. Given a spatially varying parameter field $p:\Omega\to\mathbb{R}^m$ and query locations $\{x_i\}_{i=1}^N\subset\Omega$, the model predicts mapped locations $\{u(x_i)\}_{i=1}^N$ on arbitrary structured or unstructured point sets. To avoid dependence on a fixed grid, we use a multi-resolution
geometric encoding strategy that conditions the network on coordinate-augmented samples of the parameter field. The model is trained without labeled solution data by enforcing geometry-aware constraints derived from variational energies,
diffusion-based density equalization, and quasi-conformal theory. Experimental results on quasi-conformal mapping and density-equalizing mapping problems are presented to demonstrate the effectiveness of our proposed method.
Read the full abstract ▾
Computer Vision
GEM: Generative Supervision Helps Embodied Intelligence
Ruowen Zhao, Bangguo Li, Zuyan Liu et al. (+9 more)
Embodied Vision-
Language Models (VLMs) have demonstrated impressive performance and generalization in robotics, particularly within
Vision-
Language-Action frameworks. However, a significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in
embodied environments. In this paper, we introduce GEM, a Generative-supervised
Embodied vision-
language Model designed to bridge this divide. We propose integrating a depth map generation task directly into the VLM pre-training phase. By training this generative objective jointly with the main model, we observe substantial improvements in
embodied intelligence, significantly enhancing both semantic understanding and physical operation capabilities. To support this paradigm, we curate and release GEM-4M, a comprehensive large-scale
dataset featuring a mixture of grounding,
reasoning, and planning data paired with high-quality depth supervision. Extensive experiments demonstrate that GEM achieves state-of-the-art results across diverse
embodied benchmarks. Furthermore, our deployed action model, GEM-VLA, exhibits vastly superior task execution abilities in both simulation environments and real-world evaluations. Code, models, and
datasets are available at https://zhaorw02.github.io/GEM/
Read the full abstract ▾
All authors Ruowen Zhao, Bangguo Li, Zuyan Liu, Yinan Liang, Junliang Ye, Fangfu Liu, Diankun Wu, Zhengyi Wang, Xumin Yu, Yongming Rao, Han Hu, Jun Zhu
Computer Vision
DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
Chen Shi, Jinrui Xu, Shaoshuai Shi et al. (+3 more)
Pretrained foundation models have become an important basis for end-to-end autonomous driving. In contrast to
vision-
language models pretrained primarily on static image-text pairs,
video generative models capture temporal dynamics and motion priors that are naturally suited for driving. We present DriveWAM, a driving world-action model that adapts a pretrained
video diffusion transformer into an autoregressive
video-action policy. DriveWAM organizes
video and action streams into a unified temporal
token sequence and trains them under a joint flow-matching objective, preserving the pretrained
video-generation architecture while adapting its large-scale
video priors to action generation. To incorporate high-level scene understanding, we introduce scene-evolving driving guidance, where a frozen VLM produces chunk-specific semantic intent to guide
video-action generation. To keep long-horizon rollout bounded, we further introduce selective KV
memory, which maintains bounded modality-aware
video and action
memory pools through relevance-redundancy cache selection at inference time. Experiments on NAVSIM and the PhysicalAI-Autonomous-Vehicles
benchmark show that DriveWAM achieves strong planning performance, and a data-scaling study from 4k to 100k driving clips further confirms the scaling potential of world-action modeling for end-to-end autonomous driving.
Read the full abstract ▾
All authors Chen Shi, Jinrui Xu, Shaoshuai Shi, Kehua Sheng, Bo Zhang, Li Jiang
Computer Vision
Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
Cheng Chen, Pengpeng Zeng, Yuyu Guo et al. (+3 more)
Low-Rank Adaptation (LoRA) has emerged as a promising paradigm for Continual Learning. It independently updates its low-rank factors ($A$ and $B$), creating a composite update to the full weight
matrix through their interaction. To prevent catastrophic forgetting, this update should remain orthogonal to the task-specific subspace that contains previously learned knowledge. However, we identify that this composite update systematically violates this orthogonality, reintroducing interference and undermining stability. Furthermore, naively enforcing this orthogonality compromises plasticity, disrupting the delicate stability-plasticity trade-off. To resolve these issues, we propose Janus-LoRA, a framework that restores this balance through two novel components. Specifically, we first introduce Gradient Rectification, a closed-form solution that mathematically decouples LoRA's factor updates, enforcing orthogonality against the historical knowledge subspace identified by an efficient Online Estimation. Next, to enhance plasticity, we introduce a Decoupled Margin Loss that promotes feature-level separation by pushing new feature representations away from old ones, thus creating distinct, low-interference regions for new learning. Comprehensive experiments on challenging
benchmarks demonstrate that by harmonizing parameter-level orthogonality with feature-level separation, Janus-LoRA achieves a superior balance and establishes new state-of-the-art performance.
Read the full abstract ▾
All authors Cheng Chen, Pengpeng Zeng, Yuyu Guo, Lianli Gao, Hengtao Shen, Jingkuan Song
Computer Vision
DiscoForcing: A Unified Framework for Real-Time Audio-Driven Character Control with Diffusion Forcing
Kaiyang Ji, Bingsheng Qian, Binghuan Wu et al. (+3 more)
We study real-time audio-responsive character control as a deployment-faithful problem: strictly causal, bounded-latency streaming that must generate coherent full-body motion at interactive frame rates while the audio condition can change abruptly, including tempo shifts, drops, or user edits. Prior music-to-motion systems are largely optimized for offline generation with global context, and degrade in streaming rollouts where conditioning history becomes stale or unreliable. We introduce DiscoForcing, a streaming audio-driven
diffusion framework that combines a causal music encoder that captures rhythmic structure and phase dynamics with a
diffusion-forcing sequence model trained under heterogeneous noise levels across the temporal horizon. Building on this, we design a hybrid temporal schedule and a history-guided streaming sampler to explicitly trade off responsiveness against long-horizon consistency under non-stationary audio. Implemented in an end-to-end real-time interactive system with online avatar playback and
humanoid deployment workflows, DiscoForcing delivers more stable long-horizon rollouts and sharper audio-motion
alignment than prior baselines under matched causality and latency constraints while maintaining real-time throughput.
Read the full abstract ▾
All authors Kaiyang Ji, Bingsheng Qian, Binghuan Wu, Kangyi Chen, Ye Shi, Jingya Wang
Computer Vision
SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs
Jiawei Li, Ziyi Liu, Weijie Shi et al. (+3 more)
3D object grounding localizes referred objects in a 3D scene from natural
language. Unified instance-centric 3D-LLMs aim to solve grounding together with dialog, QA, and captioning, yet many rely on a single pointer-style grounding decision that compresses a relational instruction into one selection. This is brittle for fine-grained queries where multiple same-class candidates must be ruled out by context objects and spatial relations. We propose Structured Spatial
Reasoning 3D-LLM (SSR3D-LLM), a structured grounding interface for unified 3D-LLMs. Given fixed Mask3D object proposals, the LLM writes a sequence of latent spatial
reasoning steps and
memory tokens from the query, and a geometry-aware scorer reads these latent steps in order to refine candidate rankings step by step with step-length masking. The latent steps are learned from standard
benchmark target supervision with auxiliary referential-cue supervision during training, while inference uses only the input query and Mask3D proposals. Across ReferIt3D, ScanRefer, and Multi3DRef, SSR3D-LLM achieves the strongest results among unified 3D-LLM baselines, with substantial gains over the single-pointer QPG baseline on fine-grained grounding and consistent improvements over prior unified 3D-LLMs, while preserving the default
language-task route.
Read the full abstract ▾
All authors Jiawei Li, Ziyi Liu, Weijie Shi, Long Chen, Jiajie Xu, Xiaofang Zhou
Computer Vision
SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation
Changxuan Li, Nadine Berner, Nassir Navab et al. (+2 more)
Self-supervised depth estimation from
monocular sequences relies on the joint learning of a depth and a pose network. Despite abundant research done to improve the depth network, efforts on the pose remain limited. In this context, even when depth is estimated up to scale, we highlight the importance of the
alignment between the scene scales estimated by the pose and depth nets. Then, we introduce SA4Depth, an approach to improve this
alignment and boost the depth predictions while keeping the inference time unchanged. Our proposed method uses the depth estimated during training to reproject learnable visual features across consecutive frames and refine the pose estimates by reducing feature
alignment residuals. With our method, the estimated scene scales by the separate depth and pose networks are aligned, and the prediction scale consistency is improved across different sequences. Our differentiable refinement integrates seamlessly into existing self-supervised pipelines and substantially improves their depth estimates. We demonstrate this with extensive experiments both outdoors and indoors on KITTI, Cityscapes, and NYUv2. Additionally, results on KITTI Odometry confirm the effectiveness of our pose refinement. Our code is available at https://github.com/Runningchauncey/SA4Depth .
Read the full abstract ▾
All authors Changxuan Li, Nadine Berner, Nassir Navab, Federico Tombari, Stefano Gasperini
Computer Vision
REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection
Jun Zhou, Bingwen Hu, Yaxiong Wang et al. (+4 more)
Multimodal manipulation detection aims to simultaneously identify forged image–text pairs and localize tampered regions, yet existing methods typically rely on memorizing isolated artifacts and struggle with imperceptible manipulation traces or domain shifts. Inspired by human comparative
reasoning, we reformulate this task as a reference-grounded verification problem, where authenticity is assessed by comparing a query against retrieved authentic evidence. We propose REVEAL Reference-Enabled Verification for Evidence Analysis and Localization), a framework explicitly designed for this comparative paradigm. To support this paradigm, we construct a large-scale reference library comprising 170K authentic news image–text pairs featuring over 40K public figures. Technically, REVEAL employs a difference-aware fusion mechanism to capture fine-grained discrepancies between the query and retrieved evidence. Furthermore, we introduce a task-decoupled Mixture-of-Experts (MoE) architecture to jointly execute instance-level detection and fine-grained grounding, effectively mitigating optimization conflicts between these heterogeneous objectives. Extensive experiments demonstrate that REVEAL significantly outperforms state-of-the-art methods, and notably enables training-free domain adaptation by simply updating the reference library, offering a robust and practical solution for detecting evolving misinformation. Code is available at https://anonymous.4open.science/r/REVEAL-Reference-A006.
Read the full abstract ▾
All authors Jun Zhou, Bingwen Hu, Yaxiong Wang, Zhedong Zheng, Yongzhen Wang, Yuchen Zhang, Ping Liu
Computer Vision · Cross-listed from cs.AI
Jeong Hun Yeo, Chae Won Kim, Hyeongseop Rha et al. (+1 more)
Existing Visual
Speech Recognition (VSR) systems commonly rely on left-to-right autoregressive decoding, which can force premature decisions on visually ambiguous
tokens before sufficient context is available. We propose DLLM-VSR, to the best of our knowledge, the first
Diffusion Large
Language Model (DLLM)-based VSR framework, formulating transcription as iterative masked denoising with flexible-order decoding. With confidence-based unmasking, DLLM-VSR commits high-confidence positions early and uses the committed
tokens as bidirectional context to refine ambiguous ones. To adapt DLLMs to VSR, we introduce a two-stage masked-denoising training strategy that separates visual-to-text content
alignment from length modeling. We further observe a performance gap compared with an upper-bound setting where the ground-truth transcript length is provided at inference, allowing the model to focus on transcript content decoding. To reduce this gap, we develop length-guided candidate decoding, which uses
video duration to construct plausible transcript-length hypotheses and reranks the decoded candidates using length plausibility and decoding confidence. The proposed method achieves a 19.4% word error rate on LRS3, establishing state-of-the-art performance among methods using only LRS3 as labeled training data.
Read the full abstract ▾
All authors Jeong Hun Yeo, Chae Won Kim, Hyeongseop Rha, Yong Man Ro
Computer Vision
BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers
Jungwook Seo, Yoonsik Park, Changmin Lee et al. (+1 more)
Visual data from the Web power image classifiers, which often underpin many web services, such as recommendation and content moderation. However, the raw Web data often contain spurious correlations and social biases, and neural networks are known for their tendency to learn biases present in data. This can reinforce unfairness in web services and the web data, leading to a vicious cycle. In the context of image classification, networks learn bias attributes for a specific class when a majority of images contain the same attribute only for a given class. Hence, training a fair and debiased classifier from a biased
dataset demands handling an imbalanced problem between a majority of images with bias attributes (bias-aligned samples) and a minority without (bias-conflict samples). In this work, we introduce BiasEdit, a modular framework that automatically detects bias attributes from the original
dataset and edits them to construct a debiased
dataset. Specifically, BiasEdit first detects unknown bias attributes via statistical dependence and mutual information analysis of visual-linguistic representations, and then explicitly edits those attributes using text-guided image editing to generate realistic bias-conflict samples. Unlike prior works that assume known bias attributes or relies on synthetic mixing, our method operates without manual annotations and can leverage off-the-shelf
vision-
language and editing models. BiasEdit addresses a fundamental challenge in Web-sourced visual AI, mitigating
dataset-induced bias and achieving state-of-the-art debiasing performance even when training data are fully biased.
Read the full abstract ▾
All authors Jungwook Seo, Yoonsik Park, Changmin Lee, Sungyong Baik
Computer Vision · Cross-listed from cs.RO
Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments
Julia Hindel, Simon Bultmann, Houman Masnavi et al. (+2 more)
Self-supervised online traversability estimation enables
robots to continuously learn from unlabeled open-world experiences and adapt their
navigation behavior toward safe and efficient trajectories. Existing approaches either rely on handcrafted proprioceptive traversability scores, limiting
robot-agnosticism, or cluster prior data, preventing online learning. Moreover, many continual learning methods incur substantial
memory and computational costs, hindering onboard deployment. We introduce COTRATE, an online learning framework for continuous traversability estimation from
multimodal, unlabeled
robot experience. Our method first infers robust traversability scores using a
robot-agnostic, learning-based online terrain assessment module operating on proprioceptiveand inertial signals. These scores then supervise a visual traversability network through a novel
alignment loss that associates visual
embeddings with online terrain assessments. To mitigate forgetting during continual learning with minimal overhead, we propose a diversity-aware feature selection strategythat preserves performance using a compact replay
memory. We further show that the learned traversability representation supports knowledge transfer across different
robot platforms with different locomotion kinematics. We evaluate COTRATE on a
dataset of $\approx$ 50,000 images collected with two robotic platforms across 11 outdoor terrains, and
benchmark it on
navigation tasks in three representative outdoor environments. We make the
dataset, code, and trained models publicly available.
Read the full abstract ▾
All authors Julia Hindel, Simon Bultmann, Houman Masnavi, Daniele Cattaneo, Abhinav Valada
Computer Vision
Bayesian Gated Non-Negative Contrastive Learning
Peng Cui, Jiahao Zhang and Lijie Hu
While Contrastive Learning (CL) has revolutionized self-supervised representation learning, its latent representations remain highly entangled and opaque, limiting their interpretability in safety-critical applications. We identify that a fundamental cause of this
entanglement is the reliance on deterministic similarity measures, which treat all feature dimensions equally. In compositional scenes, this creates an Optimization Conflict: common background features, such as, "blue sky", are encouraged to align in positive pairs but simultaneously repelled in negative pairs, causing gradient oscillations that hinder precise semantic disentanglement. To address this, we propose BayesNCL (
Bayesian Gated Non-Negative Contrastive Learning). Unlike standard approaches, BayesNCL introduces a probabilistic gating mechanism that dynamically filters out task-irrelevant, high-frequency common features while selectively retaining discriminative semantics. By formalizing feature selection as a variational inference problem with a sparse Bernoulli prior, our method effectively resolves the optimization conflict. Empirical experimental results on Imagenet-100 demonstrate that BayesNCL achieves a remarkable 142.1% improvement in semantic consistency compared to state-of-the-art baselines, yielding highly interpretable representations without compromising downstream task performance. Code is available at https://github.com/Cui-Peng-624/BayesNCL.
Read the full abstract ▾
Computer Vision
Anomaly as Non-Conformity via Training-Free Graph Laplacian Energy Minimization
Jungwook Seo, Minjeong Kim, Younkwan Lee et al. (+2 more)
Detecting subtle visual anomalies in images remains challenging, particularly when only normal samples are available a priori. Such unsupervised anomaly detection is typically solved by measuring feature similarity of a query patch to a
memory of normal patches. However, similarity alone does not reveal how strongly a query patch violates the structure of the normal feature
manifold. We propose a training-free
Laplacian graph energy optimization formulation, named ANoCo that scores Anomaly by the cost of Non-Conformity of a query patch to align with a fixed normal
manifold. For each query patch, we construct a bipartite query to normal graph weighted by cosine affinity, explicitly removing query-query and normal-normal edges to prevent evidence dilution. We formulate anomaly scoring as a convex
Laplacian energy with anchored normal nodes, and solve in closed form. In particular, we do not use the optimized features themselves-the anomaly score is the magnitude of the update required to satisfy normality constraints, reframing the graph
Laplacian as a non-conformity operator rather than a smoothing prior. The proposed method introduces no learnable parameters, message passing, or sampling, and has complexity comparable to a single linear solve. Across standard
benchmarks, it delivers strong image-level AUROC, stable localization maps, and improved robustness over prior methods, demonstrating the effectiveness of using optimization-induced feature drift as anomaly measure.
Read the full abstract ▾
All authors Jungwook Seo, Minjeong Kim, Younkwan Lee, Seungho Shin, Sungyong Baik
Computer Vision
VITAL: Visual-Semantic Dual Supervision for Enhanced and Interpretable Latent Reasoning in Medical MLLMs
Qiaoru Li, Shaotian Liang, Jintao Chen et al. (+4 more)
Latent
reasoning enables
reasoning over continuous hidden states rather than explicit
tokens, avoiding the
language bottleneck and inference overhead of chain-of-thought for medical VQA. However, existing methods suffer from modality collapse, insufficient visual supervision, and train-inference mismatch. Moreover, their opaque latent states offer no interpretability, which is critical in clinical applications. We propose VITAL, a latent-space
reasoning framework for medical MLLMs with visual-semantic dual supervision: an auxiliary text decoder reconstructs
reasoning chains from latent states, while a visual projector regresses ROI features from a frozen, independent medical
vision encoder. Both modules are discarded at inference with zero overhead, yet can be re-attached post-hoc for dual interpretability, providing textual and visual explanations of the
reasoning process without sacrificing efficiency. We construct a 61K
dataset spanning 9 imaging modalities, exceeding prior medical visual latent
reasoning datasets by an order of magnitude. Experiments on 7
benchmarks show that VITAL consistently and substantially outperforms the backbone, all latent
reasoning baselines, and medical MLLMs trained on far larger data, achieving state-of-the-art results competitive with trillion-parameter proprietary models.
Read the full abstract ▾
All authors Qiaoru Li, Shaotian Liang, Jintao Chen, Haoran Sun, Yuxiang Cai, Jianwei Yin, Yankai Jiang
Computer Vision
EgoRelight: Egocentric Human Capture and Illumination Recovery for Relightable and Photoreal Avatar Rendering
Jianchun Chen, Yinda Zhang, Rohit Pandey et al. (+3 more)
Mixed Reality (MR) headsets promise a future of immersive telepresence where virtual humans blend indistinguishably into real or virtual surroundings. Achieving this
vision requires a method for capturing a user's motion, estimating appearance under novel lighting, and understanding the environment - all from the constrained viewpoint of a head-mounted display (HMD). Existing approaches treat these as isolated problems: they either focus on driving avatars with baked-in lighting or rely on studio setups for relighting. In this paper, we present EgoRelight, a holistic framework for
egocentric telepresence that simultaneously captures full-body human performance, synthesizes photorealistic and relightable appearance, and estimates high dynamic range (HDR) environment maps from a single HMD. First, to ensure motion and surface reconstruction, we propose an
egocentric perception module that leverages stereo down-facing cameras to extract dense depth maps, which serve as
geometric control signals to drive a mesh-based avatar. Second, we introduce a novel neural appearance model that learns to synthesize view-dependent specular and view-independent diffuse shading separately. By employing a specialized ray-sampling strategy, our model generalizes to unseen illumination without relying on restrictive analytical BRDF priors. Third, we enable seamless avatar integration into the physical world via a test-time inverse rendering process, which recovers an HDR environment map by matching the pre-trained avatar's appearance to live
egocentric camera observations. We demonstrate our system through a social telepresence application, where remote users are coherently relit according to their physical environment. Extensive experiments show that our components and the integrated system significantly outperform state-of-the-art baselines in
geometric accuracy and rendering as well as relighting fidelity.
Read the full abstract ▾
All authors Jianchun Chen, Yinda Zhang, Rohit Pandey, Thabo Beeler, Marc Habermann, Christian Theobalt
Computer Vision
A Two-Scan Deep Learning Model for Predicting Dementia in Mild Cognitive Impairment
Sara Fin, Alireza Moayedikia, David J. White et al. (+2 more)
Predicting which people with mild cognitive impairment (MCI) will develop dementia matters for starting treatment early, yet computational work on structural
brain imaging has relied almost entirely on a single scan. We propose TAFNet, a temporal
attention fusion network that combines a baseline and a follow-up T1-weighted scan. A pretrained Siamese encoder represents each scan, and a fusion module combines the two through anatomical difference, cross-temporal
attention and joint context, mixed by a learned per-patient gate. We evaluate it on paired scans from participants with MCI in the Alzheimer's Disease Neuroimaging Initiative, taken up to two years apart, with a dementia diagnosis within three years of the first scan as the outcome. The evaluation is designed to hold up under scrutiny: conversion is defined from recorded diagnoses, the pretraining pool shares no participants with the evaluation cohort, a held-out test partition is kept apart from model development, and uncertainty is estimated by resampling participants. TAFNet discriminates converters from non-converters well on the held-out partition. It outperforms conventional single-scan networks and a model that subtracts the two scans; both differences are significant in cross-validation and have the same direction on the smaller held-out partition. With everything else held fixed, adding the follow-up scan gives a significant gain in cross-validation. Against a recurrent CNN-LSTM baseline built on the same encoder, TAFNet performs comparably, with neither model consistently ahead. Operating points chosen on validation data, in place of the default threshold, give high sensitivity at a clinically reasonable specificity.
Read the full abstract ▾
All authors Sara Fin, Alireza Moayedikia, David J. White, Uffe Kock Wiil, Alicia Troncoso
Computer Vision
Sketch2Motion: Text-driven 2D Sketch to 3D Animation via Diffusion-guided Skeleton Optimization
Gaurav Rai and Ojaswa Sharma
Animation of 2D hand-drawn sketches provides an effective medium for visual communication. However, these sketches pose challenges, particularly in handling occlusions and accurately mapping motion. While 3D animation naturally addresses these challenges, estimating 3D motion remains a very complex task. Recent approaches to converting 2D sketches to 3D animations have mainly focused on specific types of motion, such as bipedal movements and facial expressions. We propose Sketch2Motion, a
diffusion-guided framework for skeleton-based motion synthesis that combines classical character animation pipelines with deep generative priors. Our method represents motion using skeletal transformations, which are propagated to mesh deformations via linear blend skinning. To guide the resulting animation toward realistic and semantically meaningful motion, we integrate a text-to-
video diffusion model via motion-aware score-
distillation sampling (MoSDS), enabling optimization without paired motion data. Additionally, we apply physics-inspired smoothness, topological, and contact constraints to stabilize optimization and preserve motion plausibility. Further, we integrate a spring-mass simulator to introduce secondary motion effects. The proposed framework is generalized, fully differentiable, modular, and compatible with biped, quadruped, and non-living articulated characters. Experiments demonstrate that our approach produces temporally coherent, text-aligned animations that outperform baseline motion transfer methods that lack generative priors or explicit physical constraints. We will make our code and
dataset publicly available.
Read the full abstract ▾
Computer Vision
Bound-Constrained Sparse Representation for Electrical Impedance Tomography
Chun Zhang and Dong Liu
This study proposes a bound-constrained sparse representation (BC-SR) framework for electrical impedance
tomography (EIT), aimed at improving conductivity estimation without explicit regularization. BC-SR adopts a representation-driven strategy, generating conductivity from low-dimensional latent variables via an implicit composite parameterization. Structural priors are embedded using a truncated graph-
Laplacian basis, while a bound-preserving nonlinear mapping enforces admissible conductivity ranges and improves conditioning through implicit gradient modulation. The approach ensures robust convergence, even under noisy or incomplete data. Extensive validation on 2D/3D simulations, tank experiments, and in-vivo lung data shows that BC-SR improves physical consistency and structural fidelity, offering enhanced robustness compared to traditional methods. Additionally, BC-SR enables 3D time-difference EIT reconstruction, offering improved spatial resolution and a more coherent representation of 3D conductivity distributions, particularly for in-vivo lung data. This suggests potential for improved performance in EIT, particularly in clinical applications for respiratory monitoring.
Read the full abstract ▾