Computer Vision
Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization
Zhaopeng Cui, Jiarui Hu, Jingbo Liu et al. (+7 more)
Indoor visual relocalization plays a critical role in emerging spatial and
embodied AI applications. However, prior research was predominantly devoted to low-level
vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various
datasets. Our source code will be released upon acceptance.
Read the full abstract ▾
All authors Zhaopeng Cui, Jiarui Hu, Jingbo Liu, Boming Zhao, Xiyue Guo, Boyin Feng, Haocheng Peng, Yujun Shen, Hujun Bao, Guofeng Zhang
Computer Vision
UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving
Xiaowei Gao, Pengxiang Li, Yitai Cheng et al. (+4 more)
Recent
multimodal large
language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade-off between temporal
reasoning and spatial precision. Models that rely on single-frame or low-resolution inputs often miss small, distant, or partially occluded hazards, while
language-centric driving models frequently provide limited grounded evidence for their explanations. To address this gap, we propose UniDrive, a unified visual-
language and grounding framework for interpretable risk understanding in autonomous driving. UniDrive combines a temporal
reasoning branch that models scene dynamics from multi-frame visual input with a high-resolution perception branch that preserves fine-grained spatial details from the latest frame. The two branches are integrated through a gated cross-
attention fusion module, enabling dynamic context to be aligned with precise spatial evidence. Based on the fused representation, UniDrive jointly generates natural-
language risk descriptions and grounded bounding-box outputs for risk objects. Experiments on the DRAMA-
Reasoning benchmark show that UniDrive outperforms representative image-based and
video-based baselines in both captioning and risk-object grounding. In particular, UniDrive achieves the best overall performance on the validation split and demonstrates clear advantages in small-object localization, zero-shot generalization to NuScenes and BDD100K, and human-rated interpretability and trustworthiness. These results suggest that explicitly combining temporal semantics and high-resolution perception provides a stronger foundation for interpretable and safety-oriented autonomous driving systems. The code is available at https://github.com/pixeli99/unidrive-dev.
Read the full abstract ▾
All authors Xiaowei Gao, Pengxiang Li, Yitai Cheng, Ruihan Xu, James Haworth, Stephen Law, Yun Ye
Computer Vision
Adaptive Hebbian Memory Routing in Vision Transformers for Few-Shot Learning
Mohammed Yusuf Mujawar and Noorbakhsh Amiri Golilarz
Few-shot image recognition requires models to adapt to new classes from a small labeled support set. Hebbian fast-weight
memory can provide temporary associative information during an episode, but fixed
memory behavior may not be appropriate for every few-shot task. In this work, we propose Adaptive Hebbian Routing for few-shot
Vision Transformers. The method uses a lightweight MLP router to control the contribution of Hebbian
memory, the strength of
memory updates, and the retention of previous
memory from support-set features. We study Adaptive Placement, Adaptive Plasticity, and Fully Adaptive Hebbian Routing. Experiments use ViT-Small, DeiT-Small, and Swin-Tiny under 5-way 1-shot evaluation on Omniglot, CIFAR-FS, and cross-domain transfer from CIFAR-FS to Omniglot. In the direct Swin comparison, fixed and adaptive Hebbian variants use the same
memory location. Adaptive Plasticity improves the fixed Hebbian result from 96.74% to 96.92%, while Fully Adaptive Routing achieves the best result at 96.94%. The fully adaptive Swin model also reduces inference time from 16.51 ms to 14.05 ms relative to fixed Hebbian Swin. On CIFAR-FS, adaptive variants improve performance across all three backbones, and the multi-shot evaluation shows that these gains remain useful as the number of support examples increases. These results show that adaptive plasticity and adaptive
memory activation can improve few-shot
Transformer representations beyond fixed Hebbian behavior.
Read the full abstract ▾
Computer Vision
BioMedVR: Confusion-Aware Mixture-of-Prompt Experts for Biomedical Visual Reprogramming
Jiaxiang Liu, Tianxiang Hu, Juwei Guan et al. (+5 more)
Recent advances in
vision-
language models (VLMs) such as CLIP have demonstrated strong generalization across natural-image domains. However, adapting these models to biomedical imaging is non-trivial: full-model fine-tuning is computationally expensive, while medical data are often scarce and exhibit subtle, fine-grained inter-class differences, making parameter-efficient adaptation particularly critical. Visual Reprogramming (VR) offers a parameter-efficient alternative by injecting learnable perturbations into the input space, but existing VR approaches for VLMs mainly focus on positive class
prompts and overlook confusing negatives, leading to miscalibrated predictions in fine-grained medical scenarios. We present BioMedVR, the first VR-based framework for biomedical imaging, enabling few-shot adaptation of pretrained VLMs through compact learnable VR modules. To mitigate class confusion, we introduce a Confusion Minimization Mechanism that leverages LLM-generated confusion-aware attributes together with a Confusion-Suppression Loss to explicitly reduce false-positive
alignment. Moreover, the designed Mixture-of-
Prompt Experts combines a positive expert for main-class discrimination and a negative expert for confusion suppression, balanced via adaptive gating. Extensive experiments on 18
datasets, including 11 biomedical
datasets and 7 natural image
benchmarks, demonstrate that BioMedVR achieves superior accuracy and generalization, effectively bridging VR and VLMs in biomedical domains.
Read the full abstract ▾
All authors Jiaxiang Liu, Tianxiang Hu, Juwei Guan, Yujie Wu, Yusong Wang, Yao Mu, Zuozhu Liu, Mingkun Xu
Computer Vision
VSANet: View-aware Sparse Attention Network for Light Field Image Denoising
Gargi Panda, Soumitra Kundu, Saumik Bhattacharya et al. (+1 more)
Light field (LF) image denoising is challenging due to the high-dimensional structure of LF data. While noise is independent across sub-aperture images, scene content exhibits strong cross-view correlations. We introduce VSANet, a view-aware sparse
attention network for LF denoising. Specifically, we propose a view-aware sparse
attention (VSA) block that represents the 4D LF feature map as a unified spatial-angular
token space and performs cross-view aggregation via locality-sensitive hashing-based sparse
attention. This enables global feature interactions with linear complexity, effectively exploiting LF correlations across views and spatial locations. In addition, we design a feature refinement (FR) block to emphasize informative features in spatial, angular, and epipolar subspaces. The VSA and FR blocks are integrated within a sequential
attention refinement module, forming the core of VSANet. Experiments demonstrate VSANet outperforms stateof-the-art LF denoising methods.
Read the full abstract ▾
All authors Gargi Panda, Soumitra Kundu, Saumik Bhattacharya, Aurobinda Routray
Computer Vision
SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards
Sheng Xia, Zhengqin Lai, Tianxiang Jiang et al. (+4 more)
Video MLLMs often struggle with fine-grained spatio-temporal
reasoning, sometimes generating correct answers based on irrelevant frames or objects. Although outputting spatio-temporal evidence during
reasoning is a promising direction, existing RL frameworks typically rely on geometry-only (IoU) rewards, which can be sensitive to boundary perturbations and overlook semantic
alignment. To address this, we propose Semantic Evidence Reward (SER), which reformulates spatio-temporal evidence grounding as a constrained verification task. Instead of computing pixel-level overlap, SER uses a referee VLM as a local checker to evaluate model-generated evidence claims across two dimensions: relevance and localization quality, combined with a temporal penalty. This design reduces the reliance on dense box annotations and enables training directly on standard
video QA data. On the V-
STAR benchmark, SER achieves 49.6% mLGM, improving by 3.0 points over the strong evidence-grounded baseline Open-o3-
Video, demonstrating its potential in enhancing both answer accuracy and evidence grounding.
Read the full abstract ▾
All authors Sheng Xia, Zhengqin Lai, Tianxiang Jiang, Kanghui Tian, Shoujun Zhou, Bin Li, Yi Wang
Computer Vision
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
Jonas Klotz, Cassio F. Dantas, Pallavi Jain et al. (+2 more)
Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from
vision and
vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human-grounded evaluation framework that quantifies
alignment between SAE latents and human-annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention-style evaluation in
vision, we construct synCUB and synCOCO, synthetic
benchmarks of paired images that differ in exactly one attribute. We introduce Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and consistently outperforms one-to-one baselines. For functional validation, we propose a Targeted Attribute Perturbation
Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image-level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2
embeddings, we find that increased overcompleteness can reduce perturbation
alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs. Code and
datasets are available at https://github.com/JonasKlotz/sae-concept-eval.
Read the full abstract ▾
All authors Jonas Klotz, Cassio F. Dantas, Pallavi Jain, Diego Marcos, Begüm Demir
Computer Vision
Agentic Collaborative Cognition for Zero-Shot 3D Understanding
Wenxin Wang, Bo Zhang, Feng Chen et al. (+4 more)
Recent advancements have explored agentic zero-shot 3D understanding by reformulating it as
video keyframe understanding with
Multimodal Large
Language Models (MLLMs). However, existing methods face an intrinsic bottleneck due to the finite observation perspectives inherent in
videos and the implicit perception of 3D scenes. In this paper, we propose a collaborative multi-
agent framework that assigns a Planning
Agent to handle high-level viewpoint planning and supplement novel perspectives, and a Perception
Agent to explicitly summarize the 3D scene into a structured holistic cognitive map. Specifically, Planning
Agent first analyzes this cognitive map to determine query-relevant viewpoints and supplements missing critical perspectives to ensure comprehensive observation. Subsequently, Perception
Agent documents object-level attributes from these views by assigning consistent instance identifiers across viewpoints, thereby integrating fragmented observations into the holistic cognitive map. In parallel, it provides feedback to filter out mismatched candidate objects and guide subsequent viewpoint planning. Through this closed-loop iterative process, two
agents collaboratively figure out candidates until Perception
Agent determines that sufficient information has been captured to complete the task. Extensive experiments demonstrate that our method achieves state-of-the-art performance on 6
benchmarks, with improvements of 11.1% Acc@0.5 on ScanRefer, 14.6 BLEU-1 on 3D-assisted dialog, and 2.1 EM on SQA3D.
Read the full abstract ▾
All authors Wenxin Wang, Bo Zhang, Feng Chen, Zixuan Wang, Wen Li, Changsheng Li, Yinjie Lei
Computer Vision · Cross-listed from cs.RO
ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos
Pranjal Mishra, René Zurbrügg, Max Wilder-Smith et al. (+4 more)
Deploying
robots in unstructured real-world environments needs accurate, interactive models of the objects. Constructing these models at scale remains a critical bottleneck for robotic system integration. We present ArtiTwinSplat, a framework that automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D
videos, requiring no CAD models, simulation assets, or manual annotations. Our method is built on 3D Gaussian Splatting that preserve
geometric fidelity and photometric realism, coupled with an unsupervised articulation discovery pipeline that recovers part structure and joint kinematics from observed motion alone. With tracking and optimization stages our method provides stable, queryable digital twins that support real-time rendering, viewpoint control, and interactive manipulation. Unlike prior methods confined to simulation, ArtiTwinSplat operates directly on real-world observations and produces twins that are immediately usable by downstream
robot planning and learning systems. This method offers a practical, scalable pathway toward digital twin construction, lowering the integration barrier for articulated object manipulation in
embodied AI and human-
robot collaboration contexts.
Read the full abstract ▾
All authors Pranjal Mishra, René Zurbrügg, Max Wilder-Smith, Marco Hutter, Marc Pollefeys, Zuria Bauer, Hermann Blum
Computer Vision
ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
Zhentao Guo, Chen Duan, Tongkun Guan et al. (+3 more)
Despite remarkable progress in
multimodal understanding, current MLLMs still exhibit limitations in
video text understanding, particularly when semantics emerge through the integration of temporally distributed textual cues across multiple frames. This perception challenge fundamentally differs from static image text understanding, yet existing
datasets fail to capture: the vast majority of questions remain answerable from single frames, inadequately reflecting real-world
video text comprehension demands. To address this, we present ViTexQA, a large-scale
video-text QA
dataset, and FrameThinker for robust multi-frame temporal
reasoning. We build ViTexQA via a quality-controlled Chain-of-Thought (CoT) annotation pipeline boosted with temporal constraints; all its QA pairs demand cross-frame text fusion to solve, enforcing true temporal reliance. FrameThinker adopts two-stage training for explicit temporal modeling: CoT-Guided Supervised Fine-Tuning (SFT) generates frame-aware
reasoning chains, followed by Temporally-grounded Reinforcement Learning (RL) optimized with multi-frame coherence rewards. Evaluations show our method outperforms SOTA baselines on ViTexQA, lifting ROUGE-L by 6.3%.
Read the full abstract ▾
All authors Zhentao Guo, Chen Duan, Tongkun Guan, Zining Wang, Kai Zhou, Pengfei Yan
Computer Vision
EERLoss: A Novel Loss Function for Training Deep Biometric Models. A Case Study in Keystroke Dynamics
Nahuel Gonzalez, Marta Robledo-Moreno, Ivan DeAndres-Tame et al. (+2 more)
Deep learning approaches to biometric verification are commonly trained by optimizing indirect objectives, creating a misalignment between the optimization process and the primary evaluation metric, typically the Equal Error Rate (EER). This paper introduces EERLoss: a subdifferentiable, arbitrarily accurate approximation to EER for training deep biometric models. Furthermore, this framework has the potential to be adapted to optimize any specific operating point on the DET curve, enhancing its generalizability. To validate this approach, EERLoss is evaluated on a particularly demanding behavioral biometric modality: keystroke dynamics verification. This task is characterized by its high intra-class and low inter-class variability. Experiments are conducted on the large-scale KVC-onGoing
benchmark, incorporating data from over 185,000 subjects across different scenarios. A comprehensive ablation study initially demonstrates the superiority of EERLoss in comparison to existing state-of-the-art loss functions. It also converges substantially faster compared to other losses, reducing the overall training cost. Additionally, a comparison is made between the proposed loss and the KVC-winning architecture by re-training it with EERLoss, demonstrating that the proposed approach significantly outperforms the original SoTA, achieving a relative EER reduction of up to approx. 30%. This improvement on a challenging, large-scale
benchmark validates the effectiveness of EERLoss as a task-aligned training objective specifically suited for high-variance biometric traits.
Read the full abstract ▾
All authors Nahuel Gonzalez, Marta Robledo-Moreno, Ivan DeAndres-Tame, Ruben Vera-Rodriguez, Ruben Tolosana
Computer Vision
Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learning
Julien Khlaut, Charles Corbière, Baptiste Callard et al. (+9 more)
Vision-
language contrastive pretraining has become the dominant recipe for 3D medical foundation models, leveraging the large volumes of paired scans and reports produced in clinical practice. However, medical images usually span dozens of organs, and radiological reports are much longer than typical natural image captions and are composed of multiple structured sections. CLIP-style pretraining compresses this structure by encoding each modality into a single global
token, at the risk of losing important details. We introduce ConQuer (Concept Queries), an image-text pretraining method that augments CLIP's global
alignment with a set of localized
alignments, one per concept. ConQuer splits the report into concept-specific sections and learns cross-
attention queries that pool the matching image features without using any
segmentation mask or spatial supervision. Contrastive learning is then applied independently for each concept. Concepts can be any unit of semantic localization; here, they are anatomical regions, one query per organ or gross body region. As a byproduct, each query learns
attention maps focused on its concept, providing built-in spatial interpretability. We use ConQuer to train Jolia, a 3D CT foundation model on chest and abdominal CT. Jolia consistently outperforms a CLIP baseline on findings classification, report generation, and cross-center transfer, and sets a new state of the art across multiple public
benchmarks. Jolia's weights are available at https://huggingface.co/raidium/Jolia
Read the full abstract ▾
All authors Julien Khlaut, Charles Corbière, Baptiste Callard, Amaury Prat, Leo Butsanets, Antoine Saporta, Théo Danielou, Leo Machado, Korentin Le Floch, Tom Boeken, Pierre Manceron, Corentin Dancette
Computer Vision
Multilevel Stochastic Plug-and-Play for Sparse-View CT Reconstruction
Antoine De Paepe, Alexandre Bousse and Dimitris Visvikis
Sparse-view computed
tomography (SVCT) reduces radiation exposure and acquisition time, but the limited number of projection views makes the reconstruction problem severely ill-posed and leads to streak artifacts when analytical methods are used. Plug-and-Play (PnP) methods combine data fidelity with learned image priors, while
stochastic PnP methods use re-noising to reduce the mismatch between denoiser training inputs and reconstruction iterates. However, these methods often require many iterations, which limits their practical efficiency. In this work, we propose multilevel (ML)
Stochastic Plug-and-Play (ML-SPnP), a ML
stochastic PnP method for SVCT reconstruction formulated on wavelet multiresolution analysis (MRA) approximation spaces. The method combines coarse-to-fine initialization with a finite sequence of ML corrections using resolution-specific denoisers. Under suitable assumptions, the fine-level iterations following the finite ML phase inherit the convergence guarantees of the underlying
stochastic PnP
algorithm. Experiments on two CT
datasets with 20-50 projection views show that ML-SPnP achieves reconstruction quality comparable to the evaluated
stochastic PnP methods, while generally requiring less reconstruction time. These results support ML acceleration as an effective strategy for reducing the computational cost of
stochastic PnP reconstruction. The code will be made publicly available at https://github.com/antoinedepaepe/ml-spnp.
Read the full abstract ▾
Computer Vision
PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments
Zhenyang Li, Lutao Jiang, Yizhou Zhao et al. (+4 more)
Reconstructing realistic, physically plausible garments from a single image remains a fundamental challenge. Template-free methods capture surface geometry but lack explicit sewing structure for simulation; while programmatic systems are simulation-ready but constrained by predefined templates. This reveals a fundamental representation gap between
geometric reconstruction and structured garment construction. We present PatternGSL, a structured garment representation in the form of a template-free and learnable specification
language that encodes complete sewing patterns, including panel boundaries, parameterized seams, and explicit stitch
topology, in a compact and standardized form. PatternGSL preserves the physical rigor of pattern-based models while removing template dependence, elevating sewing structure as a first-class target for generative modeling. We further propose a
vision-
language framework that predicts PatternGSL specifications directly from a single image and decodes them into garments using lightweight deterministic validity handling, without optimization-based refinement or manual cleanup. In addition, we introduce PatternGSLData, the first large-scale image-to-GSL paired
dataset comprising 300K samples with complete sewing pattern annotations, enabling supervised VLM training for structured garment reconstruction. Experiments demonstrate improved pattern accuracy over prior baselines, explicit sewing-structure recovery, reliable cloth simulation, and pattern-level editing through the same deterministic decoding pipeline. Code and data-processing scripts will be released at https://lagrangeli.github.io/PatternGSL/.
Read the full abstract ▾
All authors Zhenyang Li, Lutao Jiang, Yizhou Zhao, Ying-Cong Chen, Xin Wang, Weikai Chen, Yifan Peng
Computer Vision
Quantum CT via Dynamic Interval Encoding and Prior-Balanced QUBO Reconstruction
Ao Wang, Yikuang Yuluo, Yujie Liu et al. (+7 more)
Quadratic unconstrained binary optimization (QUBO)-based
quantum computed
tomography (CT) casts reconstruction as a binary quadratic problem for
quantum annealing and hybrid
quantum–classical solvers. For grayscale CT, however, image encoding is constrained by the binary-variable budget: fixed global bit-plane encodings increase QUBO size and coupling complexity as gray-level precision improves, whereas low-bit encodings introduce quantization error. We propose a QUBO-based grayscale CT reconstruction framework that combines dynamic interval encoding with prior-balanced optimization. Each refinement round encodes active pixels only within local gray-level intervals around the current estimate, and a boundary-hit-guided update rule adaptively switches between search expansion and local refinement. To improve optimization stability, the method balances projection-domain data consistency and an edge-preserving quadratic prior before forming the final QUBO. Sparse-view and limited-angle fan-beam CT experiments show that the proposed method recovers structures and gray-level distributions more faithfully than the evaluated analytic, iterative, variational, and representation-based baselines. Expressivity analysis and ablation studies further indicate that the improvement mainly arises from effective gray-level representation through dynamic local encoding and more stable data-fidelity–prior coupling. Experiments on the D-Wave hybrid binary quadratic model (BQM) solver further demonstrate that the formulation is executable on a hardware-backed hybrid
quantum–classical backend.
Read the full abstract ▾
All authors Ao Wang, Yikuang Yuluo, Yujie Liu, Shuangyang Zhong, Yuwen Zhang, Zihao Wang, Fenglin Liu, Andreas Maier, Haijun Yu, Yixing Huang
Computer Vision
Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation
Wuming Yang, Xiang Zhang and Hongmin Zhao
Heterogeneous Knowledge
Distillation (HKD) aims to transfer knowledge across varying architectures (e.g., from
Transformer to CNN) but inherently suffers from severe training instability. We reveal that this instability stems from two highly coupled challenges: massive feature norm discrepancies that cause optimization drag, and severe gradient conflicts between the primary and
distillation objectives arising from distinct inductive biases. To achieve stable
distillation, we propose SPOFA, a framework built upon a novel Feature and Gradient Dual Stabilization mechanism. Specifically, at the feature level, we introduce a LayerNorm-based decoupling projector that explicitly decouples feature magnitude from direction, creating a bounded and stable space for semantic
alignment. At the gradient level, we propose a momentum-driven Exponential Moving Average (MEMA) dynamic scaler. By establishing a robust historical baseline of the optimization trajectory, MEMA actively evaluates instantaneous gradient conflicts and adaptively penalizes harmful
distillation signals, guaranteeing stable convergence. Importantly, SPOFA achieves this dual stabilization with an extremely lightweight parameter footprint. Extensive experiments on two mainstream
benchmarks demonstrate that SPOFA achieves state-of-the-art accuracy, significantly outperforming computationally expensive methods while introducing only minimal computational overhead compared to standard baselines.
Read the full abstract ▾
Computer Vision
Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning
Jiayi Lei, Yuandong Pu, Xingyu Han et al. (+8 more)
Text-to-image (T2I) generation models have achieved remarkable progress in producing visually realistic images from natural
language prompts. Yet it remains unclear whether their success reflects genuine causal understanding or sophisticated pattern matching over visual-textual correlations. Inspired by Russell's inductivist turkey, we introduce Counterfactual-World (CF-World), a counterfactual
benchmark designed to investigate whether text-to-image models can generate images under rules that systematically contradict real-world priors. CF-World organizes each scenario into three progressive levels: factual generation under ordinary world knowledge, explicit counterfactual generation with direct visual instructions, and implicit counterfactual generation requiring causal deduction from altered rules. We evaluate both open-source and closed-source T2I models using a
Vision Language Model (VLM)-based evaluator (CF-Eval). Furthermore, we introduce two metrics: Prior Resistance Rate (PRR), which measures a models' ability to overcome entrenched real-world priors, and
Reasoning Retention Rate (RRR), which assesses whether models can maintain
reasoning-dependent counterfactual generation without explicit visual cues. Experiments show that all models exhibit sharp degradation from factual to counterfactual settings. Further analyses suggest that these failures arise because current T2I models encode world knowledge and visual appearances as tightly coupled patterns. Consequently, their heavy reliance on frequent visual co-occurrences within the training data forces them to default to familiar commonsense priors when tasked with rendering counterfactual worlds.
Read the full abstract ▾
All authors Jiayi Lei, Yuandong Pu, Xingyu Han, Rongpeng Zhu, Jing Xu, Jinyao Wang, Zijian Zhou, Bin Fu, Yuewen Cao, Yihao Liu, Hongsheng Li
Computer Vision
PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought
Ling Li, Bowen Liu, Zinuo Zhan et al. (+5 more)
Pointing-based visual grounding requires models to precisely locate target objects by deciphering complex spatial relationships between the visual scene and pointing gestures. Traditional methods typically encode input images into static feature representations and perform
reasoning primarily within the linguistic domain, often overlooking the rich perceptual cues and explicit spatial geometry inherent in images. In this study, we aim to mitigate the cognitive vulnerability of models in interpreting gestural spatial relations by proposing PointVG-R, a
reasoning-guided Multi-modal Large
Language Model (MLLM). PointVG-R introduces
geometric-aware
reasoning for pointing-based grounding, enabling the model to think with images through the strategic integration of Reinforcement Learning (RL) and cold-start data. Specifically, we design a novel
geometric reasoning pipeline that simulates the iterative cognitive process humans employ when interpreting pointing gestures. Furthermore, we construct EgoPoint-CoT, a high-quality visual Chain-of-Thought (CoT)
dataset featuring detailed
reasoning trajectories to guide the model via Supervised Fine-Tuning (SFT) and RL. To address the varying quality of learning signals encountered during training, we further propose an Adaptive Importance Weighting strategy based on Group Variance, which dynamically adjusts reward signals to optimize the learning process. Experimental results demonstrate that PointVG-R achieves SOTA performance, outperforming the baseline by $\textbf{15.86}$ points in mIoU. Extensive ablation studies further validate the efficacy of our proposed modules. Code: https://github.com/lingli1724/PointVG-R.
Read the full abstract ▾
All authors Ling Li, Bowen Liu, Zinuo Zhan, Jianhui Zhong, Ziyu Zhu, Bingcai Wei, Kenglun Chang, Zhidong Deng
Computer Vision
ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization
Lei Xu, Haowei Wang, Shen Chen et al. (+3 more)
Multi-modal Large
Language Models (MLLMs) offer powerful
reasoning for forensic tasks, yet existing approaches utilizing exogenous
segmentation decoders often suffer from suboptimal localization. The reliance on stitched pipelines introduces information bottlenecks during backpropagation, which dilutes spatial signals and is limited by semantic priors of the segmentor. To address these limitations, we propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task. ForensicsTok directly generates spatially grounded
token sequences, enabling precise mask prediction without intermediary supervision. Specifically, we introduce a
Token Splatting Decoder (TSD) to map
tokens to binary masks via codebook-aware code smoothing, which mitigates sharp gradients from deterministic detokenizers. Furthermore, to capture diverse tampering clues, we propose a Hierarchical Expert Fusion (HEF) module that injects multi-scale features from a forensic expert model. This unified architecture effectively compensates for the lack of forensic priors in standard MLLMs. Extensive experiments on six
benchmarks show that ForensicsTok substantially improves over existing MLLM-based baselines and slightly improves over strong forensic expert baselines, while exhibiting stronger robustness to perturbations.
Read the full abstract ▾
All authors Lei Xu, Haowei Wang, Shen Chen, Taiping Yao, Bin Li, Changsheng Chen
Computer Vision
VisCritic: Visual State Comparison as Process Reward for GUI Agents
Jiachen Qian
GUI
agents powered by
vision-
language models show strong potential for automating digital tasks, yet frequently fail in long-horizon scenarios due to the absence of step-level verification. Existing process reward models verify actions through textual
reasoning alone, missing the visual nature of GUI state changes. We introduce VisCritic, a visual process reward framework that verifies
agent actions by directly comparing pre-action and post-action screenshots in visual feature space. VisCritic employs a Siamese
vision transformer to extract change-aware representations, coupled with an Action-Aware Critic Head that jointly evaluates action success, task progress, and error type. A critic-training data construction pipeline generates weakly supervised samples from existing trajectories without additional human labels for critic training. Experiments and offline analyses across five
benchmarks demonstrate that VisCritic serves as a plug-and-play enhancement for diverse GUI
agents, generally improving
benchmark metrics while providing visual diagnostic cues.
Read the full abstract ▾
Computer Vision
What Do Flow-Based Inverse Solvers Approximate? A Posterior-Transport View
Jian Xu, Delu Zeng, John Paisley et al. (+1 more)
A growing family of training-free solvers – FlowDPS, FLOWER, PnP-Flow and their
diffusion ancestors (DPS, DAPS) – repurpose a pretrained flow-matching prior to solve imaging inverse problems by adding a measurement-guidance term to the deterministic probability-flow ODE. Despite strong empirical results, what these per-step corrections actually approximate – and how far the resulting samples are from the true posterior $p(x\mid y)$ – has not been characterized. We give a posterior-transport account of flow-based inverse problem solving. Our starting point is a simple but consequential fact: for a deterministic flow prior,
Bayesian conditioning is realized entirely by a reweighting of the source distribution, not by a drift correction; pushing the reweighted source through the unmodified velocity field yields exact posterior samples. From this we show that trajectory-guidance solvers can be read as the minimum-kinetic-energy correction field needed to morph the unconditional source into the posterior, and that FlowDPS / FLOWER / PnP-Flow correspond to distinct zeroth-order / Gaussian / proximal approximations of this single object; we bound the resulting posterior bias in Wasserstein distance. A controlled $2$D study with a closed-form posterior confirms the theory decisively: source reweighting matches the true posterior to the Monte-Carlo floor on every metric, whereas trajectory guidance incurs $200$–$800\times$ larger error and collapses posterior modes, regardless of guidance strength. Guided by the analysis we propose a cheap, principled velocity-correction solver that is competitive across two in-domain priors (AFHQ, CelebA) and two out-of-distribution settings while, unlike point-estimate source-space optimizers, producing diverse posterior samples with uncertainty that correlates with reconstruction error.
Read the full abstract ▾
All authors Jian Xu, Delu Zeng, John Paisley, Qibin Zhao
Computer Vision
GeoIMO: Geometry-Driven Independent Motion Classification for Event Cameras
Anil Bayram Gogebakan, Filippo Marostica, Alessio Caviglia et al. (+2 more)
Existing automotive event
datasets rely on appearance-based annotations from frame pipelines, making them poorly suited for motion-aware event perception. We present a geometry-driven, annotation-free framework that classifies detected objects as static or independently moving by exploiting ego-motion structure directly from the event stream. A Focus of Expansion model with yaw compensation estimates global background motion, while objects are labeled as moving when local motion deviates from this prediction, as quantified by a scale-invariant residual. Temporal stabilization improves robustness across consecutive event windows. The method requires no learning, no manual motion labels, and works with any input bounding boxes. Experiments on MVSEC and the Prophesee 1 Megapixel Automotive Detection
dataset demonstrate consistent performance across diverse driving scenarios, with yaw compensation improving results during turns and a simple translational local model offering a favorable accuracy-efficiency trade-off.
Read the full abstract ▾
All authors Anil Bayram Gogebakan, Filippo Marostica, Alessio Caviglia, Alessandro Savino, Stefano Di Carlo
Computer Vision
VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection
Ling Li, Zhizhen Cai, Xinkun Wu et al. (+4 more)
Grounding deictic gestures in natural images is fundamental to AR and human-
robot collaboration, providing a basis for seamless spatial interaction. While
Transformer-based visual models have achieved significant progress in general object detection, their global
attention mechanisms often neglect micro-
geometric relationships, degrading orientation accuracy. In pointing tasks, this deficiency manifests as an inability to accurately capture the pointing ray implied by finger poses, which results in pointing drift and localization ambiguity when dealing with distant or densely packed objects. To address this, we propose VistaRef, a framework designed to explicitly enhance spatial orientation awareness. First, we develop the Local Hand Entity Modeling (LHEM) module, which incorporates hand-pose
embeddings to strengthen the model's capability to capture subtle finger deviations. Second, drawing inspiration from multi-view geometry, we construct the
Geometric Ray Modeling (GRM) module to transform implicit orientation information into explicit spatial
geometric features, guiding feature aggregation and deep fusion via
attention mechanisms. Furthermore, we introduce a novel Orientation-Consistent
Alignment Loss (OCAL) to synergistically supervise hand presence and pointing consistency, ensuring that all architectural improvements collectively serve the core objective of spatial localization. Experimental results demonstrate that VistaRef significantly outperforms the baseline, achieving a 14-point absolute gain in grounding accuracy. Qualitative analysis further confirms that VistaRef effectively models the
geometric correlation from hand to target, bridging the spatial perception gap inherent in traditional
Transformers for complex scenarios. Code: https://github.com/lingli1724/VistaRef.
Read the full abstract ▾
All authors Ling Li, Zhizhen Cai, Xinkun Wu, Ziyu Zhu, Jiaqing Lyu, Bowen Liu, Zhidong Deng
Computer Vision
RetiSEM: Generalising Causal Models for Fragmented Biomedical Data
Inam Ullah, Imran Razzak and Shoaib Jameel
Learning causal models from fragmented biomedical data is challenging because clinical, molecular, and imaging variables are often incomplete or not jointly observed. We propose RetiSEM, a domain-constrained structural equation modelling (SEM) framework for causal graph recovery and mediation analysis under limited
multimodal resources. This proposed work organises variables into biologically informed blocks, applies forbidden-edge constraints, and decomposes pathway-level effects into TE, NDE, and NIE components. We evaluate RetiSEM across ten synthetic
benchmark scenarios that vary in dimensionality, nonlinearity, causal depth, and pathway structure, together with a fragmented real-world setting that combines NHANES clinical variables with externally derived retinal representations. This approach achieves lower structural error and higher causal accuracy than unconstrained baselines across the synthetic
benchmarks. In the real-data analysis, retinal variables behave mainly as downstream biomarker-like indicators, with smaller but detectable indirect effects. These findings support our strategy as an interpretable framework for testing structured causal hypotheses in limited-resource biomedical AI. The code and resources for this work are publicly available at: https://github.com/Inamullah-Colab/ReitSEM.
Read the full abstract ▾
Computer Vision
Advancing WordArt-Oriented Scene Text Recognition: Datasets and Methods
Xingsong Ye, Yongkun Du, Jiaxin Zhang et al. (+5 more)
WordArt (artistic text) features highly customized fonts, textures, and layouts, making WordArt-oriented scene TExt Recognition (WATER) substantially more challenging than general Scene Text Recognition (STR). Existing STR
datasets and methods, typically built around regular scene text and fixed-template inputs, struggle to scale to WATER. Thus, we aim to advance this task from both data and model perspectives. On the data side, we construct a 2M synthetic
dataset, WATER-S, with the scale improved by hundreds of times compared to existing artistic text data. WATER-S consists of two complementary subsets. One rendered by an upgraded rendering pipeline (SynthWordArt), which provides highly accurate and controllable synthetic WordArt data. The other is generated by combining Qwen3-VL for
prompt mining and Z-Image for image synthesis, which improves the coverage of realistic and diverse data. On the model side, we propose WATERec. It adopts an visual encoder supporting arbitrary-shaped inputs and an autoregressive decoder to model complex layouts, structurally breaking the bottleneck of fixed-template STR on WordArt. Experiments show that this architecture outperforms prior STR methods, achieving state-of-the-art performance on irregular texts such as WordArt. Together with WATER-R, carefully reorganized from existing real STR data, our strong baseline with the new synthetic data and model design reaches 90.40% accuracy on WordArt-Bench, surpassing both general-purpose and OCR-specialized
vision-
language models by a large margin. Code and data are available at https://github.com/YesianRohn/WATER.
Read the full abstract ▾
All authors Xingsong Ye, Yongkun Du, Jiaxin Zhang, Haojie Zhang, Chong Sun, Chen Li, Jing Lyu, Zhineng Chen
Computer Vision
MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction
Peize Li, Fanhu Zeng, Tongda Xu et al. (+5 more)
In-camera JPEG previews are ubiquitous in raw image formats and provide an sRGB reference at negligible storage cost. Although existing metadata-based reconstruction frameworks can exploit this side information when recovering raw images, their context models often become computationally expensive especially at high resolution, eg, 4K raw image, given that
attention mechanisms scale quadratically with feature maps, hindering its practical application. To address these limitations, we propose MambaRaw, a JPEG-conditioned metadata-based raw image reconstruction framework that uses State Space Models (SSMs) to estimate
entropy parameters efficiently. Our key contribution comprises a Spatial-Energy Coupled Context Modeling mechanism with two lightweight modules: (1) TileMambaBlock, which performs Mamba-style selective scanning only on information-dense tiles to improve the efficiency; and (2) Energy-Aware Refinement (EAR), an identity-initialized residual module that enhance feature representation to match the long-tail energy distribution of raw signals. Extensive experiments on three camera
datasets (Sony, Olympus, Samsung) show consistent improvements over strong metadata-based baselines and set a new state of the art for JPEG-guided raw reconstruction with great efficiency. Notably, at low metadata bitrates, MambaRaw increases PSNR by 1.2–1.4 dB and reduces end-to-end coding latency by about 9%. Code is released at https://github.com/Peizeli1/MambaRaw.
Read the full abstract ▾
All authors Peize Li, Fanhu Zeng, Tongda Xu, Xingguo Xu, Xinjie Zhang, Xingtong Ge, Haotian Zhang, Yan Wang
Computer Vision
video-SALMONN-R$^3$: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding
Yixuan Li, Guangzhi Sun, Yudong Yang et al. (+1 more)
Video large
language models (LLMs) are often constrained by computation and
memory budgets, leading them to use reduced frame rates and spatial resolutions, which may cause them to miss critical information for question answering (QA). A practical and efficient solution is a two-stage paradigm: first perform coarse
video understanding to localize relevant segments, and then re-watch these segments at higher temporal or spatial fidelity. In this paper, we present
video-SALMONN-R$^3$, the first end-to-end
video-LLM that enables re-watch through reinforcement learning without relying on chain-of-thought (CoT) cold-start. This design removes the need for costly CoT data annotations and avoids CoT-based supervised fine-tuning (SFT), which can otherwise degrade the pretrained
video understanding abilities. To address the mismatch between the
reasoning-first behavior induced by re-watch and the answer-first tendency of pretrained
video-LLMs, we propose a re-answer strategy, in which the model first produces a direct answer in the first watch and then refines it after re-watching. Finally, to improve question adherence during re-watching, we propose a re-ask mechanism that re-injects the query when revisiting localized segments. Experimental results show that
video-SALMONN-R$^3$ consistently outperforms both the base model and the QA-SFT baseline, while surpassing prior re-watch-based approaches with significantly lower computational cost. Code, models, and data will be publicly released upon acceptance.
Read the full abstract ▾
All authors Yixuan Li, Guangzhi Sun, Yudong Yang, Chao Zhang
Computer Vision
Tianyu Zhu, Yingping Liang, Hesong Li et al. (+1 more)
Text-driven Referring
Video Object
Segmentation (RVOS) aims to locate and segment target objects in
videos given natural
language. However, existing models are typically trained on 2D image or
video datasets with naive
segmentation losses, which overlooks the
geometric consistency across frames and leads to weak spatial understanding. In this paper, we propose Geometry-enhanced
Language-guided
Video segmentation (GeoLaV), a two-stage framework that distills 3D
geometric knowledge from images to enhance text-driven
video segmentation. In the first stage, we perform
monocular geometry pretraining with
monocular novel-view synthesis, enabling the model to acquire geometry-consistent visual representations via spatial
alignment on large-scale single-image
datasets. In the second stage, we introduce geometry-aware
distillation and fine-tune the model on
video segmentation datasets, transferring 3D structural knowledge from a general 3D prior model. This process reinforces 3D awareness and improves both spatiotemporal coherence and
language grounding in
segmentation. Extensive experiments show that our method using only image
segmentation data already provides notable zero-shot generalization in RVOS. When combined with geometry-aware
distillation for fine-tuning on
videos, our method achieves state-of-the-art performance across multiple RVOS
benchmarks. The code is available at https://github.com/Tony1882880/GeoLaV.
Read the full abstract ▾
All authors Tianyu Zhu, Yingping Liang, Hesong Li, Ying Fu
Computer Vision
Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching
Junpeng Jing, Ronglai Zuo, Zhelun Shen et al. (+5 more)
Recent advances in stereo matching have achieved remarkable accuracy, but often rely on large models, heavy computation, or additional foundation-model priors, making them difficult to deploy on resource-constrained platforms. In contrast, efficient stereo models offer faster inference but are commonly considered less capable of strong zero-shot generalization. In this paper, we challenge this assumption by introducing Lite Any Stereo V2 (LAS2), an ultra-fast model series designed for efficient zero-shot stereo matching. LAS2 is developed from both architecture and training perspectives. Architecturally, we revisit efficient stereo design under practical deployment settings and propose a 2D-only cost aggregation framework, optimized for real inference latency rather than theoretical MACs alone. For training, we develop a three-stage strategy that combines synthetic supervision, self-
distillation, and real-world knowledge
distillation. To improve the reliability of real-world pseudo supervision, we further introduce pseudo-label filtering and an error-clamping operation, enabling smoother synthetic-to-real transfer. We instantiate LAS2 as a family of models, including feed-forward variants for different efficiency budgets and an iterative variant for higher accuracy. Extensive experiments show that LAS2 achieves state-of-the-art accuracy among efficient stereo methods while maintaining significantly lower latency. Specifically, LAS2-H achieves stronger overall zero-shot performance than the iterative method Fast-FoundationStereo, with 1.8x and 2.7x faster inference on H200 and Orin, respectively. The project page, demos, and code are available at https://tomtomtommi.github.io/LiteAnyStereoV2/.
Read the full abstract ▾
All authors Junpeng Jing, Ronglai Zuo, Zhelun Shen, Shangchen Zhou, Rolandos Alexandros Potamias, Stefanos Zafeiriou, Krystian Mikolajczyk, Jiankang Deng
Computer Vision
SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking
Mohamad Alansari, Yonathan Michael, Hasan AlMarzouqi et al. (+3 more)
We revisit the
memory update mechanism in SAM2-based visual object tracking and identify confidence-only mask selection as the dominant cause of drift under occlusion, rapid motion, and distractors. We introduce SENTRY, a training-free, plug-and-play, refine-before-write module that validates each
memory update for short-horizon temporal consistency before committing it. SENTRY aggregates diverse
segmentation hypotheses per frame, backtracks them into short tracklets, and uses neighbor-aware cycle-consistent matching against recent trajectories to favor temporally and geometrically consistent masks. It leaves the base architecture untouched, replacing confidence-driven writes with consistency-validated ones. For fair evaluation, we re-evaluate major open-source SAM2-based trackers across all available scales and
datasets, filling gaps in prior reports. Integrated into five strong baselines, SENTRY delivers consistent gains across nine
benchmarks, achieving new zero-shot SOTA on LaSOT, LaSOT_ext, GOT-10k, VOT20, VOT22, and DiDi. Despite these checks, the SAM2-L version runs at 32.8 FPS on an A100, and across compatible hosts adds only about 0.4–0.6 GB VRAM. Our results provide the first unified all-scale evaluation of SAM2-based trackers and show that enforcing temporal validity at write time stabilizes
memory-augmented tracking without retraining. Project page: https://hamadya.github.io/SENTRY/page/
Read the full abstract ▾
All authors Mohamad Alansari, Yonathan Michael, Hasan AlMarzouqi, Muzammal Naseer, Naoufel Werghi, Sajid Javed