Computer Vision
Evaluation Before Generation: A Paradigm for Robust Multimodal Sentiment Analysis with Missing Modalities
Rongfei Chen, Tingting Zhang, Xiaoyu Shen et al. (+1 more)
The missing modality problem poses a fundamental challenge in
multimodal sentiment analysis, significantly degrading model accuracy and generalization in real world scenarios. Existing approaches primarily improve robustness through
prompt learning and pre trained models. However, two limitations remain. First, the necessity of generating missing modalities lacks rigorous evaluation. Second, the structural dependencies among
multimodal prompts and their global coherence are insufficiently explored. To address these issues, a
Prompt based Missing Modality Adaptation framework is proposed. A Missing Modality Evaluator is introduced at the input stage to dynamically assess the importance of missing modalities using pretrained models and pseudo labels, thereby avoiding low quality data imputation. Building on this, a Modality invariant
Prompt Disentanglement module decomposes shared
prompts into modality specific private
prompts to capture intrinsic local correlations and improve representation quality. In addition, a Dynamic
Prompt Weighting module computes mutual information based weights from cross
attention outputs to adaptively suppress interference from missing modalities. To enhance global consistency, a Multi level
Prompt Dynamic Connection module integrates shared
prompts with self
attention outputs through residual connections, leveraging global
prompt priors to strengthen key guidance features. Extensive experiments on three public
benchmarks, including CMU MOSI, CMU MOSEI, and CH SIMS, demonstrate that the proposed framework achieves state of the art performance and stable results under diverse missing modality settings. The implementation is available at https://github.com/rongfei-chen/ProMMA
Read the full abstract ▾
All authors Rongfei Chen, Tingting Zhang, Xiaoyu Shen, Wei Zhang
Computer Vision · Cross-listed from cs.RO
Referring-Aware Visuomotor Policy Learning for Closed-Loop Manipulation
Jiahua Ma, Yiran Qin, Xin Wen et al. (+5 more)
This paper addresses a fundamental problem of visuomotor policy learning for robotic manipulation: how to enhance robustness in out-of-distribution execution errors or dynamically re-routing trajectories, where the model relies solely on the original expert demonstrations for training. We introduce the Referring-Aware Visuomotor Policy (ReV), a closed-loop framework that can adapt to unforeseen circumstances by instantly incorporating sparse referring points provided by a human or a high-level
reasoning planner. Specifically, ReV leverages the coupled
diffusion heads to preserve standard task execution patterns while seamlessly integrating sparse referring via a trajectory-steering strategy. Upon receiving a specific referring point, the global
diffusion head firstly generates a sequence of globally consistent yet temporally sparse action anchors, while identifies the precise temporal position for the referring point within this sequence. Subsequently, the local
diffusion head adaptively interpolates adjacent anchors based on the current temporal position for specific tasks. This closed-loop process repeats at every execution step, enabling real-time trajectory replanning in response to dynamic changes in the scene. In practice, rather than relying on elaborate annotations, ReV is trained only by applying targeted perturbations to expert demonstrations. Without any additional data or fine-tuning scheme, ReV achieve higher success rates across challenging simulated and real-world tasks.
Read the full abstract ▾
All authors Jiahua Ma, Yiran Qin, Xin Wen, Yixiong Li, Yuyu Sun, Yulan Guo, Liang Lin, Ruimao Zhang
Computer Vision
EchoAgent: Towards Reliable Echocardiography Interpretation with "Eyes","Hands" and "Minds"
Qin Wang, Zhiqing He, Yu Liu et al. (+8 more)
Reliable interpretation of echocardiography (Echo) is crucial for assessing cardiac function, which demands clinicians to synchronously orchestrate multiple capabilities, including visual observation (eyes), manual measurement (hands), and expert knowledge learning and
reasoning (minds). While current task-specific deep-learning approaches and
multimodal large
language models have demonstrated promise in assisting Echo analysis through automated
segmentation or
reasoning, they remain focused on restricted skills, i.e., eyes-hands or eyes-minds, thereby limiting clinical reliability and utility. To address these issues, we propose EchoAgent, an agentic system tailored for end-to-end Echo interpretation, which achieves a fully coordinated eyes-hands-minds workflow that learns, observes, operates, and reasons like a cardiac sonographer. First, we introduce an expertise-driven cognition engine where our
agent can automatically assimilate credible Echo guidelines into a structured knowledge base, thus constructing an Echo-customized mind. Second, we devise a hierarchical collaboration toolkit to endow EchoAgent with eyes-hands, which can automatically parse Echo
video streams, identify cardiac views, perform anatomical
segmentation, and quantitative measurement. Third, we integrate the perceived
multimodal evidence with the exclusive knowledge base into an orchestrated
reasoning hub to conduct explainable inferences. We evaluate EchoAgent on CAMUS and MIMIC-EchoQA
datasets, which cover 48 distinct echocardiographic views spanning 14 cardiac anatomical regions. Experimental results show that EchoAgent achieves optimal performance across diverse structure analyses, yielding overall accuracy of up to 80.00%. Importantly, EchoAgent empowers a single system with abilities to learn, observe, operate and reason like an echocardiologist, which holds great promise for reliable Echo interpretation.
Read the full abstract ▾
All authors Qin Wang, Zhiqing He, Yu Liu, Bowen Guo, Zeju Li, Miao Zhao, Wenhao Ju, Zhiling Luo, Xianhong Shu, Yi Guo, Yuanyuan Wang
Computer Vision
Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images
Xuanguang Liu, Lei Ding, Yujie Li et al. (+8 more)
Multimodal change detection (MMCD) identifies changed areas in
multimodal remote sensing data, demonstrating significant application value in land use monitoring and urban sustainable development. However, literature MMCD approaches exhibit limitations in both cross-modal interaction and exploiting modality-specific characteristics. This leads to insufficient modeling of fine-grained change information, thus hindering the precise detection of semantic changes. To address these problems, we propose STSF-Net, a framework designed for MMCD between optical and SAR images. STSF-Net jointly models modality-specific and spatio-temporal common features to enhance change representations. Specifically, modality-specific features are exploited to capture genuine semantic change signals, while spatio-temporal common features are embedded to suppress pseudo-changes caused by differences in imaging mechanisms. Furthermore, we introduce an optical and SAR feature fusion strategy that adaptively adjusts
multimodal feature importance based on semantic priors obtained from visual foundation models. Finally, we introduce the novel Delta-SN6
dataset, the first openly-accessible multiclass MMCD
benchmark consisting of very-high-resolution fully polarimetric SAR and optical images. Experimental results on Delta-SN6, BRIGHT, and Wuhan
datasets demonstrate that our method outperforms the state-of-the-art by 3.21%, 0.87%, and 1.32% in mIoU, respectively.
Read the full abstract ▾
All authors Xuanguang Liu, Lei Ding, Yujie Li, Chenguang Dai, Zhenchao Zhang, Mengmeng Li, Ziyi Yang, Yifan Sun, Yongqi Sun, Hanyun Wang, Lorenzo Bruzzone
Computer Vision
Cross-Resolution Diffusion Models via Network Pruning
Jiaxuan Ren, Junhan Zhu and Huan Wang
Diffusion models have demonstrated impressive image synthesis performance, yet many UNet-based models are trained at certain fixed resolutions. Their quality tends to degrade when generating images at out-of-training resolutions. We trace this issue to resolution-dependent parameter behaviors, where weights that function well at the default resolution can become adverse when spatial scales shift, weakening semantic
alignment and causing structural instability in the UNet architecture. Based on this analysis, this paper introduces CR-Diff, a novel method that improves the cross-resolution visual consistency by
pruning some parameters of the
diffusion model. Specifically, CR-Diff has two stages. It first performs block-wise
pruning to selectively eliminate adverse weights. Then, a pruned output amplification is conducted to further purify the pruned predictions. Empirically, extensive experiments suggest that CR-Diff can improve perceptual fidelity and semantic coherence across various
diffusion backbones and unseen resolutions, while largely preserving the performance at default resolutions. Additionally, CR-Diff supports
prompt-specific refinement, enabling quality enhancement on demand.
Read the full abstract ▾
Computer Vision
Geometrical Cross-Attention and Nonvoid Voxelization for Efficient 3D Medical Image Segmentation
Chenxin Yuan, Shoupeng Chen, Haojiang Ye et al. (+3 more)
Accurate
segmentation of 3D medical scans is crucial for clinical diagnostics and treatment planning, yet existing methods often fail to achieve both high accuracy and computational efficiency across diverse anatomies and imaging modalities. To address these challenges, we propose GCNV-Net, a novel 3D medical
segmentation framework that integrates a Tri-directional Dynamic Nonvoid Voxel
Transformer (3DNVT), a Geometrical Cross-
Attention module (GCA), and Nonvoid Voxelization. The 3DNVT dynamically partitions relevant voxels along the three orthogonal anatomical planes, namely the transverse, sagittal, and coronal planes, enabling effective modeling of complex 3D spatial dependencies. The GCA mechanism explicitly incorporates
geometric positional information during multi-scale feature fusion, significantly enhancing fine-grained anatomical
segmentation accuracy. Meanwhile, Nonvoid Voxelization processes only informative regions, greatly reducing redundant computation without compromising
segmentation quality, and achieves a 56.13% reduction in FLOPs and a 68.49% reduction in inference latency compared to conventional voxelization. We evaluate GCNV-Net on multiple widely used
benchmarks: BraTS2021, ACDC, MSD Prostate, MSD Pancreas, and AMOS2022. Our method achieves state-of-the-art
segmentation performance across all
datasets, outperforming the best existing methods by 0.65% on Dice, 0.63% on IoU, 1% on NSD, and relatively 14.5% on HD95. All results demonstrate that GCNV-Net effectively balances accuracy and efficiency, and its robustness across diverse organs, disease conditions, and imaging modalities highlights strong potential for clinical deployment.
Read the full abstract ▾
All authors Chenxin Yuan, Shoupeng Chen, Haojiang Ye, Yiming Miao, Limei Peng, Pin-Han Ho
Computer Vision
Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality
Yanming Xiu, Zhengyuan Jiang, Neil Zhenqiang Gong et al. (+1 more)
Augmented reality (AR) has rapidly expanded over the past decade. As AR becomes increasingly integrated into daily life, its security and reliability emerge as critical challenges. Among various threats, contradictory virtual content attacks, where malicious or inconsistent virtual elements are introduced into the user's view, pose a unique risk by misleading users, creating semantic confusion, or delivering harmful information. In this work, we systematically model such attacks and present ContrAR, a novel
benchmark for evaluating the robustness of
vision-
language models (VLMs) against virtual content manipulation and contradiction in AR. ContrAR contains 312 real-world AR
videos validated by 10 human participants. We further
benchmark 11 VLMs, including both commercial and open-source models. Experimental results reveal that while current VLMs exhibit reasonable understanding of contradictory virtual content, room still remains for improvement in detecting and
reasoning about
adversarial content manipulations in AR environments. Moreover, balancing detection accuracy and latency remains challenging.
Read the full abstract ▾
All authors Yanming Xiu, Zhengyuan Jiang, Neil Zhenqiang Gong, Maria Gorlatova
Computer Vision
CLIP-Guided Data Augmentation for Night-Time Image Dehazing
Xining Ge, Weijun Yuan, Gengjia Chang et al. (+2 more)
Nighttime image dehazing faces a more complex degradation pattern than its daytime counterpart, as haze scattering couples with low illumination, non-uniform lighting, and strong light interference. Under limited supervision, this complexity aggravates domain drift and training instability, since target-domain samples are scarce while naively introducing external data may weaken adaptation due to distribution mismatch. This paper presents our solution to the NTIRE 2026 Night Time Image Dehazing Challenge, built as a unified framework that integrates domain-aligned data construction, stage-wise training, and inference-time enhancement. Specifically, a pre-trained CLIP visual encoder screens candidate external samples by similarity to construct training data closer to the target domain. NAFNet is then trained in two stages, first adapting to the target domain and then expanding to broader degradation patterns. At inference time, TLC, x8 self-ensemble, and weighted snapshot fusion are combined to improve output stability. Rather than relying on complex network redesign, the proposed framework offers a practical and effective pipeline for nighttime image dehazing.
Read the full abstract ▾
All authors Xining Ge, Weijun Yuan, Gengjia Chang, Xuyang Li, Shuhong Liu
Computer Vision · Cross-listed from cs.AI
Keuntae Kim, Mingyu Kang and Yong Suk Choi
Diffusion large
language models (dLLMs) are emerging as promising alternatives to autoregressive (AR) LLMs. Recently, this paradigm has been extended to
multimodal tasks, leading to the development of
diffusion multimodal large
language models (dMLLMs). These models are expected to retain the
reasoning capabilities of LLMs while enabling faster inference through parallel generation. However, when combined with Chain-of-Thought (CoT)
reasoning, dMLLMs exhibit two critical issues. First, we observe that dMLLMs often generate the final answer
token at a very early timestep. This trend indicates that the model determines the answer before sufficient
reasoning, leading to degraded
reasoning performance. Second, during the initial timesteps, dMLLMs show minimal dependency on visual
prompts, exhibiting a fundamentally different pattern of visual information utilization compared to AR
vision-
language models. In summary, these findings indicate that dMLLMs tend to generate premature final answers without sufficiently grounding on visual inputs. To address these limitations, we propose Position and Step Penalty (PSP) and Visual
Reasoning Guidance (VRG). PSP penalizes
tokens in later positions during early timesteps, delaying premature answer generation and encouraging progressive
reasoning across timesteps. VRG, inspired by classifier-free guidance, amplifies visual grounding signals to enhance the model's
alignment with visual evidence. Extensive experiments across various dMLLMs demonstrate that our method achieves up to 7.5% higher accuracy while delivering more than 3x speedup compared to
reasoning with four times more
diffusion steps.
Read the full abstract ▾
Computer Vision
A Weak-Signal-Aware Framework for Subsurface Defect Detection: Mechanisms for Enhancing Low-SCR Hyperbolic Signatures
Wenbo Zhang, Zekun Long, Zican Liu et al. (+2 more)
Subsurface defect detection via Ground Penetrating Radar is challenged by "weak signals" faint diffraction hyperbolas with low signal-to-clutter ratios, high wavefield similarity, and
geometric degradation. Existing lightweight detectors prioritize efficiency over sensitivity, failing to preserve low-frequency structures or decouple heterogeneous clutter. We propose WSA-Net, a framework designed to enhance faint signatures through physical-feature reconstruction. Moving beyond simple parameter reduction, WSA-Net integrates four mechanisms: Signal preservation using partial convolutions; Clutter suppression via heterogeneous grouping
attention;
Geometric reconstruction to sharpen
hyperbolic arcs; Context anchoring to resolve semantic ambiguities. Evaluations on the RTSTdataset show WSA-Net achieves 0.6958 mAP@0.5 and 164 FPS with only 2.412 M parameters. Results prove that signal-centric awareness in lightweight architectures effectively reduces false negatives in infrastructure inspection.
Read the full abstract ▾
All authors Wenbo Zhang, Zekun Long, Zican Liu, Yangchen Zeng, Keyi Hu
Computer Vision · Cross-listed from cs.RO
CoEnv: Code-Driven Exploration in Real-to-Sim Digital Twins for Multi-Arm Manipulation
Li Kang, Yutao Fan, Rui Li et al. (+11 more)
General-purpose coding
agents can now write, debug, and refine entire
robot programs, but only if they have somewhere to practice. Physical hardware is a poor practice ground: exploratory rollouts are slow to reset and, when several arms share a workspace, unsafe. Generic simulation
benchmarks are safe but do not match the geometry, assets, or viewpoints of the workspace where the
robot must actually work. We present CoEx (Code-driven Exploration), which lets the
agent practice in an executable digital twin of its own deployment workspace. The twin is built by CoEnv, our real-to-sim framework, which reconstructs the physical scene from multi-view RGBD into a physics simulation. Inside it, a centralized code
agent explores at the program level, writing a complete multi-arm program, running it in the twin, and refining it against grounded verification (privileged state checks, checkpoint predicates on risky steps, and adaptive viewpoints against inter-arm occlusion). Verified programs transfer to hardware through collision-checked deployment. On five real-world multi-arm tasks with up to three heterogeneous
robots, CoEx reaches 70% average success, against 50% for the same system without the code
agent (where a VLM must choose every primitive step by step), outperforms two prior VLM-planning frameworks, and produces verified episodes at 3.8-11.7x the ablated rate. Because every verified rollout is also a demonstration, exploration leaves reusable experience behind: a $\pi_{0.5}$ policy fine-tuned purely on 200
agent-generated rollouts reaches 100% and 70% real-
robot success on two tasks, with no human demonstrations.
Read the full abstract ▾
All authors Li Kang, Yutao Fan, Rui Li, Heng Zhou, Yiran Qin, Zhemeng Zhang, Songtao Huang, Xiufeng Song, Zaibin Zhang, Bruno N.Y. Chen, Zhenfei Yin, Dongzhan Zhou, Wangmeng Zuo, Lei Bai
Computer Vision
Unifying VLM-Guided Flow Matching and Spectral Anomaly Detection for Interpretable Veterinary Diagnosis
Pu Wang, Zhixuan Mao, Jialu Li et al. (+3 more)
Automatic diagnosis of canine pneumothorax is challenged by data scarcity and the need for trustworthy models. To address this, we first introduce a public, pixel-level annotated
dataset to facilitate research. We then propose a novel diagnostic paradigm that reframes the task as a synergistic process of signal localization and spectral detection. For localization, our method employs a
Vision-
Language Model (VLM) to guide an iterative Flow Matching process, which progressively refines
segmentation masks to achieve superior boundary accuracy. For detection, the segmented mask is used to isolate features from the suspected lesion. We then apply Random
Matrix Theory (RMT), a departure from traditional classifiers, to analyze these features. This approach models healthy tissue as predictable random noise and identifies pneumothorax by detecting statistically significant outlier
eigenvalues that represent a non-random pathological signal. The high-fidelity localization from Flow Matching is crucial for purifying the signal, thus maximizing the sensitivity of our RMT detector. This synergy of generative
segmentation and first-principles statistical analysis yields a highly accurate and interpretable diagnostic system (source code is available at: https://github.com/Pu-Wang-alt/Canine-pneumothorax).
Read the full abstract ▾
All authors Pu Wang, Zhixuan Mao, Jialu Li, Zhuoran Zheng, Dianjie Lu, Youshan Zhang
Computer Vision
A Synthetic Eye Movement Dataset for Script Reading Detection: Real Trajectory Replay on a 3D Simulator
Kidus Zewde, Yuchen Zhou, Dennis Ng et al. (+6 more)
Large
vision-
language models have achieved remarkable capabilities by training on massive internet-scale data, yet a fundamental asymmetry persists: while LLMs can leverage self-supervised pretraining on abundant text and image data, the same is not true for many behavioral modalities.
Video-based behavioral data – gestures, eye movements, social signals – remains scarce, expensive to annotate, and
privacy-sensitive. A promising alternative is simulation: replace real data collection with controlled synthetic generation to produce automatically labeled data at scale. We introduce infrastructure for this paradigm applied to eye movement, a behavioral signal with applications across
vision-
language modeling, virtual reality, robotics, accessibility systems, and cognitive science. We present a pipeline for generating synthetic labeled eye movement
video by extracting real human iris trajectories from reference
videos and replaying them on a 3D eye movement simulator via headless browser automation. Applying this to the task of script-reading detection during
video interviews, we release final_
dataset_v1: 144 sessions (72 reading, 72 conversation) totaling 12 hours of synthetic eye movement
video at 25fps. Evaluation shows that generated trajectories preserve the temporal dynamics of the source data (KS D < 0.14 across all metrics). A matched frame-by-frame comparison reveals that the 3D simulator exhibits bounded sensitivity at reading-scale movements, attributable to the absence of coupled head movement – a finding that informs future simulator design. The pipeline,
dataset, and evaluation tools are released to support downstream behavioral classifier development at the intersection of behavioral modeling and
vision-
language systems.
Read the full abstract ▾
All authors Kidus Zewde, Yuchen Zhou, Dennis Ng, Neo Tiangratanakul, Tommy Duong, Ankit Raj, Yuxin Zhang, Xingyu Shen, Simiao Ren
Computer Vision
Not All Agents Matter: From Global Attention Dilution to Risk-Prioritized Game Planning
Kang Ding, Hongsong Wang, Jie Gui et al. (+1 more)
End-to-end autonomous driving resides not in the integration of perception and planning, but rather in the dynamic multi-
agent game within a unified representation space. Most existing end-to-end models treat all
agents equally, hindering the decoupling of real collision threats from complex backgrounds. To address this issue, We introduce the concept of Risk-Prioritized Game Planning, and propose GameAD, a novel framework that models end-to-end autonomous driving as a risk-aware game problem. The GameAD integrates Risk-Aware
Topology Anchoring, Strategic Payload Adapter, Minimax Risk-Aware Sparse
Attention, and Risk Consistent
Equilibrium Stabilization to enable game theoretic decision making with risk prioritized interactions. We also present the Planning Risk Exposure metric, which quantifies the cumulative risk intensity of planned trajectories over a long horizon for safe autonomous driving. Extensive experiments on the nuScenes and Bench2Drive
datasets show that our approach significantly outperforms state-of-the-art methods, especially in terms of trajectory safety.
Read the full abstract ▾
All authors Kang Ding, Hongsong Wang, Jie Gui, Lei He
Computer Vision · Cross-listed from cs.CL
Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modeling
Qiyuan Chen, Hongsen Huang, Jiahe Chen et al. (+6 more)
Vision-
language reward modeling faces a dilemma: generative approaches are interpretable but slow, while discriminative ones are efficient but act as opaque "black boxes." To bridge this gap, we propose VL-MDR (
Vision-
Language Multi-Dimensional Reward), a framework that dynamically decomposes evaluation into granular, interpretable dimensions. Instead of outputting a monolithic scalar, VL-MDR employs a visual-aware gating mechanism to identify relevant dimensions and adaptively weight them (e.g.,
Hallucination,
Reasoning) for each specific input. To support this, we curate a
dataset of 321k
vision-
language preference pairs annotated across 21 fine-grained dimensions. Extensive experiments show that VL-MDR consistently outperforms existing open-source reward models on
benchmarks like VL-RewardBench. Furthermore, we show that VL-MDR-constructed preference pairs effectively enable DPO
alignment to mitigate visual
hallucinations and improve reliability, providing a scalable solution for VLM
alignment.
Read the full abstract ▾
All authors Qiyuan Chen, Hongsen Huang, Jiahe Chen, Qian Shao, Jintai Chen, Hongxia Xu, Renjie Hua, Chuan Ren, Jian Wu
Computer Vision
Human Interaction-Aware 3D Reconstruction from a Single Image
Gwanghyun Kim, Junghun James Kim, Suh Yoon Jeon et al. (+2 more)
Reconstructing textured 3D human models from a single image is fundamental for AR/VR and digital human applications. However, existing methods mostly focus on single individuals and thus fail in multi-human scenes, where naive composition of individual reconstructions often leads to artifacts such as unrealistic overlaps, missing geometry in occluded regions, and distorted interactions. These limitations highlight the need for approaches that incorporate group-level context and interaction priors. We introduce a holistic method that explicitly models both group- and instance-level information. To mitigate perspective-induced
geometric distortions, we first transform the input into a canonical orthographic space. Our primary component, Human Group-Instance Multi-View
Diffusion (HUG-MVD), then generates complete multi-view normals and images by jointly modeling individuals and group context to resolve occlusions and proximity. Subsequently, the Human Group-Instance
Geometric Reconstruction (HUG-GR) module optimizes the geometry by leveraging explicit, physics-based interaction priors to enforce physical plausibility and accurately model inter-human contact. Finally, the multi-view images are fused into a high-fidelity texture. Together, these components form our complete framework, HUG3D. Extensive experiments show that HUG3D significantly outperforms both single-human and existing multi-human methods, producing physically plausible, high-fidelity 3D reconstructions of interacting people from a single image. Project page: https://jongheean11.github.io/HUG3D_project
Read the full abstract ▾
All authors Gwanghyun Kim, Junghun James Kim, Suh Yoon Jeon, Jason Park, Se Young Chun
Computer Vision
Few-Shot Semantic Segmentation Meets SAM3
Yi-Jen Tsai, Yen-Yu Lin and Chien-Yao Wang
Few-Shot Semantic
Segmentation (FSS) focuses on segmenting novel object categories from only a handful of annotated examples. Most existing approaches rely on extensive episodic training to learn transferable representations, which is both computationally demanding and sensitive to distribution shifts. In this work, we revisit FSS from the perspective of modern
vision foundation models and explore the potential of Segment Anything Model 3 (SAM3) as a training-free solution. By repurposing its Promptable Concept
Segmentation (PCS) capability, we adopt a simple spatial concatenation strategy that places support and query images into a shared canvas, allowing a fully frozen SAM3 to perform
segmentation without any fine-tuning or architectural changes. Experiments on PASCAL-$5^i$ and COCO-$20^i$ show that this minimal design already achieves state-of-the-art performance, outperforming many heavily engineered methods. Beyond empirical gains, we uncover that negative
prompts can be counterproductive in few-shot settings, where they often weaken target representations and lead to prediction collapse despite their intended role in suppressing distractors. These findings suggest that strong cross-image
reasoning can emerge from simple spatial formulations, while also highlighting limitations in how current foundation models handle conflicting
prompt signals. Code at: https://github.com/WongKinYiu/FSS-SAM3
Read the full abstract ▾
Computer Vision
Cross-Stage Attention Propagation for Efficient Semantic Segmentation
Beoungwoo Kang
Recent lightweight semantic
segmentation methods have made significant progress by combining compact backbones with efficient decoder heads. However, most multi-scale decoders compute
attention independently at each feature scale, introducing substantial redundancy since the resulting
attention distributions across scales are strongly correlated. We propose Cross-Stage
Attention Propagation (CSAP), a decoder framework that computes
attention at the deepest feature scale and propagates the resulting
attention maps to shallower stages, bypassing query-key computation at those stages entirely. This design preserves multi-scale contextual
reasoning while substantially reducing the decoder's computational cost. CSAP-Tiny achieves 42.9% mIoU on ADE20K with only 5.5 GFLOPs, 80.5% on Cityscapes with 21.5 GFLOPs, and 40.9% on COCO-Stuff 164K with 5.5 GFLOPs, surpassing SegNeXt-Tiny by +1.8% on ADE20K while requiring 16.8% fewer floating-point operations.
Read the full abstract ▾
Computer Vision
VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
Honghao Fu, Miao Xu, Yiwei Wang et al. (+3 more)
Scaling
multimodal large
language models (MLLMs) to long
videos is constrained by limited context windows. While
retrieval-augmented generation (RAG) is a promising remedy by organizing query-relevant visual evidence into a compact context, most existing methods (i) flatten
videos into independent segments, breaking their inherent spatio-temporal structure, and (ii) depend on explicit semantic matching, which can miss cues that are implicitly relevant to the query's intent. To overcome these limitations, we propose VideoStir, a structured and intent-aware long-
video RAG framework. It firstly structures a
video as a spatio-temporal graph at clip level, and then performs multi-hop
retrieval to aggregate evidence across distant yet contextually related events. Furthermore, it introduces an MLLM-backed intent-relevance scorer that retrieves frames based on their
alignment with the query's
reasoning intent. To support this capability, we curate IR-600K, a large-scale
dataset tailored for learning frame-query intent
alignment. Experiments show that VideoStir is competitive with state-of-the-art baselines without relying on auxiliary information, highlighting the promise of shifting long-
video RAG from flattened semantic matching to structured, intent-aware
reasoning. Codes and checkpoints are available at https://github.com/RomGai/VideoStir.
Read the full abstract ▾
All authors Honghao Fu, Miao Xu, Yiwei Wang, Dailing Zhang, Jun Liu, Yujun Cai
Computer Vision
Learning to Synergize Semantic and Geometric Priors for Limited-Data Wheat Disease Segmentation
Shijie Wang, Zijian Wang, Yadan Luo et al. (+3 more)
Wheat disease
segmentation is fundamental to precision agriculture but faces severe challenges from significant intra-class temporal variations across growth stages. Such substantial appearance shifts make collecting a representative
dataset for training from scratch both labor-intensive and impractical. To address this, we propose SGPer, a Semantic-
Geometric Prior Synergization framework that treats wheat disease
segmentation under limited data as a coupled task of disease-specific semantic perception and disease boundary localization. Our core insight is that pretrained DINOv2 provides robust category-aware semantic priors to handle appearance shifts, which can be converted into coarse spatial
prompts to guide SAM for the precise localization of disease boundaries. Specifically, SGPer designs disease-sensitive adapters with multiple disease-friendly filters and inserts them into both DINOv2 and SAM to align their pretrained representations with disease-specific characteristics. To operationalize this synergy, SGPer transforms DINOv2-derived features into dense, category-specific point
prompts to ensure comprehensive spatial coverage of all disease regions. To subsequently eliminate
prompt redundancy and ensure highly accurate mask generation, it dynamically filters these dense candidates by cross-referencing SAM's iterative mask confidence with the category-specific semantic consistency derived from DINOv2. Ultimately, SGPer distills a highly informative set of
prompts to activate SAM's
geometric priors, achieving precise and robust
segmentation that remains strictly invariant to temporal appearance changes. Extensive evaluations demonstrate that SGPer consistently achieves state-of-the-art performance on wheat disease and organ
segmentation benchmarks, especially in data-constrained scenarios.
Read the full abstract ▾
All authors Shijie Wang, Zijian Wang, Yadan Luo, Scott Chapman, Xin Yu, Zi Huang
Computer Vision · Cross-listed from cs.LG
Training Without Orthogonalization, Inference With SVD: A Gradient Analysis of Rotation Representations
Chris Choy
Recent work has shown that removing orthogonalization during training and applying it only at inference improves rotation estimation in deep learning, with empirical evidence favoring 9D representations with SVD projection. However, the theoretical understanding of why SVD orthogonalization specifically harms training, and why it should be preferred over Gram-Schmidt at inference, remains incomplete. We provide a detailed gradient analysis of SVD orthogonalization specialized to $3 \times 3$
matrices and $SO(3)$ projection. Our central result derives the exact spectrum of the SVD backward pass Jacobian: it has rank $3$ (matching the dimension of $SO(3)$) with nonzero singular values $2/(s_i + s_j)$ and condition number $\kappa = (s_1 + s_2)/(s_2 + s_3)$, creating quantifiable gradient distortion that is most severe when the predicted
matrix is far from $SO(3)$ (e.g., early in training when $s_3 \approx 0$). We further show that even stabilized SVD gradients introduce gradient direction error, whereas removing SVD from the training loop avoids this tradeoff entirely. We also prove that the 6D Gram-Schmidt Jacobian has an asymmetric spectrum: its parameters receive unequal gradient signal, explaining why 9D parameterization is preferable. Together, these results provide the theoretical foundation for training with direct 9D regression and applying SVD projection only at inference.
Read the full abstract ▾
Computer Vision
CRISP: Rank-Guided Iterative Squeezing for Robust Medical Image Segmentation under Domain Shift
Yizhou Fang, Pujin Cheng, Yixiang Liu et al. (+2 more)
Distribution shift in medical imaging remains a central bottleneck for the clinical translation of medical AI. Failure to address it can lead to severe performance degradation in unseen environments and exacerbate health inequities. Existing methods for domain adaptation are inherently limited by exhausting predefined possibilities through simulated shifts or pseudo-supervision. Such strategies struggle in the open-ended and unpredictable real world, where distribution shifts are effectively infinite. To address this challenge, we introduce an empirical law called “Rank Stability of Positive Regions”, which states that the relative rank of predicted probabilities for positive voxels remains stable under distribution shift. Guided by this principle, we propose CRISP, a parameter-free and model-agnostic framework requiring no target-domain information. CRISP is the first framework to make
segmentation based on rank rather than probabilities. CRISP simulates model behavior under distribution shift via latent feature perturbation, where voxel probability rankings exhibit two stable patterns: regions that consistently retain high probabilities (destined positives according to the principle) and those that remain low-probability (can be safely classified as negatives). Based on these patterns, we construct high-precision (HP) and high-recall (HR) priors and recursively refine them under perturbation. We then design an iterative training framework, making HP and HR progressively “squeeze” to the final
segmentation. Extensive evaluations on multi-center cardiac MRI and CT-based lung vessel
segmentation demonstrate CRISP's superior robustness, significantly outperforming state-of-the-art methods with striking HD95 reductions of up to 0.14 (7.0% improvement), 1.90 (13.1% improvement), and 8.39 (38.9% improvement) pixels across multi-center, demographic, and modality shifts, respectively.
Read the full abstract ▾
All authors Yizhou Fang, Pujin Cheng, Yixiang Liu, Xiaoying Tang, Longxi Zhou
Computer Vision
Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection
Hongsheng Li, Lingfeng Zhang, Zexian Yang et al. (+4 more)
Robust 3D object detection in adverse weather is highly challenging due to the varying reliability of different sensors. While existing
LiDAR-4D radar fusion methods improve robustness, they predominantly rely on fixed or weakly adaptive pipelines, failing to dy-namically adjust modality preferences as environmental conditions change. To bridge this gap, we reformulate multi-modal perception as a weather-conditioned branch routing problem. Instead of computing a single fused output, our framework explicitly maintains three parallel 3D feature streams: a pure
LiDAR branch, a pure 4D radar branch, and a condition-gated fusion branch. Guided by a condition
token extracted from visual and semantic
prompts, a lightweight router dynamically predicts sample-specific weights to softly aggregate these representations. Furthermore, to prevent branch collapse, we introduce a weather-supervised learning strategy with auxiliary classification and diversity regularization to enforce distinct, condition-dependent routing behaviors. Extensive experiments on the K-Radar
benchmark demonstrate that our method achieves state-of-the-art performance. Furthermore, it provides explicit and highly interpretable insights into modality preferences, transparently revealing how adaptive routing robustly shifts reliance between
LiDAR and 4D radar across diverse adverse-weather scenarios. The source code with be released.
Read the full abstract ▾
All authors Hongsheng Li, Lingfeng Zhang, Zexian Yang, Liang Li, Rong Yin, Xiaoshuai Hao, Wenbo Ding
Computer Vision
LSGS-Loc: Towards Robust 3DGS-Based Visual Localization for Large-Scale UAV Scenarios
Xiang Zhang, Tengfei Wang, Fang Xu et al. (+2 more)
Visual localization in large-scale
UAV scenarios is a critical capability for autonomous systems, yet it remains challenging due to
geometric complexity and environmental variations. While 3D Gaussian Splatting (3DGS) has emerged as a promising scene representation, existing 3DGS-based visual localization methods struggle with robust pose initialization and sensitivity to rendering artifacts in large-scale settings. To address these limitations, we propose LSGS-Loc, a novel visual localization pipeline tailored for large-scale 3DGS scenes. Specifically, we introduce a scale-aware pose initialization strategy that combines scene-agnostic relative pose estimation with explicit 3DGS scale constraints, enabling geometrically grounded localization without scene-specific training. Furthermore, in the pose refinement, to mitigate the impact of reconstruction artifacts such as blur and floaters, we develop a
Laplacian-based reliability masking mechanism that guides photometric refinement toward high-quality regions. Extensive experiments on large-scale
UAV benchmarks demonstrate that our method achieves state-of-the-art accuracy and robustness for unordered image queries, significantly outperforming existing 3DGS-based approaches. Code is available at: https://github.com/xzhang-z/LSGS-Loc
Read the full abstract ▾
All authors Xiang Zhang, Tengfei Wang, Fang Xu, Xin Wang, Zongqian Zhan
Computer Vision
Beyond Semantic Search: Towards Referential Anchoring in Composed Image Retrieval
Yuxin Yang, Yinan Zhou, Yuxin Chen et al. (+6 more)
Composed Image
Retrieval (CIR) has demonstrated significant potential by enabling flexible
multimodal queries that combine a reference image and modification text. However, CIR inherently prioritizes semantic matching, struggling to reliably retrieve a user-specified instance across contexts. In practice, emphasizing concrete instance fidelity over broad semantics is often more consequential. In this work, we propose Object-Anchored Composed Image
Retrieval (OACIR), a novel fine-grained
retrieval task that mandates strict instance-level consistency. To advance research on this task, we construct OACIRR (OACIR on Real-world images), the first large-scale, multi-domain
benchmark comprising over 160K quadruples and four challenging candidate galleries enriched with hard-negative instance distractors. Each quadruple augments the compositional query with a bounding box that visually anchors the object in the reference image, providing a precise and flexible way to ensure instance preservation. To address the OACIR task, we propose AdaFocal, a framework featuring a Context-Aware
Attention Modulator that adaptively intensifies
attention within the specified instance region, dynamically balancing focus between the anchored instance and the broader compositional context. Extensive experiments demonstrate that AdaFocal substantially outperforms existing compositional
retrieval models, particularly in maintaining instance-level fidelity, thereby establishing a robust baseline for this challenging task while opening new directions for more flexible, instance-aware
retrieval systems.
Read the full abstract ▾
All authors Yuxin Yang, Yinan Zhou, Yuxin Chen, Ziqi Zhang, Zongyang Ma, Chunfeng Yuan, Bing Li, Jun Gao, Weiming Hu
Computer Vision
LUMOS: Universal Semi-Supervised OCT Retinal Layer Segmentation with Hierarchical Reliable Mutual Learning
Yizhou Fang, Jian Zhong, Li Lin et al. (+1 more)
Optical Coherence
Tomography (OCT) layer
segmentation faces challenges due to annotation scarcity and heterogeneous label granularities across
datasets. While semi-supervised learning helps alleviate label scarcity, existing methods typically assume a fixed granularity, failing to fully exploit cross-granularity supervision. This paper presents LUMOS, a semi-supervised universal OCT retinal layer
segmentation framework based on a Dual-Decoder Network with a Hierarchical Prompting Strategy (DDN-HPS) and Reliable Progressive Multi-granularity Learning (RPML). DDN-HPS combines a dual-branch architecture with a multi-granularity prompting strategy to effectively suppress pseudo-label noise propagation. Meanwhile, RPML introduces region-level reliability weighing and a progressive training approach that guides the model from easier to more difficult tasks, ensuring the reliable selection of cross-granularity consistency targets, thereby achieving stable cross-granularity
alignment. Experiments on six OCT
datasets demonstrate that LUMOS largely outperforms existing methods and exhibits exceptional cross-domain and cross-granularity generalization capability.
Read the full abstract ▾
All authors Yizhou Fang, Jian Zhong, Li Lin, Xiaoying Tang
Computer Vision · Cross-listed from cs.CL
ICR-Drive: Instruction Counterfactual Robustness for End-to-End Language-Driven Autonomous Driving
Kaiser Hamid, Can Cui and Nade Liang
Recent progress in
vision-
language-action (VLA) models has enabled
language-conditioned driving
agents to execute natural-
language navigation commands in closed-loop simulation, yet standard evaluations largely assume instructions are precise and well-formed. In deployment, instructions vary in phrasing and specificity, may omit critical qualifiers, and can occasionally include misleading, authority-framed text, leaving instruction-level robustness under-measured. We introduce ICR-Drive, a diagnostic framework for instruction counterfactual robustness in end-to-end
language-conditioned autonomous driving. ICR-Drive generates controlled instruction variants spanning four perturbation families: Paraphrase, Ambiguity, Noise, and Misleading, where Misleading variants conflict with the
navigation goal and attempt to override intent. We replay identical CARLA routes under matched simulator configurations and seeds to isolate performance changes attributable to instruction
language. Robustness is quantified using standard CARLA Leaderboard metrics and per-family performance degradation relative to the baseline instruction. Experiments on LMDrive and BEVDriver show that minor instruction changes can induce substantial performance drops and distinct failure modes, revealing a reliability gap for deploying
embodied foundation models in safety-critical driving.
Read the full abstract ▾
Computer Vision
Can Vision-Language Models Think from the Sky? Unifying UAV Reasoning and Generation
Jintao Sun, Gangyi Ding, Donglin Di et al. (+2 more)
Vision-
Language Models have achieved strong progress in ground-view visual understanding, yet they remain brittle in high-altitude Unmanned Aerial Vehicle scenes, where objects are tiny and densely packed, textures are repetitive, and top-down orientations are ambiguous. We introduce UAVReason, a large-scale
UAV-native
dataset and evaluation suite for studying unified aerial
reasoning and generation under this nadir-view domain shift. UAVReason aligns RGB imagery, depth maps, semantic
segmentation masks, captions, and question-answer pairs within a consistent aerial domain. It contains 23.6K captioned frames, 273K VQA pairs including 68.2K two-frame temporal questions, and 188.8K cross-modal generation samples across RGB, depth, and
segmentation modalities. We further adapt UAVReason-Bagel as a unified understanding-and-generation baseline that jointly optimizes
language reasoning and dense visual generation objectives. Experiments show that general-purpose VLMs and off-the-shelf unified generators struggle with
UAV-native grounding, while UAVReason-Bagel substantially improves over its pretrained counterpart, increasing VQA-1F F1 from 0.394 to 0.711, VQA-2F F1 from 0.427 to 0.822, and heading-aware VQA F1 from 0.798 to 0.973. For generation, it improves
segmentation mIoU to 0.143 and reduces KID from 0.078 to 0.048 for depth-
segmentation-text-conditioned RGB synthesis. More importantly, our ablations reveal a bidirectional synergy between synthesis and
reasoning. Dense generation objectives improve temporal semantic consistency, while
language-level
reasoning regularizes sparse-condition image synthesis. These results suggest that unified
reasoning and generation provide effective geometry-aware structural priors for physically grounded aerial intelligence. All data, code, and evaluation tools will be released.
Read the full abstract ▾
All authors Jintao Sun, Gangyi Ding, Donglin Di, Hu Zhang, Zhedong Zheng
Computer Vision
3DTurboQuant: Training-Free Near-Optimal Quantization for 3D Reconstruction Models
Jae Joong Lee
Every existing method for compressing 3D Gaussian Splatting, NeRF, or
transformer-based 3D reconstructors requires learning a data-dependent codebook through per-scene fine-tuning. We show this is unnecessary. The parameter vectors that dominate storage in these models, 45-dimensional spherical harmonics in 3DGS and 1024-dimensional key-value vectors in
DUSt3R, fall in a dimension range where a single random rotation transforms any input into coordinates with a known Beta distribution. This makes precomputed, data-independent Lloyd-Max quantization near-optimal, within a factor of 2.7 of the information-theoretic lower bound. We develop 3D, deriving (1) a dimension-dependent criterion that predicts which parameters can be quantized and at what bit-width before running any experiment, (2) norm-separation bounds connecting quantization MSE to rendering PSNR per scene, (3) an entry-grouping strategy extending rotation-based quantization to 2-dimensional hash grid features, and (4) a composable
pruning-quantization pipeline with a closed-form compression ratio. On NeRF Synthetic, 3DTurboQuant compresses 3DGS by 3.5x with 0.02dB PSNR loss and
DUSt3R KV caches by 7.9x with 39.7dB pointmap fidelity. No training, no codebook learning, no calibration data. Compression takes seconds. The code will be released (https://github.com/JaeLee18/3DTurboQuant)
Read the full abstract ▾
Computer Vision
Rethinking IRSTD: Single-Point Supervision Guided Encoder-only Framework is Enough for Infrared Small Target Detection
Rixiang Ni, Boyang Li, Jun Chen et al. (+6 more)
Infrared small target detection (IRSTD) aims to separate small targets from clutter backgrounds. Extensive research is dedicated to the pixel-level supervision-guided "encoder-decoder"
segmentation paradigm. Although having achieved promising performance, they neglect the fact that small targets only occupy a few pixels and are usually accompanied with blurred boundary caused by clutter backgrounds. Based on this observation, we argue that the first principle of IRSTD should be target localization instead of separating all target region accompanied with indistinguishable background noise. In this paper, we reformulate IRSTD as a centroid regression task and propose a novel Single-Point Supervision guided Infrared Probabilistic Response Encoding method (namely, SPIRE), which is indeed challenging due to the mismatch between reduced supervision network and equivalent output. Specifically, we first design a Point-Response Prior Supervision (PRPS), which transforms single-point annotations into probabilistic response map consistent with infrared point-target response characteristics, with a High-Resolution Probabilistic Encoder (HRPE) that enables encoder-only, end-to-end regression without decoder reconstruction. By preserving high-resolution features and increasing effective supervision density, SPIRE alleviates optimization instability under sparse target distributions. Finally, extensive experiments on various IRSTD
benchmarks, including SIRST-UAVB and SIRST4 demonstrate that SPIRE achieves competitive target-level detection performance with consistently low false alarm rate (Fa) and significantly reduced computational cost. Code is publicly available at: https://github.com/NIRIXIANG/SPIRE-IRSTD.
Read the full abstract ▾
All authors Rixiang Ni, Boyang Li, Jun Chen, Yonghao Li, Feiyu Ren, Yuji Wang, Haoyang Yuan, Wujiao He, Wei An