Computer Vision
RSTNet: Enhancing Small-Target Recognition in Noisy SAR Imagery via Robust Feature Learning and Distribution-Aware Regression
Xiaojing Zhao, Shiyang Li, Zenan Chu et al. (+5 more)
SAR supports all-day-and-night oceanic observation, yet vessel identification from SAR images is hampered by speckle noise, intricate land-sea backgrounds and dim miniature vessels, yielding numerous false identifications and missed targets. We develop an SAR-adaptive stable detection model RSTNet based on YOLOv8. A large-kernel channel-separated denoising unit eliminates noise and reserves delicate vessel features; parallel patch-aware
attention enhances multi-scale feature extraction for miniature objects; NWD loss substitutes conventional IoU loss to achieve accurate bounding box regression. The proposed model outperforms the original YOLOv8 on the SSDD
dataset with 97.0% precision, 95.1% recall and 98.9% mAP@0.5. Validations on the HRSID
dataset verify its favorable generalization capacity for coastal miniature vessels. Therefore, our work delivers an effective technical scheme for ocean observation imaging with noisy miniature targets. The source code is available at https://github.com/renhcmhx/SAR.git.
Read the full abstract ▾
All authors Xiaojing Zhao, Shiyang Li, Zenan Chu, Ying Zhang, Peinan Hao, Tianzi Yan, Jiajia Chen, Huicong Ning
Computer Vision
Footprint-Guided Exemplar-Free Continual Histopathology Report Generation
Pratibha Kumari, Daniel Reisenbüchler, Afshin Bozorgpour et al. (+3 more)
Rapid progress in
vision-
language modeling has enabled pathology report generation from gigapixel whole-slide images, but most approaches assume static training with simultaneous access to all data. In clinical deployment, however, new organs, institutions, and reporting conventions emerge over time, and sequential fine-tuning can cause catastrophic forgetting. We introduce an exemplar-free continual learning framework for WSI-to-report generation that avoids storing raw slides or patch exemplars. The core idea is a compact domain footprint built in a frozen patch-
embedding space: a small codebook of representative morphology
tokens together with slide-level co-occurrence summaries and lightweight patch-count priors. These footprints support generative replay by synthesizing pseudo-WSI representations that reflect domain-specific morphological mixtures, while a teacher snapshot provides pseudo-reports to supervise the updated model without retaining past data. To address shifting reporting conventions, we distill domain-specific linguistic characteristics into a compact style descriptor and use it to steer generation. At inference, the model identifies the most compatible descriptor directly from the slide signal, enabling domain-agnostic setup without requiring explicit domain identifiers. Evaluated across multiple public continual learning
benchmarks, our approach outperforms exemplar-free and limited-buffer rehearsal baselines, highlighting footprint-based generative replay as a practical solution for deployment in evolving clinical settings.
Read the full abstract ▾
All authors Pratibha Kumari, Daniel Reisenbüchler, Afshin Bozorgpour, yousef Sadegheih, Priyankar Choudhary, Dorit Merhof
Computer Vision
Action-Geometry Prediction with 3D Geometric Prior for Bimanual Manipulation
Chongyang Xu, Haipeng Li, Shen Cheng et al. (+4 more)
Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions. However, existing methods typically rely on 2D features with limited spatial awareness, or require explicit point clouds that are difficult to obtain reliably in real-world settings. At the same time, recent 3D
geometric foundation models show that accurate and diverse 3D structure can be reconstructed directly from RGB images in a fast and robust manner. We leverage this opportunity and propose a framework that builds bimanual manipulation directly on a pre-trained 3D
geometric foundation model. Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses
diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap. By explicitly predicting how the 3D scene will evolve together with the action sequence, the policy gains strong spatial understanding and predictive capability using only RGB observations. We evaluate our method both in simulation on the RoboTwin
benchmark and in real-world
robot executions. Our approach consistently outperforms 2D-based and point-cloud-based baselines, achieving state-of-the-art performance in manipulation success, inter-arm coordination, and 3D spatial prediction accuracy. Code is available at https://github.com/Chongyang-99/GAP.git.
Read the full abstract ▾
All authors Chongyang Xu, Haipeng Li, Shen Cheng, Jingyu Hu, Haoqiang Fan, Ziliang Feng, Shuaicheng Liu
Computer Vision
See, Act, Adapt: Active Perception for Unsupervised Cross-Domain Visual Adaptation via Personalized VLM-Guided Agent
Tianci Tang, Tielong Cai, Hongwei Wang et al. (+1 more)
Pre-trained perception models excel in generic image domains but degrade significantly in novel environments like indoor scenes. The conventional remedy is fine-tuning on downstream data which incurs catastrophic forgetting of prior knowledge and demands costly, scene-specific annotations. We propose a paradigm shift through Sea$^2$ (See, Act, Adapt): rather than adapting the perception modules themselves, we adapt how they are deployed through an intelligent pose-control
agent. Sea$^2$ keeps all perception modules frozen, requiring no downstream labels during training, and uses only scalar perceptual feedback to navigate the
agent toward informative viewpoints. Specially, we transform a
vision-
language model (VLM) into a low-level pose controller through a two-stage training pipeline: first fine-tuning it on rule-based exploration trajectories that systematically probe indoor scenes, and then refining the policy via unsupervised reinforcement learning that constructs rewards from the perception module's outputs and confidence. Unlike prior active perception methods that couple exploration with specific models or collect data for retraining them, Sea$^2$ directly leverages off-the-shelf perception models for various tasks without the need for retraining. We conducted experiments on three visual perception tasks, including visual grounding,
segmentation and 3D box estimation, with performance improvements of 13.54%, 15.92% and 27.68% respectively on
dataset ReplicaCAD.
Read the full abstract ▾
All authors Tianci Tang, Tielong Cai, Hongwei Wang, Gaoang Wang
Computer Vision · Cross-listed from eess.IV
BiM-GeoAttn-Net: Linear-Time Depth Modeling with Geometry-Aware Attention for 3D Aortic Dissection CTA Segmentation
Yuan Zhang, Lei Liu, Jialin Zhang et al. (+3 more)
Accurate
segmentation of aortic dissection (AD) lumens in CT angiography (CTA) is essential for quantitative morphological assessment and clinical decision-making. However, reliable 3D delineation remains challenging due to limited long-range context modeling, which compromises inter-slice coherence, and insufficient structural discrimination under low-contrast conditions. To address these limitations, we propose BiM-GeoAttn-Net, a lightweight framework that integrates linear-time depth-wise state-space modeling with geometry-aware vessel refinement. Our approach is featured by Bidirectional Depth Mamba (BiM) to efficiently capture cross-slice dependencies and Geometry-Aware Vessel
Attention (GeoAttn) module that employs orientation-sensitive anisotropic filtering to refine tubular structures and sharpen ambiguous boundaries. Extensive experiments on a multi-source AD CTA
dataset demonstrate that BiM-GeoAttn-Net achieves a Dice score of 93.35% and an HD95 of 12.36 mm, outperforming representative CNN-,
Transformer-, and SSM-based baselines in overlap metrics while maintaining competitive boundary accuracy. These results suggest that coupling linear-time depth modeling with geometry-aware refinement provides an effective, computationally efficient solution for robust 3D AD
segmentation.
Read the full abstract ▾
All authors Yuan Zhang, Lei Liu, Jialin Zhang, Ya-Nan Zhang, Ling Wang, Nan Mu
Computer Vision · Cross-listed from cs.AI
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
Yiyang Fang, Wenke Huang, Pei Fu et al. (+5 more)
Multimodal Large
Language Models (MLLMs) have shown remarkable progress in visual
reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of human
emotions. Existing approaches based on supervised fine-tuning often suffer from limited generalization and poor interpretability, while reinforcement learning methods such as Group Relative Policy Optimization fail to align with the intrinsic characteristics of emotional cognition. To address these challenges, we propose Reflective Reinforcement Learning for Emotional
Reasoning (EMO-R3), a framework designed to enhance the emotional
reasoning ability of MLLMs. Specifically, we introduce Structured Emotional Thinking to guide the model to perform step-by-step emotional
reasoning in a structured and interpretable manner, and design a Reflective Emotional Reward that enables the model to re-evaluate its
reasoning based on visual-text consistency and emotional coherence. Extensive experiments demonstrate that EMO-R3 significantly improves both the interpretability and emotional intelligence of MLLMs, achieving superior performance across multiple visual emotional understanding
benchmarks.
Read the full abstract ▾
All authors Yiyang Fang, Wenke Huang, Pei Fu, Yihao Yang, Kehua Su, Zhenbo Luo, Jian Luan, Mang Ye
Computer Vision · Cross-listed from eess.IV
FluoCLIP: Stain-Aware Focus Quality Assessment in Fluorescence Microscopy
Hyejin Park, Jiwon Yoon, Sumin Park et al. (+5 more)
Accurate focus quality assessment (FQA) in fluorescence microscopy is challenging due to stain-dependent optical variations that induce heterogeneous focus behavior across images. Existing methods, however, treat focus quality as a stain-agnostic problem, assuming a shared global ordering. We formulate stain-aware FQA for fluorescence microscopy, showing that focus-rank relationships vary substantially across stains due to stain-dependent imaging characteristics and invalidate this assumption. To support this formulation, we introduce FluoMix, the first
dataset for stain-aware FQA spanning multiple tissues, fluorescent stains, and focus levels. We further propose FluoCLIP, a two-stage
vision-
language framework that grounds stain semantics and enables stain-conditioned ordinal
reasoning for focus prediction, effectively decoupling stain representation from ordinal structure. By explicitly modeling stain-dependent focus behavior, FluoCLIP consistently outperforms both conventional FQA methods and recent
vision-
language baselines, demonstrating strong generalization across diverse fluorescence microscopy conditions. Code and
dataset are publicly available at https://fluoclip.github.io/.
Read the full abstract ▾
All authors Hyejin Park, Jiwon Yoon, Sumin Park, Suree Kim, Sinae Jang, Eunsoo Lee, Dongmin Kang, Dongbo Min
Computer Vision
Fourier Angle Alignment for Oriented Object Detection in Remote Sensing
Changyu Gu, Linwei Chen, Lin Gu et al. (+1 more)
In remote sensing rotated object detection, mainstream methods suffer from two bottlenecks, directional incoherence at detector neck and task conflict at detecting head. Ulitising
fourier rotation equivariance, we introduce
Fourier Angle
Alignment, which analyses angle information through frequency spectrum and aligns the main direction to a certain orientation. Then we propose two plug and play modules : FAAFusion and FAA Head. FAAFusion works at the detector neck, aligning the main direction of higher-level features to the lower-level features and then fusing them. FAA Head serves as a new detection head, which pre-aligns RoI features to a canonical angle and adds them to the original features before classification and regression. Experiments on DOTA-v1.0, DOTA-v1.5 and HRSC2016 show that our method can greatly improve previous work. Particularly, our method achieves new state-of-the-art results of 78.72% mAP on DOTA-v1.0 and 72.28% mAP on DOTA-v1.5
datasets with single scale training and testing, validating the efficacy of our approach in remote sensing object detection. The code is made publicly available at https://github.com/gcy0423/
Fourier-Angle-
Alignment .
Read the full abstract ▾
All authors Changyu Gu, Linwei Chen, Lin Gu, Ying Fu
Computer Vision
Diffusion Probe: Generated Image Result Prediction Using CNN Probes
Benlei Cui, Bukun Huang, Zhizeng Ye et al. (+7 more)
Text-to-image (T2I)
diffusion models lack an efficient mechanism for early quality assessment, leading to costly trial-and-error in multi-generation scenarios such as
prompt iteration,
agent-based generation, and flow-grpo. We reveal a strong correlation between early
diffusion cross-
attention distributions and final image quality. Based on this finding, we introduce
Diffusion Probe, a framework that leverages internal cross-
attention maps as predictive signals. We design a lightweight predictor that maps statistical properties of early-stage cross-
attention extracted from initial denoising steps to the final image's overall quality. This enables accurate
forecasting of image quality across diverse evaluation metrics long before full synthesis is complete. We validate
Diffusion Probe across a wide range of settings. On multiple T2I models, across early denoising windows, resolutions, and quality metrics, it achieves strong correlation (PCC > 0.7) and high classification performance (AUC-ROC > 0.9). Its reliability translates into practical gains. By enabling early quality-aware decisions in workflows such as
prompt optimization, seed selection, and accelerated RL training, the probe supports more targeted sampling and avoids computation on low-potential generations. This reduces computational overhead while improving final output quality.
Diffusion Probe is model-agnostic, efficient, and broadly applicable, offering a practical solution for improving T2I generation efficiency through early quality prediction. Code: https://github.com/Alibaba-VELLDEPTH/DiffusionProbe
Read the full abstract ▾
All authors Benlei Cui, Bukun Huang, Zhizeng Ye, Xuemei Dong, Tuo Chen, Hui Xue, Dingkang Yang, Longtao Huang, Jingqun Tang, Haiwen Hong
Computer Vision · Cross-listed from eess.IV
VesselBridge3D: A Foundation Model Adaptation Framework for Label-Efficient 3D Vessel Segmentation
Kirato Yoshihara, Yohei Sugawara, Yuta Tokuoka et al. (+1 more)
State-of-the-art vessel
segmentation methods typically require large-scale annotated
datasets and suffer from severe performance degradation under domain shifts. In clinical practice, however, acquiring extensive annotations for every new scanner or protocol is unfeasible. To address this, we propose VesselBridge3D, a foundation model adaptation framework that bridges frozen
vision foundation models and volumetric vessel
segmentation through lightweight 3D adaptation modules. The framework combines a lightweight 3D Adapter, a multi-scale 3D Aggregator, and Z-channel
embedding for efficient adaptation to volumetric medical images. We instantiate VesselBridge3D with three frozen foundation encoders (DINOv3, MedSAM, and MedGemma) and evaluate it on the TopCoW (ID) and Lausanne (OOD)
datasets. In the extreme low-data regime with 5 training samples, our method achieved a Dice score of 43.42%, marking a 30% relative improvement over the state-of-the-art nnU-Net (33.41%) and outperforming other
Transformer-based baselines by up to 45%. The proposed framework was effective across all evaluated frozen foundation encoders, with DINOv3 yielding the best performance in the most label-efficient settings. Furthermore, in the out-of-distribution setting, our model demonstrated superior robustness, achieving a 50% relative improvement over nnU-Net (21.37% vs. 14.22%), which suffered from severe domain overfitting. Ablation studies confirmed the effectiveness of the proposed 3D adaptation modules. Our results demonstrate that VesselBridge3D is an effective framework for label-efficient 3D vessel
segmentation under data scarcity and domain shifts.
Read the full abstract ▾
All authors Kirato Yoshihara, Yohei Sugawara, Yuta Tokuoka, Lihang Hong
Computer Vision · Cross-listed from eess.IV
VideoPulse: Neonatal heart rate and peripheral capillary oxygen saturation (SpO2) estimation from contact free video
Deependra Dewagiri, Kamesh Anuradha, Pabadhi Liyanage et al. (+6 more)
Remote photoplethysmography (rPPG) enables contact free monitoring of vital signs and is especially valuable for neonates, since conventional methods often require sustained skin contact with adhesive probes that can irritate fragile skin and increase infection control burden. We present VideoPulse, a neonatal
dataset and an end to end pipeline that estimates neonatal heart rate and peripheral capillary oxygen saturation (SpO2) from facial
video. VideoPulse contains 157 recordings totaling 2.6 hours from 52 neonates with diverse face orientations. Our pipeline performs face
alignment and artifact aware supervision using denoised pulse oximeter signals, then applies 3D CNN backbones for heart rate and SpO2 regression with label distribution smoothing and weighted regression for SpO2. Predictions are produced in 2 second windows. On the NBHR neonatal
dataset, we obtain heart rate MAE 2.97 bpm using 2 second windows (2.80 bpm at 6 second windows) and SpO2 MAE 1.69 percent. Under cross
dataset evaluation, the NBHR trained heart rate model attains 5.34 bpm MAE on VideoPulse, and fine tuning an NBHR pretrained SpO2 model on VideoPulse yields MAE 1.68 percent. These results indicate that short unaligned neonatal
video segments can support accurate heart rate and SpO2 estimation, enabling low cost non invasive monitoring in neonatal intensive care.
Read the full abstract ▾
All authors Deependra Dewagiri, Kamesh Anuradha, Pabadhi Liyanage, Helitha Kulatunga, Pamuditha Somarathne, Udaya S. K. P. Miriya Thanthrige, Nishani Lucas, Anusha Withana, Joshua P. Kulasingham
Computer Vision · Cross-listed from cs.LG
OPTIAGENT: A Physics-Driven Agentic Framework for Automated Optical Design
Yuyu Geng, Lei Sun, Yao Gao et al. (+6 more)
Optical design is the process of configuring optical elements to precisely manipulate light for high-fidelity imaging. It is inherently a highly non-convex optimization problem that relies heavily on human heuristic expertise and domain-specific knowledge. While Large
Language Models (LLMs) possess extensive optical knowledge, their capabilities in leveraging the knowledge in designing lens system remain significantly constrained. This work represents the first attempt to employ LLMs in the field of optical design. We bridge the expertise gap by enabling users without formal optical training to successfully develop functional lens systems. Concretely, we curate a comprehensive
dataset, named OptiDesignQA, which encompasses both classical lens systems sourced from standard optical textbooks and novel configurations generated by automated design
algorithms for training and evaluation. Furthermore, we inject domain-specific optical expertise into the LLM through a hybrid objective of full-system synthesis and lens completion. To align the model with optical principles, we employ Group Relative Policy Optimization Done Right (DrGRPO) guided by Optical Lexicographic Reward for physics-driven policy
alignment. This reward system incorporates structural format rewards, physical feasibility rewards, light-manipulation accuracy, and LLM-based heuristics. Finally, our model integrates with specialized optical optimization routines for end-to-end fine-tuning and precision refinement. We
benchmark our proposed method against both traditional optimization-based automated design
algorithms and LLM counterparts, and experimental results show the superiority of our method.
Read the full abstract ▾
All authors Yuyu Geng, Lei Sun, Yao Gao, Xinxin Hu, Zhonghua Yi, Xiaolong Qian, Weijian Hu, Jian Bai, Kaiwei Wang
Computer Vision
Learning Accurate Segmentation Purely from Self-Supervision
Zuyao You, Zuxuan Wu and Yu-Gang Jiang
Accurately segmenting objects without any manual annotations remains one of the core challenges in computer
vision. In this work, we introduce Selfment, a fully self-supervised framework that segments foreground objects directly from raw images without human labels, pretrained
segmentation models, or any post-processing. Selfment first constructs patch-level affinity graphs from self-supervised features and applies NCut to obtain an initial coarse foreground–background separation. We then introduce Iterative Patch Optimization (IPO), a feature-space refinement procedure that progressively enforces spatial coherence and semantic consistency through iterative patch clustering. The refined masks are subsequently used as supervisory signals to train a lightweight
segmentation head with contrastive and region-consistency objectives, allowing the model to learn stable and transferable object representations. Despite its simplicity and complete absence of manual supervision, Selfment sets new state-of-the-art (SoTA) results across multiple
benchmarks. It achieves substantial improvements on $F_{\max}$ over previous unsupervised saliency detection methods on ECSSD ($+4.0\%$), HKUIS ($+4.6\%$), and PASCAL-S ($+5.7\%$). Moreover, without any additional fine-tuning, Selfment demonstrates remarkable zero-shot generalization to camouflaged object detection tasks (e.g., $0.910$ $S_m$ on CHAMELEON and $0.792$ $F_{\beta}^{\omega}$ on CAMO), outperforming all existing unsupervised approaches and even rivaling the SoTA fully supervised methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
Neural Image Space Tessellation effect
Youyang Du, Junqiu Zhu, Zheng Zeng et al. (+2 more)
We present Neural Image Space Tessellation effect (NIST), a lightweight screen-space post-processing approach for reducing the faceted silhouettes of low-poly renderings. Instead of tessellating primitives, creating new geometry, or modifying the underlying mesh, NIST uses the low-poly rendering result together with simple auxiliary G-buffer attributes to learn geometry-guided smoothing of object contours in image space. At its core, NIST first deforms image-space contours implicitly and then learns to reassign appearance in the whole image-space, including the deformed regions, preserving texture continuity and avoiding seam artifacts. Experiments show that NIST reduces visually apparent
geometric faceting and produces smooth, coherent silhouettes close to tessellation-based smoothing references, with a nearly constant per-frame cost in our tested settings. To the best of our knowledge, NIST is the first work to move the solution of low-poly silhouette faceting from the pre-rendering geometry stage to a post-rendering screen-space stage.
Read the full abstract ▾
All authors Youyang Du, Junqiu Zhu, Zheng Zeng, Lu Wang, Lingqi Yan
Computer Vision · Cross-listed from eess.IV
Unsupervised Causal Prototypical Networks for De-biased Interpretable Dermoscopy Diagnosis
Junhao Jia, Yueyi Wu, Huangwei Chen et al. (+4 more)
Despite the success of deep learning in dermoscopy image analysis, its inherent black-box nature hinders clinical trust, motivating the use of prototypical networks for case-based visual transparency. However, inevitable selection bias in clinical data often drives these models toward shortcut learning, where environmental confounders are erroneously encoded as predictive prototypes, generating spurious visual evidence that misleads medical decision-making. To mitigate these confounding effects, we propose CausalProto, an Unsupervised Causal Prototypical Network that fundamentally purifies the visual evidence chain. Framed within a Structural Causal Model, we employ an Information Bottleneck-constrained encoder to enforce strict unsupervised orthogonal disentanglement between pathological features and environmental confounders. By mapping these decoupled representations into independent prototypical spaces, we leverage the learned spurious dictionary to perform
backdoor adjustment via do-calculus, transforming complex causal interventions into efficient expectation pooling to marginalize environmental noise. Extensive experiments on multiple dermoscopy
datasets demonstrate that CausalProto achieves superior diagnostic performance and consistently outperforms standard black box models, while simultaneously providing transparent and high purity visual interpretability without suffering from the traditional accuracy compromise.
Read the full abstract ▾
All authors Junhao Jia, Yueyi Wu, Huangwei Chen, Haodong Jing, Haishuai Wang, Jiajun Bu, Lei Wu
Computer Vision · Cross-listed from cs.HC
Shape vs. Context: Examining Human–AI Gaps in Ambiguous Japanese Character Recognition
Daichi Haraguchi
High text recognition performance does not guarantee that
Vision-
Language Models (VLMs) share human-like decision patterns when resolving ambiguity. We investigate this behavioral gap by directly comparing humans and VLMs using continuously interpolated Japanese character shapes generated via a $\beta$-VAE. We estimate decision boundaries in a single-character recognition (shape-only task) and evaluate whether VLM responses align with human judgments under shape in context (i.e.,
embedding an ambiguous character near the human decision boundary in word-level context). We find that human and VLM decision boundaries differ in the shape-only task, and that shape in context can improve human
alignment in some conditions. These results highlight qualitative behavioral differences, offering foundational insights toward human–VLM
alignment benchmarking.
Read the full abstract ▾
Computer Vision
U-Mind: A Unified Framework for Real-Time Multimodal Interaction with Audiovisual Generation
Xiang Deng, Feng Gao, Yong Zhang et al. (+5 more)
Full-stack
multimodal interaction in real-time is a central goal in building intelligent
embodied agents capable of natural, dynamic communication. However, existing systems are either limited to unimodal generation or suffer from degraded
reasoning and poor cross-modal
alignment, preventing coherent and perceptually grounded interactions. In this work, we introduce U-Mind, the first unified system for high-intelligence
multimodal dialogue that supports real-time generation and jointly models
language,
speech, motion, and
video synthesis within a single interactive loop. At its core, U-Mind implements a Unified
Alignment and
Reasoning Framework that addresses two key challenges: enhancing cross-modal synchronization via a segment-wise
alignment strategy, and preserving
reasoning abilities through Rehearsal-Driven Learning. During inference, U-Mind adopts a text-first decoding pipeline that performs internal chain-of-thought planning followed by temporally synchronized generation across modalities. To close the loop, we implement a real-time
video rendering framework conditioned on pose and
speech, enabling expressive and synchronized visual feedback. Extensive experiments demonstrate that U-Mind achieves state-of-the-art performance on a range of
multimodal interaction tasks, including question answering, instruction following, and motion generation, paving the way toward intelligent, immersive conversational
agents.
Read the full abstract ▾
All authors Xiang Deng, Feng Gao, Yong Zhang, Youxin Pang, Xu Xiaoming, Zhuoliang Kang, Xiaoming Wei, Yebin Liu
Computer Vision
UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking
Hao Wu, Xudong Wang, Jialiang Zhang et al. (+5 more)
One-stream
Transformer-based trackers achieve advanced performance in visual object tracking but suffer from significant computational overhead that hinders real-time deployment. While
token pruning offers a path to efficiency, existing methods are fragmented. They typically prune the search region, dynamic template, and static template in isolation, overlooking critical inter-component dependencies, which yields suboptimal
pruning and degraded accuracy. To address this, we introduce UTPTrack, a simple and Unified
Token Pruning framework that, for the first time, jointly compresses all three components. UTPTrack employs an
attention-guided,
token type-aware strategy to holistically model redundancy, a design that seamlessly supports unified tracking across
multimodal and
language-guided tasks within a single model. Extensive evaluations on 10
benchmarks demonstrate that UTPTrack achieves a new state-of-the-art in the accuracy-efficiency trade-off for
pruning-based trackers,
pruning 65.4% of
vision tokens in RGB-based tracking and 67.5% in unified tracking while preserving 99.7% and 100.5% of baseline performance, respectively. This strong performance across both RGB and
multimodal scenarios underlines its potential as a robust foundation for future research in efficient visual tracking. Code will be released at https://github.com/EIT-NLP/UTPTrack.
Read the full abstract ▾
All authors Hao Wu, Xudong Wang, Jialiang Zhang, Junlong Tong, Xinghao Chen, Junyan Lin, Yunpu Ma, Xiaoyu Shen
Computer Vision
A Difference-in-Difference Approach to Detecting AI-Generated Images
Xinyi Qi, Kai Ye, Chengchun Shi et al. (+3 more)
Diffusion models are able to produce AI-generated images that are almost indistinguishable from real ones. This raises concerns about their potential misuse and poses substantial challenges for detecting them. Many existing detectors rely on reconstruction error – the difference between the input image and its reconstructed version – as the basis for distinguishing real from fake images. However, these detectors become less effective as modern AI-generated images become increasingly similar to real ones. To address this challenge, we propose a novel difference-in-difference method. Instead of directly using the reconstruction error (a first-order difference), we compute the difference in reconstruction error – a second-order difference – for variance reduction and improving detection accuracy. Extensive experiments demonstrate that our method achieves strong generalization performance, enabling reliable detection of AI-generated images in the era of generative AI.
Read the full abstract ▾
All authors Xinyi Qi, Kai Ye, Chengchun Shi, Ying Yang, Hongyi Zhou, Jin Zhu
Computer Vision · Cross-listed from cs.RO
StemVLA:An Open-Source Vision-Language-Action Model with Future 3D Spatial Geometry Knowledge and 4D Historical Representation
Jiasong Xiao, Yutao She, Kai Li et al. (+2 more)
Vision-
language-action (VLA) models integrate visual observations and
language instructions to predict
robot actions, demonstrating promising generalization in manipulation tasks. However, most existing approaches primarily rely on direct mappings from 2D visual inputs to action sequences, without explicitly modeling the underlying 3D spatial structure or temporal world dynamics. Such representations may limit spatial
reasoning and long-horizon decision-making in dynamic environments. To address this limitation, we propose StemVLA, a novel framework that explicitly incorporates both future-oriented 3D spatial knowledge and historical 4D spatiotemporal representations into action prediction. First, instead of relying solely on observed images, StemVLA forecasts structured 3D future spatial-
geometric world knowledge, enabling the model to anticipate upcoming scene geometry and object configurations. Second, to capture temporal consistency and motion dynamics, we feed historical image frames into a pretrained
video-geometry
transformer backbone to extract implicit 3D world representations, and further aggregate them across time using a temporal
attention module, termed VideoFormer [20], forming a unified 4D historical spatiotemporal representation. By jointly modeling 2D observations, predicted 3D future structure, and aggregated 4D temporal dynamics, StemVLA enables more comprehensive world understanding for
robot manipulation. Extensive experiments in simulation demonstrate that Stem-VLA achieves an average accuracy of 92.0% across the LIBERO subsets, and 86.0% on the long-horizon LIBERO-Long subset.
Read the full abstract ▾
All authors Jiasong Xiao, Yutao She, Kai Li, Yuyang Sha, Ziang Cheng
Computer Vision
Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?
Hongbo Jiang, Jie Li, Yunhang Shen et al. (+4 more)
Unified
Multimodal Large
Language Models (U-MLLMs) integrate understanding and generation within a single architecture. However, existing evaluations typically assess these capabilities separately, overlooking semantic equivalence, i.e., the ability to manifest consistent
reasoning results regardless of the output modality. In this work, we investigate whether current U-MLLMs satisfy this premise. We observe that while models demonstrate robust textual
reasoning, they fail to maintain semantic equivalence when required to render the same results in the image modality. To rigorously diagnose this discrepancy, we introduce VGUBench, a framework to decouple
reasoning logic from generation fidelity. VGUBench comprises three diagnostic tasks: (1)Textual Generative Understanding, establishing a baseline for
reasoning accuracy in textual response; (2)Visual Generative Understanding, evaluating the ability to generate visual responses that represent the correct answer; and (3)a Visual Rendering control task, which assesses the ability to directly render explicit visual descriptions into images without complex
reasoning. Our evaluation reveals a significant disparity: despite strong performance in textual understanding and visual rendering, U-MLLMs exhibit a marked performance collapse when required to generate visual answers to questions. Furthermore, we find a negligible correlation between visual answering performance and basic rendering quality. These results suggest that the failure stems not from insufficient generation fidelity, but from a breakdown in cross-modal semantic
alignment. We provide diagnostic insights to address this challenge in future Unified Generation and Understanding Models.
Read the full abstract ▾
All authors Hongbo Jiang, Jie Li, Yunhang Shen, Pingyang Dai, Xing Sun, Haoyu Cao, Liujuan Cao
Computer Vision
EgoGraph: Temporal Knowledge Graph for Egocentric Video Understanding
Shitong Sun, Ke Han, Yukai Huang et al. (+2 more)
Ultra-long
egocentric videos spanning multiple days present significant challenges for
video understanding. Existing approaches still rely on fragmented local processing and limited temporal modeling, restricting their ability to reason over such extended sequences. To address these limitations, we introduce EgoGraph, a training-free and dynamic knowledge-graph construction framework that explicitly encodes long-term, cross-entity dependencies in
egocentric video streams. EgoGraph employs a novel
egocentric schema that unifies the extraction and abstraction of core entities, such as people, objects, locations, and events, and structurally reasons about their attributes and interactions, yielding a significantly richer and more coherent semantic representation than traditional clip-based
video models. Crucially, we develop a temporal relational modeling strategy that captures temporal dependencies across entities and accumulates stable long-term
memory over multiple days, enabling complex temporal
reasoning. Extensive experiments on the EgoLifeQA and EgoR1-bench
benchmarks demonstrate that EgoGraph achieves state-of-the-art performance on long-term
video question answering, validating its effectiveness as a new paradigm for ultra-long
egocentric video understanding.
Read the full abstract ▾
All authors Shitong Sun, Ke Han, Yukai Huang, Weitong Cai, Jifei Song
Computer Vision · Cross-listed from cs.RO
A Reliable Indoor Navigation System for Humans Using AR-based Technique
Vijay U.Rathod, Manav S.Sharma, Shambhavi Verma et al. (+3 more)
Reliable
navigation systems are not available indoors, such as in campuses and small areas. Users must depend on confusing, time-consuming static signage or floor maps. In this paper, an AR-based technique has been applied to campus and small-site
navigation, where Vuforia Area Target is used for environment modeling. AI
navigation's NavMesh component is used for
navigation purposes, and the A*
algorithm is used within this component for shortest path calculation. Compared to Dijkstra's
algorithm, it can reach a solution about two to three times faster for smaller search spaces. In many cases, Dijkstra's
algorithm has difficulty performing well in high-complexity environments where
memory usage grows and processing times increase. Compared to older approaches such as GPS, real-time processing and AR overlays can be combined to provide intuitive directions for users while dynamically updating the path in response to environmental changes. Experimental results indicate significantly improved
navigation accuracy, better user experience, and greater efficiency compared to traditional methods. These results show that AR technology integrated with existing pathfinding
algorithms is feasible and scalable, making it a user-friendly solution for indoor
navigation. Although highly effective in limited and defined indoor spaces, further optimization of NavMesh is required for large or highly dynamic environments.
Read the full abstract ▾
All authors Vijay U.Rathod, Manav S.Sharma, Shambhavi Verma, Aadi Joshi, Sachin Aage, Sujal Shahane
Computer Vision
HiDrop: Hierarchical Vision Token Reduction in MLLMs via Late Injection, Concave Pyramid Pruning, and Early Exit
Hao Wu, Yingqi Fan, Jinyang Dai et al. (+3 more)
The quadratic computational cost of processing
vision tokens in
Multimodal Large
Language Models (MLLMs) hinders their widespread adoption. While progressive
vision token pruning offers a promising solution, current methods misinterpret shallow layer functions and use rigid schedules, which fail to unlock the full efficiency potential. To address these issues, we propose HiDrop, a framework that aligns
token pruning with the true hierarchical function of MLLM layers. HiDrop features two key innovations: (1) Late Injection, which bypasses passive shallow layers to introduce visual
tokens exactly where active fusion begins; and (2) Concave Pyramid
Pruning with an Early Exit mechanism to dynamically adjust
pruning rates across middle and deep layers. This process is optimized via an inter-layer similarity measure and a differentiable top-k operator. To ensure practical efficiency, HiDrop further incorporates persistent positional encoding, FlashAttention-compatible
token selection, and parallel decoupling of
vision computation to eliminate hidden overhead associated with dynamic
token reduction. Extensive experiments show that HiDrop compresses about 90% visual
tokens while matching the original performance and accelerating training by 1.72 times. Our work not only sets a new state-of-the-art for efficient MLLM training and inference but also provides valuable insights into the hierarchical nature of
multimodal fusion. The code is released at https://github.com/EIT-NLP/HiDrop.
Read the full abstract ▾
All authors Hao Wu, Yingqi Fan, Jinyang Dai, Junlong Tong, Yunpu Ma, Xiaoyu Shen
Computer Vision
Towards Source-Aware Object Swapping with Initial Noise Perturbation
Jiahui Zhan, Xianbing Sun, Xiangnan Zhu et al. (+4 more)
Object swapping aims to replace a source object in a scene with a reference object while preserving object fidelity, scene fidelity, and object-scene harmony. Existing methods either require per-object finetuning and slow inference or rely on extra paired data that mostly depict the same object across contexts, forcing models to rely on background cues rather than learning cross-object
alignment. We propose SourceSwap, a self-supervised and source-aware framework that learns cross-object
alignment. Our key insight is to synthesize high-quality pseudo pairs from any image via a frequency-separated perturbation in the initial-noise space, which alters appearance while preserving pose, coarse shape, and scene layout, requiring no
videos, multi-view data, or additional images. We then train a dual U-Net with full-source conditioning and a noise-free reference encoder, enabling direct inter-object
alignment, zero-shot inference without per-object finetuning, and lightweight iterative refinement. We further introduce SourceBench, a high-quality
benchmark with higher resolution, more categories, and richer interactions. Experiments demonstrate that SourceSwap achieves superior fidelity, stronger scene preservation, and more natural harmony, and it transfers well to edits such as subject-driven refinement and face swapping.
Read the full abstract ▾
All authors Jiahui Zhan, Xianbing Sun, Xiangnan Zhu, Yikun Ji, Ruitong Liu, Liqing Zhang, Jianfu Zhang
Computer Vision
Any Model, Any Place, Any Time: Get Remote Sensing Foundation Model Embeddings On Demand
Dingqi Ye, Daniel Kiv, Wei Hu et al. (+2 more)
The remote sensing community is witnessing a rapid growth of foundation models, which provide powerful
embeddings for a wide range of downstream tasks. However, practical adoption and fair comparison remain challenging due to substantial heterogeneity in model release formats, platforms and interfaces, and input data specifications. These inconsistencies significantly increase the cost of obtaining, using, and benchmarking
embeddings across models. To address this issue, we propose rs-embed, a Python library that offers a unified, region of interst (ROI) centric interface: with a single line of code, users can retrieve
embeddings from any supported model for any location and any time range. The library also provides efficient batch processing to enable large-scale
embedding generation and evaluation. The code is available at: https://github.com/cybergis/rs-embed
Read the full abstract ▾
All authors Dingqi Ye, Daniel Kiv, Wei Hu, Jimeng Shi, Shaowen Wang
Computer Vision
Nazia Hossain, Xintong Jiang, Yu Tian et al. (+3 more)
Fine-grained crop-weed
segmentation is essential for enabling targeted herbicide application in precision agriculture. However, existing deep learning models struggle to generalize across heterogeneous agricultural environments due to reliance on
dataset-specific visual features. We propose
Vision-
Language Weed
Segmentation (VL-WS), a novel framework that addresses this limitation by grounding pixel-level
segmentation in semantically aligned, domain-invariant representations. Our architecture employs a dual-encoder design, where frozen Contrastive
Language-Image Pretraining (CLIP)
embeddings and task-specific spatial features are fused and modulated via Feature-wise Linear Modulation (FiLM) layers conditioned on natural
language captions. This design enables image level textual descriptions to guide channel-wise feature refinement while preserving fine-grained spatial localization. Unlike prior works restricted to training and evaluation on single-source
datasets, VL-WS is trained on a unified corpus that includes close-range ground imagery (robotic platforms) and high-altitude
UAV imagery, covering diverse crop types, weed species, growth stages, and sensing conditions. Experimental results across four
benchmark datasets demonstrate the effectiveness of our framework, with VL-WS achieving a mean Dice score of 91.64% and outperforming the CNN baseline by 4.98%. The largest gains occur on the most challenging weed class, where VL-WS attains 80.45% Dice score compared to 65.03% for the best baseline, representing a 15.42% improvement. VL-WS further maintains stable weed
segmentation performance under limited target-domain supervision, indicating improved generalization and data efficiency. These findings highlight the potential of
vision-
language alignment to enable scalable, label-efficient
segmentation models deployable across diverse real-world agricultural domains.
Read the full abstract ▾
All authors Nazia Hossain, Xintong Jiang, Yu Tian, Philippe Seguin, O. Grant Clark, Shangpeng Sun
Computer Vision
Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering
Ao Li, Rui Liu, Mingjie Li et al. (+6 more)
Automated radiology report generation using
vision-
language models (VLMs) is limited by the risk of prior-comparison
hallucination, where the model generates historical findings unsupported by the current study. We address this challenge with a training-free, inference-time control framework termed Semantically Decoupled Latent Steering (SDLS). Unlike generic activation steering, which often suffers from semantic
entanglement, our approach constructs a semantic-free intervention vector via large
language model (LLM)-driven semantic decomposition followed by $QR$-based orthogonalization. This orthogonalization step is critical. It leverages
geometric constraints to filter out the clinical semantics often entangled in standard principal component analysis (PCA) directions, ensuring that the steering vector targets only the “historical comparison" axis. We validate our method on the BiomedGPT foundation model, demonstrating that it overcomes the trade-off between
hallucination suppression and clinical accuracy. Extensive experiments on MIMIC-CXR, and zero-shot transfer evaluation on CheXpert Plus and IU-Xray, demonstrate the robustness of our approach. Quantitative evaluations on MIMIC-CXR show that our approach significantly reduces the probability of historical
hallucinations (FilBERT score decreases from 0.2373 to 0.1889) and improves clinical label fidelity (CheXpert macro-F1 increases from 0.2242 to 0.3208). Supplementary evaluations confirm that the structural integrity of the clinical narrative is maintained.
Read the full abstract ▾
All authors Ao Li, Rui Liu, Mingjie Li, Sheng Liu, Lei Wang, Xiaodan Liang, Lina Yao, Xiaojun Chang, Lei Xing
Computer Vision
ProtoDCS: Towards Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models
Wei Luo, Yangfan Ou, Jin Deng et al. (+4 more)
Large-scale
Vision-
Language Models (VLMs) exhibit strong zero-shot recognition, yet their real-world deployment is challenged by distribution shifts. While Test-Time Adaptation (TTA) can mitigate this, existing VLM-based TTA methods operate under a closed-set assumption, failing in open-set scenarios where test streams contain both
covariate-shifted in-distribution (csID) and out-of-distribution (csOOD) data. This leads to a critical difficulty: the model must discriminate unknown csOOD samples to avoid interference while simultaneously adapting to known csID classes for accuracy. Current open-set TTA (OSTTA) methods rely on hard thresholds for separation and
entropy minimization for adaptation. These strategies are brittle, often misclassifying ambiguous csOOD samples and inducing overconfident predictions, and their parameter-update mechanism is computationally prohibitive for VLMs. To address these limitations, we propose Prototype-based Double-Check Separation (ProtoDCS), a robust framework for OSTTA that effectively separates csID and csOOD samples, enabling safe and efficient adaptation of VLMs to csID data. Our main contributions are: (1) a novel double-check separation mechanism employing probabilistic Gaussian Mixture Model (GMM) verification to replace brittle thresholding; and (2) an evidence-driven adaptation strategy utilizing uncertainty-aware loss and efficient prototype-level updates, mitigating overconfidence and reducing computational overhead. Extensive experiments on CIFAR-10/100-C and Tiny-ImageNet-C demonstrate that ProtoDCS achieves state-of-the-art performance, significantly boosting both known-class accuracy and OOD detection metrics. Code will be available at https://github.com/O-YangF/ProtoDCS.
Read the full abstract ▾
All authors Wei Luo, Yangfan Ou, Jin Deng, Zeshuai Deng, Xiquan Yan, Zhiquan Wen, Mingkui Tan
Computer Vision
3D Modality-Aware Pre-training for Vision-Language Model in MRI Multi-organ Abnormality Detection
Haowen Zhu, Ning Yin and Xiaogen Zhou
Vision-
language models (VLMs) show strong potential for complex diagnostic tasks in medical imaging. However, applying VLMs to multi-organ medical imaging introduces two principal challenges: (1) modality-specific
vision-
language alignment and (2) cross-modal feature fusion. In this work, we propose MedMAP, a Medical Modality-Aware Pretraining framework that enhances
vision-
language representation learning in 3D MRI. MedMAP comprises a modality-aware
vision-
language alignment stage and a fine-tuning stage for multi-organ abnormality detection. During the pre-training stage, the modality-aware encoders implicitly capture the joint modality distribution and improve
alignment between visual and textual representations. We then fine-tune the pre-trained
vision encoders (while keeping the text encoder frozen) for downstream tasks. To this end, we curated MedMoM-MRI3D, comprising 7,392 3D MRI volume-report pairs spanning twelve MRI modalities and nine abnormalities tailored for various 3D medical analysis tasks. Extensive experiments on MedMoM-MRI3D demonstrate that MedMAP significantly outperforms existing VLMs in 3D MRI-based multi-organ abnormality detection. Our code is available at https://github.com/RomantiDr/MedMAP.
Read the full abstract ▾