Computer Vision
TopoVST: Toward Topology-fidelitous Vessel Skeleton Tracking
Yaoyu Liu, Minghui Zhang, Junjun He et al. (+1 more)
Automatic extraction of vessel skeletons is crucial for many clinical applications. However, achieving topologically faithful delineation of thin vessel skeletons remains highly challenging, primarily due to frequent discontinuities and the presence of spurious skeleton segments. To address these difficulties, we propose TopoVST, a
topology-fidelitious vessel skeleton tracker. TopoVST constructs multi-scale sphere graphs to sample the input image and employs graph neural networks to jointly estimate tracking directions and vessel radii. The utilization of multi-scale representations is enhanced through a gating-based feature fusion mechanism, while the issue of class imbalance during training is mitigated by
embedding a geometry-aware weighting scheme into the directional loss. In addition, we design a wave-propagation-based skeleton tracking
algorithm that explicitly mitigates the generation of spurious skeletons through space-occupancy filtering. We evaluate TopoVST on two vessel
datasets with different geometries. Extensive comparisons with state-of-the-art baselines demonstrate that TopoVST achieves competitive performance in both overlapping and topological metrics. Our source code is available at: https://github.com/EndoluminalSurgicalVision-IMR/TopoVST.
Read the full abstract ▾
All authors Yaoyu Liu, Minghui Zhang, Junjun He, Yun Gu
Computer Vision · Cross-listed from cs.RO
PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning
Yinfeng Gao, Qichao Zhang, Deqing Liu et al. (+8 more)
End-to-end autonomous driving policies based on Imitation Learning (IL) often struggle in closed-loop execution due to the misalignment between inadequate open-loop training objectives and real driving requirements. While Reinforcement Learning (RL) offers a solution by directly optimizing driving goals via reward signals, the rendering-based training environments introduce the rendering gap and are inefficient due to high computational costs. To overcome these challenges, we present a novel Pseudo-simulation-based RL method for closed-loop end-to-end autonomous driving, PerlAD. Based on offline
datasets, PerlAD constructs a pseudo-simulation that operates in vector space, enabling efficient, rendering-free trial-and-error training. To bridge the gap between static
datasets and dynamic closed-loop environments, PerlAD introduces a prediction world model that generates reactive
agent trajectories conditioned on the ego vehicle's plan. Furthermore, to facilitate efficient planning, PerlAD utilizes a hierarchical decoupled planner that combines IL for lateral path generation and RL for longitudinal speed optimization. Comprehensive experimental results demonstrate that PerlAD achieves state-of-the-art performance on the Bench2Drive
benchmark, surpassing the previous E2E RL method by 10.29% in Driving Score without requiring expensive online interactions. Additional evaluations on the DOS
benchmark further confirm its reliability in handling safety-critical occlusion scenarios.
Read the full abstract ▾
All authors Yinfeng Gao, Qichao Zhang, Deqing Liu, Zhongpu Xia, Guang Li, Kun Ma, Guang Chen, Hangjun Ye, Long Chen, Da-Wei Ding, Dongbin Zhao
Computer Vision
Balancing Saliency and Coverage: Semantic Prominence-Aware Budgeting for Visual Token Compression in VLMs
Jaehoon Lee, Mingi Jung, Soohyuk Jang et al. (+3 more)
Large
Vision-
Language Models (VLMs) achieve strong
multimodal understanding capabilities by leveraging high-resolution visual inputs, but the resulting large number of visual
tokens creates a major computational bottleneck. Recent work mitigates this issue through visual
token compression, typically compressing
tokens based on saliency, diversity, or a fixed combination of both. We observe that the distribution of semantic prominence varies substantially across samples, leading to different optimal trade-offs between local saliency preservation and global coverage. This observation suggests that applying a static compression strategy across all samples can be suboptimal. Motivated by this insight, we propose PromPrune, a sample-adaptive visual
token selection framework composed of semantic prominence-aware budget allocation and a two-stage selection pipeline. Our method adaptively balances local saliency preservation and global coverage according to the semantic prominence distribution of each sample. By allocating
token budgets between locally salient regions and globally diverse regions, our method maintains strong performance even under high compression ratios. On LLaVA-NeXT-7B, our approach reduces FLOPs by 88% and prefill latency by 22% while preserving 97.5% of the original accuracy.
Read the full abstract ▾
All authors Jaehoon Lee, Mingi Jung, Soohyuk Jang, Seungryong Yoo, Dahuin Jung, Sungroh Yoon
Computer Vision
PASTE: Physics-Aware Scattering Topology Embedding Framework for SAR Object Detection
Jiacheng Chen, Yuxuan Xiong and Haipeng Wang
Current deep learning-based object detection for Synthetic Aperture Radar (SAR) imagery mainly adopts optical image methods, treating targets as texture patches while ignoring inherent electromagnetic scattering mechanisms. Though scattering points have been studied to boost detection performance, most methods still rely on amplitude-based statistical models. Some approaches introduce frequency-domain information for scattering center extraction, but they suffer from high computation cost and poor compatibility with diverse
datasets. Thus, effectively
embedding scattering topological information into modern detection frameworks remains challenging. To solve these problems, this paper proposes the Physics-Aware Scattering
Topology Embedding Framework (PASTE), a novel closed-loop architecture for comprehensive scattering prior integration. By building the full pipeline from
topology generation, injection to joint supervision, PASTE elegantly integrates scattering physics into modern SAR detectors. Specifically, it designs a scattering keypoint generation and automatic annotation scheme based on the Attributed Scattering Center (ASC) model to produce scalable and physically consistent priors. A scattering
topology injection module guides multi-scale feature learning, and a scattering prior supervision strategy constrains network optimization by aligning predictions with scattering center distributions. Experiments on real
datasets show that PASTE is compatible with various detectors and brings relative mAP gains of 2.9% to 11.3% over baselines with acceptable computation overhead. Visualization of scattering maps verifies that PASTE successfully embeds scattering topological priors into feature space, clearly distinguishing target and background scattering regions, thus providing strong interpretability for results.
Read the full abstract ▾
Computer Vision
SpiralDiff: Spiral Diffusion with LoRA for RGB-to-RAW Conversion Across Cameras
Huanjing Yue, Shangbin Xie, Cong Cao et al. (+4 more)
RAW images preserve superior fidelity and rich scene information compared to RGB, making them essential for tasks in challenging imaging conditions. To alleviate the high cost of data collection, recent RGB-to-RAW conversion methods aim to synthesize RAW images from RGB. However, they overlook two key challenges: (i) the reconstruction difficulty varies with pixel intensity, and (ii) multi-camera conversion requires camera-specific adaptation. To address these issues, we propose SpiralDiff, a
diffusion-based framework tailored for RGB-to-RAW conversion with a signal-dependent noise weighting strategy that adapts reconstruction fidelity across intensity levels. In addition, we introduce CamLoRA, a camera-aware lightweight adaptation module that enables a unified model to adapt to different camera-specific ISP characteristics. Extensive experiments on four
benchmark datasets demonstrate the superiority of SpiralDiff in RGB-to-RAW conversion quality and its downstream benefits in RAW-based object detection. Our code and model are available at https://github.com/Chuancy-TJU/SpiralDiff.
Read the full abstract ▾
All authors Huanjing Yue, Shangbin Xie, Cong Cao, Qian Wu, Lei Zhang, Lei Zhao, Jingyu Yang
Computer Vision
LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models
Soumyaratna Debnath, Bui Duc Manh, Zinan Liu et al. (+1 more)
Vision-
Language Models (VLMs) typically assume a uniform spatial fidelity across the entire field of view of visual inputs, dedicating equal precision to even the uninformative regions. By contrast, human
vision is neither uniform nor static; it is adaptive, selective, and resource-efficient. In light of this, we present the first systematic analysis of bio-inspired visual representation methods, providing insights for more efficient and adaptive VLMs. We propose LLMind (Looking Like the Mind), a novel training-free framework that mimics foveated encoding and cortical magnification in human
vision to achieve adaptive, efficient representations for VLMs under tight pixel budgets. Our key idea is to explore a Bio-inspired Adaptive Sampling Strategy (BASS), enabling a Mobius-parameterized module that performs non-uniform sampling while preserving global scene structure. On top of BASS, we introduce closed-loop semantic feedback (CSF) via test-time adaptation to align perceptual saliency with textual information from the frozen VLM. We evaluate LLMind against uniform and other sampling baselines across diverse scene-level and region-guided visual question answering
benchmarks. The results show dramatic gains, with average improvements of +20% on VQAv2, +38% on Seed-Bench, and +37% on A-OKVQA compared to uniform sampling under tight pixel budgets. More surprisingly, LLMind retains up to 82%, 92%, and 97% of the full-resolution performance using only 1%, 3%, and 5% of the pixels, respectively. Moreover, LLMind is lightweight, plug-and-play, and compatible with existing VLMs without requiring architectural changes.
Read the full abstract ▾
All authors Soumyaratna Debnath, Bui Duc Manh, Zinan Liu, Lin Wang
Computer Vision
RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation
Linfei Li, Lin Zhang and Ying Shen
Visual-
language grounding aims to establish semantic correspondences between natural
language and visual entities, enabling models to accurately identify and localize target objects based on textual instructions. Existing VLG approaches focus on coarse-grained, object-level localization, while traditional robotic grasping methods rely predominantly on
geometric cues and lack
language guidance, which limits their applicability in
language-driven manipulation scenarios. To address these limitations, we propose the RealVLG framework, which integrates the RealVLG-11B
dataset and the RealVLG-R1 model to unify real-world visual-
language grounding and grasping tasks. RealVLG-11B
dataset provides multi-granularity annotations including bounding boxes,
segmentation masks, grasp poses, contact points, and human-verified fine-grained
language descriptions, covering approximately 165,000 images, over 800 object instances, 1.3 million
segmentation, detection, and
language annotations, and roughly 11 billion grasping examples. Building on this
dataset, RealVLG-R1 employs Reinforcement Fine-tuning on pretrained large-scale
vision-
language models to predict bounding boxes,
segmentation masks, grasp poses, and contact points in a unified manner given natural
language instructions. Experimental results demonstrate that RealVLG supports zero-shot perception and manipulation in real-world unseen environments, establishing a unified semantic-visual
multimodal benchmark that provides a comprehensive data and evaluation platform for
language-driven robotic perception and grasping policy learning. All data and code are publicly available at https://github.com/lif314/RealVLG-R1.
Read the full abstract ▾
Computer Vision
Video Detector: A Dual-Phase Vision-Based System for Real-Time Traffic Intersection Control and Intelligent Transportation Analysis
Mustafa Fatih Şen, Halûk Gümüşkaya and Şenol Pazar
Urban traffic management increasingly requires intelligent sensing systems capable of adapting to dynamic traffic conditions without costly infrastructure modifications.
Vision-based vehicle detection has therefore become a key technology for modern intelligent transportation systems. This study presents
Video Detector (VD), a dual-phase
vision-based traffic intersection management system designed as a flexible and cost-effective alternative to traditional inductive loop detectors. The framework integrates a real-time module (VD-RT) for intersection control with an offline analytical module (VD-Offline) for detailed traffic behavior analysis. Three system configurations were implemented using SSD Inception v2, Faster R-CNN Inception v2, and CenterNet ResNet-50 V1 FPN, trained on
datasets totaling 108,000 annotated images across 6-10 vehicle classes. Experimental results show detection performance of up to 90% test accuracy and 29.5 mAP@0.5, while maintaining real-time throughput of 37 FPS on HD
video streams. Field deployments conducted in collaboration with Istanbul IT and Smart City Technologies Inc. (ISBAK) demonstrate stable operation under diverse environmental conditions. The system supports virtual loop detection, vehicle counting, multi-object tracking, queue estimation, speed analysis, and multiclass vehicle classification, enabling comprehensive intersection monitoring without the need for embedded road sensors. The annotated
dataset and training pipeline are publicly released to support reproducibility. These results indicate that the proposed framework provides a scalable and deployable
vision-based solution for intelligent transportation systems and smart-city traffic management.
Read the full abstract ▾
Computer Vision
From Horizontal to Rotated: Cross-View Object Geo-Localization with Orientation Awareness
Chenlin Fu, Ao Gong and Yingying Zhu
Cross-View object geo-localization (CVOGL) aims to precisely determine the geographic coordinates of a query object from a ground or drone perspective by referencing a satellite map.
Segmentation-based approaches offer high precision but require prohibitively expensive pixel-level annotations, whereas more economical detection-based methods suffer from lower accuracy. This performance disparity in detection is primarily caused by two factors: the poor
geometric fit of Horizontal Bounding Boxes (HBoxes) for oriented objects and the degradation in precision due to feature map scaling. Motivated by these, we propose leveraging Rotated Bounding Boxes (RBoxes) as a natural extension of the detection-based paradigm. RBoxes provide a much tighter
geometric fit to oriented objects. Building on this, we introduce OSGeo, a novel geo-localization framework, meticulously designed with a multi-scale perception module and an orientation-sensitive head to accurately regress RBoxes. To support this scheme, we also construct and release CVOGL-R, the first
dataset with precise RBox annotations for CVOGL. Extensive experiments demonstrate that our OSGeo achieves state-of-the-art performance, consistently matching or even surpassing the accuracy of leading
segmentation-based methods but with an annotation cost that is over an order of magnitude lower.
Read the full abstract ▾
Computer Vision
AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving
Wenhui Huang, Songyan Zhang, Qihang Huang et al. (+6 more)
Integrating
vision-
language models (VLMs) into end-to-end (E2E) autonomous driving (AD) systems has shown promise in improving scene understanding. However, existing integration strategies suffer from several limitations: they either struggle to resolve distribution misalignment between
reasoning and action spaces, underexploit the general
reasoning capabilities of pretrained VLMs, or incur substantial inference latency during action policy generation, which degrades driving performance. To address these challenges, we propose AutoMoT in this work, an end-to-end AD framework that unifies
reasoning and action generation within a single
vision-
language-action (VLA) model. Our approach leverages a mixture-of-
transformer (MoT) architecture with joint
attention sharing, which preserves the general
reasoning capabilities of pre-trained VLMs while enabling efficient fast-slow inference through asynchronous execution at different task frequencies. Extensive experiments on multiple
benchmarks, under both open- and closed-loop settings, demonstrate that AutoMoT achieves competitive performance compared to state-of-the-art methods. We further investigate the functional boundary of pre-trained VLMs in AD, examining when AD-tailored fine-tuning is necessary. Our results show that pre-trained VLMs can achieve competitive multi-task scene understanding performance through semantic prompting alone, while fine-tuning remains essential for action-level tasks such as decision-making and trajectory planning. We refer to https://automot-website.github.io/ for the demonstration
videos and qualitative results.
Read the full abstract ▾
All authors Wenhui Huang, Songyan Zhang, Qihang Huang, Zhidong Wang, Zhiqi Mao, Collister Chua, Zhan Chen, Long Chen, Chen Lv
Computer Vision
From Artefact to Insight: Efficient Low-Rank Adaptation of BrushNet for Scanning Probe Microscopy Image Restoration
Ziwei Wei, Yao Shen, Wanheng Lu et al. (+2 more)
Scanning Probe Microscopy or SPM offers nanoscale resolution but is frequently marred by structured artefacts such as line scan dropout, gain induced noise, tip convolution, and phase hops. While most available methods treat SPM artefact removal as isolated denoising or interpolation tasks, the generative inpainting perspective remains largely unexplored. In this work, we introduce a
diffusion based inpainting framework tailored to scientific grayscale imagery. By fine tuning less than 0.2 percent of BrushNet weights with rank constrained low rank adaptation (LoRA), we adapt a pretrained
diffusion model using only 7390 artefact, clean pairs distilled from 739 experimental scans. On our forthcoming public SPM InpBench
benchmark, the LoRA enhanced model lifts the Peak Signal to Noise Ratio or PSNR by 6.61 dB and halves the Learned Perceptual Image Patch Similarity or LPIPS relative to zero-shot inference, while matching or slightly surpassing the accuracy of full retraining, trainable on a single GPU instead of four high-
memory cards. The approach generalizes across various SPM image channels including height, amplitude and phase, faithfully restores subtle structural details, and suppresses
hallucination artefacts inherited from natural image priors. This lightweight framework enables efficient, scalable recovery of irreplaceable SPM images and paves the way for a broader
diffusion model adoption in nanoscopic imaging analysis.
Read the full abstract ▾
All authors Ziwei Wei, Yao Shen, Wanheng Lu, Ghim Wei Ho, Kaiyang Zeng
Computer Vision
Personalized Federated Learning with Residual Fisher Information for Medical Image Segmentation
Meilu Zhu, Yuxing Li, Zhiwei Wang et al. (+1 more)
Federated learning enables multiple clients (institutions) to collaboratively train machine learning models without sharing their private data. To address the challenge of data heterogeneity across clients, personalized
federated learning (pFL) aims to learn customized models for each client. In this work, we propose pFL-ResFIM, a novel pFL framework that achieves client-adaptive personalization at the parameter level. Specifically, we introduce a new metric, Residual Fisher Information
Matrix (ResFIM), to quantify the sensitivity of model parameters to domain discrepancies. To estimate ResFIM for each client model under
privacy constraints, we employ a spectral transfer strategy that generates simulated data reflecting the domain styles of different clients. Based on the estimated ResFIM, we partition model parameters into domain-sensitive and domain-invariant components. A personalized model for each client is then constructed by aggregating only the domain-invariant parameters on the server. Extensive experiments on public
datasets demonstrate that pFL-ResFIM consistently outperforms state-of-the-art methods, validating its effectiveness.
Read the full abstract ▾
All authors Meilu Zhu, Yuxing Li, Zhiwei Wang, Edmund Y. Lam
Computer Vision
DamageArbiter: A Multimodal Arbitration Framework for Disaster Damage Assessment from Street-View Imagery
Yifan Yang, Lei Zou, Wenjing Gong et al. (+6 more)
Analyzing street-view imagery with computer
vision models offers a promising approach for rapid, hyperlocal disaster damage assessment, but existing approaches typically rely on black-box pre-trained
vision models, which lack interpretability and reliability. This study proposes DamageArbiter, a
multimodal disagreement-driven arbitration framework designed to improve the accuracy and reliability of street-view-based damage assessment. DamageArbiter leverages the complementary strengths of unimodal and
multimodal models and employs a lightweight logistic regression meta-classifier to arbitrate cases in which model predictions disagree. Using 2,556 post-disaster street-view images, paired with manually generated or large
language model (LLM)-generated text descriptions, we systematically compared DamageArbiter with fine-tuned unimodal (image-only and text-only) models and CLIP-based
multimodal models in terms of classification performance and overconfidence errors. Results show that DamageArbiter improved accuracy to 75.85% and the Matthews correlation coefficient (MCC) to 0.6188, compared with the best-performing text-only baseline (63.07% accuracy, 0.4126 MCC), image-only baseline (74.33% accuracy, 0.5947 MCC), and CLIP baseline (74.22% accuracy, 0.5915 MCC). The overconfidence analysis further reveals that DamageArbiter substantially reduced the overconfidence error from 70.58% for the best-performing baseline, the image-only ViT model, to 16.45%. Overall, this study demonstrates that accuracy alone is insufficient for evaluating disaster damage classification models and highlights the importance of measuring overconfidence errors as part of model reliability assessment. DamageArbiter thus offers a more reliable framework for rapid, hyperlocal disaster damage assessment from street-view imagery.
Read the full abstract ▾
All authors Yifan Yang, Lei Zou, Wenjing Gong, Kani Fu, Zongrong Li, Siqin Wang, Bing Zhou, Heng Cai, Hao Tian
Computer Vision · Cross-listed from eess.IV
Halfway to 3D: Ensembling 2.5D and 3D Models for Robust COVID-19 CT Diagnosis
Tuan-Anh Yang, Bao V. Q. Bui, Chanh-Quang Vo-Van et al. (+1 more)
We propose a deep learning framework for COVID-19 detection and disease classification from chest CT scans that integrates both 2.5D and 3D representations to capture complementary slice-level and volumetric information. The 2.5D branch processes multi-view CT slices (axial, coronal, sagittal) using a DINOv3
vision transformer to extract robust visual features, while the 3D branch employs a ResNet-18 architecture to model volumetric context and is pretrained with Variance Risk Extrapolation (VREx) followed by supervised contrastive learning to improve cross-source robustness. Predictions from both branches are combined through logit-level ensemble inference. Experiments on the PHAROS-AIF-MIH
benchmark demonstrate the effectiveness of the proposed approach: for binary COVID-19 detection, the ensemble achieves 94.48% accuracy and a 0.9426 Macro F1-score, outperforming both individual models, while for multi-class disease classification the 2.5D DINOv3 model achieves the best performance with 79.35% accuracy and a 0.7497 Macro F1-score. These results highlight the benefit of combining pretrained slice-based representations with volumetric modeling for robust multi-source medical imaging analysis. Code is available at https://github.com/HySonLab/PHAROS-AIF-MIH
Read the full abstract ▾
All authors Tuan-Anh Yang, Bao V. Q. Bui, Chanh-Quang Vo-Van, Truong-Son Hy
Computer Vision
SemanticFace: Semantic Facial Action Estimation via Semantic Distillation in Interpretable Space
Zejian Kang, Kai Zheng, Yuanchen Fei et al. (+3 more)
Facial action estimation from a single image is often formulated as predicting or fitting parameters in compact expression spaces, which lack explicit semantic interpretability. However, many practical applications, such as avatar control and human-computer interaction, require interpretable facial actions that correspond to meaningful muscle movements. In this work, we propose SemanticFace, a framework for facial action estimation in the interpretable ARKit blendshape space that reformulates coefficient prediction as structured semantic
reasoning. SemanticFace adopts a two-stage semantic
distillation paradigm: it first derives structured semantic supervision from ground-truth ARKit coefficients and then distills this knowledge into a
multimodal large
language model to predict interpretable facial action coefficients from images. Extensive experiments demonstrate that
language-aligned semantic supervision improves both coefficient accuracy and perceptual consistency, while enabling strong cross-identity generalization and robustness to large domain shifts, including cartoon faces.
Read the full abstract ▾
All authors Zejian Kang, Kai Zheng, Yuanchen Fei, Wentao Yang, Hongyuan Zou, Xiangru Huang
Computer Vision
Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projection
Yewon Han, Yumin Seol, EunGyung Kong et al. (+2 more)
Existing jailbreak defence frameworks for Large
Vision-
Language Models often suffer from a safety utility tradeoff, where strengthening safety inadvertently degrades performance on general visual-grounded
reasoning tasks. In this work, we investigate whether safety and utility are inherently antagonistic objectives. We focus on a modality induced bias direction consistently observed across
datasets, which arises from suboptimal coupling between the Large
Language Model backbone and visual encoders. We further demonstrate that this direction undermines performance on both tasks. Leveraging this insight, we propose Two Birds, One Projection, an efficient inference time jailbreak defence that projects cross-modal features onto the null space of the identified bias direction to remove the corresponding components. Requiring only a single forward pass, our method effectively breaks the conventional tradeoff, simultaneously improving both safety and utility across diverse
benchmarks.
Read the full abstract ▾
All authors Yewon Han, Yumin Seol, EunGyung Kong, Minsoo Jo, Taesup Kim
Computer Vision
RadarXFormer: Robust Object Detection via Cross-Dimension Fusion of 4D Radar Spectra and Images for Autonomous Driving
Yue Sun, Yeqiang Qian, Zhe Wang et al. (+3 more)
Reliable perception is essential for autonomous driving systems to operate safely under diverse real-world traffic conditions. However, camera- and
LiDAR-based perception systems suffer from performance degradation under adverse weather and lighting conditions, limiting their robustness and large-scale deployment in intelligent transportation systems. Radar-
vision fusion provides a promising alternative by combining the environmental robustness and cost efficiency of millimeter-wave (mmWave) radar with the rich semantic information captured by cameras. Nevertheless, conventional 3D radar measurements lack height resolution and remain highly sparse, while emerging 4D mmWave radar introduces elevation information but also brings challenges such as signal noise and large data volume. To address these issues, this paper proposes RadarXFormer, a 3D object detection framework that enables efficient cross-modal fusion between 4D radar spectra and RGB images. Instead of relying on sparse radar point clouds, RadarXFormer directly leverages raw radar spectra and constructs an efficient 3D representation that reduces data volume while preserving complete 3D spatial information. The "X" highlights the proposed cross-dimension (3D-2D) fusion mechanism, in which multi-scale 3D spherical radar feature cubes are fused with complementary 2D image feature maps. Experiments on the K-Radar
dataset demonstrate improved detection accuracy and robustness under challenging conditions while maintaining real-time inference capability.
Read the full abstract ▾
All authors Yue Sun, Yeqiang Qian, Zhe Wang, Tianhui Li, Chunxiang Wang, Ming Yang
Computer Vision
RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models
Ravi Ranjan, Utkarsh Grover, Xiaomin Lin et al. (+1 more)
Transformer based
diffusion and
vision-
language models have achieved remarkable success; yet, efficiently removing undesirable or sensitive information without retraining remains a central challenge for model safety and compliance. We introduce Ratio-Aware Zero/One-step Optimized Retentive unlearning (RAZOR), a lightweight, model-agnostic unlearning framework that generalizes forgetting updates to coordinated multi-layer and multi-head edits within
transformer backbones. RAZOR identifies the most important layers and
attention heads by measuring how much they contribute to forgetting the target data while preserving useful knowledge. Then, it updates these parts of the model using a carefully regularized rule to avoid harming overall performance. The set of edited components grows gradually, ensuring precise unlearning without over-editing or damaging unrelated capabilities. We evaluate RAZOR on CLIP, Stable
Diffusion, and
vision-
language models (VLMs) using widely adopted unlearning
benchmarks covering identity, style, and object erasure tasks. Our results show that RAZOR achieves highly accurate and stable forgetting, even under quantization. This approach offers stronger retention and better efficiency than prior methods. Notably, it also operates significant faster than conventional techniques. These results demonstrate that RAZOR is a practical and scalable solution for safe, adaptive unlearning in
transformer-based
vision models.
Read the full abstract ▾
All authors Ravi Ranjan, Utkarsh Grover, Xiaomin Lin, Agoritsa Polyzou
Computer Vision
M2IR: Proactive All-in-One Image Restoration via Mamba-style Modulation and Mixture-of-Experts
Shiwei Wang, Yongzhen Wang, Bingwen Hu et al. (+3 more)
While
Transformer-based architectures have dominated recent advances in all-in-one image restoration, they remain fundamentally reactive: propagating degradations rather than proactively suppressing them. In the absence of explicit suppression mechanisms, degraded signals interfere with feature learning, compelling the decoder to balance artifact removal and detail preservation, thereby increasing model complexity and limiting adaptability. To address these challenges, we propose M2IR, a novel restoration framework that proactively regulates degradation propagation during the encoding stage and efficiently eliminates residual degradations during decoding. Specifically, the Mamba-Style
Transformer (MST) block performs pixel-wise selective state modulation to mitigate degradations while preserving structural integrity. In parallel, the Adaptive Degradation Expert Collaboration (ADEC) module utilizes degradation-specific experts guided by a DA-CLIP-driven router and complemented by a shared expert to eliminate residual degradations through targeted and cooperative restoration. By integrating the MST block and ADEC module, M2IR transitions from passive reaction to active degradation control, effectively harnessing learned representations to achieve superior generalization, enhanced adaptability, and refined recovery of fine-grained details across diverse all-in-one image restoration
benchmarks. Our source codes are available at https://github.com/Im34v/M2IR.
Read the full abstract ▾
All authors Shiwei Wang, Yongzhen Wang, Bingwen Hu, Liyan Zhang, Xiao-Ping Zhang, Mingqiang Wei
Computer Vision · Cross-listed from cs.RO
Ego to World: Collaborative Spatial Reasoning in Embodied Systems via Reinforcement Learning
Heng Zhou, Li Kang, Yiran Qin et al. (+12 more)
Understanding the world from distributed, partial viewpoints is a fundamental challenge for
embodied multi-
agent systems. Each
agent perceives the environment through an ego-centric view that is often limited by occlusion and ambiguity. To study this problem, we introduce the Ego-to-World (E2W)
benchmark, which evaluates a
vision-
language model's ability to fuse heterogeneous viewpoints across three tasks: (i) global counting, (ii) relational location
reasoning, and (iii) action-oriented grasping that requires predicting view-specific image coordinates. To address this setting, we propose CoRL, a two-stage framework that combines Chain-of-Thought supervised fine-tuning with reinforcement learning using Group-Relative Policy Optimization. Its core component, the Cross-View Spatial Reward (CVSR), provides dense task-aligned feedback by linking
reasoning steps to visual evidence, ensuring coherent cross-view entity resolution, and guiding the model toward correct final predictions. Experiments on E2W show that CoRL consistently surpasses strong proprietary and open-source baselines on both
reasoning and perception-grounding metrics, while ablations further confirm the necessity of each CVSR component. Beyond that, CoRL generalizes to external spatial
reasoning benchmarks and enables effective real-world multi-
robot manipulation with calibrated multi-camera rigs, demonstrating cross-view localization and successful grasp-and-place execution. Together, E2W and CoRL provide a principled foundation for learning world-centric scene understanding from distributed, ego-centric observations, advancing collaborative
embodied AI.
Read the full abstract ▾
All authors Heng Zhou, Li Kang, Yiran Qin, Xiufeng Song, Ao Yu, Zilu Zhang, Haoming Song, Kaixin Xu, Yuchen Fan, Dongzhan Zhou, Xiaohong Liu, Ruimao Zhang, Philip Torr, Lei Bai, Zhenfei Yin
Computer Vision
HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory System
Kailin Lyu, Kangyi Wu, Pengna Li et al. (+9 more)
LLM-based
agents have demonstrated impressive zero-shot performance in
vision-
language navigation (VLN) tasks. However, most zero-shot methods primarily rely on closed-source LLMs as navigators, which face challenges related to high
token costs and potential data leakage risks. Recent efforts have attempted to address this by using open-source LLMs combined with a spatiotemporal CoT framework, but they still fall far short compared to closed-source models. In this work, we identify a critical issue,
Navigation Amnesia, through a detailed analysis of the
navigation process. This issue leads to
navigation failures and amplifies the gap between open-source and closed-source methods. To address this, we propose HiMemVLN, which incorporates a Hierarchical
Memory System into a
multimodal large model to enhance visual perception recall and long-term localization, mitigating the amnesia issue and improving the
agent's
navigation performance. Extensive experiments in both simulated and real-world environments demonstrate that HiMemVLN achieves nearly twice the performance of the open-source state-of-the-art method. The code is available at https://github.com/lvkailin0118/HiMemVLN.
Read the full abstract ▾
All authors Kailin Lyu, Kangyi Wu, Pengna Li, Xiuyu Hu, Qingyi Si, Cui Miao, Ning Yang, Zihang Wang, Long Xiao, Lianyu Hu, Jingyuan Sun, Ce Hao
Computer Vision
Global Truncated Loss Minimization for Robust and Threshold-Resilient Geometric Estimation
Tianyu Huang, Liangzu Peng, Xinyue Zhang et al. (+5 more)
To achieve outlier-robust
geometric estimation, robust objective functions are generally employed to mitigate the influence of outliers. The widely used consensus maximization(CM) is highly robust when paired with global branch-and-bound(BnB) search. However, CM relies solely on inlier counts and is sensitive to the inlier threshold. Besides, the discrete nature of CM leads to loose bounds, necessitating extensive BnB iterations and computation cost. Truncated losses(TL), another continuous alternative, leverage residual information more effectively and could potentially overcome these issues. But to our knowledge, no prior work has systematically explored globally minimizing TL with BnB and its potential for enhanced threshold resilience or search efficiency. In this work, we propose GTM, the first unified BnB-based framework for globally-optimal TL loss minimization across diverse
geometric problems. GTM involves a hybrid solving design: given an n-dimensional problem, it performs BnB search over an (n-1)-dimensional subspace while the remaining 1D variable is solved by bounding the objective function. Our hybrid design not only reduces the search space, but also enables us to derive Lipschitz-continuous bounding functions that are general, tight, and can be efficiently solved by a classic global Lipschitz solver named DIRECT, which brings further acceleration. We conduct a systematic evaluation on various BnB-based methods for CM and TL on the robust linear regression problem, showing that GTM enjoys remarkable threshold resilience and the highest efficiency compared to baseline methods. Furthermore, we apply GTM on different
geometric estimation problems with diverse residual forms. Extensive experiments demonstrate that GTM achieves state-of-the-art outlier-robustness and threshold-resilience while maintaining high efficiency across these estimation tasks.
Read the full abstract ▾
All authors Tianyu Huang, Liangzu Peng, Xinyue Zhang, Tongfan Guan, Jinhu Dong, Haoang Li, Laurent Kneip, Yun-Hui Liu
Computer Vision
Face-to-Face: A Video Dataset for Multi-Person Interaction Modeling
Ernie Chu and Vishal M. Patel
Modeling the reactive tempo of human conversation remains difficult because most audio-visual
datasets portray isolated speakers delivering short monologues. We introduce Face-to-Face with Jimmy Fallon (F2F-JF), a 70-hour, 14k-clip
dataset of two-person talk-show exchanges that preserves the sequential dependency between a guest turn and the host's response. A semi-automatic pipeline combines multi-person tracking,
speech diarization, and lightweight human verification to extract temporally aligned host/guest tracks with tight crops and metadata that are ready for downstream modeling. We showcase the
dataset with a reactive,
speech-driven digital avatar task in which the host
video during $[t_1,t_2]$ is generated from their audio plus the guest's preceding
video during $[t_0,t_1]$. Conditioning a MultiTalk-style
diffusion model on this cross-person visual context yields small but consistent
Emotion-FID and FVD gains while preserving lip-sync quality relative to an audio-only baseline. The
dataset, preprocessing recipe, and baseline together provide an end-to-end blueprint for studying dyadic, sequential behavior, which we expand upon throughout the paper.
Dataset and code are available at https://face2face2026.github.io.
Read the full abstract ▾
Computer Vision
Mind-of-Director: Multi-modal Agent-Driven Film Previsualization via Collaborative Decision-Making
Shufeng Nan, Mengtian Li, Sixiao Zheng et al. (+3 more)
We present Mind-of-Director, a multi-modal
agent-driven framework for film previz that models the collaborative decision-making process of a film production team. Given a creative idea, Mind-of-Director orchestrates multiple specialized
agents to produce previz sequences within the game engine. The framework consists of four cooperative modules: Script Development, where
agents draft and refine the screenplay iteratively; Virtual Scene Design, which transforms text into semantically aligned 3D environments; Character Behaviour Control, which determines character blocking and motion; and Camera Planning, which optimizes framing, movement, and composition for cinematic camera effects. A real-time visual editing system built in the game engine further enables interactive inspection and synchronized timeline adjustment across scenes, behaviours, and cameras. Extensive experiments and human evaluations show that Mind-of-Director generates high-quality, semantically grounded previz sequences in approximately 25 minutes per idea, demonstrating the effectiveness of
agent collaboration for both automated prototyping and human-in-the-loop filmmaking.
Read the full abstract ▾
All authors Shufeng Nan, Mengtian Li, Sixiao Zheng, Yuwei Lu, Han Zhang, Yanwei Fu
Computer Vision
High-Fidelity 3D Facial Avatar Synthesis with Controllable Fine-Grained Expressions
Yikang He, Jichao Zhang, Wei Wang et al. (+2 more)
Facial expression editing methods can be mainly categorized into two types based on their architectures: 2D-based and 3D-based methods. The former lacks 3D face modeling capabilities, making it difficult to edit 3D factors effectively. The latter has demonstrated superior performance in generating high-quality and view-consistent renderings using single-view 2D face images. Although these methods have successfully used animatable models to control facial expressions, they still have limitations in achieving precise control over fine-grained expressions. To address this issue, in this paper, we propose a novel approach by simultaneously refining both the latent code of a pretrained 3D-Aware GAN model for texture editing and the expression code of the driven 3DMM model for mesh editing. Specifically, we introduce a Dual Mappers module, comprising Texture Mapper and
Emotion Mapper, to learn the transformations of the given latent code for textures and the expression code for meshes, respectively. To optimize the Dual Mappers, we propose a Text-Guided Optimization method, leveraging a CLIP-based objective function with expression text
prompts as targets, while integrating a SubSpace Projection mechanism to project the text
embedding to the expression subspace such that we can have more precise control over fine-grained expressions. Extensive experiments and comparative analyses demonstrate the effectiveness and superiority of our proposed method.
Read the full abstract ▾
All authors Yikang He, Jichao Zhang, Wei Wang, Nicu Sebe, Yao Zhao
Computer Vision
Zero-Shot Reconstruction of Animatable 3D Avatars with Cloth Dynamics from a Single Image
Joohyun Kwon, Geonhee Sim and Gyeongsik Moon
Existing single-image 3D human avatar methods primarily rely on rigid joint transformations, limiting their ability to model realistic cloth dynamics. We present DynaAvatar, a zero-shot framework that reconstructs animatable 3D human avatars with motion-dependent cloth dynamics from a single image. Trained on large-scale multi-person motion
datasets, DynaAvatar employs a
Transformer-based feed-forward architecture that directly predicts dynamic 3D Gaussian deformations without subject-specific optimization. To overcome the scarcity of dynamic captures, we introduce a static-to-dynamic knowledge transfer strategy: a
Transformer pretrained on large-scale static captures provides strong
geometric and appearance priors, which are efficiently adapted to motion-dependent deformations through lightweight LoRA fine-tuning on dynamic captures. We further propose the DynaFlow loss, an optical flow-guided objective that provides reliable motion-direction
geometric cues for cloth dynamics in rendered space. Finally, we reannotate the missing or noisy SMPL-X fittings in existing dynamic capture
datasets, as most public dynamic capture
datasets contain incomplete or unreliable fittings that are unsuitable for training high-quality 3D avatar reconstruction models. Experiments demonstrate that DynaAvatar produces visually rich and generalizable animations, outperforming prior methods.
Read the full abstract ▾
Computer Vision
AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvas
Longhui Yuan
Multi-person identity-preserving generation requires binding multiple reference faces to specified locations under a text
prompt. Strong identity/layout conditions often trigger copy-paste shortcuts and weaken
prompt-driven controllability. We present AnyPhoto, a
diffusion-
transformer finetuning framework with (i) a RoPE-aligned location canvas plus location-aligned
token pruning for spatial grounding, (ii) AdaLN-style identity-adaptive modulation from face-recognition
embeddings for persistent identity injection, and (iii) identity-isolated
attention to prevent cross-identity interference. Training combines conditional flow matching with an
embedding-space face similarity loss, together with reference-face replacement and location-canvas degradations to discourage shortcuts. On MultiID-Bench, AnyPhoto improves identity similarity while reducing copy-paste tendency, with gains increasing as the number of identities grows. AnyPhoto also supports
prompt-driven stylization with accurate placement, showing great potential application value.
Read the full abstract ▾
Computer Vision
SSR: A Training-Free Approach for Streaming 3D Reconstruction
Hui Deng, Yuxin Mao, Yuxin He et al. (+1 more)
Streaming 3D reconstruction demands long-horizon state updates under strict latency constraints, yet stateful recurrent models often suffer from
geometric drift as errors accumulate over time. We revisit this problem from a Grassmannian
manifold perspective: the latent persistent state can be viewed as a subspace representation, i.e., a point evolving on a Grassmannian
manifold, where temporal coherence implies the state trajectory should remain on (or near) this
manifold.Based on this view, we propose Self-expressive Sequence Regularization (SSR), a plug-and-play, training-free operator that enforces Grassmannian sequence regularity during inference.Given a window of historical states, SSR computes an analytical affinity
matrix via the self-expressive property and uses it to regularize the current update, effectively pulling noisy predictions back toward the
manifold-consistent trajectory with minimal overhead. Experiments on long-sequence
benchmarks demonstrate that SSR consistently reduces drift and improves reconstruction quality across multiple streaming 3D reconstruction tasks.
Read the full abstract ▾
All authors Hui Deng, Yuxin Mao, Yuxin He, Yuchao Dai
Computer Vision
Topology-Preserving Polygon Augmentation for Segmentation in Structured Visual Domains
Sudip Laudari and Sang Hun Baek
Geometric data augmentation is widely used in
segmentation workflows, but polygon annotations are often assumed to remain valid after transformation. This assumption can fail in structured domains such as architectural floorplan analysis, where a region may contain an interior void encoded as part of a single ordered polygon chain. Cropping or clipping can remove bridge vertices in this chain, causing one semantic region to split into disconnected components. We propose a lightweight
topology-preserving augmentation strategy that repairs missing adjacency relations in index space while preserving the original vertex order. The method adds minimal overhead and can be integrated into existing preprocessing workflows. Experiments show that the proposed approach achieves near-perfect Cyclic Adjacency Preservation (CAP) across common
geometric transformations and improves annotation consistency in polygon-based
segmentation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
LiDAR-EVS: Enhance Extrapolated View Synthesis for 3D Gaussian Splatting with Pseudo-LiDAR Supervision
Yiming Huang, Xin Kang, Sipeng Zhang et al. (+3 more)
3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time
LiDAR and camera synthesis in autonomous driving simulation. However, simulating
LiDAR with 3DGS remains challenging for extrapolated views beyond the training trajectory, as existing methods are typically trained on single-traversal sensor scans, suffer from severe overfitting and poor generalization to novel ego-vehicle paths. To enable reliable simulation of
LiDAR along unseen driving trajectories without external multi-pass data, we present
LiDAR-EVS, a lightweight framework for robust extrapolated-view
LiDAR simulation in autonomous driving. Designed to be plug-and-play,
LiDAR-EVS readily extends to diverse
LiDAR sensors and neural rendering baselines with minimal modification. Our framework comprises two key components: (1) pseudo extrapolated-view point cloud supervision with multi-frame
LiDAR fusion, view transformation, occlusion curling, and intensity adjustment; (2) spatially-constrained dropout regularization that promotes robustness to diverse trajectory variations encountered in real-world driving. Extensive experiments demonstrate that
LiDAR-EVS achieves SOTA performance on extrapolated-view
LiDAR synthesis across three
datasets, making it a promising tool for data-driven simulation, closed-loop evaluation, and synthetic data generation in autonomous driving systems.
Read the full abstract ▾
All authors Yiming Huang, Xin Kang, Sipeng Zhang, Hongliang Ren, Weihua Zhang, Junjie Lai