Computer Vision
DCG ReID: Disentangling Collaboration and Guidance Fusion Representations for Multi-modal Vehicle Re-Identification
Aihua Zheng, Ya Gao, Shihao Li et al. (+2 more)
Multi-modal vehicle Re-Identification (ReID) aims to leverage complementary information from RGB, Near Infrared (NIR), and Thermal Infrared (TIR) modalities to retrieve the same vehicle. The challenges of multi-modal vehicle ReID arise from the uncertainty of modality quality distribution induced by inherent discrepancies across modalities, resulting in distinct conflicting fusion requirements for data with balanced and unbalanced quality distributions. Existing methods handle all multi-modal data within a single fusion model, overlooking the different needs of the two data types and making it difficult to decouple the conflict between intra-class consistency and inter-modal heterogeneity. To this end, we propose Disentangle Collaboration and Guidance Fusion Representations for Multi-modal Vehicle ReID (DCG-ReID). Specifically, to disentangle heterogeneous quality-distributed modal data without mutual interference, we first design the Dynamic Confidence-based Disentangling Weighting (DCDW) mechanism: dynamically reweighting three-modal contributions via interaction-derived modal confidence to build a disentangled fusion framework. Building on DCDW, we develop two scenario-specific fusion strategies: (1) for balanced quality distributions, Collaboration Fusion Module (CFM) mines pairwise consensus features to capture shared discriminative information and boost intra-class consistency; (2) for unbalanced distributions, Guidance Fusion Module (GFM) implements differential amplification of modal discriminative disparities to reinforce dominant modality advantages, guide auxiliary modalities to mine complementary discriminative info, and mitigate inter-modal divergence to boost multi-modal joint decision performance. Extensive experiments on three multi-modal ReID
benchmarks (WMVeID863, MSVR310, RGBNT100) validate the effectiveness of our method. Code will be released upon acceptance.
Read the full abstract ▾
All authors Aihua Zheng, Ya Gao, Shihao Li, Chenglong Li, Jin Tang
Computer Vision
Zoom-IQA: Image Quality Assessment with Reliable Region-Aware Reasoning
Guoqiang Liang, Jianyi Wang, Zhonghua Wu et al. (+2 more)
Image Quality Assessment (IQA) is a long-standing problem in computer
vision. Previous methods typically focus on predicting numerical scores without explanation or providing low-level descriptions lacking precise scores. Recent
reasoning-based
vision language models (VLMs) have shown strong potential for IQA by jointly generating quality descriptions and scores. However, existing VLM-based IQA methods often suffer from unreliable
reasoning due to their limited capability of integrating visual and textual cues. In this work, we introduce Zoom-IQA, a VLM-based IQA model to explicitly emulate key cognitive behaviors: uncertainty awareness, region
reasoning, and iterative refinement. Specifically, we present a two-stage training pipeline: 1) supervised fine-tuning (SFT) on our Grounded-Rationale-IQA (GR-IQA)
dataset to teach the model to ground its assessments in key regions, and 2) reinforcement learning (RL) for dynamic policy exploration, stabilized by our KL-Coverage regularizer to prevent
reasoning and scoring diversity collapse, with a Progressive Re-sampling Strategy for mitigating annotation bias. Extensive experiments show that Zoom-IQA achieves improved robustness, explainability, and generalization. The application to downstream tasks, such as image restoration, further demonstrates the effectiveness of Zoom-IQA.
Read the full abstract ▾
All authors Guoqiang Liang, Jianyi Wang, Zhonghua Wu, Shangchen Zhou, Chen Change Loy
Computer Vision
TA-Prompting: Enhancing Video Large Language Models for Dense Video Captioning via Temporal Anchors
Wei-Yuan Cheng, Kai-Po Chang, Chi-Pin Huang et al. (+2 more)
Dense
video captioning aims to interpret and describe all temporally localized events throughout an input
video. Recent state-of-the-art methods leverage large
language models (LLMs) to provide detailed moment descriptions for
video data. However, existing VideoLLMs remain challenging in identifying precise event boundaries in untrimmed
videos, causing the generated captions to be not properly grounded. In this paper, we propose TA-Prompting, which enhances VideoLLMs via Temporal Anchors that learn to precisely localize events and
prompt the VideoLLMs to perform temporal-aware
video event understanding. During inference, in order to properly determine the output caption sequence from an arbitrary number of events presented within a
video, we introduce an event coherent sampling strategy to select event captions with sufficient coherence across temporal events and cross-modal similarity with the given
video. Through extensive experiments on
benchmark datasets, we show that our TA-Prompting is favorable against state-of-the-art VideoLLMs, yielding superior performance on dense
video captioning and temporal understanding tasks including moment
retrieval and temporalQA.
Read the full abstract ▾
All authors Wei-Yuan Cheng, Kai-Po Chang, Chi-Pin Huang, Fu-En Yang, Yu-Chiang Frank Wang
Computer Vision
Towards Agnostic and Holistic Universal Image Segmentation with Bit Diffusion
Jakob Lønborg Christensen, Morten Rieger Hannemose, Anders Bjorholm Dahl et al. (+1 more)
This paper introduces a
diffusion-based framework for universal image
segmentation, making agnostic
segmentation possible without depending on mask-based frameworks and instead predicting the full
segmentation in a holistic manner. We present several key adaptations to
diffusion models, which are important in this discrete setting. Notably, we show that a location-aware palette with our 2D gray code ordering improves performance. Adding a final tanh activation function is crucial for discrete data. On optimizing
diffusion parameters, the sigmoid loss weighting consistently outperforms alternatives, regardless of the prediction type used, and we settle on x-prediction. While our current model does not yet surpass leading mask-based architectures, it narrows the performance gap and introduces unique capabilities, such as principled ambiguity modeling, that these models lack. All models were trained from scratch, and we believe that combining our proposed improvements with large-scale pretraining or promptable conditioning could lead to competitive models.
Read the full abstract ▾
All authors Jakob Lønborg Christensen, Morten Rieger Hannemose, Anders Bjorholm Dahl, Vedrana Andersen Dahl
Computer Vision · Cross-listed from eess.IV
Lesion Segmentation in FDG-PET/CT Using Swin Transformer U-Net 3D: A Robust Deep Learning Framework
Shovini Guha and Dwaipayan Nandi
Accurate and automated lesion
segmentation in Positron Emission
Tomography / Computed
Tomography (PET/CT) imaging is essential for cancer diagnosis and therapy planning. This paper presents a Swin
Transformer UNet 3D (SwinUNet3D) framework for lesion
segmentation in Fluorodeoxyglucose Positron Emission
Tomography / Computed
Tomography (FDG-PET/CT) scans. By combining shifted window self-
attention with U-Net style skip connections, the model captures both global context and fine anatomical detail. We evaluate SwinUNet3D on the AutoPET III FDG
dataset and compare it against a baseline 3D U-Net. Results show that SwinUNet3D achieves a Dice score of 0.88 and IoU of 0.78, surpassing 3D U-Net (Dice 0.48, IoU 0.32) while also delivering faster inference times. Qualitative analysis demonstrates improved detection of small and irregular lesions, reduced false positives, and more accurate PET/CT fusion. While the framework is currently limited to FDG scans and trained under modest GPU resources, it establishes a strong foundation for future multi-tracer, multi-center evaluations and benchmarking against other
transformer-based architectures. Overall, SwinUNet3D represents an efficient and robust approach to PET/CT lesion
segmentation, advancing the integration of
transformer-based models into oncology imaging workflows.
Read the full abstract ▾
Computer Vision
Breaking Self-Attention Failure: Rethinking Query Initialization for Infrared Small Target Detection
Yuteng Liu, Duanni Meng, Yimian Dai et al. (+3 more)
Infrared small target detection (IRSTD) faces significant challenges due to low signal-to-noise ratios, extremely small target sizes, and complex cluttered backgrounds. Although DETR-based detectors benefit from global context modeling, their query initialization can become unreliable in IRSTD because only a few encoder
tokens correspond to targets, while the majority describe the background. We revisit this phenomenon and reveal that the target-relevant
embeddings of IRST are inevitably overwhelmed by dominant background features due to the self-
attention mechanism, leading to unreliable query initialization and inaccurate target localization. To address this issue, we propose SEF-DETR, a novel framework that refines query initialization through Patch-wise Spectral Screening (PSS), Frequency-Routed Examination (FRE), and Reliability-Consistency-aware Fusion (RCF). PSS encodes the radial and directional energy distributions of local
Fourier spectra and aggregates overlapping patch predictions into a target-relevant density map. Guided by this map, FRE performs sparse deformable re-examination after each encoder layer, while retaining content-driven sampling to distinguish true targets from frequency-domain false alarms. RCF further re-ranks candidate queries according to spatial-frequency consistency and frequency reliability. Extensive experiments on three public IRSTD
datasets demonstrate that SEF-DETR achieves superior detection performance over state-of-the-art methods with low computational overhead, providing a robust and efficient DETR-based solution for IRSTD.
Read the full abstract ▾
All authors Yuteng Liu, Duanni Meng, Yimian Dai, Maoxun Yuan, Xingxing Wei, Bo li
Computer Vision
DGA-Net: Enhancing SAM with Depth Prompting and Graph-Anchor Guidance for Camouflaged Object Detection
Yuetong Li, Qing Zhang, Yilin Zhao et al. (+2 more)
To fully exploit depth cues in Camouflaged Object Detection (COD), we present DGA-Net, a specialized framework that adapts the Segment Anything Model (SAM) via a novel “depth prompting" paradigm. Distinguished from existing approaches that primarily rely on sparse
prompts (e.g., points or boxes), our method introduces a holistic mechanism for constructing and propagating dense depth
prompts. Specifically, we propose a Cross-modal Graph Enhancement (CGE) module that synthesizes RGB semantics and depth
geometric within a heterogeneous graph to form a unified guidance signal. Furthermore, we design an Anchor-Guided Refinement (AGR) module. To counteract the inherent information decay in feature hierarchies, AGR forges a global anchor and establishes direct non-local pathways to broadcast this guidance from deep to shallow layers, ensuring precise and consistent
segmentation. Quantitative and qualitative experimental results demonstrate that our proposed DGA-Net outperforms the state-of-the-art COD methods.
Read the full abstract ▾
All authors Yuetong Li, Qing Zhang, Yilin Zhao, Gongyang Li, Zeming Liu
Computer Vision
SketchThinker-R1: Towards Efficient Sketch-Style Reasoning in Large Multimodal Models
Ruiyang Zhang, Dongzhan Zhou and Zhedong Zheng
Despite the empirical success of extensive, step-by-step
reasoning in large
multimodal models, long
reasoning processes inevitably incur substantial computational overhead, i.e., in terms of higher
token costs and increased response time, which undermines inference efficiency. In contrast, humans often employ sketch-style
reasoning: a concise, goal-directed cognitive process that prioritizes salient information and enables efficient problem-solving. Inspired by this cognitive efficiency, we propose SketchThinker-R1, which incentivizes sketch-style
reasoning ability in large
multimodal models. Our method consists of three primary stages. In the Sketch-Mode Cold Start stage, we convert standard long
reasoning process into sketch-style
reasoning and finetune base
multimodal model, instilling initial sketch-style
reasoning capability. Next, we train SketchJudge Reward Model, which explicitly evaluates thinking process of model and assigns higher scores to sketch-style
reasoning. Finally, we conduct Sketch-Thinking Reinforcement Learning under supervision of SketchJudge to further generalize sketch-style
reasoning ability. Experimental evaluation on four
benchmarks reveals that our SketchThinker-R1 achieves over 64% reduction in
reasoning token cost without compromising final answer accuracy. Qualitative analysis further shows that sketch-style
reasoning focuses more on key cues during problem solving.
Read the full abstract ▾
Computer Vision
Topology-aware Pathological Consistency Matching for Weakly-Paired IHC Virtual Staining
Mingzhou Jiang, Jiaying Zhou, Nan Zeng et al. (+5 more)
Immunohistochemical (IHC) staining provides crucial molecular characterization of tissue samples and plays an indispensable role in the clinical examination and diagnosis of cancers. However, compared with the commonly used Hematoxylin and Eosin (H&E) staining, IHC staining involves complex procedures and is both time-consuming and expensive, which limits its widespread clinical use. Virtual staining converts H&E images to IHC images, offering a cost-effective alternative to clinical IHC staining. Nevertheless, using adjacent slides as ground truth often results in weakly-paired data with spatial misalignment and local deformations, hindering effective supervised learning. To address these challenges, we propose a novel
topology-aware framework for H&E-to-IHC virtual staining. Specifically, we introduce a
Topology-aware Consistency Matching (TACM) mechanism that employs graph contrastive learning and topological perturbations to learn robust matching patterns despite spatial misalignments, ensuring structural consistency. Furthermore, we propose a
Topology-constrained Pathological Matching (TCPM) mechanism that aligns pathological positive regions based on node importance to enhance pathological consistency. Extensive experiments on two
benchmarks across four staining tasks demonstrate that our method outperforms state-of-the-art approaches, achieving superior generation quality with higher clinical relevance.
Read the full abstract ▾
All authors Mingzhou Jiang, Jiaying Zhou, Nan Zeng, Mickael Li, Qijie Tang, Chao He, Huazhu Fu, Honghui He
Computer Vision
StableDPT: Temporal Stable Monocular Video Depth Estimation
Ivan Sobko, Hayko Riemenschneider, Markus Gross et al. (+1 more)
Applying single image
Monocular Depth Estimation (MDE) models to
video sequences introduces significant temporal instability and flickering artifacts. We propose a novel approach that adapts any state-of-the-art image-based (depth) estimation model for
video processing by integrating a new temporal module - trainable on a single GPU in a few days. Our architecture StableDPT builds upon an off-the-shelf
Vision Transformer (ViT) encoder and enhances the Dense Prediction
Transformer (DPT) head. The core of our contribution lies in the temporal layers within the head, which use an efficient cross-
attention mechanism to integrate information from keyframes sampled across the entire
video sequence. This allows the model to capture global context and inter-frame relationships leading to more accurate and temporally stable depth predictions. Furthermore, we propose a novel inference strategy for processing
videos of arbitrary length avoiding the scale misalignment and redundant computations associated with overlapping windows used in other methods. Evaluations on multiple
benchmark datasets demonstrate improved temporal consistency, competitive state-of-the-art performance and on top 2x faster processing in real-world scenarios.
Read the full abstract ▾
All authors Ivan Sobko, Hayko Riemenschneider, Markus Gross, Christopher Schroers
Computer Vision
Textile IR: A Bidirectional Intermediate Representation for Physics-Aware Fashion CAD
Petteri Teikari and Neliana Fuenmayor
We introduce Textile IR, a bidirectional intermediate representation that connects manufacturing-valid CAD, physics-based simulation, and lifecycle assessment for fashion design. Unlike existing siloed tools where pattern software guarantees sewable outputs but understands nothing about drape, and physics simulation predicts behaviour but cannot automatically fix patterns, Textile IR provides the semantic glue for integration through a seven-layer Verification Ladder – from cheap syntactic checks (pattern closure, seam compatibility) to expensive physics validation (drape simulation, stress analysis). The architecture enables bidirectional feedback: simulation failures suggest pattern modifications; material substitutions update sustainability estimates in real time; uncertainty propagates across the pipeline with explicit confidence bounds. We formalise fashion engineering as constraint satisfaction over three domains and demonstrate how Textile IR's scene-graph representation enables AI systems to manipulate garments as structured programs rather than pixel arrays. The framework addresses the compound uncertainty problem: when measurement errors in material testing, simulation approximations, and LCA database gaps combine, sustainability claims become unreliable without explicit uncertainty tracking. We propose six research priorities and discuss deployment considerations for fashion SMEs where integrated workflows reduce specialised engineering requirements. Key contribution: a formal representation that makes engineering constraints perceptible, manipulable, and immediately consequential – enabling designers to navigate sustainability, manufacturability, and aesthetic tradeoffs simultaneously rather than discovering conflicts after costly physical prototyping.
Read the full abstract ▾
Computer Vision
DreamStyle: A Unified Framework for Video Stylization
Mengtian Li, Jinshu Chen, Songtao Zhao et al. (+3 more)
Video stylization, an important downstream task of
video generation models, has not yet been thoroughly explored. Its input style conditions typically include text, style image, and stylized first frame. Each condition has a characteristic advantage: text is more flexible, style image provides a more accurate visual anchor, and stylized first frame makes long-
video stylization feasible. However, existing methods are largely confined to a single type of style condition, which limits their scope of application. Additionally, their lack of high-quality
datasets leads to style inconsistency and temporal flicker. To address these limitations, we introduce DreamStyle, a unified framework for
video stylization, supporting (1) text-guided, (2) style-image-guided, and (3) first-frame-guided
video stylization, accompanied by a well-designed data curation pipeline to acquire high-quality paired
video data. DreamStyle is built on a vanilla Image-to-
Video (I2V) model and trained using a Low-Rank Adaptation (LoRA) with
token-specific up
matrices that reduces the confusion among different condition
tokens. Both qualitative and quantitative evaluations demonstrate that DreamStyle is competent in all three
video stylization tasks, and outperforms the competitors in style consistency and
video quality.
Read the full abstract ▾
All authors Mengtian Li, Jinshu Chen, Songtao Zhao, Wanquan Feng, Pengqi Tu, Qian He
Computer Vision
EarthVL: A Progressive Earth Vision-Language Understanding and Generation Framework
Junjue Wang, Yanfei Zhong, Zihang Chen et al. (+3 more)
Earth
vision has achieved milestones in geospatial object recognition but lacks exploration in object-relational
reasoning, limiting comprehensive scene understanding. To address this, a progressive Earth
vision-
language understanding and generation framework is proposed, including a multi-task
dataset (EarthVLSet) and a semantic-guided network (EarthVLNet). Focusing on city planning applications, EarthVLSet includes 10.9k sub-meter resolution remote sensing images, land-cover masks, and 761.5k textual pairs involving both multiple-choice and open-ended visual question answering (VQA) tasks. In an object-centric way, EarthVLNet is proposed to progressively achieve semantic
segmentation, relational
reasoning, and comprehensive understanding. The first stage involves land-cover
segmentation to generate object semantics for VQA guidance. Guided by pixel-wise semantics, the object awareness based large
language model (LLM) performs relational
reasoning and knowledge summarization to generate the required answers. As for optimization, the numerical difference loss is proposed to dynamically add difference penalties, addressing the various objects' statistics. Three
benchmarks, including semantic
segmentation, multiple-choice, and open-ended VQA demonstrated the superiorities of EarthVLNet, yielding three future directions: 1)
segmentation features consistently enhance VQA performance even in cross-
dataset scenarios; 2) multiple-choice tasks show greater sensitivity to the
vision encoder than to the
language decoder; and 3) open-ended tasks necessitate advanced
vision encoders and
language decoders for an optimal performance. We believe this
dataset and method will provide a beneficial
benchmark that connects ”image-mask-text”, advancing geographical applications for Earth
vision.
Read the full abstract ▾
All authors Junjue Wang, Yanfei Zhong, Zihang Chen, Zhuo Zheng, Ailong Ma, Liangpei Zhang
Computer Vision
AbductiveMLLM: Boosting Visual Abductive Reasoning Within MLLMs
Boyu Chang, Qi Wang, Xi Guo et al. (+3 more)
Visual abductive
reasoning (VAR) is a challenging task that requires AI systems to infer the most likely explanation for incomplete visual observations. While recent MLLMs develop strong general-purpose
multimodal reasoning capabilities, they fall short in abductive inference, as compared to human beings. To bridge this gap, we draw inspiration from the interplay between verbal and pictorial abduction in human cognition, and propose to strengthen abduction of MLLMs by mimicking such dual-mode behavior. Concretely, we introduce AbductiveMLLM comprising of two synergistic components: REASONER and IMAGINER. The REASONER operates in the verbal domain. It first explores a broad space of possible explanations using a blind LLM and then prunes visually incongruent hypotheses based on cross-modal causal
alignment. The remaining hypotheses are introduced into the MLLM as targeted priors, steering its
reasoning toward causally coherent explanations. The IMAGINER, on the other hand, further guides MLLMs by emulating human-like pictorial thinking. It conditions a text-to-image
diffusion model on both the input
video and the REASONER's output
embeddings to "imagine" plausible visual scenes that correspond to verbal explanation, thereby enriching MLLMs' contextual grounding. The two components are trained jointly in an end-to-end manner. Experiments on standard VAR
benchmarks show that AbductiveMLLM achieves state-of-the-art performance, consistently outperforming traditional solutions and advanced MLLMs.
Read the full abstract ▾
All authors Boyu Chang, Qi Wang, Xi Guo, Zhixiong Nan, Yazhou Yao, Tianfei Zhou
Computer Vision
ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration
Xu Zhang, Huan Zhang, Guoli Wang et al. (+2 more)
All-in-One Image Restoration (AiOIR) has advanced significantly, offering promising solutions for complex real-world degradations. However, most existing approaches rely heavily on degradation-specific representations, often resulting in oversmoothing and artifacts. To address this, we propose ClearAIR, a novel AiOIR framework inspired by Human Visual Perception (HVP) and designed with a hierarchical, coarse-to-fine restoration strategy. First, leveraging the global priority of early HVP, we employ a
Multimodal Large
Language Model (MLLM)-based Image Quality Assessment (IQA) model for overall evaluation. Unlike conventional IQA, our method integrates cross-modal understanding to more accurately characterize complex, composite degradations. Building upon this overall assessment, we then introduce a region awareness and task recognition pipeline. A semantic cross-
attention, leveraging semantic guidance unit, first produces coarse semantic
prompts. Guided by this regional context, a degradation-aware module implicitly captures region-specific degradation characteristics, enabling more precise local restoration. Finally, to recover fine details, we propose an internal clue reuse mechanism. It operates in a self-supervised manner to mine and leverage the intrinsic information of the image itself, substantially enhancing detail restoration. Experimental results show that ClearAIR achieves superior performance across diverse synthetic and real-world
datasets.
Read the full abstract ▾
All authors Xu Zhang, Huan Zhang, Guoli Wang, Qian Zhang, Lefei Zhang
Computer Vision
AnyDepth: Depth Estimation Made Easy
Zeyu Ren, Zeyu Zhang, Wukai Li et al. (+2 more)
Monocular depth estimation aims to recover the depth information of 3D scenes from 2D images. Recent work has made significant progress, but its reliance on large-scale
datasets and complex decoders has limited its efficiency and generalization ability. In this paper, we propose a lightweight and data-centric framework for zero-shot
monocular depth estimation. We first adopt DINOv3 as the visual encoder to obtain high-quality dense features. Secondly, to address the inherent drawbacks of the complex structure of the DPT, we design the Simple Depth
Transformer (SDT), a compact
transformer-based decoder. Compared to the DPT, it uses a single-path feature fusion and upsampling process to reduce the computational overhead of cross-scale feature fusion, achieving higher accuracy while reducing the number of parameters by approximately 85%-89%. Furthermore, we propose a quality-based filtering strategy to filter out harmful samples, thereby reducing
dataset size while improving overall training quality. Extensive experiments on five
benchmarks demonstrate that our framework surpasses the DPT in accuracy. This work highlights the importance of balancing model design and data quality for achieving efficient and generalizable zero-shot depth estimation. Code: https://github.com/AIGeeksGroup/AnyDepth. Website: https://aigeeksgroup.github.io/AnyDepth.
Read the full abstract ▾
All authors Zeyu Ren, Zeyu Zhang, Wukai Li, Qingxiang Liu, Hao Tang
Computer Vision
Toward Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups
Hyungtae Lim, Minkyun Seo, Luca Carlone et al. (+1 more)
Some deep learning-based point cloud registration methods struggle with zero-shot generalization, often requiring target-domain retraining, fine-tuning, or
dataset-specific metric parameter tuning for new environments. We identify three critical limitations: (a) fixed user-defined parameters (e.g., voxel size, search radius) that fail to generalize across varying scales, (b) learned keypoint detectors exhibit poor cross-domain transferability, and (c) absolute coordinates amplify scale mismatches between
datasets. To address these three issues, we present BUFFER-X, a registration framework whose descriptor model is trained once and then applied without parameter tuning or target-domain retraining, which achieves zero-shot generalization through: (a)
geometric bootstrapping for automatic hyperparameter estimation, (b) distribution-aware farthest point sampling to replace learned detectors, and (c) patch-level coordinate normalization to ensure scale consistency. Our approach employs hierarchical multi-scale matching to extract correspondences across local, middle, and global receptive fields, enabling robust registration in diverse environments. For efficiency-critical applications, we present BUFFER-X-Lite from a holistic system perspective, integrating the BUFFER-X front-end and adaptive early exit with the fast and robust pose solver to reduce total computation time by 43% relative to BUFFER-X while preserving comparable accuracy. We evaluate on a comprehensive
benchmark comprising 12
datasets spanning object-scale, indoor, and outdoor scenes, including cross-sensor registration between heterogeneous
LiDAR configurations. Results demonstrate that our approach generalizes effectively across diverse scales, scenes, and sensor setups. Code: https://github.com/MIT-SPARK/BUFFER-X.
Read the full abstract ▾
All authors Hyungtae Lim, Minkyun Seo, Luca Carlone, Jaesik Park
Computer Vision
D$^3$R-DETR: DETR with Dual-Domain Density Refinement for Tiny Object Detection in Aerial Images
Zixiao Wen, Zhen Yang, Xianjie Bao et al. (+4 more)
Detecting tiny objects plays a vital role in remote sensing intelligent interpretation, as these objects often carry critical information for downstream applications. However, due to the extremely limited pixel information and significant variations in object density, mainstream
Transformer-based detectors often suffer from slow convergence and inaccurate query-object matching. To address these challenges, we propose D$^3$R-DETR, a novel DETR-based detector with Dual-Domain Density Refinement. By fusing spatial and frequency domain information, our method refines low-level feature maps and utilizes their rich details to predict more accurate object density map, thereby guiding the model to precisely localize tiny objects. Extensive experiments on the AI-TOD-v2
dataset demonstrate that D$^3$R-DETR outperforms existing state-of-the-art detectors for tiny object detection.
Read the full abstract ▾
All authors Zixiao Wen, Zhen Yang, Xianjie Bao, Lei Zhang, Xiantai Xiang, Wenshuai Li, Yuhan Liu
Computer Vision
Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench
Zanting Ye, Xiaolong Niu, Xuanbin Wu et al. (+14 more)
While
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable proficiency in tasks such as abnormality detection and report generation for anatomical modalities, their capability in functional imaging remains largely unexplored. In this work, we identify and quantify a fundamental functional perception gap: the inability of current
vision encoders to decode functional tracer biodistribution independent of morphological priors. Identifying Positron Emission
Tomography (PET) as the quintessential modality to investigate this disconnect, we introduce PET-Bench, the first large-scale functional imaging
benchmark comprising 52,308 hierarchical QA pairs from 9,732 multi-site, multi-tracer PET studies. Extensive evaluation of 19 state-of-the-art MLLMs reveals a critical safety hazard termed the Chain-of-Thought (CoT)
hallucination trap. We observe that standard CoT prompting, widely considered to enhance
reasoning, paradoxically decouples linguistic generation from visual evidence in PET, producing clinically fluent but factually ungrounded diagnoses. To resolve this, we propose Atomic Visual
Alignment (AVA), a simple fine-tuning strategy that enforces the mastery of low-level functional perception prior to high-level diagnostic
reasoning. Our results demonstrate that AVA effectively bridges the perception gap, transforming CoT from a source of
hallucination into a robust inference tool and improving diagnostic accuracy by up to 14.83%. Code and data are available at https://github.com/yezanting/PET-Bench.
Read the full abstract ▾
All authors Zanting Ye, Xiaolong Niu, Xuanbin Wu, Xu Han, Shengyuan Liu, Jing Hao, Zhihao Peng, Hao Sun, Jieqin Lv, Fanghu Wang, Yanchao Huang, Hubing Wu, Yixuan Yuan, Habib Zaidi, Arman Rahmim, Yefeng Zheng, Lijun Lu
Computer Vision · Cross-listed from cs.SD
Omni2Sound: Towards Unified Video-Text-to-Audio Generation
Yusheng Dai, Zehua Chen, Yuxuan Jiang et al. (+4 more)
Training a unified model integrating
video-to-audio (V2A), text-to-audio (T2A), and joint
video-text-to-audio (VT2A) generation offers significant application flexibility, yet faces two unexplored foundational challenges: (1) the scarcity of high-quality audio captions with tight V-A-T
alignment, leading to severe semantic conflict between
multimodal conditions, and (2) cross-task and intra-task competition, manifesting as an adverse V2A-T2A performance trade-off and modality bias in the VT2A task. First, to address data scarcity, we introduce SoundAtlas, a large-scale
dataset (470k pairs) that significantly outperforms existing
benchmarks and even human experts in quality. Powered by a novel agentic pipeline, it integrates
Vision-to-
Language Compression to mitigate visual bias of MLLMs, a Junior-Senior
Agent Handoff for a 5$\times$ cost reduction, and rigorous Post-hoc Filtering to ensure fidelity. Consequently, SoundAtlas delivers semantically rich and temporally detailed captions with tight V-A-T
alignment. Second, we propose Omni2Sound, a unified VT2A
diffusion model supporting flexible input modalities. To resolve the inherent cross-task and intra-task competition, we design a three-stage multi-task progressive training schedule that converts cross-task competition into joint optimization and mitigates modality bias in the VT2A task, maintaining both audio-visual
alignment and off-screen audio generation faithfulness. Finally, we construct VGGSound-Omni, a comprehensive
benchmark for unified evaluation, including challenging off-screen tracks. With a standard DiT backbone, Omni2Sound achieves unified SOTA performance across all three tasks within a single model, demonstrating strong generalization across
benchmarks with heterogeneous input conditions.
Read the full abstract ▾
All authors Yusheng Dai, Zehua Chen, Yuxuan Jiang, Baolong Gao, Qiuhong Ke, Jianfei Cai, Jun Zhu
Computer Vision
HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps
Xuchang Zhong, Xu Cao, Jinke Feng et al. (+1 more)
Visual localization on standard-definition (SD) maps has emerged as a promising low-cost and scalable solution for autonomous driving. However, existing regression-based approaches often overlook inherent
geometric priors, resulting in suboptimal training efficiency and limited localization accuracy. In this paper, we propose a novel homography-guided pose
estimator network for fine-grained visual localization between multi-view images and standard-definition (SD) maps. We construct input pairs that satisfy a homography constraint by projecting ground-view features into the BEV domain and enforcing semantic
alignment with map features. Then we leverage homography relationships to guide feature fusion and restrict the pose outputs to a valid feasible region, which significantly improves training efficiency and localization accuracy compared to prior methods relying on
attention-based fusion and direct 3-DoF pose regression. To the best of our knowledge, this is the first work to unify BEV semantic
reasoning with homography learning for image-to-map localization. Furthermore, by explicitly modeling homography transformations, the proposed framework naturally supports cross-resolution inputs, enhancing model flexibility. Extensive experiments on the nuScenes
dataset demonstrate that our approach significantly outperforms existing state-of-the-art visual localization methods. Code and pretrained models will be publicly released to foster future research.
Read the full abstract ▾
All authors Xuchang Zhong, Xu Cao, Jinke Feng, Hao Fang
Computer Vision
Foreground-Aware Dataset Distillation via Dual-Path Patch Selection
Mingzhuo Li, Guang Li, Longzhen Li et al. (+4 more)
In this paper, we propose a foreground-aware
dataset distillation method that performs content-adaptive dual-path patch selection. With the rising computational cost of training large-scale deep models,
dataset distillation has emerged as a promising approach for constructing compact synthetic
datasets that retain knowledge from large original
datasets. Although optimization-based methods can achieve strong performance, they often incur considerable computational and
memory costs and may produce synthetic images that generalize poorly across architectures. Recent non-optimization-based methods alleviate these limitations by constructing distilled images from real-image patches. However, their inflexible patch-construction strategies may either retain redundant background content or discard important foreground structures. To address this issue, we propose a foreground-aware
dataset distillation framework with content-adaptive dual-path patch selection. We first use Grounded SAM2 to estimate the foreground occupancy of each image and derive category-specific routing thresholds from the corresponding class-wise distributions. Each image is routed to one of two complementary paths: the candidate-crop selection path for images with relatively sparse foregrounds, or the direct transformation path for images with relatively dominant foregrounds. This dual-path mechanism balances foreground concentration and preservation, reducing redundant background content while retaining the structural integrity of foreground objects at the
dataset level. Experiments on CIFAR-10, CIFAR-100, ImageNette, and ImageWoof with two network architectures and three
distillation budgets demonstrate that the proposed method outperforms the baseline method RDED in nearly all evaluated settings.
Read the full abstract ▾
All authors Mingzhuo Li, Guang Li, Longzhen Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Computer Vision · Cross-listed from cs.RO
Loop Closure using AnyLoc Visual Place Recognition in DPV-SLAM
Wenzheng Zhang, Kazuki Adachi, Yoshitaka Hara et al. (+1 more)
Loop closure is crucial for maintaining the accuracy and consistency of visual SLAM. We propose a method to improve loop closure performance in DPV-SLAM. Our approach integrates AnyLoc, a learning-based visual place recognition technique, as a replacement for the classical Bag of Visual Words (BoVW) loop detection method. In contrast to BoVW, which relies on handcrafted features, AnyLoc utilizes deep feature representations, enabling more robust image
retrieval across diverse viewpoints and lighting conditions. Furthermore, we propose an adaptive mechanism that dynamically adjusts similarity threshold based on environmental conditions, removing the need for manual tuning. Experiments on both indoor and outdoor
datasets demonstrate that our method significantly outperforms the original DPV-SLAM in terms of loop closure accuracy and robustness. The proposed method offers a practical and scalable solution for enhancing loop closure performance in modern SLAM systems.
Read the full abstract ▾
All authors Wenzheng Zhang, Kazuki Adachi, Yoshitaka Hara, Sousuke Nakamura
Computer Vision
Robust Mesh Saliency Ground Truth Acquisition in VR via View Cone Sampling and Manifold Diffusion
Guoquan Zheng, Jie Hao, Huiyu Duan et al. (+7 more)
As the complexity of 3D digital content grows exponentially, understanding human visual
attention is critical for optimizing rendering and processing resources. Therefore, reliable 3D mesh saliency ground truth (GT) is essential for human-centric visual modeling in virtual reality (VR). However, existing VR eye-tracking frameworks are fundamentally bottlenecked by their underlying acquisition and generation mechanisms. The reliance on zero-area single ray sampling (SRS) fails to capture contextual features, leading to severe texture aliasing and discontinuous saliency signals. And the conventional application of Euclidean smoothing propagates saliency across disconnected physical gaps, resulting in semantic confusion on complex 3D
manifolds. This paper proposes a robust framework to address these limitations. We first introduce a view cone sampling (VCS) strategy, which simulates the human foveal receptive field via Gaussian-distributed ray bundles to improve sampling robustness for complex
topologies. Furthermore, a hybrid
Manifold-Euclidean constrained
diffusion (HCD)
algorithm is developed, fusing
manifold geodesic constraints with Euclidean scales to ensure topologically-consistent saliency propagation. We demonstrate the improvement in performance over baseline methods and the benefits for downstream tasks through subjective experiments and qualitative and quantitative methods. By mitigating "topological short-
circuits" and aliasing, our framework provides a high-fidelity 3D
attention acquisition paradigm that aligns with natural human perception, offering a more accurate and robust baseline for 3D mesh saliency research.
Read the full abstract ▾
All authors Guoquan Zheng, Jie Hao, Huiyu Duan, Long Tang, Shuo Yang, Yucheng Zhu, Yongming Han, Liang Yuan, Patrick Le Callet, Guangtao Zhai
Computer Vision
MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos
Taeyeon Kim, Youngju Na, Jumin Lee et al. (+3 more)
Transferring articulated motion from
monocular videos to rigged 3D characters is challenging due to pose ambiguity in 2D observations and morphological differences between source and target. Existing approaches often follow a reconstruct-then-retarget paradigm, tying transfer quality to intermediate 3D reconstruction and limiting applicability to categories with parametric templates. We propose MorphGS, a framework that formulates motion retargeting as a target-driven analysis-by-synthesis problem, directly optimizing target morphology and pose through image-space supervision. A rig-coupled morphology parameterization factorizes character identity from time-varying joint rotations, while dense 2D-3D correspondences and synthesized views provide complementary structural and multi-view guidance. Experiments on synthetic
benchmarks and real-world
videos show consistent improvements over baselines. Project page: https://xodus777.github.io/MorphGS/
Read the full abstract ▾
All authors Taeyeon Kim, Youngju Na, Jumin Lee, Sebin Lee, Minhyuk Sung, Sung-Eui Yoon
Computer Vision
GRRE: Leveraging G-Channel Removed Reconstruction Error for Robust Detection of AI-Generated Images
Shuman He, Xiehua Li, Xioaju Yang et al. (+2 more)
The rapid progress of generative models, particularly
diffusion models and GANs, has greatly increased the difficulty of distinguishing synthetic images from real ones. Although numerous detection methods have been proposed, their accuracy often degrades when applied to images generated by novel or unseen generative models, highlighting the challenge of achieving strong generalization. To address this challenge, we introduce a novel detection paradigm based on channel removal reconstruction. Specifically, we observe that when the green (G) channel is removed from real images and reconstructed, the resulting reconstruction errors differ significantly from those of AI-generated images. Building upon this insight, we propose G-channel Removed Reconstruction Error (GRRE), a simple yet effective method that exploits this discrepancy for robust AI-generated image detection. Extensive experiments demonstrate that GRRE consistently achieves high detection accuracy across multiple generative models, including those unseen during training. Compared with existing approaches, GRRE not only maintains strong robustness against various perturbations and post-processing operations but also exhibits superior cross-model generalization. These results highlight the potential of channel-removal-based reconstruction as a powerful forensic tool for safeguarding image authenticity in the era of generative AI.
Read the full abstract ▾
All authors Shuman He, Xiehua Li, Xioaju Yang, Yang Xiong, Keqin Li
Computer Vision
DreamLoop: Controllable Cinemagraph Generation from a Single Photograph
Aniruddha Mahapatra, Long Mai, Cusuh Ham et al. (+1 more)
Cinemagraphs, which combine static photographs with selective, looping motion, offer unique artistic appeal. Generating them from a single photograph in a controllable manner is particularly challenging. Existing image-animation techniques are restricted to simple, low-frequency motions and operate only in narrow domains with repetitive textures like water and smoke. In contrast, large-scale
video diffusion models are not tailored for cinemagraph constraints and lack the specialized data required to generate seamless, controlled loops. We present DreamLoop, a controllable
video synthesis framework dedicated to generating cinemagraphs from a single photo without requiring any cinemagraph training data. Our key idea is to adapt a general
video diffusion model by training it on two objectives: temporal bridging and motion conditioning. This strategy enables flexible cinemagraph generation. During inference, by using the input image as both the first- and last- frame condition, we enforce a seamless loop. By conditioning on static tracks, we maintain a static background. Finally, by providing a user-specified motion path for a target object, our method provides intuitive control over the animation's trajectory and timing. To our knowledge, DreamLoop is the first method to enable cinemagraph generation for general scenes with flexible and intuitive controls. We demonstrate that our method produces high-quality, complex cinemagraphs that align with user intent, outperforming existing approaches.
Read the full abstract ▾
All authors Aniruddha Mahapatra, Long Mai, Cusuh Ham, Feng Liu