Computer Vision
Adapting 2D Multi-Modal Large Language Model for 3D CT Image Analysis
Yang Yu, Dunyuan Xu, Yaoqian Li et al. (+3 more)
3D medical image analysis is of great importance in disease diagnosis and treatment. Recently,
multimodal large
language models (MLLMs) have exhibited robust perceptual capacity, strong cross-modal
alignment, and promising generalizability. Therefore, they have great potential to improve the performance of medical report generation (MRG) and medical visual question answering (MVQA), which serve as two important tasks in clinical scenarios. However, due to the scarcity of 3D medical images, existing 3D medical MLLMs suffer from insufficiently pretrained
vision encoder and inability to extract customized image features for different kinds of tasks. In this paper, we propose to first transfer a 2D MLLM, which is well trained with 2D natural images, to support 3D medical volumetric inputs while reusing all of its pre-trained parameters. To enable the
vision encoder to extract tailored image features for various tasks, we then design a Text-Guided Hierarchical MoE (TGH-MoE) framework, which can distinguish tasks under the guidance of the text
prompt. Furthermore, we propose a two-stage training strategy to learn both task-shared and task-specific image features. As demonstrated empirically, our method outperforms existing 3D medical MLLMs in both MRG and MVQA tasks. Our code will be released once this paper is accepted.
Read the full abstract ▾
All authors Yang Yu, Dunyuan Xu, Yaoqian Li, Xiaomeng Li, Jinpeng Li, Pheng-Ann Heng
Computer Vision
SMFormer: Empowering Self-supervised Stereo Matching via Foundation Models and Data Augmentation
Yun Wang, Zhengjie Yang, Jiahao Zheng et al. (+3 more)
Recent self-supervised stereo matching methods have made significant progress. They typically rely on the photometric consistency assumption, which presumes corresponding points across views share the same appearance. However, this assumption could be compromised by real-world disturbances, resulting in invalid supervisory signals and a significant accuracy gap compared to supervised methods. To address this issue, we propose SMFormer, a framework integrating more reliable self-supervision guided by the
Vision Foundation Model (VFM) and data augmentation. We first incorporate the VFM with the Feature Pyramid Network (FPN), providing a discriminative and robust feature representation against disturbance in various scenarios. We then devise an effective data augmentation mechanism that ensures robustness to various transformations. The data augmentation mechanism explicitly enforces consistency between learned features and those influenced by illumination variations. Additionally, it regularizes the output consistency between disparity predictions of strong augmented samples and those generated from standard samples. Experiments on multiple mainstream
benchmarks demonstrate that our SMFormer achieves state-of-the-art (SOTA) performance among self-supervised methods and even competes on par with supervised ones. Remarkably, in the challenging Booster
benchmark, SMFormer even outperforms some SOTA supervised methods, such as CFNet.
Read the full abstract ▾
All authors Yun Wang, Zhengjie Yang, Jiahao Zheng, Zhanjie Zhang, Dapeng Oliver Wu, Yulan Guo
Computer Vision
Are Pretrained Image Matchers Good Enough for SAR-Optical Satellite Registration?
Isaac Corley, Alex Stoken and Gabriele Berton
Cross-modal optical–SAR (Synthetic Aperture Radar) registration is a bottleneck in remote-sensing disaster response. Modern image matchers are developed and benchmarked almost exclusively on natural-image domains. We evaluate twenty-four pretrained matcher configurations in a zero-shot setting, with no fine-tuning or domain adaptation on satellite or SAR data. The evaluation spans SpaceNet9 and two additional cross-modal
benchmarks under a deterministic protocol that uses tiled large-image inference, robust
geometric filtering, and tie-point-grounded metrics. Our results show uneven transfer: matchers with explicit cross-modal training do not uniformly outperform those without it. XoFTR (trained for visible–thermal matching) and RoMa achieve the lowest reported mean tie-point error at $3.0$ px on the labeled SpaceNet9 training scenes. RoMa achieves this result without any cross-modal training. MatchAnything-ELoFTR ($3.4$ px), trained on synthetic cross-modal pairs, is close behind. XoFTR also runs roughly an order of magnitude faster than the RoMa family ($0.4$ s vs. $5.0$ s per pair). RoMa's performance is consistent with the hypothesis that frozen DINOv2 features confer robustness to large appearance shifts. Deployment protocol choices (geometry model, tile size, inlier gating) change mean error by up to $33\times$ for a single matcher. This shift can exceed the effect of swapping matchers within our evaluated protocol ablations. Affine geometry alone reduces mean error from $12.3$ to $9.7$ px.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
ReaLiTy and LADS: A Unified Framework and Dataset Suite for LiDAR Adaptation Across Sensors and Adverse Weather Conditions
Vivek Anand, Bharat Lohani, Rakesh Mishra et al. (+1 more)
Reliable
LiDAR perception requires robustness across sensors, environments, and adverse weather. However, existing
datasets rarely provide physically consistent observations of the same scene under varying sensor configurations and weather conditions, limiting systematic analysis of domain shifts. This work presents ReaLiTy, a unified physics-informed framework that transforms
LiDAR data to match target sensor specifications and weather conditions. The framework integrates physically grounded cues with a learning-based module to generate realistic intensity patterns, while a physics-based weather model introduces consistent
geometric and radiometric degradations. Building on this framework, we introduce the
LiDAR Adaptation
Dataset Suite (LADS), a collection of physically consistent, transformation-ready point clouds with one-to-one correspondence to original
datasets. Experiments demonstrate improved cross-domain consistency and realistic weather effects. ReaLiTy and LADS provide a reproducible foundation for studying
LiDAR adaptation and simulation-driven perception in intelligent transportation systems.
Read the full abstract ▾
All authors Vivek Anand, Bharat Lohani, Rakesh Mishra, Gaurav Pandey
Computer Vision
A3-FPN: Asymptotic Content-Aware Pyramid Attention Network for Dense Visual Prediction
Meng'en Qin, Yu Song, Quanling Zhao et al. (+3 more)
Learning multi-scale representations is the common strategy to tackle object scale variation in dense prediction tasks. Although existing feature pyramid networks have greatly advanced visual recognition, inherent design defects inhibit them from capturing discriminative features and recognizing small objects. In this work, we propose Asymptotic Content-Aware Pyramid
Attention Network (A3-FPN), to augment multi-scale feature representation via the asymptotically disentangled framework and content-aware
attention modules. Specifically, A3-FPN employs a horizontally-spread column network that enables asymptotically global feature interaction and disentangles each level from all hierarchical representations. In feature fusion, it collects supplementary content from the adjacent level to generate position-wise offsets and weights for context-aware resampling, and learns deep context reweights to improve intra-category similarity. In feature reassembly, it further strengthens intra-scale discriminative feature learning and reassembles redundant features based on information content and spatial variation of feature maps. Extensive experiments on MS COCO, VisDrone2019-DET and Cityscapes demonstrate that A3-FPN can be easily integrated into state-of-the-art CNN and
Transformer-based architectures, yielding remarkable performance gains. Notably, when paired with OneFormer and Swin-L backbone, A3-FPN achieves 49.6 mask AP on MS COCO and 85.6 mIoU on Cityscapes. Codes are available at https://github.com/mason-ching/A3-FPN.
Read the full abstract ▾
All authors Meng'en Qin, Yu Song, Quanling Zhao, Xiaodong Yang, Yingtao Che, Xiaohui Yang
Computer Vision · Cross-listed from cs.AI
Ruijia Li, Mingzi Zhang, Zengyi Yu et al. (+2 more)
As
Vision-
Language Models (VLMs) become integral to educational decision-making, ensuring their
fairness is paramount. However, current text-centric evaluations neglect the visual modality, leaving an unregulated channel for latent social biases. To bridge this gap, we present Edu-MMBias, a systematic
auditing framework grounded in the tri-component model of attitudes from social psychology. This framework diagnoses bias across three hierarchical dimensions: cognitive, affective, and behavioral. Utilizing a specialized generative pipeline that incorporates a self-correct mechanism and human-in-the-loop verification, we synthesize contamination-resistant student profiles to conduct a holistic stress test on state-of-the-art VLMs. Our extensive audit reveals critical, counter-intuitive patterns: models exhibit a compensatory class bias favoring lower-status narratives while simultaneously harboring deep-seated health and racial stereotypes. Crucially, we find that visual inputs act as a safety
backdoor, triggering a resurgence of biases that bypass text-based
alignment safeguards and revealing a systematic misalignment between latent cognition and final decision-making. The contributions of this paper are available at: https://anonymous.4open.science/r/EduMMBias-63B2.
Read the full abstract ▾
All authors Ruijia Li, Mingzi Zhang, Zengyi Yu, Yuang Wei, Bo Jiang
Computer Vision
Radiology Report Generation for Low-Quality X-Ray Images
Hongze Zhu, Chen Hu, Jiaxuan Jiang et al. (+6 more)
Vision-
Language Models (VLMs) have significantly advanced automated Radiology Report Generation (RRG). However, existing methods implicitly assume high-quality inputs, overlooking the noise and artifacts prevalent in real-world clinical environments. Consequently, current models exhibit severe performance degradation when processing suboptimal images. To bridge this gap, we propose a robust report generation framework explicitly designed for image quality variations. We first introduce an Automated Quality Assessment
Agent (AQAA) to identify low-quality samples within the MIMIC-CXR
dataset and establish the Low-quality Radiology Report Generation (LRRG)
benchmark. To tackle degradation-induced shifts, we propose a novel Dual-loop Training Strategy leveraging bi-level optimization and gradient consistency. This approach ensures the model learns quality-agnostic diagnostic features by aligning gradient directions across varying quality regimes. Extensive experiments demonstrate that our approach effectively mitigates model performance degradation caused by image quality deterioration. The code and data will be released upon acceptance.
Read the full abstract ▾
All authors Hongze Zhu, Chen Hu, Jiaxuan Jiang, Hong Liu, Yawen Huang, Ming Hu, Tianyu Wang, Zhijian Wu, Yefeng Zheng
Computer Vision · Cross-listed from cs.RO
Device-Conditioned Neural Architecture Search for Efficient Robotic Manipulation
Yiming Wu, Huan Wang, Zhenghao Chen et al. (+2 more)
The growing complexity of visuomotor policies poses significant challenges for deployment with heterogeneous robotic hardware constraints. However, most existing model-efficient approaches for robotic manipulation are device- and model-specific, lack generalizability, and require time-consuming per-device optimization during the adaptation process. In this work, we propose a unified framework named Device-Conditioned Quantization-For-All (DC-QFA) which amortizes deployment effort with the device-conditioned quantization-aware training and hardware-constrained architecture search. Specifically, we introduce a single supernet that spans a rich design space over network architectures and mixed-precision bit-widths. It is optimized with latency- and
memory-aware regularization, guided by per-device lookup tables. With this supernet, for each target platform, we can perform a once-for-all lightweight search to select an optimal subnet without any per-device re-optimization, which enables more generalizable deployment across heterogeneous hardware, and substantially reduces deployment time. To improve long-horizon stability under low precision, we further introduce multi-step on-policy
distillation to mitigate error accumulation during closed-loop execution. Extensive experiments on three representative policy backbones, such as DiffusionPolicy-T, MDT-V, and OpenVLA-OFT, demonstrate that our DC-QFA achieves $2\text{-}3\times$ acceleration on edge devices, consumer-grade GPUs, and cloud platforms, with negligible performance drop in task success. Real-world evaluations on an Inovo
robot equipped with a force/torque sensor further validates that our low-bit DC-QFA policies maintain stable, contact-rich manipulation even under severe quantization.
Read the full abstract ▾
All authors Yiming Wu, Huan Wang, Zhenghao Chen, Ge Yuan, Dong Xu
Computer Vision
Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
Yibo Yan, Mingdong Ou, Yi Cao et al. (+5 more)
Multi-vector models dominate Visual Document
Retrieval (VDR) due to their fine-grained matching capabilities, but their high storage and computational costs present a major barrier to practical deployment. In this paper, we propose ColChunk, a plug-and-play framework that introduces
multimodal late chunking to construct efficient, contextualized multi-vectors. Unlike existing
pruning or fixed-
token approaches, ColChunk employs hierarchical clustering on patch-level
embeddings, fused with a 2D position prior to ensure spatial-semantic coherence. This adaptive grouping allows for a content-aware representation that preserves global context while drastically reducing the vector count. Evaluations across 24 VDR
datasets demonstrate ColChunk achieves over a 90% reduction in storage requirements while simultaneously delivering a 9-point average improvement in nDCG@5 across representative single-vector models. ColChunk provides a practical solution for balancing
retrieval accuracy and efficiency in visual document systems.
Read the full abstract ▾
All authors Yibo Yan, Mingdong Ou, Yi Cao, Jiahao Huo, Xin Zou, Shuliang Liu, James Kwok, Xuming Hu
Computer Vision
Semantic Manipulation Localization
Zhenshan Tan, Chenhan Lu, Yuxiang Huang et al. (+6 more)
Image Manipulation Localization (IML) aims to identify edited regions in an image. However, with the increasing use of modern image editing and generative models, many manipulations no longer exhibit obvious low-level artifacts. Instead, they often involve subtle but meaning-altering edits to an object's attributes, state, or relationships while remaining highly consistent with the surrounding content. This makes conventional IML methods less effective because they mainly rely on artifact detection rather than semantic sensitivity. To address this issue, we introduce Semantic Manipulation Localization (SML), a new task that focuses on localizing subtle semantic edits that significantly change image interpretation. We further construct a dedicated fine-grained
benchmark for SML using a semantics-driven manipulation pipeline with pixel-level annotations. Based on this task, we propose TRACE (Targeted
Reasoning of Attributed Cognitive Edits), an end-to-end framework that models semantic sensitivity through three progressively coupled components: semantic anchoring, semantic perturbation sensing, and semantic-constrained
reasoning. Specifically, TRACE first identifies semantically meaningful regions that support image understanding, then injects perturbation-sensitive frequency cues to capture subtle edits under strong visual consistency, and finally verifies candidate regions through joint
reasoning over semantic content and semantic scope. Extensive experiments show that TRACE consistently outperforms existing IML methods on our
benchmark and produces more complete, compact, and semantically coherent localization results. These results demonstrate the necessity of moving beyond artifact-based localization and provide a new direction for image forensics in complex semantic editing scenarios.
Read the full abstract ▾
All authors Zhenshan Tan, Chenhan Lu, Yuxiang Huang, Ziwen He, Xiang Zhang, Yuzhe Sha, Xianyi Chen, Tianrun Chen, Zhangjie Fu
Computer Vision
Improving Deep Learning-Based Target Volume Auto-Delineation for Adaptive MR-Guided Radiotherapy in Head and Neck Cancer: Impact of a Volume-Aware Dice Loss
Sogand Beirami, Zahra Esmaeilzadeh, Ahmed Gomaa et al. (+9 more)
Background: Manual delineation of target volumes in head and neck cancer (HNC) remains a significant bottleneck in radiotherapy planning, characterized by high inter-observer variability and time consumption. This study evaluates the integration of a Volume-Aware (VA) Dice loss function into a self-configuring deep learning framework to enhance the auto-
segmentation of primary tumors (PT) and metastatic lymph nodes (LN) for adaptive MR-guided radiotherapy. We investigate how volume-sensitive weighting affects the detection of small, anatomically complex nodal metastases compared to conventional loss functions. Methods: Utilizing the HNTS-MRG 2024
dataset, we implemented an nnU-Net ResEnc M architecture. We conducted a multi-label
segmentation task, comparing a standard Dice loss baseline against two Volume-Aware configurations: a "Dual Mask" setup (VA loss on both PT and LN) and a "Selective LN Mask" setup (VA loss on LN only). Evaluation metrics included volumetric Dice scores, surface-based metrics (SDS, MSD, HD95), and lesion-wise binary detection sensitivity and precision. Results: The Selective LN Mask configuration achieved the highest LN Volumetric Dice Score (0.758 vs. 0.734 baseline) and significantly improved LN Lesion-Wise Detection Sensitivity (84.93% vs. 81.80%). However, a critical trade-off was observed; PT detection precision declined significantly in the selective setup (63.65% vs. 81.27%). The Dual Mask configuration provided the most balanced performance across both targets, maintaining primary tumor precision at 82.04% while improving LN sensitivity to 83.46%. Conclusions: A volume-sensitive loss function mitigated the under-representation of small metastatic lesions in HNC. While selective weighting yielded the best nodal detection, a dual-mask approach is required in multi-label tasks to maintain
segmentation accuracy for larger primary tumor volumes.
Read the full abstract ▾
All authors Sogand Beirami, Zahra Esmaeilzadeh, Ahmed Gomaa, Pluvio Stephan, Ishita Sheth, Thomas Weissmann, Juliane Szkitsak, Philipp Schubert, Yixing Huang, Annette Schwarz, Stefanie Corradini, Florian Putz
Computer Vision
VGA-Bench: A Unified Benchmark and Multi-Model Framework for Video Aesthetics and Generation Quality Evaluation
Longteng Jiang, DanDan Zheng, Qianqian Qiao et al. (+7 more)
The rapid advancement of AIGC-based
video generation has underscored the critical need for comprehensive evaluation frameworks that go beyond traditional generation quality metrics to encompass aesthetic appeal. However, existing
benchmarks remain largely focused on technical fidelity, leaving a significant gap in holistic assessment-particularly with respect to perceptual and artistic qualities. To address this limitation, we introduce VGA-Bench, a unified
benchmark for joint evaluation of
video generation quality and aesthetic quality. VGA-Bench is built upon a principled three-tier taxonomy: Aesthetic Quality, Aesthetic Tagging, and Generation Quality, each decomposed into multiple fine-grained sub-dimensions to enable systematic assessment. Guided by this taxonomy, we design 1,016 diverse
prompts and generate a large-scale
dataset of over 60,000
videos using 12
video generation models, ensuring broad coverage across content, style, and artifacts. To enable scalable and automated evaluation, we annotate a subset of the
dataset via human labeling and develop three dedicated multi-task neural assessors: VAQA-Net for aesthetic quality prediction, VTag-Net for automatic aesthetic tagging, and VGQA-Net for generation and basic quality attributes. Extensive experiments demonstrate that our models achieve reliable
alignment with human judgments, offering both accuracy and efficiency. We release VGA-Bench as a public
benchmark to foster research in AIGC evaluation, with applications in content moderation, model debugging, and generative model optimization.
Read the full abstract ▾
All authors Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang, Huaye Wang, Yihang Bo, Bao Peng, Jingdong Chen, Jun Zhou, Xin Jin
Computer Vision
PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit–Explicit Optimization
Dongli Wu, Jingyu Hu, Ka-Hei Hui et al. (+4 more)
Existing methods for single-image 3D indoor scene generation often produce visually plausible results that violate real-world physical constraints, limiting their reliability in robotics,
embodied AI, and virtual environment design. To bridge this gap, we introduce a Physics Evaluator that assesses four aspects (
geometric priors, contact, stability, and deployability) through nine sub-constraints, establishing a
benchmark for evaluating physical consistency. Our analysis reveals that even state-of-the-art methods produce scenes with substantial physical inconsistencies despite their visual plausibility. To address these limitations, we propose a framework that incorporates feedback from the Physics Evaluator into both training and inference to improve the physical plausibility of generated scenes. Specifically, PhyMix comprises two components: (i) implicit
alignment via Scene-GRPO, a critic-free group-relative preference optimization strategy that uses physical evaluations to guide the generator toward physically plausible layouts, and (ii) explicit refinement via verifier-guided agentic repair, which translates evaluator diagnostics into executable scene-editing skills to correct residual violations at inference time. PhyMix unifies physical evaluation, preference-based
alignment, and agentic inference-time correction to improve physical plausibility while retaining the observed scene content. Experiments show improvements in reconstruction quality and physical plausibility, while qualitative examples on stylized and real-world images illustrate the applicability of the framework beyond synthetic inputs.
Read the full abstract ▾
All authors Dongli Wu, Jingyu Hu, Ka-Hei Hui, Xiaobao Wei, Chengwen Luo, Jianqiang Li, Zhengzhe Liu
Computer Vision
A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection
Nojod M. Alotaibi and Areej M. Alhothali
Major depressive disorder (MDD) is a prevalent mental disorder associated with complex neurobiological changes that cannot be fully captured using a single imaging modality. The use of
multimodal magnetic resonance imaging (MRI) provides a more comprehensive understanding of
brain changes by combining structural and functional data. Despite this, the effective integration of these modalities remains challenging. In this study, we propose a dual cross-
attention-based
multimodal fusion framework that explicitly models bidirectional interactions between structural MRI (sMRI) and resting-state functional MRI (rs-fMRI) representations. The proposed approach is tested on the large-scale REST-meta-MDD
dataset using both structural and functional
brain atlas configurations. Numerous experiments conducted under a 10-fold stratified cross-validation demonstrated that the proposed fusion
algorithm achieves robust and competitive performance across all atlas types. The proposed method consistently outperforms conventional feature-level concatenation for functional atlases, while maintaining comparable performance for structural atlases. The most effective dual cross-
attention multimodal model obtained 84.71% accuracy, 86.42% sensitivity, 82.89% specificity, 84.34% precision, and 85.37% F1-score. These findings emphasize the importance of explicitly modeling cross-modal interactions for
multimodal neuroimaging-based MDD classification.
Read the full abstract ▾
Computer Vision
Dual-Branch Remote Sensing Infrared Image Super-Resolution
Xining Ge, Gengjia Chang, Weijun Yuan et al. (+6 more)
Remote sensing infrared image super-resolution aims to recover sharper thermal observations from low-resolution inputs while preserving target contours, scene layout, and radiometric stability. Unlike visible-image super-resolution, thermal imagery is weakly textured and more sensitive to unstable local sharpening, which makes complementary local and global modeling especially important. This paper presents our solution to the NTIRE 2026 Infrared Image Super-Resolution Challenge, a dual-branch system that combines a HAT-L branch and a MambaIRv2-L branch. The inference pipeline applies test-time local conversion on HAT, eight-way self-ensemble on MambaIRv2, and fixed equal-weight image-space fusion. We report both the official challenge score and a reproducible evaluation on 12 synthetic times-four thermal samples derived from Caltech Aerial RGB-Thermal, on which the fused output outperforms either single branch in PSNR, SSIM, and the overall Score. The results suggest that infrared super-resolution benefits from explicit complementarity between locally strong
transformer restoration and globally stable state-space modeling.
Read the full abstract ▾
All authors Xining Ge, Gengjia Chang, Weijun Yuan, Zhan Li, Zhanglu Chen, Boyang Yao, Yihang Chen, Yifan Deng, Shuhong Liu
Computer Vision
VGGT-HPE: Reframing Head Pose Estimation as Relative Pose Prediction
Vasiliki Vasileiou, Panagiotis P. Filntisis, Petros Maragos et al. (+1 more)
Monocular head pose estimation is traditionally formulated as direct regression from a single image to an absolute pose. This paradigm forces the network to implicitly internalize a
dataset-specific canonical reference frame. In this work, we argue that predicting the relative rigid transformation between two observed head configurations is a fundamentally easier and more robust formulation. We introduce VGGT-HPE, a relative head pose
estimator built upon a general-purpose geometry foundation model. Finetuned exclusively on synthetic facial renderings, our method sidesteps the need for an implicit anchor by reducing the problem to estimating a
geometric displacement from an explicitly provided anchor with a known pose. As a practical benefit, the relative formulation also allows the anchor to be chosen at test time - for instance, a near-neutral frame or a temporally adjacent one - so that the prediction difficulty can be controlled by the application. Despite zero real-world training data, VGGT-HPE achieves state-of-the-art results on the BIWI
benchmark, outperforming established absolute regression methods trained on mixed and real
datasets. Through controlled easy- and hard-pair
benchmarks, we also systematically validate our core hypothesis: relative prediction is intrinsically more accurate than absolute regression, with the advantage scaling alongside the difficulty of the target pose. Project page and code: https://vasilikivas.github.io/VGGT-HPE
Read the full abstract ▾
All authors Vasiliki Vasileiou, Panagiotis P. Filntisis, Petros Maragos, Kostas Daniilidis
Computer Vision
Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation
Ruibin Li, Tao Yang, Fangzhou Ai et al. (+4 more)
Streaming
video generation (SVG) distills a pretrained bidirectional
video diffusion model into an autoregressive model equipped with sliding window
attention (SWA). However, SWA inevitably loses distant history during long
video generation, and its computational overhead remains a critical challenge to real-time deployment. In this work, we propose Hybrid Forcing, which jointly optimizes temporal information retention and computational efficiency through a hybrid
attention design. First, we introduce lightweight linear temporal
attention to preserve long-range dependencies beyond the sliding window. In particular, we maintain a compact key-value state to incrementally absorb evicted
tokens, retaining temporal context with negligible
memory and computational overhead. Second, we incorporate block-sparse
attention into the local sliding window to reduce redundant computation within short-range modeling, reallocating computational capacity toward more critical dependencies. Finally, we introduce a decoupled
distillation strategy tailored to the hybrid
attention design. A few-step initial
distillation is performed under dense
attention, then the
distillation of our proposed linear temporal and block-sparse
attention is activated for streaming modeling, ensuring stable optimization. Extensive experiments on both short- and long-form
video generation
benchmarks demonstrate that Hybrid Forcing consistently achieves state-of-the-art performance. Notably, our model achieves real-time, unbounded 832x480
video generation at 29.5 FPS on a single NVIDIA H100 GPU without quantization or model compression. The source code and trained models are available at https://github.com/leeruibin/hybrid-forcing.
Read the full abstract ▾
All authors Ruibin Li, Tao Yang, Fangzhou Ai, Tianhe Wu, Shilei Wen, Bingyue Peng, Lei Zhang
Computer Vision
Degradation-Consistent Paired Training for Robust AI-Generated Image Detection
Zongyou Yang, Yinghan Hou and Xiaokun Yang
AI-generated image detectors suffer significant performance degradation under real-world image corruptions such as JPEG compression, Gaussian blur, and resolution downsampling. We observe that state-of-the-art methods, including B-Free, treat degradation robustness as a byproduct of data augmentation rather than an explicit training objective. In this work, we propose Degradation-Consistent Paired Training (DCPT), a simple yet effective training strategy that explicitly enforces robustness through paired consistency constraints. For each training image, we construct a clean view and a degraded view, then impose two constraints: a feature consistency loss that minimizes the cosine distance between clean and degraded representations, and a prediction consistency loss based on symmetric KL divergence that aligns output distributions across views. DCPT adds zero additional parameters and zero inference overhead. Experiments on the Synthbuster
benchmark (9 generators, 8 degradation conditions) demonstrate that DCPT improves the degraded-condition average accuracy by 9.1 percentage points compared to an identical baseline without paired training, while sacrificing only 0.9% clean accuracy. The improvement is most pronounced under JPEG compression (+15.7% to +17.9%). Ablation further reveals that adding architectural components leads to overfitting on limited training data, confirming that training objective improvement is more effective than architectural augmentation for degradation robustness.
Read the full abstract ▾
Computer Vision
ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents
Dongjie Huo, Haoyun Liu, Guoqing Liu et al. (+10 more)
Current
embodied intelligent systems still face a substantial gap between high-level
reasoning and low-level physical execution in open-world environments. Although
Vision-
Language-Action (VLA) models provide strong perception and intuitive responses, their open-loop nature limits long-horizon performance.
Agents incorporating System 2 cognitive mechanisms improve planning, but usually operate in closed sandboxes with predefined toolkits and limited real-system control. OpenClaw provides a localized runtime with full system privileges, but lacks the
embodied control architecture required for long-duration, multi-
robot execution. We therefore propose ABot-Claw, an
embodied extension of OpenClaw that integrates: 1) a unified embodiment interface with capability-driven scheduling for heterogeneous
robot coordination; 2) a visual-centric cross-embodiment
multimodal memory for persistent context retention and grounded
retrieval; and 3) a critic-based closed-loop feedback mechanism with a generalist reward model for online progress evaluation, local correction, and replanning. With a decoupled architecture spanning the OpenClaw layer, shared service layer, and
robot embodiment layer, ABot-Claw enables real-world interaction, closes the loop from natural
language intent to physical action, and supports progressively self-evolving robotic
agents in open, dynamic environments.
Read the full abstract ▾
All authors Dongjie Huo, Haoyun Liu, Guoqing Liu, Dekang Qi, Zhiming Sun, Maoguo Gao, Jianxin He, Yandan Yang, Xinyuan Chang, Feng Xiong, Xing Wei, Zhiheng Ma, Mu Xu
Computer Vision
Mining Attribute Subspaces for Efficient Fine-tuning of 3D Foundation Models
Yu Jiang, Hanwen Jiang, Ahmed Abdelkader et al. (+4 more)
With the emergence of 3D foundation models, there is growing interest in fine-tuning them for downstream tasks, where LoRA is the dominant fine-tuning paradigm. As 3D
datasets exhibit distinct variations in texture, geometry, camera motion, and lighting, there are interesting fundamental questions: 1) Are there LoRA subspaces associated with each type of variation? 2) Are these subspaces disentangled (i.e., orthogonal to each other)? 3) How do we compute them effectively? This paper provides answers to all these questions. We introduce a robust approach that generates synthetic
datasets with controlled variations, fine-tunes a LoRA adapter on each
dataset, and extracts a LoRA sub-space associated with each type of variation. We show that these subspaces are approximately disentangled. Integrating them leads to a reduced LoRA subspace that enables efficient LoRA fine-tuning with improved prediction accuracy for downstream tasks. In particular, we show that such a reduced LoRA subspace, despite being derived entirely from synthetic data, generalizes to real
datasets. An ablation study validates the effectiveness of the choices in our approach.
Read the full abstract ▾
All authors Yu Jiang, Hanwen Jiang, Ahmed Abdelkader, Wen-Sheng Chu, Brandon Y. Feng, Zhangyang Wang, Qixing Huang
Computer Vision
Global monitoring of methane point sources using deep learning on hyperspectral radiance measurements from EMIT
Vishal V. Batchu, Michelangelo Conserva, Alex Wilson et al. (+5 more)
Anthropogenic methane (CH4) point sources are critical drivers of near-term
climate forcing, safety hazards, and system-inefficiencies. Space-based imaging spectroscopy is an emerging tool for identifying emissions globally, but existing approaches largely rely on manual plume identification. Here, we present the Methane Analysis and Plume Localization with EMIT (MAPL-EMIT) model, an end-to-end
vision transformer framework that leverages the complete radiance spectrum from the Earth Surface Mineral
Dust Source Investigation (EMIT) instrument to jointly retrieve methane enhancements across all pixels within a scene. This approach brings together spectral information with spatial context to significantly lower detection limits. MAPL-EMIT simultaneously supports enhancement quantification, plume delineation, and source localization, even for overlapping plumes. The model was trained on 3.6 million physics-based synthetic plumes injected into global EMIT radiance data. Synthetic evaluation confirms the model's ability to identify plumes with high recall and precision and to capture weaker plumes relative to existing matched-filter approaches. On real-world
benchmarks, MAPL-EMIT captures 84% of known hand-annotated NASA-L2B plume complexes across a test set of 1084 EMIT granules, while capturing roughly 1.5 times as many plausible plumes compared to human analysts. Further verification against coincident airborne data, top-emitting landfills, and controlled release experiments confirms the model's ability to identify previously uncaptured sources. By incorporating model-generated metrics such as spectral fit scores and estimated noise levels, the framework can limit false-positives. Overall, MAPL-EMIT enables high-throughput implementation on the full EMIT data catalog, shifting methane monitoring to a rapid, scalable paradigm for global plume mapping at the facility scale.
Read the full abstract ▾
All authors Vishal V. Batchu, Michelangelo Conserva, Alex Wilson, Anna M. Michalak, Varun Gulshan, Philip G. Brodrick, Andrew K. Thorpe, Christopher V. Arsdale
Computer Vision
Particle Diffusion Matching: Random Walk Correspondence Search for the Alignment of Standard and Ultra-Widefield Fundus Images
Kanggeon Lee, Soochahn Lee and Kyoung Mu Lee
We propose a robust
alignment technique for Standard Fundus Images (SFIs) and Ultra-Widefield Fundus Images (UWFIs), which are challenging to align due to differences in scale, appearance, and the scarcity of distinctive features. Our method, termed Particle
Diffusion Matching (PDM), performs
alignment through an iterative Random Walk Correspondence Search (RWCS) guided by a
diffusion model. At each iteration, the model estimates displacement vectors for particle points by considering local appearance, the structural distribution of particles, and an estimated global transformation, enabling progressive refinement of correspondences even under difficult conditions. PDM achieves state-of-the-art performance across multiple retinal image
alignment benchmarks, showing substantial improvement on a primary
dataset of SFI-UWFI pairs and demonstrating its effectiveness in real-world clinical scenarios. By providing accurate and scalable correspondence estimation, PDM overcomes the limitations of existing methods and facilitates the integration of complementary retinal image modalities. This
diffusion-guided search strategy offers a new direction for improving downstream supervised learning, disease diagnosis, and multi-modal image analysis in ophthalmology.
Read the full abstract ▾
Computer Vision
Active Diffusion Matching: Score-based Iterative Alignment of Cross-Modal Retinal Images
Kanggeon Lee, Su Jeong Song, Soochahn Lee et al. (+1 more)
Objective: The study aims to address the challenge of aligning Standard Fundus Images (SFIs) and Ultra-Widefield Fundus Images (UWFIs), which is difficult due to their substantial differences in viewing range and the amorphous appearance of the retina. Currently, no specialized method exists for this task, and existing image
alignment techniques lack accuracy. Methods: We propose Active
Diffusion Matching (ADM), a novel cross-modal
alignment method. ADM integrates two interdependent score-based
diffusion models to jointly estimate global transformations and local deformations via an iterative Langevin Markov chain. This approach facilitates a
stochastic, progressive search for optimal
alignment. Additionally, custom sampling strategies are introduced to enhance the adaptability of ADM to given input image pairs. Results: Comparative experimental evaluations demonstrate that ADM achieves state-of-the-art
alignment accuracy. This was validated on two
datasets: a private
dataset of SFI-UWFI pairs and a public
dataset of SFI-SFI pairs, with mAUC improvements of 5.2 and 0.4 points on the private and public
datasets, respectively, compared to existing state-of-the-art methods. Conclusion: ADM effectively bridges the gap in aligning SFIs and UWFIs, providing an innovative solution to a previously unaddressed challenge. The method's ability to jointly optimize global and local
alignment makes it highly effective for cross-modal image
alignment tasks. Significance: ADM has the potential to transform the integrated analysis of SFIs and UWFIs, enabling better clinical utility and supporting learning-based image enhancements. This advancement could significantly improve diagnostic accuracy and patient outcomes in ophthalmology.
Read the full abstract ▾
All authors Kanggeon Lee, Su Jeong Song, Soochahn Lee, Kyoung Mu Lee
Computer Vision
MatRes: Zero-Shot Test-Time Model Adaptation for Simultaneous Matching and Restoration
Kanggeon Lee, Soochahn Lee and Kyoung Mu Lee
Real-world image pairs often exhibit both severe degradations and large viewpoint changes, making image restoration and
geometric matching mutually interfering tasks when treated independently. In this work, we propose MatRes, a zero-shot test-time adaptation framework that jointly improves restoration quality and correspondence estimation using only a single low-quality and high-quality image pair. By enforcing conditional similarity at corresponding locations, MatRes updates only lightweight modules while keeping all pretrained components frozen, requiring no offline training or additional supervision. Extensive experiments across diverse combinations show that MatRes yields significant gains in both restoration and
geometric alignment compared to using either restoration or matching models alone. MatRes offers a practical and widely applicable solution for real-world scenarios where users commonly capture multiple images of a scene with varying viewpoints and quality, effectively addressing the often-overlooked mutual interference between matching and restoration.
Read the full abstract ▾
Computer Vision
Attention-Guided Dual-Stream Learning for Group Engagement Recognition: Fusing Transformer-Encoded Motion Dynamics with Scene Context via Adaptive Gating
Saniah Kayenat Chowdhury and Muhammad E.H. Chowdhury
Student engagement is crucial for improving learning outcomes in group activities. Highly engaged students perform better both individually and contribute to overall group success. However, most existing automated engagement recognition methods are designed for online classrooms or estimate engagement at the individual level. Addressing this gap, we propose DualEngage, a novel two-stream framework for group-level engagement recognition from in-classroom
videos. It models engagement as a joint function of both individual and group-level behaviors. The primary stream models person-level motion dynamics by detecting and tracking students, extracting dense optical flow with the Recurrent All-Pairs Field Transforms network, encoding temporal motion patterns using a
transformer encoder, and finally aggregating per-student representations through
attention pooling into a unified representation. The secondary stream captures scene-level spatiotemporal information from the full
video clip, leveraging a pretrained three-dimensional Residual Network. The two-stream representations are combined via softmax-gated fusion, which dynamically weights each stream's contribution based on the joint context of both features. DualEngage learns a joint representation of individual actions with overarching group dynamics. We evaluate the proposed approach using fivefold cross-validation on the Classroom Group Engagement
Dataset developed by Ocean University of China, achieving an average classification accuracy of 0.9621+/-0.0161 with a macro-averaged F1 of 0.9530+/-0.0204. To understand the contribution of each branch, we further conduct an ablation study comparing single-stream variants against the two-stream model. This work is among the first in classroom engagement recognition to adopt a dual-stream design that explicitly leverages motion cues as an
estimator.
Read the full abstract ▾
Computer Vision
DocRevive: A Unified Pipeline for Document Text Restoration
Kunal Purkayastha, Ayan Banerjee, Josep Llados et al. (+1 more)
In Document Understanding, the challenge of reconstructing damaged, occluded, or incomplete text remains a critical yet unexplored problem. Subsequent document understanding tasks can benefit from a document reconstruction process. In response, this paper presents a novel unified pipeline combining state-of-the-art Optical Character Recognition (OCR), advanced image analysis, masked
language modeling, and
diffusion-based models to restore and reconstruct text while preserving visual integrity. We create a synthetic
dataset of 30{,}078 degraded document images that simulates diverse document degradation scenarios, setting a
benchmark for restoration tasks. Our pipeline detects and recognizes text, identifies degradation with an occlusion detector, and uses an inpainting model for semantically coherent reconstruction. A
diffusion-based module seamlessly reintegrates text, matching font, size, and
alignment. To evaluate restoration quality, we propose a Unified Context Similarity Metric (UCSM), incorporating edit, semantic, and length similarities with a contextual predictability measure that penalizes deviations when the correct text is contextually obvious. Our work advances document restoration, benefiting archival research and digital preservation while setting a new standard for text reconstruction. The OPRB
dataset and code are available at \href{https://huggingface.co/
datasets/kpurkayastha/OPRB}{Hugging Face} and \href{https://github.com/kunalpurkayastha/DocRevive}{Github} respectively.
Read the full abstract ▾
All authors Kunal Purkayastha, Ayan Banerjee, Josep Llados, Umapada Pal
Computer Vision
Spotlight and Shadow: Attention-Guided Dual-Anchor Introspective Decoding for MLLM Hallucination Mitigation
Yebo Wu, Han Jin, Zhijiang Guo et al. (+1 more)
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable
reasoning capabilities yet continue to suffer from
hallucination, where generated text contradicts visual content. In this paper, we introduce Dual-Anchor Introspective Decoding (DaID), a novel contrastive decoding framework that dynamically calibrates each
token generation by mining the model's internal perceptual discrepancies. Specifically, DaID identifies a Spotlight layer to amplify visual factual signals and a Shadow layer to suppress textual inertia. By leveraging visual
attention distributions to guide this dual-anchor selection process, our method ensures precise,
token-specific adaptation. Experimental results across multiple
benchmarks and MLLMs demonstrate that DaID significantly mitigates
hallucination while enhancing general
reasoning capabilities.
Read the full abstract ▾
All authors Yebo Wu, Han Jin, Zhijiang Guo, Li Li
Computer Vision
Armin Gerami, Seyedehanita Madani and Ramani Duraiswami
Multimodal Transformers serve as the backbone for state-of-the-art
vision-
language models, yet their quadratic
attention complexity remains a critical barrier to scalability. In this work, we investigate the viability of Linear
Attention (LA) as a high-efficiency alternative within
multimodal frameworks. By integrating LA, we reduce the computational overhead from quadratic to linear relative to sequence length while preserving competitive performance. We evaluate our approach across ViT-S/16, ViT-B/16, and ViT-L/16 architectures trained on the LAION-400M
dataset, with validation focused on ImageNet-21K zero-shot accuracy. Our systematic evaluation demonstrates that Linear
Attention not only yields significant computational savings but also adheres to the same scaling laws as standard softmax
attention. These findings position Linear
Attention as a robust, scalable solution for next-generation
multimodal Transformers tasked with processing increasingly large and complex
datasets.
Read the full abstract ▾
Computer Vision
U$^{2}$Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation
Xunpei Sun, Wenwei Lin, Yi Chang et al. (+1 more)
Unsupervised optical flow methods typically lack reliable uncertainty estimation, limiting their robustness and interpretability. We propose U$^{2}$Flow, the first recurrent unsupervised framework that jointly estimates optical flow and per-pixel uncertainty. The core innovation is a decoupled learning strategy that derives uncertainty supervision from augmentation consistency via a Laplace-based maximum likelihood objective, enabling stable training without ground truth. The predicted uncertainty is further integrated into the network to guide adaptive flow refinement and dynamically modulate the regional smoothness loss. Furthermore, we introduce an uncertainty-guided bidirectional flow fusion mechanism that enhances robustness in challenging regions. Extensive experiments on KITTI and Sintel demonstrate that U$^{2}$Flow achieves state-of-the-art performance among unsupervised methods while producing highly reliable uncertainty maps, validating the effectiveness of our joint estimation paradigm. The code is available at https://github.com/sunzunyi/U2FLOW.
Read the full abstract ▾
All authors Xunpei Sun, Wenwei Lin, Yi Chang, Gang Chen
Computer Vision
Intra-finger Variability of Diffusion-based Latent Fingerprint Generation
Noor Hussein, Anil K. Jain and Karthik Nandakumar
The primary goal of this work is to systematically evaluate the intra-finger variability of synthetic fingerprints (particularly latent prints) generated using a state-of-the-art
diffusion model. Specifically, we focus on enhancing the latent style diversity of the generative model by constructing a comprehensive latent style bank curated from seven diverse
datasets, which enables the precise synthesis of latent prints with over 40 distinct styles encapsulating different surfaces and processing techniques. We also implement a semi-automated framework to understand the integrity of fingerprint ridges and minutiae in the generated impressions. Our analysis indicates that though the generation process largely preserves the identity, a small number of local inconsistencies (addition and removal of minutiae) are introduced, especially when there are poor quality regions in the reference image. Furthermore, mismatch between the reference image and the chosen style
embedding that guides the generation process introduces global inconsistencies in the form of hallucinated ridge patterns. These insights highlight the limitations of existing synthetic fingerprint generators and the need to further improve these models to simultaneously enhance both diversity and identity consistency.
Read the full abstract ▾