Computer Vision
On the Robustness of Diffusion-Based Image Compression to Bit-Flip Errors
Amit Vaisman, Gal Pomerants and Raz Lapid
Modern image compression methods are typically optimized for the rate–distortion–perception trade-off, whereas their robustness to bit-level corruption is rarely examined. We show that
diffusion-based compressors built on the Reverse Channel Coding (RCC) paradigm are substantially more robust to bit flips than classical and learned codecs. We further introduce a more robust variant of Turbo-DDCM that significantly improves robustness while only minimally affecting the rate–distortion–perception trade-off. Our findings suggest that RCC-based compression can yield more resilient compressed representations, potentially reducing reliance on error-correcting codes in highly noisy environments.
Read the full abstract ▾
Computer Vision
ASSR-Net: Anisotropic Structure-Aware and Spectrally Recalibrated Network for Hyperspectral Image Fusion
Qiya Song, Hongzhi Zhou, Lishan Tan et al. (+2 more)
Hyperspectral image fusion aims to reconstruct high-spatial-resolution hyperspectral images (HR-HSI) by integrating complementary information from multi-source inputs. Despite recent progress, existing methods still face two critical challenges: (1) inadequate reconstruction of anisotropic spatial structures, resulting in blurred details and compromised spatial quality; and (2) spectral distortion during fusion, which hinders fine-grained spectral representation. To address these issues, we propose ASSR-Net: an Anisotropic Structure-Aware and Spectrally Recalibrated Network for Hyperspectral Image Fusion. ASSR-Net adopts a two-stage fusion strategy comprising anisotropic structure-aware spatial enhancement (ASSE) and hierarchical prior-guided spectral calibration (HPSC). In the first stage, a directional perception fusion module adaptively captures structural features along multiple orientations, effectively reconstructing anisotropic spatial patterns. In the second stage, a spectral recalibration module leverages the original low-resolution HSI as a spectral prior to explicitly correct spectral deviations in the fused results, thereby enhancing spectral fidelity. Extensive experiments on various
benchmark datasets demonstrate that ASSR-Net consistently outperforms state-of-the-art methods, achieving superior spatial detail preservation and spectral consistency.
Read the full abstract ▾
All authors Qiya Song, Hongzhi Zhou, Lishan Tan, Renwei Dian, Shutao Li
Computer Vision
FoleyDesigner: Immersive Stereo Foley Generation with Precise Spatio-Temporal Alignment for Film Clips
Mengtian Li, Kunyan Dai, Yi Ding et al. (+4 more)
Foley art plays a pivotal role in enhancing immersive auditory experiences in film, yet manual creation of spatio-temporally aligned audio remains labor-intensive. We propose FoleyDesigner, a novel framework inspired by professional Foley workflows, integrating film clip analysis, spatio-temporally controllable Foley generation, and professional audio mixing capabilities. FoleyDesigner employs a multi-
agent architecture for precise spatio-temporal analysis. It achieves spatio-temporal
alignment through latent
diffusion models trained on spatio-temporal cues extracted from
video frames, combined with large
language model (LLM)-driven hybrid mechanisms that emulate post-production practices in film industry. To address the lack of high-quality stereo audio
datasets in film, we introduce FilmStereo, the first professional stereo audio
dataset containing spatial metadata, precise timestamps, and semantic annotations for eight common Foley categories. For applications, the framework supports interactive user control while maintaining seamless integration with professional pipelines, including 5.1-channel Dolby Atmos systems compliant with ITU-R BS.775 standards, thereby offering extensive creative flexibility. Extensive experiments demonstrate that our method achieves superior spatio-temporal
alignment compared to existing baselines, with seamless compatibility with professional film production standards. The project page is available at https://gekiii996.github.io/FoleyDesigner/ .
Read the full abstract ▾
All authors Mengtian Li, Kunyan Dai, Yi Ding, Ruobing Ni, Ying Zhang, Wenwu Wang, Zhifeng Xie
Computer Vision
Single-Stage Signal Attenuation Diffusion Model for Low-Light Image Enhancement and Denoising
Ying Liu, Junchao Zhang and Caiyun Wu
Diffusion models excel at image restoration via probabilistic modeling of forward noise addition and reverse denoising, and their ability to handle complex noise while preserving fine details makes them well-suited for Low-Light Image Enhancement (LLIE). Mainstream
diffusion based LLIE methods either adopt a two-stage pipeline or an auxiliary correction network to refine U-Net outputs, which severs the intrinsic link between enhancement and denoising and leads to suboptimal performance owing to inconsistent optimization objectives. To address these issues, we propose the Signal Attenuation
Diffusion Model (SADM), a novel
diffusion process that integrates the signal attenuation mechanism into the
diffusion pipeline, enabling simultaneous brightness adjustment and noise suppression in a single stage. Specifically, the signal attenuation coefficient simulates the inherent signal attenuation of low-light degradation in the forward noise addition process, encoding the physical priors of low-light degradation to explicitly guide reverse denoising toward the concurrent optimization of brightness recovery and noise suppression, thereby eliminating the need for extra correction modules or staged training relied on by existing methods. We validate that our design maintains consistency with Denoising
Diffusion Implicit Models(DDIM) via multi-scale pyramid sampling, balancing interpretability, restoration quality, and computational efficiency.
Read the full abstract ▾
Computer Vision
Beyond Semantics: Disentangling Information Scope in Sparse Autoencoders for CLIP
Yusung Ro, Jaehyun Choi and Junmo Kim
Sparse Autoencoders (SAEs) have emerged as a powerful tool for interpreting the internal representations of CLIP
vision encoders, yet existing analyses largely focus on the semantic meaning of individual features. We introduce information scope as a complementary dimension of interpretability that characterizes how broadly an SAE feature aggregates visual evidence, ranging from localized, patch-specific cues to global, image-level signals. We observe that some SAE features respond consistently across spatial perturbations, while others shift unpredictably with minor input changes, indicating a fundamental distinction in their underlying scope. To quantify this, we propose the Contextual Dependency Score (CDS), which separates positionally stable local scope features from positionally variant global scope features. Our experiments show that features of different information scopes exert systematically different influences on CLIP's predictions and confidence. These findings establish information scope as a critical new axis for understanding CLIP representations and provide a deeper diagnostic view of SAE-derived features.
Read the full abstract ▾
Computer Vision
GaussianGrow: Geometry-aware Gaussian Growing from 3D Point Clouds with Text Guidance
Weiqi Zhang, Junsheng Zhou, Haotian Geng et al. (+4 more)
3D Gaussian Splatting has demonstrated superior performance in rendering efficiency and quality, yet the generation of 3D Gaussians still remains a challenge without proper
geometric priors. Existing methods have explored predicting point maps as
geometric references for inferring Gaussian primitives, while the unreliable estimated geometries may lead to poor generations. In this work, we introduce GaussianGrow, a novel approach that generates 3D Gaussians by learning to grow them from easily accessible 3D point clouds, naturally enforcing
geometric accuracy in Gaussian generation. Specifically, we design a text-guided Gaussian growing scheme that leverages a multi-view
diffusion model to synthesize consistent appearances from input point clouds for supervision. To mitigate artifacts caused by fusing neighboring views, we constrain novel views generated at non-preset camera poses identified in overlapping regions across different views. For completing the hard-to-observe regions, we propose to iteratively detect the camera pose by observing the largest un-grown regions in point clouds and inpainting them by inpainting the rendered view with a pretrained 2D
diffusion model. The process continues until complete Gaussians are generated. We extensively evaluate GaussianGrow on text-guided Gaussian generation from synthetic and even real-scanned point clouds. Project Page: https://weiqi-zhang.github.io/GaussianGrow
Read the full abstract ▾
All authors Weiqi Zhang, Junsheng Zhou, Haotian Geng, Kanle Shi, Shenkun Xu, Yi Fang, Yu-Shen Liu
Computer Vision
MPM: Mutual Pair Merging for Efficient Vision Transformers
Simon Ravé, Pejman Rasti and David Rousseau
Decreasing sequence length is a common way to accelerate
transformers, but prior
token reduction work often targets classification and reports proxy metrics rather than end-to-end latency. For semantic
segmentation,
token reduction is further constrained by the need to reconstruct dense, pixel-aligned features, and on modern accelerators the overhead of computing merge maps can erase expected gains. We propose Mutual Pair Merging (MPM), a training-free
token aggregation module that forms mutual nearest-neighbor pairs in cosine space, averages each pair, and records a merge map enabling a gather-based reconstruction before the decoder so that existing
segmentation heads can be used unchanged. MPM introduces no learned parameters and no continuous compression knob (no keep-rate or threshold). The speed-accuracy trade-off is set by a discrete insertion schedule. We
benchmark end-to-end latency on an NVIDIA H100 GPU (with and without FlashAttention-2) and a Raspberry Pi 5 across standard
segmentation datasets. On ADE20K, MPM reduces per-image latency by up to 60% for ViT-Tiny on Raspberry Pi 5, and increases throughput by up to 20% on H100 with FlashAttention-2 while keeping the mIoU drop below 3%. These results suggest that simple, reconstruction-aware, training-free
token merging can translate into practical wall-clock gains for
segmentation when overhead is explicitly accounted for.
Read the full abstract ▾
Computer Vision
In Depth We Trust: Reliable Monocular Depth Supervision for Gaussian Splatting
Wenhui Xiao, Ethan Goan, Rodrigo Santa Cruz et al. (+4 more)
Using accurate depth priors in 3D Gaussian Splatting helps mitigate artifacts caused by sparse training data and textureless surfaces. However, acquiring accurate depth maps requires specialized acquisition systems. Foundation
monocular depth estimation models offer a cost-effective alternative, but they suffer from scale ambiguity, multi-view inconsistency, and local
geometric inaccuracies, which can degrade rendering performance when applied naively. This paper addresses the challenge of reliably leveraging
monocular depth priors for Gaussian Splatting (GS) rendering enhancement. To this end, we introduce a training framework integrating scale-ambiguous and noisy depth priors into
geometric supervision. We highlight the importance of learning from weakly aligned depth variations. We introduce a method to isolate ill-posed geometry for selective
monocular depth regularization, restricting the propagation of depth inaccuracies into well-reconstructed 3D structures. Extensive experiments across diverse
datasets show consistent improvements in
geometric accuracy, leading to more faithful depth estimation and higher rendering quality across different GS variants and
monocular depth backbones tested.
Read the full abstract ▾
All authors Wenhui Xiao, Ethan Goan, Rodrigo Santa Cruz, David Ahmedt-Aristizabal, Olivier Salvado, Clinton Fookes, Leo Lebrat
Computer Vision
Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs
Chongyu Wang, Ting Huang, Chunyu Sun et al. (+3 more)
Multimodal Large
Language Models (MLLMs) have achieved remarkable progress in 2D visual tasks but still struggle to understand physical space in real-world visual streams. Recently, feed-forward
geometric foundation models that implicitly extract
geometric priors from visual inputs have offered a promising direction. However, many existing geometry-aware MLLMs extract features from a single deep encoder layer and perform one-shot fusion at the
language-model input, potentially overlooking complementary information across
geometric encoder layers, limiting access to fine-grained
geometric cues, and hindering progressive cross-modal adaptation. To address these limitations, we propose GUIDE (
Geometric Unrolling Inside MLLM Early-layers), a progressive framework for integrating
geometric priors. GUIDE samples and aligns multi-level features from the
geometric encoder and sequentially injects them into the early layers of the MLLM. This design enables the MLLM to continuously access multi-granularity
geometric cues and progressively integrate them with visual representations during layer-wise semantic abstraction. GUIDE further introduces a dual context-aware gating mechanism at both the
token and layer levels, selectively regulating
geometric information to reduce redundant injection and interference with pretrained representations. Extensive experiments on spatial
reasoning and 3D scene understanding
benchmarks, including VSI-Bench, ScanRefer, and Scan2Cap, validate the effectiveness and cross-task applicability of GUIDE. Our 5B and 9B models achieve average scores of 71.5 and 72.1 on VSI-Bench, respectively.
Read the full abstract ▾
All authors Chongyu Wang, Ting Huang, Chunyu Sun, Xinyu Ning, Di Wang, Hao Tang
Computer Vision
CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration
Xuecong Liu, Mengzhu Ding, Zixuan Sun et al. (+2 more)
We present Consistent-Recurrent Feature Flow
Transformer (CRFT), a unified coarse-to-fine framework based on feature flow learning for robust cross-modal image registration. CRFT learns a modality-independent feature flow representation within a
transformer-based architecture that jointly performs feature
alignment and flow estimation. The coarse stage establishes global correspondences through multi-scale feature correlation, while the fine stage refines local details via hierarchical feature fusion and adaptive spatial
reasoning. To enhance
geometric adaptability, an iterative discrepancy-guided
attention mechanism with a Spatial
Geometric Transform (SGT) recurrently refines the flow field, progressively capturing subtle spatial inconsistencies and enforcing feature-level consistency. This design enables accurate
alignment under large affine and scale variations while maintaining structural coherence across modalities. Extensive experiments on diverse cross-modal
datasets demonstrate that CRFT consistently outperforms state-of-the-art registration methods in both accuracy and robustness. Beyond registration, CRFT provides a generalizable paradigm for
multimodal spatial correspondence, offering broad applicability to remote sensing, autonomous
navigation, and medical imaging. Code and
datasets are publicly available at https://github.com/NEU-Liuxuecong/CRFT.
Read the full abstract ▾
All authors Xuecong Liu, Mengzhu Ding, Zixuan Sun, Zhang Li, Xichao Teng
Computer Vision
3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models
Xinye Zheng, Fei Wang, Yiqi Nie et al. (+5 more)
Reconstructing 3D scenes from smoke-degraded multi-view images is particularly difficult because smoke introduces strong scattering effects, view-dependent appearance changes, and severe degradation of cross-view consistency. To address these issues, we propose a framework that integrates visual priors with efficient 3D scene modeling. We employ Nano-Banana-Pro to enhance smoke-degraded images and provide clearer visual observations for reconstruction and develop Smoke-GS, a medium-aware 3D Gaussian Splatting framework for smoke scene reconstruction and restoration-oriented novel view synthesis. Smoke-GS models the scene using explicit 3D Gaussians and introduces a lightweight view-dependent medium branch to capture direction-dependent appearance variations caused by smoke. Our method preserves the rendering efficiency of 3D Gaussian Splatting while improving robustness to smoke-induced degradation. Results demonstrate the effectiveness of our method for generating consistent and visually clear novel views in challenging smoke environments.
Read the full abstract ▾
All authors Xinye Zheng, Fei Wang, Yiqi Nie, Kun Li, Junjie Chen, Jiaqi Zhao, Yanyan Wei, Zhiliang Wu
Computer Vision
SnapFlow: One-Step Action Generation for Flow-Matching VLAs via Progressive Self-Distillation
Wuyang Luan, Junhui Li, Weiguang Zhao et al. (+2 more)
Vision-
language-action (VLA) models with flow-matching action heads, such as {\pi}0 and {\pi}0.5, have become a standard recipe for generalist
robot manipulation, but every action chunk is produced by integrating a learned velocity field over about ten sequential action-expert evaluations, so action generation dominates model latency and caps the achievable control frequency. Existing remedies either shrink each evaluation through layer
distillation or
pruning while leaving the sequential solver in place, or import one-step objectives from image generation that were designed for training from scratch, rely on an exponential-moving-average teacher or an external critic, and regress finite-span targets onto the conditional velocity, whose variance around the marginal field biases the learned map. We introduce SnapFlow, a self-
distillation method that adapts a pretrained flow-matching VLA to one-step action generation. SnapFlow mixes local flow-matching supervision with shortcut samples whose target is the model's own two-step Euler endpoint under stop-gradient, and adds a single zero-initialized destination-time projection so that one action expert serves both objectives while the pretrained predictor is preserved at initialization; no separate teacher network or critic is required. On {\pi}0.5, whose 10-NFE teacher operates near ceiling, the archived one-NFE SnapFlow checkpoint reaches 98.75% LIBERO success versus 97.75% for the teacher; under a separate paired evaluation of {\pi}0, the one-NFE model surpasses its teacher by 10.5 percentage points. SnapFlow also lowers mean and tail action error on held-out data, cuts batch-one model latency by about 3.3x, and transfers to shelf
retrieval on a dual-arm
humanoid.
Read the full abstract ▾
All authors Wuyang Luan, Junhui Li, Weiguang Zhao, Rui Ma, Tieru Wu
Computer Vision
Probing Intrinsic Medical Task Relationships: A Contrastive Learning Perspective
Jonas Muth, Zdravko Marinov and Simon Reiß
While much of the medical computer
vision community has focused on advancing performance for specific tasks, the underlying relationships between tasks, i.e., how they relate, overlap, or differ on a representational level, remain largely unexplored. Our work explores these intrinsic relationships between medical
vision tasks, specifically, we investigate 30 tasks, such as semantic tasks (e.g.,
segmentation and detection), image generative tasks (e.g., denoising, inpainting, or colorization), and image transformation tasks (e.g.,
geometric transformations). Our goal is to probe whether a data-driven representation space can capture an underlying structure of tasks across a variety of 39
datasets from wildly different medical imaging modalities, including computed
tomography,
magnetic resonance, electron microscopy, X-ray ultrasound and more. By revealing how tasks relate to one another, we aim to provide insights into their fundamental properties and interconnectedness. To this end, we introduce Task-Contrastive Learning (TaCo), a contrastive learning framework designed to embed tasks into a shared representation space. Through TaCo, we map these heterogeneous tasks from different modalities into a joint space and analyze their properties: identifying which tasks are distinctly represented, which blend together, and how iterative alterations to tasks are reflected in the
embedding space. Our work provides a foundation for understanding the intrinsic structure of medical
vision tasks, offering a deeper understanding of task similarities and their interconnected properties in
embedding spaces.
Read the full abstract ▾
Computer Vision
Analogical Reasoning as a Doctor: A Foundation Model for Gastrointestinal Endoscopy Diagnosis
Peixi Peng, Housheng Xie, Yanling Wei et al. (+5 more)
Gastrointestinal diseases impose a growing global health burden, and endoscopy is a primary tool for early diagnosis. However, routine endoscopic image interpretation still suffers from missed lesions and limited efficiency. Although AI-assisted diagnosis has shown promise, existing models often lack generalizability, adaptability, robustness, and scalability because of limited medical data, domain shift, and heterogeneous annotations. To address these challenges, we develop RATNet, a foundation model for gastrointestinal endoscopy imaging based on analogical
reasoning. RATNet acquires and transfers knowledge from heterogeneous expert annotations across five gastrointestinal endoscopy
datasets through a cyclic pre-training strategy. Its architecture consists of an encoder, a relevance-knowledge acquisition and transfer (RAT) module, a projector, and a multi-task head, and supports fine-tuning, linear probing, and zero-shot transfer. Evaluations show that RATNet outperforms existing foundation models, including GastroNet and GastroVision, across six scenarios: diagnosis of common gastrointestinal diseases, few-shot learning for rare diseases, zero-shot transfer to new medical sites, robustness under long-tailed disease distributions, adaptation to novel diseases, and
privacy-preserving deployment via
federated learning. Its advantage comes from an analogical
reasoning mechanism that matches image-derived posterior knowledge to a learned prior knowledge base and transfers relative knowledge to guide diagnosis, improving generalization and resistance to bias. RATNet is open and cost-effective, supports automatic integration of heterogeneous annotations without manual label unification, and reduces data acquisition costs, making it a practical foundation for intelligent gastrointestinal diagnosis, especially in resource-limited settings.
Read the full abstract ▾
All authors Peixi Peng, Housheng Xie, Yanling Wei, Guangcong Ruan, Xiaoyang Zou, Qian Cao, Yongjian Nian, Guoyan Zheng
Computer Vision
PanopticQuery: Unified Query-Time Reasoning for 4D Scenes
Ruilin Tang, Yang Zhou, Zhong Ye et al. (+3 more)
Understanding dynamic 4D environments through natural
language queries requires not only accurate scene reconstruction but also robust semantic grounding across space, time, and viewpoints. While recent methods using neural representations have advanced 4D reconstruction, they remain limited in contextual
reasoning, especially for complex semantics such as interactions, temporal actions, and spatial relations. A key challenge lies in transforming noisy, view-dependent predictions into globally consistent 4D interpretations. We introduce PanopticQuery, a framework for unified query-time
reasoning in 4D scenes. Our approach builds on 4D Gaussian Splatting for high-fidelity dynamic reconstruction and introduces a multi-view semantic consensus mechanism that grounds natural
language queries by aggregating 2D semantic predictions across multiple views and time frames. This process filters inconsistent outputs, enforces
geometric consistency, and lifts 2D semantics into structured 4D groundings via neural field optimization. To support evaluation, we present Panoptic-L4D, a new
benchmark for
language-based querying in dynamic scenes. Experiments demonstrate that PanopticQuery sets a new state of the art on complex
language queries, effectively handling attributes, actions, spatial relationships, and multi-object interactions. A
video demonstration is available in the supplementary materials.
Read the full abstract ▾
All authors Ruilin Tang, Yang Zhou, Zhong Ye, Wenxi Liu, Yan Huang, Shengfeng He
Computer Vision
Towards Athlete Fatigue Assessment from Association Football Videos
Xavier Bou, Nathan Correger, Alexandre Cloots et al. (+7 more)
Fatigue monitoring is central in association football due to its links with injury risk and tactical performance. However, objective fatigue-related indicators are commonly derived from subjective self-reported metrics, biomarkers derived from laboratory tests, or, more recently, intrusive sensors such as heart monitors or GPS tracking data. This paper studies whether
monocular broadcast
videos can provide spatio-temporal signals of sufficient quality to support fatigue-oriented analysis. Building on state-of-the-art Game State Reconstruction methods, we extract player trajectories in pitch coordinates and propose a novel kinematics processing
algorithm to obtain temporally consistent speed and acceleration estimates from reconstructed tracks. We then construct acceleration–speed (A-S) profiles from these signals and analyze their behavior as fatigue-related performance indicators. We evaluate the full pipeline on the public SoccerNet-GSR
benchmark, considering both 30-second clips and a complete 45-minute half to examine short-term reliability and longer-term temporal consistency. Our results indicate that
monocular GSR can recover kinematic patterns that are compatible with A-S profiling while also revealing sensitivity to trajectory noise, calibration errors, and temporal discontinuities inherent to broadcast footage. These findings support
monocular broadcast
video as a low-cost basis for fatigue analysis and delineate the methodological challenges for future research.
Read the full abstract ▾
All authors Xavier Bou, Nathan Correger, Alexandre Cloots, Cédric Gavage, Silvio Giancola, Cédric Schwartz, François Delvaux, Rudi Cloots, Marc Van Droogenbroeck, Anthony Cioppa
Computer Vision
Letian Bai, Chengyu Tao and Juan Du
Multi-view anomaly detection aims to identify surface defects on complex objects using observations captured from multiple viewpoints. However, existing unsupervised methods often suffer from feature inconsistency arising from viewpoint variations and modality discrepancies. To address these challenges, we propose a Semantic and
Geometric Alignment Network (SGANet), a unified framework for
multimodal multi-view anomaly detection that effectively combines semantic and
geometric alignment to learn physically coherent feature representations across viewpoints and modalities. SGANet consists of three key components. The Selective Cross-view Feature Refinement Module (SCFRM) selectively aggregates informative patch features from adjacent views to enhance cross-view feature interaction. The Semantic-Structural Patch
Alignment (SSPA) enforces semantic
alignment across modalities while maintaining structural consistency under viewpoint transformations. The Multi-View
Geometric Alignment (MVGA) further aligns geometrically corresponding patches across viewpoints. By jointly modeling feature interaction, semantic and structural consistency, and global
geometric correspondence, SGANet effectively enhances anomaly detection performance in
multimodal multi-view settings. Extensive experiments on the SiM3D and Eyecandies
datasets demonstrate that SGANet achieves state-of-the-art performance in both anomaly detection and localization, validating its effectiveness in realistic industrial scenarios.
Read the full abstract ▾
Computer Vision
A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting
Yongchuan Cui and Peng Liu
Remote sensing imagery suffers from clouds, haze, noise, resolution limits, and sensor heterogeneity. Existing restoration and fusion approaches train separate models per degradation type. In this work, we present
Language-conditioned Large-scale Remote Sensing restoration model (LLaRS), the first unified foundation model for multi-modal and multi-task remote sensing low-level
vision. LLaRS employs Sinkhorn-Knopp optimal transport to align heterogeneous bands into semantically matched slots, routes features through three complementary mixture-of-experts layers (convolutional experts for spatial patterns, channel-mixing experts for spectral fidelity, and
attention experts with low-rank adapters for global context), and stabilizes joint training via step-level dynamic weight adjustment. To train LLaRS, we construct LLaRS1M, a million-scale multi-task
dataset spanning eleven restoration and enhancement tasks, integrating real paired observations and controlled synthetic degradations with diverse natural
language prompts. Experiments show LLaRS consistently outperforms seven competitive models, and parameter-efficient finetuning experiments demonstrate strong transfer capability and adaptation efficiency on unseen data. Repo: https://github.com/yc-cui/LLaRS
Read the full abstract ▾
Computer Vision
DetailVerifyBench: A Benchmark for Dense Hallucination Localization in Long Image Captions
Xinran Wang, Yuxuan Zhang, Xiao Zhang et al. (+7 more)
Accurately detecting and localizing
hallucinations is a critical task for ensuring high reliability of image captions. In the era of
Multimodal Large
Language Models (MLLMs), captions have evolved from brief sentences into comprehensive narratives, often spanning hundreds of words. This shift exponentially increases the challenge: models must now pinpoint specific erroneous spans or words within extensive contexts, rather than merely flag response-level inconsistencies. However, existing
benchmarks lack the fine granularity and domain diversity required to evaluate this capability. To bridge this gap, we introduce DetailVerifyBench, a rigorous
benchmark comprising 1,000 high-quality images across five distinct domains. With an average caption length of over 200 words and dense,
token-level annotations of multiple
hallucination types, it stands as the most challenging
benchmark for precise
hallucination localization in the field of long image captioning to date. Our
benchmark is available at https://zyx-hhnkh.github.io/DetailVerifyBench/.
Read the full abstract ▾
All authors Xinran Wang, Yuxuan Zhang, Xiao Zhang, Haolong Yan, Muxi Diao, Songyu Xu, Zhonghao Yan, Hongbing Li, Kongming Liang, Zhanyu Ma
Computer Vision
FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos
Alexandros Delitzas, Chenyangguang Zhang, Alexey Gavryushin et al. (+8 more)
We present FunRec, a method for reconstructing functional 3D digital twins of indoor scenes directly from
egocentric RGB-D interaction
videos. Unlike existing methods on articulated reconstruction, which rely on controlled setups, multi-state captures, or CAD priors, FunRec operates directly on in-the-wild human interaction sequences to recover interactable 3D scenes. It automatically discovers articulated parts, estimates their kinematic parameters, tracks their 3D motion, and reconstructs static and moving geometry in canonical space, yielding simulation-compatible meshes. Across new real and simulated
benchmarks, FunRec surpasses prior work by a large margin, achieving up to +50 mIoU improvement in part
segmentation, 5-10 times lower articulation and pose errors, and significantly higher reconstruction accuracy. We further demonstrate applications on URDF/USD export for simulation, hand-guided affordance mapping and
robot-scene interaction.
Read the full abstract ▾
All authors Alexandros Delitzas, Chenyangguang Zhang, Alexey Gavryushin, Tommaso Di Mario, Boyang Sun, Rishabh Dabral, Leonidas Guibas, Christian Theobalt, Marc Pollefeys, Francis Engelmann, Daniel Barath
Computer Vision
Semantic-Topological Graph Reasoning for Language-Guided Pulmonary Screening
Chenyu Xue, Yiran Liu, Mian Zhou et al. (+2 more)
Medical image
segmentation driven by free-text clinical instructions is a critical frontier in computer-aided diagnosis. However, existing
multimodal and foundation models struggle with the semantic ambiguity of clinical reports and fail to disambiguate complex anatomical overlaps in low-contrast scans. Furthermore, fully fine-tuning these massive architectures on limited medical
datasets invariably leads to severe overfitting. To address these challenges, we propose a novel Semantic-Topological Graph
Reasoning (STGR) framework for
language-guided pulmonary screening. Our approach elegantly synergizes the
reasoning capabilities of large
language models (LLaMA-3-V) with the zero-shot delineation of
vision foundation models (MedSAM). Specifically, we introduce a Text-to-
Vision Intent
Distillation (TVID) module to extract precise diagnostic guidance. To resolve anatomical ambiguity, we formulate mask selection as a dynamic graph
reasoning problem, where candidate lesions are modeled as nodes and edges capture spatial and semantic affinities. To ensure deployment feasibility, we introduce a Selective Asymmetric Fine-Tuning (SAFT) strategy that updates less than 1% of the parameters. Rigorous 5-fold cross-validation on the LIDC-IDRI and LNDb
datasets demonstrates that our framework establishes a new state-of-the-art. Notably, it achieves an 81.5% Dice Similarity Coefficient (DSC) on LIDC-IDRI, outperforming leading LLM-based tools like LISA by over 5%. Crucially, our SAFT strategy acts as a powerful regularizer, yielding exceptional cross-fold stability (0.6% DSC variance) and paving the way for robust, context-aware clinical deployment.
Read the full abstract ▾
All authors Chenyu Xue, Yiran Liu, Mian Zhou, Jionglong Su, Zhixiang Lu
Computer Vision
Evaluation of Randomization through Style Transfer for Enhanced Domain Generalization
Dustin Eisenhardt, Timothy Schaumlöffel, Alperen Kantarci et al. (+1 more)
Deep learning models for computer
vision often suffer from poor generalization when deployed in real-world settings, especially when trained on synthetic data due to the well-known Sim2Real gap. Despite the growing popularity of style transfer as a data augmentation strategy for domain generalization, the literature contains unresolved contradictions regarding three key design axes: the diversity of the style pool, the role of texture complexity, and the choice of style source. We present a systematic empirical study that isolates and evaluates each of these factors for driving scene understanding, resolving inconsistencies in prior work. Our findings show that (i) expanding the style pool yields larger gains than repeated augmentation with few styles, (ii) texture complexity has no significant effect when the pool is sufficiently large, and (iii) diverse artistic styles outperform domain-aligned alternatives. Guided by these insights, we derive StyleMixDG (Style-Mixing for Domain Generalization), a lightweight, model-agnostic augmentation recipe that requires no architectural modifications or additional losses. Evaluated on the GTAV $\rightarrow$ {BDD100k, Cityscapes, Mapillary Vistas}
benchmark, StyleMixDG demonstrates consistent improvements over strong baselines, confirming that the empirically identified design principles translate into practical gains. The code will be released on GitHub.
Read the full abstract ▾
All authors Dustin Eisenhardt, Timothy Schaumlöffel, Alperen Kantarci, Gemma Roig
Computer Vision · Cross-listed from cs.CE
INTERACT: An AI-Driven Extended Reality Framework for Accesible Communication Featuring Real-Time Sign Language Interpretation and Emotion Recognition
Nikolaos D. Tantaroudas, Andrew J. McCracken, Ilias Karachalios et al. (+1 more)
Video conferencing has become central to professional collaboration, yet most platforms offer limited support for deaf, hard-of-hearing, and
multilingual users. The World Health Organisation estimates that over 430 million people worldwide require rehabilitation for disabling hearing loss, a figure projected to exceed 700 million by 2050. Conventional accessibility measures remain constrained by high costs, limited availability, and logistical barriers, while Extended Reality (XR) technologies open new possibilities for immersive and inclusive communication. This paper presents INTERACT (Inclusive Networking for Translation and
Embodied Real-Time Augmented Communication Tool), an AI-driven XR platform that integrates real-time
speech-to-text conversion, International Sign
Language (ISL) rendering through 3D avatars,
multilingual translation, and
emotion recognition within an immersive virtual environment. Built on the CORTEX2 framework and deployed on Meta Quest 3 headsets, INTERACT combines Whisper for
speech recognition, NLLB for
multilingual translation, RoBERTa for
emotion classification, and Google MediaPipe for gesture extraction. Pilot evaluations were conducted in two phases, first with technical experts from academia and industry, and subsequently with members of the deaf community. The trials reported 92% user satisfaction, transcription accuracy above 85%, and 90%
emotion-detection precision, with a mean overall experience rating of 4.6 out of 5.0 and 90% of participants willing to take part in further testing. The results highlight strong potential for advancing accessibility across educational, cultural, and professional settings. An extended version of this work, including full pilot data and implementation details, has been published as an Open Research Europe article [Tantaroudas et al., 2026a].
Read the full abstract ▾
All authors Nikolaos D. Tantaroudas, Andrew J. McCracken, Ilias Karachalios, Evangelos Papatheou
Computer Vision
ID-Selection: Importance-Diversity Based Visual Token Selection for Efficient LVLM Inference
Zhaohong Huang, Wenjing Liu, Yuxin Zhang et al. (+2 more)
Recent advances have explored visual
token pruning to accelerate the inference of large
vision-
language models (LVLMs). However, existing methods often struggle to balance
token importance and diversity: importance-based methods tend to retain redundant
tokens, whereas diversity-based methods may overlook informative ones. This trade-off becomes especially problematic under high reduction ratios, where preserving only a small subset of visual
tokens is critical. To address this issue, we propose ID-Selection, a simple yet effective
token selection strategy for efficient LVLM inference. The key idea is to couple importance estimation with diversity-aware iterative selection: each
token is first assigned an importance score, after which high-scoring
tokens are selected one by one while the scores of similar
tokens are progressively suppressed. In this way, ID-Selection preserves informative
tokens while reducing redundancy in a unified selection process. Extensive experiments across 5 LVLM backbones and 16 main
benchmarks demonstrate that ID-Selection consistently achieves superior performance and efficiency, especially under extreme
pruning ratios. For example, on LLaVA-1.5-7B, ID-Selection prunes 97.2% of visual
tokens, retaining only 16
tokens, while reducing inference FLOPs by over 97% and preserving 91.8% of the original performance, all without additional training.
Read the full abstract ▾
All authors Zhaohong Huang, Wenjing Liu, Yuxin Zhang, Fei Chao, Rongrong Ji
Computer Vision · Cross-listed from cs.RO
Uncovering Linguistic Fragility in Vision-Language-Action Models via Diversity-Aware Red Teaming
Baoshun Tong, Haoran He, Ling Pan et al. (+2 more)
Vision-
Language-Action (VLA) models have achieved remarkable success in robotic manipulation. However, their robustness to linguistic nuances remains a critical, under-explored safety concern, posing a significant safety risk to real-world deployment. Red teaming, or identifying environmental scenarios that elicit catastrophic behaviors, is an important step in ensuring the safe deployment of
embodied AI
agents. Reinforcement learning (RL) has emerged as a promising approach in automated red teaming that aims to uncover these vulnerabilities. However, standard RL-based adversaries often suffer from severe mode collapse due to their reward-maximizing nature, which tends to converge to a narrow set of trivial or repetitive failure patterns, failing to reveal the comprehensive landscape of meaningful risks. To bridge this gap, we propose a novel Diversity-Aware
Embodied Red Teaming (DAERT) framework, to expose the vulnerabilities of VLAs against linguistic variations. Our design is based on evaluating a uniform policy, which is able to generate a diverse set of challenging instructions while ensuring its attack effectiveness, measured by execution failures in a physical simulator. We conduct extensive experiments across different robotic
benchmarks against two state-of-the-art VLAs, including $\pi_0$ and OpenVLA. Our method consistently discovers a wider range of more effective
adversarial instructions that reduce the average task success rate from 93.33% to 5.85%, demonstrating a scalable approach to stress-testing VLA
agents and exposing critical safety blind spots before real-world deployment.
Read the full abstract ▾
All authors Baoshun Tong, Haoran He, Ling Pan, Yang Liu, Liang Lin
Computer Vision
RABC-Net: Reliability-Aware Annotation-Free Skin Lesion Segmentation for Low-Resource Dermoscopy
Yujie Yao, Yuhaohang He, Junjie Huang et al. (+6 more)
Pixel-level annotation is costly in low-resource dermoscopy. We present RABC-Net, a reliability-aware annotation-free
segmentation system that combines pseudo-label reliability learning, restricted target-domain adaptation, and Reliability-Adaptive Boundary Calibration (RABC). The system decouples reliability learning from deployment: uncertainty-aware pseudo-label interaction shapes robust representations during training, while the image-only inference path is preserved and RABC performs local logit-space calibration from boundary confidence, uncertainty, and foreground probability. No manual masks are used for training or target-domain adaptation; validation labels, when available, are used only for final operating-point selection. Across ISIC-2017, ISIC-2018, and PH2, RABC-Net achieves macro-average DICE/JAC of 86.58%/79.47% and consistent matched-protocol results. Controlled within-study analyses show that RABC provides localized gains over nonlearned boundary correction, while the overall result comes from the full reliability-aware system. Adaptation updates only 3.50% of model parameters, image-only inference runs at 87.4 FPS, and the selected operating points use $\sigma=0$ on all three
datasets, indicating that learned calibration avoids extra smoothing at deployment.
Read the full abstract ▾
All authors Yujie Yao, Yuhaohang He, Junjie Huang, Zhou Liu, Jiangzhao Li, Yan Qiao, Wen Xiao, Yunsen Liang, Xiaofan Li
Computer Vision
Purify-then-Align: Towards Robust Human Sensing under Modality Missing with Knowledge Distillation from Noisy Multimodal Teacher
Pengcheng Weng, Yanyu Qian, Yangxin Xu et al. (+1 more)
Robust
multimodal human sensing must overcome the critical challenge of missing modalities. Two principal barriers are the Representation Gap between heterogeneous data and the Contamination Effect from low-quality modalities. These barriers are causally linked, as the corruption introduced by contamination fundamentally impedes the reduction of representation disparities. In this paper, we propose PTA, a novel "Purify-then-Align" framework that solves this causal dependency through a synergistic integration of meta-learning and knowledge
diffusion. To purify the knowledge source, PTA first employs a meta-learning-driven weighting mechanism that dynamically learns to down-weight the influence of noisy, low-contributing modalities. Subsequently, to align different modalities, PTA introduces a
diffusion-based knowledge
distillation paradigm in which an information-rich clean teacher, formed from this purified consensus, refines the features of each student modality. The ultimate payoff of this "Purify-then-Align" strategy is the creation of exceptionally powerful single-modality encoders imbued with cross-modal knowledge. Comprehensive experiments on the large-scale MM-Fi and XRF55
datasets, under pronounced Representation Gap and Contamination Effect, demonstrate that PTA achieves state-of-the-art performance and significantly improves the robustness of single-modality models in diverse missing-modality scenarios.
Read the full abstract ▾
All authors Pengcheng Weng, Yanyu Qian, Yangxin Xu, Fei Wang
Computer Vision
WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrieval
Yizhuo Xu, Chaojian Yu, Yuanjie Shao et al. (+3 more)
Composed Image
Retrieval (CIR) task aims to retrieve target images based on reference images and modification texts. Current CIR methods primarily rely on fine-tuning
vision-
language pre-trained models. However, we find that these approaches commonly suffer from severe overfitting, posing challenges for CIR with limited triplet data. To better understand this issue, we present a systematic study of overfitting in VLP-based CIR, revealing a significant and previously overlooked generalization gap across different models and
datasets. Motivated by these findings, we introduce WRF4CIR, a Weight-Regularized Fine-tuning network for CIR. Specifically, during the fine-tuning process, we apply
adversarial perturbations to the model weights for regularization, where these perturbations are generated in the opposite direction of gradient descent. Intuitively, WRF4CIR increases the difficulty of fitting the training data, which helps mitigate overfitting in CIR under limited triplet supervision. Extensive experiments on
benchmark datasets demonstrate that WRF4CIR significantly narrows the generalization gap and achieves substantial improvements over existing methods.
Read the full abstract ▾
All authors Yizhuo Xu, Chaojian Yu, Yuanjie Shao, Tongliang Liu, Qinmu Peng, Xinge You
Computer Vision
High-Resolution Single-Shot Polarimetric Imaging Made Easy
Shuangfan Zhou, Chu Zhou, Heng Guo et al. (+4 more)
Polarization-based
vision has gained increasing
attention for providing richer physical cues beyond RGB images. While achieving single-shot capture is highly desirable for practical applications, existing Division-of-Focal-Plane (DoFP) sensors inherently suffer from reduced spatial resolution and artifacts due to their spatial multiplexing mechanism. To overcome these limitations without sacrificing the snapshot capability, we propose EasyPolar, a multi-view polarimetric imaging framework. Our system is grounded in the physical insight that three independent intensity measurements are sufficient to fully characterize linear polarization. Guided by this, we design a triple-camera setup consisting of three synchronized RGB cameras that capture one unpolarized view and two polarized views with distinct orientations. Building upon this hardware design, we further propose a confidence-guided polarization reconstruction network to address the potential misalignment in multi-view fusion. The network performs multi-modal feature fusion under a confidence-aware physical guidance mechanism, which effectively suppresses warping-induced artifacts and enforces explicit
geometric constraints on the solution space. Experimental results demonstrate that our method achieves high-quality results and benefits various downstream tasks.
Read the full abstract ▾
All authors Shuangfan Zhou, Chu Zhou, Heng Guo, Youwei Lyu, Boxin Shi, Zhanyu Ma, Imari Sato
Computer Vision
Physics-Aligned Spectral Mamba: Decoupling Semantics and Dynamics for Few-Shot Hyperspectral Target Detection
Luqi Gong, Qixin Xie, Yue Chen et al. (+4 more)
Meta-learning facilitates few-shot hyperspectral target detection (HTD), but adapting deep backbones remains challenging. Full-parameter fine-tuning is inefficient and prone to overfitting, and existing methods largely ignore the frequency-domain structure and spectral band continuity of hyperspectral data, limiting spectral adaptation and cross-domain generalization.To address these challenges, we propose SpecMamba, a parameter-efficient and frequency-aware framework that decouples stable semantic representation from agile spectral adaptation. Specifically, we introduce a Discrete Cosine Transform Mamba Adapter (DCTMA) on top of frozen
Transformer representations. By projecting spectral features into the frequency domain via DCT and leveraging Mamba's linear-complexity state-space recursion, DCTMA explicitly captures global spectral dependencies and band continuity while avoiding the redundancy of full fine-tuning. Furthermore, to address prototype drift caused by limited sample sizes, we design a Prior-Guided Tri-Encoder (PGTE) that allows laboratory spectral priors to guide the optimization of the learnable adapter without disrupting the stable semantic feature space. Finally, a Self-Supervised Pseudo-Label Mapping (SSPLM) strategy is developed for test-time adaptation, enabling efficient decision boundary refinement through uncertainty-aware sampling and dual-path consistency constraints. Extensive experiments on multiple public
datasets demonstrate that SpecMamba consistently outperforms state-of-the-art methods in detection accuracy and cross-domain generalization.
Read the full abstract ▾
All authors Luqi Gong, Qixin Xie, Yue Chen, Ziqiang Chen, Fanda Fan, Shuai Zhao, Chao Li