Computer Vision
Diffusion-guided Generalizable Enhancer for Urban Scene Reconstruction
Henry Che, Jingkang Wang, Yun Chen et al. (+3 more)
Urban scene reconstruction from real-world observations has emerged as a powerful tool for self-driving development and testing. While current neural rendering approaches achieve high-fidelity rendering along the recorded trajectories, their quality degrades significantly under large viewpoint shifts, limiting the applicability for closed-loop simulation. Recent works have shown promising results in using
diffusion models to enhance quality at these challenging viewpoints and distill improvements back into 3D representations. However, they often require costly per-scene optimization, and the distilled representations remain fragile and fail to generalize beyond limited synthesized views. To address these limitations, we propose GenRe, a novel
diffusion-guided generalizable enhancer for urban scene reconstruction. GenRe takes as input any pretrained 3D Gaussian representation and fixes the deficiencies within a few minutes. By learning to distill generative priors across diverse scenes, GenRe produces robust and high-fidelity representation efficiently that generalizes reliably to challenging unseen viewpoints (e.g., lane change). Experiments show that GenRe outperforms existing methods in both quality and efficiency and benefits various downstream tasks, enabling robust and scalable sensor simulation for autonomous driving.
Read the full abstract ▾
All authors Henry Che, Jingkang Wang, Yun Chen, Ze Yang, Sivabalan Manivasagam, Raquel Urtasun
Computer Vision
The Neglected Baseline in Model Interpretation
Yongjin Cui and Xiaohui Fan
We observe that existing model interpretation methods generally ignore the baseline, and such neglect often results in imprecise or even incorrect interpretation. In this paper, we reformulate the task of model interpretation and the interpretation principles for model interpretation results to demonstrate the importance of the baseline. We further unify gradient-based methods, Integrated Gradients (IG) methods, and Taylor expansion, clarifying the connections among them and explicitly identifying the baseline for each method. On this basis, we analyze the flaws and errors in related model interpretation methods (IG, LayerCAM, ODAM, Difference Map). We advocate evaluating the quality of model interpretation results precisely through the attribution error between the attribution result and the attribution target, rather than adopting flawed evaluation methods, such as those based on marginal-effect or the assumption of perfect model performance. We revise IG and develope a model interpretation method with a clear and reasonable baseline, achieving better results. Our method supports model interpretation based on features from any layer. Interpretation based on features from different layers are all reasonable, and the differences among these results reflect varying degrees of feature extraction at different feature extraction stages.
Read the full abstract ▾
Computer Vision
Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence
Xingyue Wang, Bo Liu, Meng Wang et al. (+4 more)
Visual Question Answering (VQA) holds great promise for clinical support, particularly in ophthalmology, where retinal fundus photography is essential for diagnosis. However, ophthalmic VQA
benchmarks primarily emphasize answer accuracy, neglecting the explicit visual evidence necessary for clinical interpretability. In this work, we introduce FundusGround, a new
benchmark for clinically interpretable ophthalmic VQA with spatially-grounded lesion evidence. Specifically, we propose a three-stage pipeline that collects 10,719 fundus images with 15,595 image-level meticulously annotated lesions. To ensure anatomical consistency and clinical validity, all lesions are spatially localized using the Early Treatment Diabetic Retinopathy Study (ETDRS) grid, enabling standardized mapping to nine clinically meaningful retinal regions. Built upon this structured lesion evidence, 72,706 questions are then generated spanning four formats: open-ended, closed-ended, single-choice, and multiple-choice. We further
benchmark multiple general- and medical- large
vision-
language models using dual metrics for answer accuracy and lesion-level
reasoning. The experiments demonstrate that incorporating lesion-level visual evidence consistently improves model performance and transparency, highlighting the necessity of explicit spatial grounding for reliable and explainable ophthalmic VQA.
Read the full abstract ▾
All authors Xingyue Wang, Bo Liu, Meng Wang, Zhixuan Zhang, Chengcheng Zhu, Huazhu Fu, Jiang Liu
Computer Vision
From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding
Yandi Wang, Libin Zhan, Ziwei Huang et al. (+5 more)
Extracting structured information from visual documents (Visual Information Extraction, VIE) is a cornerstone of business automation. While recent
Multimodal Large
Language Models (MLLMs) have shown promising capabilities, existing
benchmarks suffer from critical limitations in scale and realism, lack semantic granularity, and fail to cover diverse document types. To bridge this gap, we introduce ReceiptBench, a large-scale, human-annotated
benchmark consisting of 10k diverse receipts, organizing information extraction into four hierarchical sub-tasks: (1) Basic Perception for raw text spotting, (2) Format Normalization for strictly following standardization instructions, (3) Semantic
Reasoning for inferring implicit attributes from context, and (4) Structure Parsing for handling nested line items. Furthermore, we propose a two-stage training framework incorporating Metric-Aware Group Relative Policy Optimization (GRPO), which translates rigorous evaluation constraints into reinforcement learning signals to enhance structural consistency. Extensive experiments demonstrate that our method yields state-of-the-art performance, surpassing leading proprietary models on complex
reasoning tasks. We release our
datasets and code at https://github.com/wwwT0ri/ReceiptBench.
Read the full abstract ▾
All authors Yandi Wang, Libin Zhan, Ziwei Huang, Tiancheng Luo, Yuxuan Jiang, Wang Dong, Leilei Gan, Jun Chen
Computer Vision
Translating Signals to Languages for sEMG-Based Activity Recognition
Ming Wang, Haoxuan Qu, Qiuhong Ke et al. (+3 more)
Surface electromyography (sEMG) signal-based activity recognition has attracted increasing research
attention in recent years. To develop accurate sEMG signal-based activity recognizers, numerous approaches have been proposed. Some studies focus on designing larger and more expressive model architectures to enhance the representational capacity of sEMG signals, while others aim to enrich model priors through large-scale pretraining, thereby improving recognition performance. Recently, large
language models (LLMs) have shown remarkable generalization and
reasoning capabilities in natural
language processing, whose implicit knowledge, learned from extensive linguistic descriptions of actions, opens new possibilities for interpreting sEMG signals and inferring activity intentions. Motivated by this, we propose LLM-sEMG, a novel framework that leverages LLMs as sEMG activity recognizers. Within this framework, we design a
language-oriented mapping mechanism that converts continuous sEMG sequences into sEMG
language, integrating several strategies to further facilitate the signal-to-
language mapping process. Extensive experiments demonstrate that the proposed framework achieves highly accurate sEMG signal-based activity recognition using large
language models.
Read the full abstract ▾
All authors Ming Wang, Haoxuan Qu, Qiuhong Ke, Wei Zhou, Hossein Rahmani, Jun Liu
Computer Vision
Zi Ye, Yibin Wen, Xiaoya Fan et al. (+10 more)
Agricultural decision-making increasingly requires
multimodal systems that can transform visual observations into reliable, executable actions. However, existing agricultural
multimodal benchmarks mainly evaluate final-answer correctness and provide limited support for assessing whether models can use external tools to complete precision-sensitive workflows. In this paper, we introduce AgroTools, a
benchmark for evaluating tool-augmented
multimodal agents in agriculture. AgroTools contains 539 question-answer instances paired with 1,097 heterogeneous agricultural images, spanning five task families and an executable environment of 14 agricultural tools. Each query is annotated with structured tool-use traces, enabling a dual-view evaluation of both process-level execution quality and outcome-level task success. We
benchmark 9 open-source and 4 closed-source
multimodal large
language models on AgroTools. Results show that current models remain far from reliable in agricultural tool-use settings, with clear bottlenecks in tool planning, argument generation, execution recovery, and final-answer synthesis. We hope AgroTools will support future research on
multimodal agents for high-precision agricultural applications. The
benchmark and evaluation are available at https://huggingface.co/
datasets/AgroTools/AgroTools.
Read the full abstract ▾
All authors Zi Ye, Yibin Wen, Xiaoya Fan, Xinyu Zhang, Jing Wu, Kun Zeng, Zurong Mai, Jiarui Zhang, Bohan Shi, Juepeng Zheng, Jianxi Huang, Yutong Lu, Haohuan Fu
Computer Vision
GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction
Corentin Dumery, David Colmenares, Alexander Fix et al. (+3 more)
Eye tracking (ET) is a foundational technology for advanced AR/VR applications. However, training ET models for every new ET device is challenging: real data collection is costly and time-consuming, while existing synthetic data generation methods lack realism. To remove the need for additional data collection while maintaining data quality, we introduce a data-driven 3D prior that models the distribution of human eyes across diverse identities, gaze directions, and light settings. This model, which we coin GazePrior, then enables sparse-input 3D reconstruction of annotated data collected with previous ET devices, which can in turn be rendered from the cameras of any target ET device. Our approach synthesizes data with the realism, diversity and ground-truth accuracy of real data collection without its prohibitive costs. Our experiments demonstrate that ET models trained with our synthesized data outperform previous zero-shot methods, achieving higher accuracy and robustness.
Read the full abstract ▾
All authors Corentin Dumery, David Colmenares, Alexander Fix, Pascal Fua, Ali Behrooz, Jogendra Kundu
Computer Vision
VEELA: A Clinically-Constrained Benchmark for Liver Vessel Segmentation in Computed Tomography Angiography
Ziya Ata Yaz{\i}c{\i}, N. Sinem Gezer, İlkay Öksüz et al. (+19 more)
Accurate
segmentation of hepatic and portal vessels in contrast-enhanced computed
tomography angiography (CTA) remains challenging due to complex vascular
topology, peripheral visibility limitations, and acquisition-induced ambiguities. While existing public
datasets offer valuable
benchmarks, few include clinically realistic annotation constraints. We introduce VEELA (Vessel Extraction and Extrication for Liver Analysis), a rigorously curated liver vessel
dataset derived from 40 CTA scans inherited from the CHAOS grand-challenge cohort. All vessels were manually delineated slice-by-slice under multi-expert consensus, using a strict visibility-driven annotation policy and avoiding anatomically inferred interpolation. This design explicitly captures anatomical variability and imaging-related uncertainty. As a continuation of the CHAOS challenge, VEELA enables reproducible cross-
benchmark evaluation while extending the scope to fine-grained hepatic and portal vessel
segmentation. We further establish a standardized benchmarking framework and analyze complementary evaluation metrics, including
topology-aware (clDice), overlap-based (IoU), boundary-sensitive (NSD), and geometry-aware (area, length) measures. Our results demonstrate that different metrics capture distinct aspects of vascular integrity, underscoring the necessity of multi-perspective evaluation for clinically meaningful vessel
segmentation. VEELA is publicly released to facilitate reproducible research and support the development of robust vascular
segmentation methods. Researchers can access the evaluation metrics,
dataset, and submission platform at https://www.synapse.org/Synapse:syn65471967.
Read the full abstract ▾
All authors Ziya Ata Yaz{\i}c{\i}, N. Sinem Gezer, İlkay Öksüz, İlker Özgür Koska, Tuğçe Toprak, Pervin Bulucu, Ufuk Beşenk, A. Emre Kavur, Pierre-Henri Conze, Haz{\i}m Kemal Ekenel, Oğuz Dicle, Mustafa Ege Şeker, Mustafa Said Kartal, Ariorad Moniri, Orhan Özkan, Osman Faruk Bayram, Hakan Polat, Musa Balc{\i}, Ece Tuğba Cebeci, Baran C{\i}lga, Kardelen Peçenek, M. Alper Selver
Computer Vision
QuantSR+: Pushing the Limit of Quantized Image Super-Resolution Networks
Haotong Qin, Xudong Ma, Xianglong Liu et al. (+4 more)
Low-bit quantization is widely used to compress super-resolution (SR) models and reduce storage and computation costs for deployment on resource-limited devices. However, when SR models are pushed to ultra-low precision (2-4 bits), performance can drop sharply due to diminished representational capacity and the detail-sensitive nature of SR. To address these issues, we propose QuantSR+, a unified framework that improves quantization operators, network design, and training optimization, achieving better trade-offs between accuracy and efficiency than prior low-bit SR methods. QuantSR+ mainly relies on three technical contributions: (1) Redistribution-driven Bit Determination (RBD), which reshapes quantization distributions in both forward and backward passes to preserve representation fidelity; (2) Quantized Slimmable Architecture (QSA), which begins with an over-parameterized model and progressively prunes less critical blocks to meet efficiency budgets while pushing the accuracy performance; and (3) Slimming-guided Function-localized
Distillation (SFD), which enforces block-aware feature
alignment via a direct loss and a progressive, function-local training schedule to capture quantization effects better and speed up convergence. Extensive experiments show that QuantSR+ achieves state-of-the-art performance against both specialized quantized SR methods and generic quantization approaches. For SwinIR-S on Urban100 (x4), it improves PSNR by 0.29 dB over the 2-bit SOTA baseline. Meanwhile, it delivers strong efficiency gains at 2-bit, reducing operations by up to 87.9% and storage by 89.4%. QuantSR+ is effective for both convolutional and
transformer-based SR models, indicating broad applicability.
Read the full abstract ▾
All authors Haotong Qin, Xudong Ma, Xianglong Liu, Jie Luo, Jinyang Guo, Michele Magno, Yulun Zhang
Computer Vision
Bernini: Latent Semantic Planning for Video Diffusion
Bernini Team: Chenchen Liu, Junyi Chen, Lei Li et al. (+8 more)
Multimodal large
language models (MLLMs) and
diffusion models have each reached remarkable maturity: MLLMs excel at
reasoning over heterogeneous
multimodal inputs with strong semantic grounding, while
diffusion models synthesize images and
videos with photorealistic fidelity. We argue that these two families can be unified through a simple division of labor: MLLMs perform semantic planning, while
diffusion models render pixels from high-level semantic guidance and low-level visual features. Building on this idea, we propose Bernini, a unified framework for
video generation and editing. An MLLM-based planner predicts the target semantic representation directly in the ViT
embedding space, and a DiT-based renderer synthesizes pixels conditioned on this plan, augmented by text features and, for editing, source VAE features for detail preservation. Because semantics serve as the interface, the planner and renderer can be trained separately and only lightly co-trained, preserving the pretrained strengths of both components while keeping training efficient. To better handle multiple visual inputs, we introduce Segment-Aware 3D Rotary Positional
Embedding (SA-3D RoPE), and further incorporate chain-of-thought
reasoning in the planner to better transfer understanding into generation. Bernini achieves state-of-the-art performance across a wide range of
video generation and editing
benchmarks, with the MLLM's pretrained understanding translating into strong generalization on challenging editing tasks.
Read the full abstract ▾
All authors Bernini Team: Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan
Computer Vision
4D-GSW: Kinematic-Aware Spatio-Temporal Consistent Watermarking for 4D Gaussian Splatting
Sifan Zhou, Hang Zhang, Yuhang Wang et al. (+1 more)
While 4D Gaussian Splatting (4DGS) has revolutionized high-fidelity dynamic reconstruction, safeguarding the intellectual property of these assets remains an open challenge. Conventional steganographic techniques often neglect the underlying kinematic
manifolds, triggering non-physical artifacts such as severe temporal flickering and "FVD collapse". To address this, we propose 4D-GSW, a kinematic-aware watermarking framework designed to embed robust copyright information while preserving high spatio-temporal consistency. Unlike prior 4D steganography that primarily focuses on opacity-guided invisibility, our approach explicitly addresses the physical coherence of motion trajectories. We introduce a Spatio-Temporal
Curvature (STC) metric to identify "Dynamic Instants," adaptively gating watermark gradient injection to shield critical motion
manifolds from non-physical perturbations. To ensure global coherence across complex deformations, we formulate a joint HMM-MRF energy minimization model that synchronizes watermark phases within both temporal trajectories and spatial neighborhoods. Furthermore, an anisotropic gradient routing mechanism ensures that watermark
embedding remains strictly decoupled from photometric reconstruction fidelity. Extensive experiments have demonstrated the superior performance of our method in robustly hiding watermarks while resisting various attacks and maintaining high rendering quality and spatiotemporal consistency.
Read the full abstract ▾
All authors Sifan Zhou, Hang Zhang, Yuhang Wang, Ming Li
Computer Vision
3D LULC classification using multispectral LiDAR and deep learning: current and prospective schemes
Narges Takhtkeshha, Aldino Rizaldy, Markus Hollaus et al. (+3 more)
Land Use Land Cover (LULC) classification is essential for national 3D mapping, geospatial analysis, and sustainable planning. Multispectral (MS)
LiDAR provides synchronized spatial-spectral information, and deep learning (DL) enables 3D point cloud semantic
segmentation; however, adoption is limited by the lack of publicly available urban and suburban MS
LiDAR datasets aligned with National Mapping and Cadastral Agencies (NMCAs) classification schemes. This study addresses these gaps by introducing L1 and L2 NMCA-aligned LULC classification schemes and a new
benchmark MS
LiDAR dataset. We evaluate seven state-of-the-art DL models and perform spectral ablation studies at both levels of detail. Results show that Point
Transformer V3 achieves the best performance, with mIoU of 79.4% (L1, 8 classes) and 58.9% (L2, 20 classes) using a dual-wavelength
LiDAR system (532 nm and 1064 nm). Ablation results show that multispectral information improves performance over geometry-only inputs, with gains of 1.1 percentage points at L1 and 7.8 points at L2. These results highlight the value of
LiDAR reflectance for fine-grained material discrimination and support the
evolution of NMCA LULC schemes toward higher semantic detail. The Loosdorf-MSL
dataset contributes a new
benchmark for consistent national and international LULC mapping.
Read the full abstract ▾
All authors Narges Takhtkeshha, Aldino Rizaldy, Markus Hollaus, Juha Hyyppä, Fabio Remondino, Gottfried Mandlburger
Computer Vision
Robustness of breast lesion segmentation under MRI undersampling improves with k-space-aware deep learning
Lukas T. Rotkopf, Marco Schlimbach, Julius C. Holzschuh et al. (+3 more)
Purpose: To assess whether breast lesion
segmentation can be learned directly from acquired MRI k-space, and whether doing so improves robustness when data are accelerated or noisy. Materials and Methods: This retrospective study used public breast dynamic contrast-enhanced MRI (DCE-MRI)
datasets with acquired and synthetic k-space, together with a within-
dataset synthetic control. We compared four 3D U-Net variants: a hybrid k-space-to-image model, a native k-space model, and magnitude and complex image-space baselines. Models were evaluated under increasing undersampling and added complex Gaussian k-space noise. The primary outcome was patient-level Dice similarity coefficient under cross-validation, with the hybrid model prespecified as the main comparison against the magnitude image-space baseline. Results: At full sampling, the hybrid and image-space models performed similarly. As acceleration increased, the hybrid model retained substantially more
segmentation accuracy and significantly outperformed the magnitude image-space baseline across moderate to high undersampling levels. The same pattern was observed when noise was added directly to k-space: the hybrid model degraded more slowly, whereas the image-space baseline failed under heavier noise. This advantage was reproduced in the within-
dataset synthetic control. Feature analysis suggested that the k-space stage and image-space stage played complementary roles, with frequency-domain filtering concentrated before image-domain lesion localization. Conclusion: K-space-aware deep learning improves the robustness of breast lesion
segmentation under MRI undersampling and k-space noise, while matching image-space methods at full sampling.
Read the full abstract ▾
All authors Lukas T. Rotkopf, Marco Schlimbach, Julius C. Holzschuh, Heinz-Peter Schlemmer, Jens Kleesiek, Moritz Rempe
Computer Vision
PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models
Jose Edgar Hernandez Cancino Estrada, Mauro Díaz Lupone, Žiga Emeršič et al. (+3 more)
Identity-conditioned
diffusion models enable high-quality and identity-consistent face generation, but they also raise severe
privacy concerns, as models may continue to synthesize individuals despite their right to be forgotten. While machine unlearning has been extensively studied for concept and data removal, identity unlearning remains largely unexplored, particularly in models conditioned directly on identity
embeddings rather than text
prompts. In this work, we study identity unlearning in Arc2Face, a state-of-the-art identity-conditioned latent
diffusion model for face generation, and introduce Proximity-guided Identity Unlearning (PIU), an anchor-guided framework for identity unlearning. Specifically, we formulate identity removal as an identity replacement objective that reassigns the source identity to a selected anchor identity in the learned identity space, and we complement it with a proximity-based anchor selection strategy motivated by the geometry of ArcFace representations. We further show that effective unlearning can be achieved through localized fine-tuning of a small subset of identity-sensitive cross-
attention layers. Experiments across multiple target identities show that our framework effectively suppresses generation of the target identity while preserving realism and identity consistency for retained identities, as validated by improved performance on unlearning and image-quality metrics, together with qualitative evaluation. The source code for the PIU identity unlearning framework is made publicly available at https://github.com/edgarcancinoe/piu-unlearning.
Read the full abstract ▾
All authors Jose Edgar Hernandez Cancino Estrada, Mauro Díaz Lupone, Žiga Emeršič, Vitomir Štruc, Peter Peer, Darian Tomašević
Computer Vision
Detection of Virus and Small Cell Patches in Foci Images Using Switchable Convolution and Feature Pyramid Networks
Amrita Singh and Snehasis Mukherjee
Accurate detection and counting of virus patches in focus-forming unit (FFU) images, also known as foci images, are important for quantifying viral infection and analyzing cellular structures. This task is challenging because biomedical targets often vary substantially in size, density, contrast, and shape. In this paper, we propose an enhanced YOLOv2-based detector that integrates a Feature Pyramid Network (FPN) to improve multi-scale feature representation. We also incorporate a switchable atrous convolution mechanism to adapt the receptive field for fine-grained targets in dense microscopy images. The proposed method is evaluated on biomedical foci image
datasets for virus patch and small
cell patch detection. For small
cell patch detection, the model achieves a mean average precision (mAP) of 40.5% at a 25% Intersection over Union (IoU) threshold. For FFU virus patch detection, the model achieves an mAP of 68%. These results indicate that combining FPN-based feature fusion with switchable convolution improves the suitability of YOLOv2 for specialized biomedical object detection tasks
Read the full abstract ▾
Computer Vision
Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability
Xiang Chen, Yuxian Dong, Chao Li et al. (+6 more)
Vision-
language models (VLMs) have achieved strong performance across diverse
multimodal tasks, but their
adversarial robustness in visible-infrared (VIS-IR) scenarios remains underexplored. This gap is critical because VIS-IR sensing is widely used in real-world perception systems to support reliable understanding under challenging imaging conditions. To address this cross-modal threat setting, we propose CFGPatch, a curved-edge fractal
geometric adversarial patch framework for attacking VIS-IR VLMs. CFGPatch builds on triangular fractal geometry and replaces rigid straight-edged primitives with Bezier-curved elements, preserving multi-scale fractal self-similarity while introducing smoother contours, richer directional variation, and more flexible shape deformation. In addition, we design a modality-specific Fraser-spiral rendering mechanism to inject fine-grained texture distortions and misleading perceptual cues into visible and infrared images. By coupling global curved-fractal geometry with local spiral-based appearance interference, CFGPatch disrupts both shape perception and texture interpretation. We further adopt expectation over transformation (EOT) to improve robustness against common image-level transformations. Extensive experiments show that CFGPatch effectively fools VIS-IR VLMs and consistently outperforms standard patch baselines in attack effectiveness and robustness. Moreover,
adversarial samples optimized for zero-shot classification transfer well to image captioning and visual question answering, demonstrating strong cross-task transferability and generalizability across downstream tasks.
Read the full abstract ▾
All authors Xiang Chen, Yuxian Dong, Chao Li, Chengyin Hu, Jiaju Han, Fengyu Zhang, Yiwei Wei, Jiahuan Long, Jiujiang Guo
Computer Vision · Cross-listed from cs.RO
Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors
Jiahe Chen, ZiRui Wang, Feiyu Jia et al. (+7 more)
Whole-body
Humanoid-Object Interaction (HOI) is bottlenecked by the scarcity of high-fidelity 3D data. While
video generative priors offer a promising alternative, existing methods suffer from Representation Misalignment due to their reliance on
geometric priors (e.g., explicit CAD models), and Retargeting Complexity arising from intensive morphing and morphological mismatch. We propose Imagine2Real, a zero-shot HOI framework for flexible, geometry-free interaction. To resolve misalignment, we formulate
robot and object motions as unified 4D point trajectories. To overcome retargeting complexity, our Keypoints Tracker tracks only sparse critical points (base, hands, and object), entirely bypassing the error-amplifying retargeting process. To maintain natural gaits despite these sparse signals, we utilize the latent space of a Behavior Foundation Model (BFM) as the tracker's search domain. Using a progressive training strategy, Imagine2Real learns robust behaviors with simple tracking rewards, enabling zero-shot physical deployment within a motion capture(mocap) system.
Read the full abstract ▾
All authors Jiahe Chen, ZiRui Wang, Feiyu Jia, Xiao Chen, Xiaojie Niu, Weishuai Zeng, Tianfan Xue, Xiaowei Zhou, Jiangmiao Pang, Jingbo Wang
Computer Vision
MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering
Junbin Xiao, Jiajun Chen, Tianxiang Sun et al. (+2 more)
Long streaming
video QA remains challenging due to growing visual
tokens and limited
reasoning length of large
language models (LLMs). KV-caching stores the Key-Value (KV) of the historical
tokens via LLM prefill and enables more efficient streaming QA. However, existing methods cache every one or two frames, causing redundant
memory usage and losing fine-grained spatial details within frame or temporal contexts across frames. This paper proposes MuKV, a method that features a multi-grained KV cache compression module and a semi-hierarchical
retrieval approach to improve both efficiency and accuracy for long streaming VideoQA. For the offline KV cache, MuKV extracts visual representations at patch-, frame-, and segment-levels. The multiple levels of granularity preserve both local cues and global temporal context, while maintaining efficiency with a dual signal
token compression mechanism guided by self-
attention and frequency. For online QA, MuKV designs a semi-hierarchical
retrieval method to retrieve relevant KV caches for answer generation. Experiments on long-streaming VideoQA
benchmarks show that MuKV significantly improves answer accuracy, without sacrificing
memory and online QA efficiency. Moreover, our compression mechanism alone brings consistent benefits across answer accuracy,
memory, and QA efficiency over baselines, showcasing highly effective contribution.
Read the full abstract ▾
All authors Junbin Xiao, Jiajun Chen, Tianxiang Sun, Xun Yang, Angela Yao
Computer Vision · Cross-listed from cs.NI
Impact of Atmospheric Turbulence and Pointing Error on Earth Observation
Celia Sánchez-de-Miguel, Antonio M. Mercado-Martínez, Beatriz Soret et al. (+2 more)
Earth Observation (EO) imagery is often degraded by atmospheric
turbulence and pointing jitter; yet, these effects are rarely considered in
datasets used to train AI-based detection models. Based on prior work, this paper presents an enhanced image simulator that enables the incorporation of vertical-path atmospheric
turbulence and satellite pointing jitter, arising from platform and sensor vibrations, to generate physically realistic distorted images. As a case study, vessel detection is evaluated using YOLOv8 and RetinaNet on images generated by the proposed simulator under different levels of
turbulence and pointing errors. Results show that YOLOv8 recall decreases from 91% under ideal conditions to 60% in the presence of weak
turbulence, and falls below 40% under strong
turbulence or jitter. In contrast, RetinaNet demonstrates greater robustness, maintaining approximately 75% recall across degraded conditions. These results highlight the importance of incorporating realistic physical degradations into EO training
datasets to ensure reliable performance of AI-based models in operational environments, as demonstrated in maritime surveillance applications.
Read the full abstract ▾
All authors Celia Sánchez-de-Miguel, Antonio M. Mercado-Martínez, Beatriz Soret, Antonio Jurado-Navas, Miguel Castillo-Vázquez
Computer Vision · Cross-listed from cs.LG
An Evidence Hierarchy for Bayesian Object Classification via OSINT-Aided Heterogeneous Sensor Fusion
Jan Nausner and Michael Hubner
Heterogeneous sensor fusion is vital for detecting, localizing, and classifying CBRNE threats. However, individual sensors are often only capable of detecting a subset of relevant threats with varying reliability or can even provide only indirect threat indications, making threat classification challenging. Furthermore, high clutter rates on the sensor side present a great challenge for fusion systems. Additionally, the limited availability of high quality
datasets hinders the advancement of learning-based detection and classification models in smart sensors. To mitigate these sensor related shortcomings, a context-aware and domain knowledge-enhanced fusion process is proposed. First, a novel evidence hierarchy is established that enables modeling of direct, indicative, and contextual information. Second, contextual information about the environment is introduced into the fusion process, by collecting, processing, and exploiting OSINT inputs. Third, all levels of the evidence hierarchy are used to craft a
Bayesian threat type classification mechanism with domain knowledge-informed priors. The proposed methodology is evaluated in simulated scenarios, and the results demonstrate the benefit of the proposed fusion approach in terms of robustness to clutter and prior mismatch, with an overall classification accuracy of up to 95%.
Read the full abstract ▾
Computer Vision
Direct content-based retrieval from music scores images
Noelia Luna-Barahona, Antonio Ríos-Vila, Félix Fuentes-Hurtado et al. (+2 more)
The digitization of musical scores plays a crucial role in their preservation and accessibility, yet information
retrieval still depends mainly on metadata searches, such as by title or composer. Content based search in music score images remains underexplored compared to text documents, despite its potential value for musicians, musicologists, and educators. This work contributes to the field by first studying which characteristics of a score are most relevant for search and by defining a systematic method to build query
datasets from any annotated corpus. We also consider diverse methods for content-based search on music score images, ranging from transcription-based approaches relying on Optical Music Recognition (OMR), to a transcription-free
Transformer model trained to recognize queries directly from score images, and a text-prompted Large
Language Model. Our experiments evaluate these models on four corpora exhibiting diverse characteristics in terms of
dataset size, image quality, and typesetting mechanisms. Overall, each method excels under different conditions: OMR-based pipelines achieve higher in-domain
retrieval, whereas transcription-free models handle domain variability more effectively.
Read the full abstract ▾
All authors Noelia Luna-Barahona, Antonio Ríos-Vila, Félix Fuentes-Hurtado, David Rizo, Jorge Calvo-Zaragoza
Computer Vision
D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities
Danish Ali, Ajmal Mian, Naveed Akhtar et al. (+1 more)
Accurate
brain tumor
segmentation using multi-parametric MRI is critical for effective treatment planning. However, in clinical settings, complete acquisition of all MRI sequences is not always possible. The absence of certain MRI modalities results in substantial performance degradation in existing
segmentation methods, which typically rely on naive feature concatenation or direct fusion strategies. To address this limitation, we propose a novel
segmentation model D3Seg which is designed to maintain stable performance under missing-modality settings. D3Seg introduces Multi-hop Modality Graph Fusion (MMGF) to model higher-order inter-modality dependencies, a lightweight
diffusion-based imputation mechanism to compensate for missing T1ce and FLAIR feature representations in latent space, and probability-space decision refinement to mitigate dominant-class overconfidence and improve delineation of underrepresented tumor subregions. We evaluate the proposed D3Seg model on BraTS 2023 Glioma as the primary
benchmark and further test it on a subset of the external BraTS 2023 Meningioma cohort to assess generalization across tumor pathologies. The results are compared with the state-of-the-art models under different missing-modality conditions. The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing-modality configurations compared to the current state-of-the-art model on BraTS Glioma
dataset. Cross-cohort evaluation on BraTS Meningioma
dataset demonstrates the generalizability of the proposed model, showing consistent improvements in the challenging TC and ET regions, with approximately 1.5-3.0% and 1.5-6.5% gains respectively across several missing-modality configurations.
Read the full abstract ▾
All authors Danish Ali, Ajmal Mian, Naveed Akhtar, Ghulam Mubashar Hassan
Computer Vision
REACH: Hand Pose Estimation from Room Corners
Shu Nakamura, Ryo Kawahara, Genki Kinoshita et al. (+4 more)
We introduce a novel 3D hand pose
estimator that can accurately recover the shape and pose of people's hands in a room from afar, typically from fixed cameras at room corners, in extremely low-resolution and frequently occluded views. Our key idea is to fully leverage hand-body coordination, its temporal progression, and multiview observations. We achieve this with a novel
Transformer-based model, in which hand and body configurations are modeled through correlations between their visual features expressed as per-view
tokens, and their temporal coordination is exploited in an autoregressive manner. We introduce a novel
dataset, which we refer to as REACH, Room-Environment
dataset Annotated with Chest cameras for Hand pose estimation, to train and test our method. REACH is a first-of-its-kind large-scale hand pose
dataset that captures accurate hand movements of 50 participants across a wide variety of daily activities. In order to avoid interfering with natural movements while annotating the hands with accurate shape and pose, we leverage concealed chest cameras. Through extensive experiments, including comparative studies with existing methods, we show that our model, REACH-Net, achieves highly accurate 3D hand pose estimation from afar. These results broaden the horizon of 3D hand pose estimation, especially towards "in-the-wild" continuous human behavior analysis.
Read the full abstract ▾
All authors Shu Nakamura, Ryo Kawahara, Genki Kinoshita, Ryosuke Hirai, Yasutomo Kawanishi, Shohei Nobuhara, Ko Nishino
Computer Vision
A Robust Semantic Segmentation Pipeline for the CVPR 2026 8th UG2+ Challenge Track 2
Jinming Chai, Libo Yan, Licheng Jiao et al. (+1 more)
This report presents our solution for the WeatherProof
Dataset Challenge, namely CVPR 2026 8th UG2+ Challenge Track 2: Semantic
Segmentation in Adverse Weather. For the semantic
segmentation task under adverse weather conditions, we propose a semi-supervised
segmentation pipeline. Our method is trained exclusively on the WeatherProof
dataset, without using any additional external data. Specifically, we adopt UniMatch V2 as the baseline model and treat all degraded-weather images as unlabeled data for semi-supervised training, thereby fully exploiting the data distribution provided by the challenge. During inference, we further apply test-time augmentation to improve the robustness and
segmentation accuracy of the final predictions. The code is publicly available at: https://github.com/ylb888/weatherproof-challenge-unimatchv2.
Read the full abstract ▾
All authors Jinming Chai, Libo Yan, Licheng Jiao, Fang Liu
Computer Vision
GALAR-TemporalNet v2: Anatomy-Guided Dual-Branch Temporal Classification with Bidirectional Mamba and Dual-Graph GCN for Video Capsule Endoscopy – after competition results
Jiye Won, Seangmin Lee and Soon Ki Jung
Video Capsule Endoscopy (VCE) poses a challenging multi-label temporal classification problem, requiring simultaneous localization of 8 anatomical regions and detection of 9 pathological findings across tens of thousands of frames. We present GALAR-TemporalNet v2, a hierarchical temporal model that addresses three core challenges: extreme class imbalance, long-range temporal dependencies, and pathology–anatomy
entanglement. Our architecture combines windowed self-
attention for local modeling, a Dual-Graph GCN for global frame relationships, and Bidirectional Mamba for selective boundary context encoding. A novel anatomy prototype residual pathway decouples pathological deviation signals from normal organ appearance, and a frame-level GCN skip connection stabilizes training of visually confusable rare classes. The competition version, GALAR-TemporalNet, achieved an overall mAP@0.5 of 0.2644 and mAP@0.95 of 0.2353 on the RARE-
VISION test set. Following the competition, the redesigned GALAR-TemporalNet v2 – incorporating a restructured pathology branch, refined loss functions, and extended post-processing – improved these results to mAP@0.5 of 0.3409 and mAP@0.95 of 0.3333.
Read the full abstract ▾
Computer Vision
EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning
Kailin Zhuang, Jiawei Wu and Zhi Jin
Multimodal Large
Language Model (MLLM)-driven image restoration
agent demonstrates effectiveness in degradation coupling scenarios by flexibly selecting tools and determining removal orders. However, its zero-shot planning often fails without experience, necessitating severe trial-and-error overhead to achieve satisfactory outcomes. Currently, two paradigms are employed to address this issue, yet a dilemma persists: Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradations. In contrast, training-free methods utilize explicit experience storage for compatibility but still incur trial-and-error overhead due to naive experience. To resolve the dilemma, we propose EvoIR-
Agent, which first systematically formulates the experience components of a training-free image restoration
agent. Subsequently, a hierarchical experience pool is constructed, which enables coarse-to-fine guidance for diverse tools and removal orders. Furthermore, a self-evolving mechanism is introduced to update the pool from scratch using accumulated records, thereby greatly improving performance and efficiency. Extensive experiments reveal that EvoIR-
Agent achieves a significant lead in the full reference metrics and yields a Pareto-optimal balance, which outperforms state-of-the-art methods in performance and is 61% more efficient.
Read the full abstract ▾
Computer Vision
Zero-Shot Temporal Action Localization Through Textual Guidance
Benedetta Liberatori, Alessandro Conti, Lorenzo Vaquero et al. (+3 more)
Zero-shot temporal action localization (ZS-TAL) consists of classifying and localizing actions in untrimmed
videos, where action classes are unseen at training time. Existing work uses
Vision and
Language Models (VLMs), taking advantage of their strong zero-shot transfer capabilities. Yet, these models face evident challenges with fine-grained action classification, making it difficult to directly use them to distinguish between the presence and absence of an action. Most current methods for ZS-TAL address these challenges by training models on large-scale
video datasets, which require annotated data and often result in limited generalization performance. Recently, approaches discarding the use of labeled data have emerged as an alternative. Following this direction, we propose a novel approach, “Textual Guidance for finer localization of actions in
videos” (TEGU), that compensates for the lack of supervision from training data by exploiting rich textual information derived from large
language models and structured text extracted from captions. This additional linguistic context can improve fine-grained discrimination by providing richer cues about fine-grained action differences within
videos. We validate the effectiveness of the proposed method by conducting experiments on the THUMOS14 and the ActivityNet-v1.3
datasets. Our results show that, by exploiting rich textual information for improved action localization, TEGU outperforms state-of-the-art ZS-TAL approaches that do not involve training
Read the full abstract ▾
All authors Benedetta Liberatori, Alessandro Conti, Lorenzo Vaquero, Paolo Rota, Yiming Wang, Elisa Ricci
Computer Vision
OSS: Open Suturing Skills Vision-Based Assessment Challenge 2024-2025
Hanna Hoffmann, Setareh Bady, Claas de Boer et al. (+54 more)
Achieving high levels of
surgical skill through effective training is essential for optimal patient outcomes. Automated, data-driven skill assessment holds significant potential to improve
surgical training. While machine learning-based methods are increasingly popular for assessing skills in minimally invasive surgery, their application to open surgery remains limited. We present the results of a dedicated MICCAI challenge designed to
benchmark and advance
vision-based skill assessment in open surgery. The challenge
dataset comprises
videos of an open suturing training task recorded with a static GoPro camera in a dry-lab setting, with instrument trajectories available in addition to the primary
video modality. The OSS Challenge was hosted over two consecutive years, comprising two and three independent tasks, respectively: (1) classifying skill level into four classes, (2) predicting the full Objective Structured Assessment of Technical Skills across eight categories, and (3) tracking hands and
surgical tools. Participants submitted diverse solutions including deep learning-based
video models, tracking-driven methods, and hybrid approaches. General-purpose spatiotemporal
video models consistently achieved the strongest performance, though conceptually diverse approaches reached competitive levels when well-executed. Predicting fine-grained OSATS scores remains challenging but benefits substantially from increased training data. Keypoint tracking proves difficult given frequent occlusions and out-of-frame instances, limiting current applicability for motion-based skill analysis. This work
benchmarks innovative and diverse solutions for
surgical skill assessment, highlighting both the promise and current limitations of
video-based evaluation in open surgery and identifying critical directions for advancing automated skill assessment toward clinical impact.
Read the full abstract ▾
All authors Hanna Hoffmann, Setareh Bady, Claas de Boer, Max Kirchner, Jan Egger, Rainer Röhrig, Frank Hölzle, Lennart Johannes Gruber, Kunpeng Xie, Marlon Neuhaus, Victor Alves, Guilherme Barbosa, Leonardo Barroso, João Carvalho, Hao Chen, Gabriella d'Albenzio, André Ferreira, Nuno Gomes, Yuichiro Hayashi, Kousuke Hirasawa, Rebecca Hisey, Seungjae Hong, Seoi Jeong, Tiago Jesus, Daehong Kang, Satoshi Kasai, Shunsuke Kikuchi, Takayuki Kitasaka, Satoshi Kondo, Hyoun-Joong Kong, Youngbin Kong, Atsushi Kouno, Shlomi Laufer, Kyu Eun Lee, Bining Long, Nooshin Maghsoodi, Hiroki Matsuzaki, Evangelos Mazomenos, Ori Meiraz, Kensaku Mori, Marina Music, Masahiro Oda, Roi Papo, Jieun Park, Rafael Piexoto, Saeid Rezaei, Mariana Ribeiro, Soyeon Shin, Yang Shu, Idan Smoller, Danail Stoyanov, Yihui Wang, Xinkai Zhao, Sebastian Bodenstedt, Isabel Funke, Stefanie Speidel, Behrus Hinrichs-Puladi
Computer Vision
Ultra-High-Definition Image Quality Assessment via Graph Representation Learning
Shaode Yu, Enqi Chen, Ming Huang et al. (+4 more)
Blind image quality assessment (BIQA) for ultrahighdefinition (UHD) images remains challenging because native-resolution inference is computationally expensive, whereas aggressive resizing or isolated cropping may suppress scale-sensitive distortions and weaken the relationship between local artifacts and global scene context. This paper aims to improve UHD-BIQA by explicitly modeling the structural dependencies among sampled image regions rather than treating them as independent views, and a graph representation learning framework UHD-GCN-BIQA is proposed. The framework samples aspect-ratio-aligned patches from each UHD image, encodes them as graph nodes, and constructs a hybrid k-nearest-neighbor graph using spatial proximity and feature similarity. Residual graph convolution is used to propagate contextual information across regions, and gated
attention pooling aggregates patchlevel evidence into an imagelevel quality prediction. An exponential moving average normalized multiobjective loss function is adopted to stabilize the joint optimization of regression, correlation, and ranking objectives. Experiments on the UHD-IQA
benchmark show that UHD-GCN-BIQA achieves PLCC = 0.7784, SRCC = 0.8019, and RMSE = 0.0519, obtaining competitive correlation performance and the lowest RMSE among the compared methods. These results indicate that graph-based region relation modeling is effective for UHD image quality assessment, particularly for improving absolute quality score estimation under high-resolution visual content.
Read the full abstract ▾
All authors Shaode Yu, Enqi Chen, Ming Huang, Xuemin Ren, Songnan Zhao, Zhicheng Zhang, Qiurui Sun
Computer Vision
No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos
Matteo Balice, Yanik Kunzi, Chenyangguang Zhang et al. (+3 more)
Recent feed-forward 3D gaussian splatting methods have made dramatic progress on individual aspects of 3D scene reconstruction, but no existing method jointly addresses dynamic content, multi-view input, and unknown camera poses in a single feed-forward pass. Methods that handle dynamics either require accurate camera poses or accept only
monocular input; pose-free multi-view methods address only static scenes; and per-scene optimization methods bridge some of these gaps but at minutes-to-hours cost per scene. We introduce NoPo4D, the first feed-forward system that addresses this empty quadrant. Building on a pretrained geometry backbone and recent 4D Gaussian frameworks, NoPo4D introduces a velocity decomposition that splits Gaussian motion into per-pixel image-plane shifts and depth changes, allowing direct supervision from pseudo ground-truth optical flow on the 2D component. This sidesteps both the differentiable rendering that couples prior posed methods to pose accuracy and the 3D motion ground truth that prior pose-free methods require. The system is rounded out by a bidirectional motion encoder for cross-view and cross-frame feature aggregation, and view-dependent opacity that mitigates cross-view and cross-timestep Gaussian misalignments. On four multi-view dynamic
benchmarks, NoPo4D consistently outperforms prior feed-forward baselines, and with an optional post-optimization stage surpasses per-scene optimization methods, while running orders of magnitude faster.
Read the full abstract ▾
All authors Matteo Balice, Yanik Kunzi, Chenyangguang Zhang, Matteo Matteucci, Marc Pollefeys, Sunghwan Hong