Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Computer Vision

cs.CV Watch Computer Vision

Friday 19 June

Computer Vision · Cross-listed from eess.IV

Unsupervised Susceptibility Distortion Correction of EPI without Calibration Scans via Image Translation-Based Registration

Wooseung Kim and Sung-Hong Park

Functional magnetic resonance imaging (fMRI) utilizes echo-planar imaging (EPI) to capture blood-oxygen-level-dependent (BOLD) signals with high temporal resolution. However, EPI is inherently sensitive to magnetic field inhomogeneities, resulting in susceptibility-induced geometric distortions along the phase-encoding (PE) direction. To correct these distortions, conventional approaches rely on additional calibration scans, such as field maps or reverse PE acquisitions, which are not always available in practice. To overcome this limitation, we propose SACRED, a calibration scan-free susceptibility distortion correction framework that corrects geometric distortions via image translation-based registration using only a routinely acquired anatomical T1-weighted (T1w) image and a unidirectional PE BOLD image. SACRED employs an invertible neural network as the image translation backbone to bridge the contrast gap between BOLD and T1w images while enforcing structural consistency through a modality independent neighborhood descriptor. This design enables the use of a mono-contrast similarity objective to train the registration network in an unsupervised manner without requiring distortion-corrected BOLD images. In addition, we incorporate test-time adaptation (TTA) to further enhance performance on out-of-distribution (OOD) data at inference time. SACRED was evaluated on one in-distribution (ID) dataset and two OOD datasets, and was compared with representative fMRI distortion correction methods. The results demonstrate that SACRED significantly outperforms competing methods on both ID and OOD datasets, exhibiting robustness to scanner and population shifts, partly enabled by TTA. The code will be made publicly available upon acceptance.
Read the full abstract

Computer Vision

The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detection

Serdar Ozsoy, Lars Doorenbos, Federico Spurio et al. (+2 more)

Procedural mistake detection is important for quality control and user assistance across many disciplines. Recent work in this field has achieved significant gains by using the reasoning capabilities of Video-Language Models (VLMs) as components within multi-stage pipelines, which consist of separate modules for supervised temporal action segmentation, error detection, and explainability. Consequently, they remain dependent on tailored training datasets and require task-specific training, limiting their wider applicability. To remedy this, we introduce zero-shot procedural mistake detection and propose a unified Zero-shot Procedural Mistake detection (ZeProM) framework that jointly solves procedural mistake detection and temporal action segmentation with a single pre-trained VLM. By evaluating our framework on two canonical mistake detection benchmarks, EgoPER and CaptainCook4D, we find that ZeProM can perform these tasks successfully, while approaching, or even outperforming, the performance of fully supervised methods. For instance, we achieve a 4.4 point improvement in EDA and a 2.0 point improvement in F1@.5 on average over all five EgoPER tasks compared to the strongest supervised methods. Overall, our results show the potential of unified methods for procedural mistake detection, and we hope this will steer the field away from highly complex pipelines and toward more generally applicable solutions.
Read the full abstract

All authors Serdar Ozsoy, Lars Doorenbos, Federico Spurio, Gianpiero Francesca, Juergen Gall

Computer Vision

A Smart Classroom Behavior Analysis Framework with a New Highly Congested Classroom Dataset

Wei Xu, Maoxiang Chu, Yuelong Fan et al. (+5 more)

Student behavior detection is important for intelligent classroom analysis but remains challenging in large-class scenarios due to dense instance co-occurrence, asymmetric occlusion, depth-wise scale variation, and fine-grained semantic degradation in distant targets. Existing classroom behavior datasets and general-purpose detectors are insufficient to characterize and address these challenges. This paper constructs the Highly Congested Classroom Behavior (HCCB) dataset, containing 50,229 student behavior instances across seven categories: reading, writing, heads up, sleeping, looking around, bowing head, and using phone. HCCB provides a challenging benchmark that integrates dense distributions, severe occlusion, scale variation, and fine-grained behavioral semantics. To address these issues, we propose ODER-HSFNet, a YOLO-based detection framework tailored to highly crowded classrooms. At its core, ODER-HSFNet introduces three task-specific innovations: the Occlusion-aware Deformable Edge Rectifier (ODER), which strengthens boundary evidence under occlusion; the Hypergraph-State Spatial Fusion (HSSF) module, which integrates local structure enhancement, state-space contextual modeling, and high-order relation aggregation; and the Occlusion-Calibrated Detection Head (OCDetect), which suppresses low-quality Pre-NMS candidates and reduces false positives from occlusion boundaries and neighboring instances. Experiments on two classroom behavior detection datasets show that ODER-HSFNet outperforms mainstream YOLO-series methods, achieving 60.60%/80.12% mAP50:95/mAP50 on HCCB and 57.36%/74.65% on SCB-D3-S. Ablation studies further verify the effectiveness of the proposed design for highly crowded classroom behavior detection.
Read the full abstract

All authors Wei Xu, Maoxiang Chu, Yuelong Fan, Guanghao Liao, Yinxiang Yu, Zhi Chen, Haotian Wang, Yutian Zhu

Computer Vision

Compressing History into Memory: Distilling Transformers into Recurrent Transformers

Philippe Weinzaepfel, Christian Wolf, Mert Bülent Sariyildiz et al. (+2 more)

Transformers are AI's workhorse but their computational cost becomes prohibitive when processing long sequences. We target long-horizon streaming vision and robotics applications, where it is particularly impractical to store and maintain a history of observations. Recurrent Transformers address this limitation by maintaining fixed-size memory but their performance lags behind that of transformers operating over the full observation history. We argue that this gap does not stem from architectural limitations, but from differences in how these models learn to compress past information. Without access to an observation history, recurrent models must explicitly decide what to retain in memory at each step, a significantly harder learning problem. In this work, we propose a distillation approach that transfers the compression strategy of a classical full-history transformer to a recurrent variant. We enable this by designing a teacher model that explicitly compresses its observation history into a fixed-size bottleneck representation and directly supervise the student's memory with this bottleneck representation, effectively aligning the two compression mechanisms. We show that this approach allows to train a recurrent latent robotic memory with linear-time complexity on the Mem-RPE task while substantially narrowing the performance gap to full-history transformers. We additionally validate the same principle on streaming visual question answering (VQA) and observe improved recurrent predictions thanks to memory distillation
Read the full abstract

All authors Philippe Weinzaepfel, Christian Wolf, Mert Bülent Sariyildiz, Guillaume Bono, Gianluca Monaci

Computer Vision · Cross-listed from cs.RO

LOGOS: LiDAR-Only Gaussian Elevation Splatting for Unified Tiny Obstacle Segmentation

Nan Ming, Yeqiang Qian, Chunxiang Wang et al. (+1 more)

Robust obstacle segmentation is essential for the safety of intelligent robots, where LiDAR-based perception systems play a fundamental role in the robot-environment interaction. While extensive LiDAR-based approaches have demonstrated high performance on common obstacles in urban scenarios, their results on tiny obstacles such as curbs, gravel, and potholes remain unsatisfactory due to the significant similarity between tiny obstacles and inherent road undulations. Moreover, their segmentation accuracy even deteriorates sharply when the LiDAR scans suffer from degradation in challenging off-road scenes. To overcome these bottlenecks, we propose LOGOS, a LiDAR-only unified tiny obstacle segmentation system, which models the road surface as a continuous mixture of 2D Gaussian primitives and distinguishes tiny obstacles via high-presicion elevation estimation. Unlike existing Gaussian splatting methods that rely on iterative RGB training, LOGOS is a backpropagation-free LiDAR-only approach. It directly estimates Gaussian parameters via a freespace-aware initialization by incrementally pruning non-road primitives using smoothness constraints. Subsequently, pointwise signed distances are computed via a novel normal-aware elevation splatting function, ensuring robustness to both flat and sloped terrains. We evaluate LOGOS on a highly heterogeneous benchmark of point cloud frames collected from urban mobility scenarios and mining haulage off-road environments. These data are practically acquired using different LiDAR sensors and exhibit large variations in point density, terrain roughness, and obstacle types. Experiments on the road and off-road scenes demonstrate that LOGOS significantly outperforms other state-of-the-art methods, particularly in degraded point cloud regions and challenging off-road scenarios, while maintaining real-time efficiency.
Read the full abstract

All authors Nan Ming, Yeqiang Qian, Chunxiang Wang, Ming Yang

Computer Vision · Cross-listed from eess.IV

A Skin-Tone-Aware Dual-Representation Remote Photoplethysmography Framework for Contactless Respiratory Rate Estimation

Trishna Saikia, Anup Kumar Gupta, Puneet Gupta et al. (+1 more)

Respiratory rate is a vital indicator of pulmonary and cardiovascular health, yet conventional methods for estimating respiratory rate are often intrusive due to their contact-based nature. Remote photoplethysmography offers a promising non-contact alternative and has been widely used for heart rate estimation; however, its potential for respiratory rate estimation remains underexplored. Existing methods typically adapt green and chrominance-based projections originally designed for heart rate estimation, which only partially capture respiratory dynamics. Most prior work focuses on the Eulerian representation with fixed or empirically selected RGB projections. To address these gaps, we propose a skin-tone-aware dynamic RGB signal projection that captures respiratory information. To mitigate the sensitivity of the Lagrangian representation to non-respiratory motion, we introduce a denoising network for motion-based remote photoplethysmography signals. We further design a phase-independent contrastive loss that enables Eulerian and Lagrangian representations to collaboratively learn respiratory rate information. We also introduce RR-rPPG, a respiratory-rate facial video dataset with Indian demographic representation. We evaluate the method on RR-rPPG and the publicly available COHFACE dataset, where it consistently outperforms comparison methods and achieves up to a 42.1% reduction in mean absolute error across the evaluated settings. The proposed framework demonstrates the effectiveness of jointly leveraging skin-tone-aware Eulerian and denoised Lagrangian representations for contactless respiratory rate estimation from facial videos. In addition, RR-rPPG contributes a diverse benchmark resource for future research in remote respiratory monitoring. The code and dataset will be made publicly available upon paper acceptance.
Read the full abstract

All authors Trishna Saikia, Anup Kumar Gupta, Puneet Gupta, Pasi Liljeberg

Computer Vision · Cross-listed from cs.RO

Decoupling the Declarative from the Procedural in Vision-Language-Action Models

Nikolaos Tsagkas, Andreas Sochopoulos, Chris Xiaoxuan Lu et al. (+2 more)

Deploying generalist robotic agents in the real world requires transferable skills. Specifically, a policy trained to clone a behavior from object-specific demonstrations must generalize beyond that object, otherwise data collection requirements become intractable. Recently, fine-tuning of pre-trained billion-parameter Vision-Language Models (VLMs), initially on large-scale robot datasets and then on fewer scenario-specific demonstrations, has emerged as the predominant paradigm for designing Vision-Language-Action (VLA) models. While these policies achieve state-of-the-art manipulation performance in-distribution, they remain brittle to minor spatial, semantic, and task variations. In this work, we address the inability of current models to decouple the declarative (i.e., concepts and entity semantics) from the procedural knowledge (i.e., how to do something) encoded in their parameters, which is a fundamental bottleneck for zero-shot skill transfer to novel objects. To address this, we propose w$^{2}$VLA, a new VLA model with restructured information flow. Rather than feeding all multimodal tokens from the VLM encoder into a large, opaque transformer-based action expert, our approach modulates the robot state sequence with visual, spatial, and skill information in a compositional and interpretable manner. Unlike popular, state-of-the-art VLAs, we show that our modular approach successfully decouples knowledge representations, enabling robust behavior cloning and unprecedented zero-shot skill transfer capabilities across dissimilar, unseen objects.
Read the full abstract

All authors Nikolaos Tsagkas, Andreas Sochopoulos, Chris Xiaoxuan Lu, Oisin Mac Aodha, Alexandros Kouris

Computer Vision

Semi-Supervised Vision-Language-Action Model

Hongyang He, Jiuming Liu and Victor Sanchez

Vision-Language-Action (VLA) models enable robots to predict actions directly from visual observations and language instructions, but adapting them to new environments still depends on costly action-labeled demonstrations. To reduce this dependence, we study semi-supervised VLA adaptation under limited supervision signals, where only a small portion of trajectories contain robot actions and the remaining trajectories provide action-unlabeled vision-language observations. Unlike standard semi-supervised learning, the missing supervision is an embodied action signal that must be visually grounded, language-consistent, physically feasible, and temporally stable. To address this problem, we propose SemiVLA, a self-distilled teacher-student framework that learns from reliable pseudo-actions on unlabeled trajectories. SemiVLA introduces a VLA-specific reliability controller to assess vision-language alignment, action feasibility, and temporal transition consistency, and further updates the teacher through a Bottleneck-Projected Alignment Update to avoid noisy feedback contamination. With OpenVLA as the backbone, SemiVLA consistently improves multiple PEFT strategies across LIBERO and CALVIN. Under 10% labeled trajectories, SemiVLA with Selective LoRA achieves 89.0% average success on LIBERO, outperforming supervised LoRA by 8.0 points without extra inference cost.
Read the full abstract

Computer Vision · Cross-listed from cs.RO

ASCII Art Turns LLMs into VLA Controllers

Yitao Jiang, Roy Xing, Luyang Zhao et al. (+3 more)

Vision–Language–Action (VLA) controllers are often built by extending vision–language models (VLMs) with action supervision, relying on multimodal backbones with large data and compute requirements. We demonstrate that a text-only large language model (LLM) can be adapted into a VLA-style controller when visual observations are rendered into a text input using an ASCII representation. This ASCII-as-vision interface enables existing training and deployment stacks for LLMs to efficiently condition on visual state, follow natural-language instructions, and produce constrained, executable actions. We fine-tune and compare multiple LLMs and VLMs across model families and scales, using both expert demonstrations from a planning-based teacher, as well as DAgger for iterative improvement. In a 2D manipulation benchmark, in both simulation and on a physical manipulator, the resulting controllers can identify task-relevant entities and plan feasible action sequences. Our results suggest that ASCII rendering can serve as a lightweight, interpretable modality bridge from images to text, complementing conventional VLA pipelines, and opening directions for VLA research with text-only backbones.
Read the full abstract

All authors Yitao Jiang, Roy Xing, Luyang Zhao, Brian Plancher, Muhao Chen, Devin Balkcom

Computer Vision

Native space based pipelines outperform template space based pipeline in subcortical segmentation

Tomás Lima, Daniel Novák and Eduard Bakštein

Accurate segmentation of subcortical regions is critical for neurosurgical planning and functional research. Most automated methods rely on template space coregistration, which may compromise patient-specific accuracy, particularly in small structures. We identify a need to evaluate whether native space approaches offer a measurable advantage, which we evaluate in the context of movement disorders. We developed two UNet-based segmentation pipelines of the Subthalamic Nucleus (STN) - a common surgical target in Parkinson's Disease - and the neighbouring Red Nucleus (RN) and Substantia Nigra (SN). We collected 7T and 3T MRI data from five public datasets. The pipelines were evaluated in the native-space against manual labels. We further investigated the effect of the template resolution. Motivated by the hypothesis that models may better learn target boundaries in higher field, we tested the transferability of 7T-trained models to 3T clinical images, and whether synthetic 3T training data - generated via a disentangled representation learning method - could help bridging this domain gap. On held-out 7T data, the native pipeline consistently outperformed the template one. For the STN, native-space Dice reached 0.775 +- 0.055 versus 0.713 +- 0.051 (1 mm template), with HD95 of 0.79 +- 0.24 mm versus 1.17 +- 1.10 mm, respectively. Similar advantages were observed for the RN and SN. Increasing template resolution did not improve accuracy. When applied to 3T images, all models showed a considerable performance drop. Adding synthetic 3T data yielded only modest improvements, though without degrading 7T performance. Native-space segmentation is preferable for applications requiring patient specific anatomical fidelity, such as the surgical planning in PD. Bridging the 7T-to-3T domain gap remains an open challenge, motivating future work on domain adaptation tailored to subcortical structures.
Read the full abstract

Computer Vision

Technical Report for ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Exploring Query-Based Segmentation and Increased Spatial Context for Outdoor Scene Understanding

David Pascual-Hernández, Roberto Calvo-Palomino, Inmaculada Mora-Jiménez et al. (+1 more)

In this report, we present our submission to the GOOSE 2D Fine-Grained Semantic Segmentation Challenge, organized as part of the Workshop on Field Robotics at ICRA 2026. The challenge combines data from the GOOSE and GOOSE-Ex datasets, which comprise more than 13k images captured from 4 distinct camera setups, annotated using a hierarchical taxonomy of 56 fine-grained classes and 11 broader categories. Starting from SegFormer as a baseline, we progressively improve segmentation performance through increased training crop sizes, a transition to the query-based Mask2Former architecture, and test-time augmentation. Our experiments show that query-based segmentation significantly outperforms the baseline model. Furthermore, increasing the crop size used during training yields substantial gains, highlighting the relevance of preserving scene context for fine-grained semantic disambiguation. Our final submission, using test-time augmentation, achieves an mIoU of 69.6% on the challenge test set, providing a strong baseline for fine-grained semantic segmentation in outdoor environments. To facilitate reproducibility and future research, code and weights will be made publicly available at https://github.com/RoboticsLabURJC/outdoor-fine-grained-segmentation .
Read the full abstract

All authors David Pascual-Hernández, Roberto Calvo-Palomino, Inmaculada Mora-Jiménez, Jose María Cañas-Plaza

Computer Vision · Cross-listed from cs.CL

Precision Recall Controllable Radiology Report Generation via Hybrid Natural Language and Clinical Reward Learning

Ling Chen, Ruinan Jin, Jun Luo et al. (+6 more)

Automated radiology report generation (RRG) has gained increasing attention because it can reduce the heavy workload of clinical report writing. However, most existing methods mainly optimize for natural language generation (NLG) metrics that focus on language fluency, while providing little control over clinically important factors such as precision and recall. As consequence, generated reports may be fluent but not well aligned with different clinical needs. To address this challenge, we propose a reinforcement learning framework for precision recall controllable RRG, where a control parameter explicitly adjusts the trade-off between clinical precision and recall during inference. This design allows the model to flexibly generate reports according to different clinical requirements. To ensure clinical correctness, we introduce a clinical reward into the training objective, which helps improve clinical efficacy (CE) beyond standard language-based optimization. In addition, we apply a group-relative training strategy that normalizes rewards within each training group, reducing reward variance and improving training stability. Extensive experiments on the MIMIC-CXR dataset show that our method consistently outperforms state-of-the-art approaches in both NLG and CE evaluation metrics, while providing reliable control over the CE precision recall trade-off.
Read the full abstract

All authors Ling Chen, Ruinan Jin, Jun Luo, Hanliang Chen, Quirin Strotzer, Rongkai Yan, Yuan Xue, Luciano Prevedello, Dufan Wu

Computer Vision

Synergistic Dual-Branch Adaptation for Multi-modal Generalized Category Discovery

Yuxun Qu, Minyu Zhou, Yongqiang Tang et al. (+2 more)

Generalized Category Discovery (GCD) aims to classify old categories and discover new ones from unlabeled data. Recent multi-modal approaches introduce retrieved or synthesized texts into a dual-branch architecture to provide semantic cues complementary to visual features. However, the cross-modal synergy in existing dual-branch methods remains coarse and incomplete: the two modalities are encoded independently with the bias and noise in the derived text left unaddressed during encoding, and existing mutual learning strategies operate only on global class-level anchors, lacking fine-grained relational supervision. To address these limitations, we propose the Synergistic Dual-Branch Adaptation (SDBA) framework, which serves as a plug-and-play enhancement compatible with existing dual-branch methods such as GET and TextGCD. SDBA comprises two components: the cross-modal synergistic adapter inserts lightweight adapters into both branches and further injects visual information into the text adapter at each encoder layer to enhance text feature learning during encoding; the neighborhood mutual learning module enforces consistent local neighborhood distributions between the two branches via bidirectional KL divergence, providing fine-grained relational supervision for both old and new classes. Extensive experiments on six benchmarks demonstrate state-of-the-art performance, and consistent improvements on different baselines validate the broad scalability of the proposed framework.
Read the full abstract

All authors Yuxun Qu, Minyu Zhou, Yongqiang Tang, Chenyang Zhang, Wensheng Zhang

Computer Vision

MIRCaps: A Large-Scale Mixed-Domain Dataset with Image-Level and Region-Level Captions for Fine-Grained Vision-Language Learning

Arlindo Luciano Tulumba Roberto and Hyungjoon Kim

Despite recent progress in Vision-Language Models (VLMs), mixed-domain image-caption datasets for both general-purpose and CCTV-based video surveillance systems remain limited. To address this gap, we introduce a large-scale multimodal dataset comprising 141,364 images, 981,947 image-level captions, 1,742,264 region-level captions, and 1,391,779 bounding box annotations. Each image is associated with an average of seven image-level captions describing different aspects of the overall scene, as well as seven region-level captions for each annotated bounding box. These complementary caption types are designed to help VLMs learn fine-grained visual attributes, including object categories, estimated sizes, colors, actions, states, and surrounding environmental context. We demonstrate the effectiveness of the dataset on two important downstream tasks: image captioning and object detection. Experimental results show that lightweight VLMs, including SmolVLM-256M-Instruct, BLIP, BLIP2, and Qwen2.5-VL 3B-Instruct, can be effectively fine-tuned using our dataset. Our dataset and code are publicly available at https://zenodo.org/records/20418601.
Read the full abstract

Computer Vision · Cross-listed from eess.IV

2D Versus 3D Diffusion for In Silico Training of Interventional X-ray AI Models

Sampath Rapuri, Jeremy Ko, Benjamin D. Killeen et al. (+2 more)

The ability to synthesize realistic X-ray images has catalyzed the development of AI models for X-ray image-guided procedures, which otherwise suffer from a lack of available annotated data. Prior work has demonstrated the effectiveness of mechanistic simulation of digitally reconstructed radiographs (DRRs) as a training data source for a myriad of tasks, including segmentation and anatomical landmark detection, with comparable or superior performance to real data training. However, mechanistic DRR synthesis still relies on the availability of annotated high-resolution anatomical models. Deriving these from CT images of real patients or specimens imposes an undesirable bottleneck on data quantity and variability. In this work, we explore two methods for synthesizing training data: (1) a 3D conditional latent diffusion model that generates CT volumes to use as inputs for mechanistic DRR generation without real, 3D anatomical models, and (2) a view-conditioned 2D diffusion model that produces synthetic X-rays. In controlled experiments, we demonstrate that synthetic 2D diffusion-based X-rays can be used to train an anatomical landmark detection model that generalized to real X-ray images with performance rivaling that of a model trained on real X-ray images. Thus, we provide preliminary evidence that synthetic, 2D diffusion-based training data can substitute for real X-ray data, identifying a promising avenue towards generating large, diverse datasets for training robust AI models in interventional X-ray imaging.
Read the full abstract

All authors Sampath Rapuri, Jeremy Ko, Benjamin D. Killeen, Russell H. Taylor, Mathias Unberath

Computer Vision · Cross-listed from cs.RO

Robot Self-Improvement via Human-Video Dynamics Models

Hanzhi Chen, Anran Zhang, Simon Schaefer et al. (+5 more)

A central question in robot learning is how to acquire skills from the kinds of data that humans learn from: passive observation, embodied practice, and the experience of failure. Human videos provide the first of these in abundance, and prior work has shown they can initialize useful policies. Far less clear is whether they can support the second and third: whether priors extracted from human videos can ground a robot's own attempts well enough to evaluate them, correct them, and improve from them. In this work, we show that human videos can be used to learn embodiment-agnostic action, dynamics, and value representations that transfer across robot embodiments, providing the predictive foundation required for robots to autonomously improve from their own rollouts and failures. We introduce Dynamics-Guided Action Correction (DGAC), a training-free approach that leverages these adapted models to repair failed states: each failure becomes a query for which the learned models propose and rank corrective actions, turning failures into supervision for the next policy update. Across seven real-world manipulation tasks spanning both a mobile manipulator and a static manipulator arm, our approach improves success rates from 40% to 81% across multiple policy backbones, demonstrating cross-embodiment robot self-improvement from human-video priors. These results show that human priors and robot failures can be combined to enable scalable autonomous policy improvement. Project page: https://ethz-mrl.github.io/robot-self-improvement-website/.
Read the full abstract

All authors Hanzhi Chen, Anran Zhang, Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers, Oier Mees, Stefan Leutenegger

Computer Vision · Cross-listed from cs.LG

VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

Florian Seligmann, Emiliyan Gospodinov, Enes Ulas Dincer et al. (+1 more)

Vision-language-action models (VLAs) achieve state-of-the-art performance on many robotic manipulation tasks, yet they can still behave unpredictably in out-of-distribution scenarios. Runtime failure detection is therefore essential for the safe real-world deployment of VLAs.However, existing task failure detectors require computationally expensive action sampling, are based on architectural assumptions that limit their applicability to VLAs, or need access to failure rollouts. We propose VLA-FAIL, a lightweight and broadly applicable failure detection framework for VLAs that combines two novel failure detectors with minimal overhead, without requiring failure data. The first, last-layer Mahalanobis distance (LLMD), detects out-of-distribution states by measuring token-wise deviations in last-layer features relative to the training data. The second, action chunk consistency (ACC), exploits the temporal overlap induced by receding-horizon control and detects failures when consecutive action chunks become inconsistent. To capture the trade-off between detection accuracy and detection latency, we introduce AUCPDT, a threshold-independent metric that jointly evaluates precision, recall, and detection time. Through extensive real-world and simulation experiments, we demonstrate that LLMD and ACC capture complementary failure modes whose combination enables reliable and early failure detection across diverse tasks, frequently outperforming significantly more expensive baseline methods. Project page: https://vla-fail.github.io/vla-fail-2026
Read the full abstract

All authors Florian Seligmann, Emiliyan Gospodinov, Enes Ulas Dincer, Gerhard Neumann

Computer Vision

EnTrust: Modeling Inter-Modal Conflict for Trustworthy Multimodal Medical Image Analysis

Dwarikanath Mahapatra, Abhijit Das, Behzad Bozorgtabar et al. (+5 more)

Multimodal medical imaging fuses complementary anatomical and functional information, yet modalities frequently disagree in pathologically heterogeneous regions. Current segmentation models handle this in one of two inadequate ways: deterministic fusion that averages away disagreement, or post-hoc uncertainty estimation decoupled from the fusion process that produces it. Both obscure the clinically critical question: why is this prediction unreliable? We present EnTrust, a framework that treats inter-modal conflict as the primary source of predictive uncertainty. Our EnFuse module decomposes multimodal features into three disentangled components: shared anatomical consensus (F_c), modality-specific cues (F_{u,m}), and spatially localized conflict signals (F_{cf}), with independence enforced via a cross-covariance objective. This structured decomposition conditions SegDiff, a diffusion-based generative segmentation model whose sampled hypotheses diverge specifically in regions of modal disagreement. TrustMap then translates this hypothesis divergence into calibrated, pixel-wise uncertainty using ensemble entropy, conflict-guided perturbation probing, and a learned calibration head, enabling clinicians to understand not only where predictions are uncertain, but why. Across four benchmarks spanning brain, cardiac, lesion, and oncology domains, EnTrust achieves state-of-the-art segmentation accuracy while reducing calibration error by 40% compared to the strongest baseline. Notably, it outperforms 5x deep ensembles using a single model at roughly half the memory footprint. Code and checkpoints are available at https://github.com/GenMI-Lab/EnTrust.git.
Read the full abstract

All authors Dwarikanath Mahapatra, Abhijit Das, Behzad Bozorgtabar, Zongyuan Ge, Sudipta Roy, Deepak Nayak, Mauricio Reyes, Imran Razzak

Computer Vision

OSOG: A Differentiable, Physics-Informed Synthetic Data Engine for Micro-Optical Environments

Caio Silva

Deep learning in computational microscopy is severely constrained by the scarcity of densely annotated datasets. While synthetic data generation has bridged this gap in macroscopic computer vision, traditional graphics engines rely on geometric ray-tracing, failing to capture the micro-optical phenomena required for microscopy. Conversely, while wave-optics formulations exist, rendering them computationally tractable at the scale required for deep learning remains a massive systems challenge. To address this, we introduce the Optical Synthetic Object Generator (OSOG), a high-performance, fully differentiable forward-modeling engine. Drawing on established physical models of diffraction and phase retardation, OSOG maps continuous Optical Path Difference (OPD) calculations into a highly optimized, PyTorch-native Structure-of-Arrays (SoA) architecture. We validate this computational framework across three axes: First, object detection models (YOLOv11-OBB) trained purely on OSOG-generated data achieve robust zero-shot transfer to real-world highly occluded Lysozyme micrographs. Second, we introduce DiffOSOG, demonstrating that the engine's end-to-end differentiability allows for the exact recovery of continuous optical parameters via curriculum-guided inverse rendering. Finally, OSOG bypasses the $\mathcal{O}(N)$ bottlenecks of sequential ray-tracing, demonstrating sub-linear scaling by synthesizing 40,000 complex wave-optic particles in under 50 milliseconds (\>20 FPS). By providing a fast, scalable, and physically grounded tensor pipeline, OSOG enables true real-time, on-the-fly dataset generation.
Read the full abstract

Computer Vision

FLM-Occ: Feed-forward Likelihood Maximization for Efficient Indoor Occupancy Prediction

Guangcheng Chen, Lihuang Fang, Huaqi Tao et al. (+3 more)

Recent indoor occupancy prediction methods adopt Gaussian primitives as a sparse 3D representation for computational efficiency. However, their training relies on voxel classification, which imposes only local constraints and lacks global supervision on the distribution of the primitives. Therefore, they inevitably predict spurious primitives in empty regions, undermining both representational and computational efficiency. To address this, we propose Feed-forward Likelihood Maximization (FLM), a novel framework that reformulates occupancy prediction as voxel distribution estimation. In FLM, a network is trained to predict a mixture model that maximizes the likelihood over ground-truth occupied voxels in a feed-forward manner. To enable end-to-end training of networks and voxelization of a standard mixture model, we define mixture weights as normalized primitive volumes to implicitly enforce simplex constraints and derive novel voxelization formulas. Based on FLM, our FLM-Occ, a novel method that is capable of relocating randomly initialized primitives over long distances to model a scene. On Occ-ScanNet, FLM-Occ achieves superior accuracy using only 32 superquadrics, 2.7% of the prior SoTA, while running 3.7 times faster.
Read the full abstract

All authors Guangcheng Chen, Lihuang Fang, Huaqi Tao, Yicheng He, Li He, Hong Zhang

Computer Vision

Graph-of-Differences: Anatomy-Structured Difference Alignment for Medical Image Re-Identification

Nichula Wasalathilaka, Abhijit Das, Imran Razzak et al. (+1 more)

Medical image re-identification (MedReID) enables longitudinal patient linkage but remains vulnerable to shortcut learning and often produces decisions that clinicians cannot audit against named anatomy. We propose Graph-of-Differences (GoD), which grounds identity comparisons in explicit anatomical structure. Each image is represented as an anatomy graph whose nodes correspond to named anatomical regions; given an image pair, soft node correspondence is established, and differences are computed over matched anatomy. A graph-level difference alignment objective ties these anatomy-matched differences to the global backbone difference, ensuring the retrieval signal is anchored in homologous structures rather than arbitrary spatial tokens. Explanations are defined over named graph nodes and quantitatively audited via node insertion/deletion tests, replacing unstable pixel heatmaps with verifiable structure-level evidence. On internal benchmarks, GoD improves Rank-1 by +7.1 pp on fundus and +3.1 pp on CXR over a strong frozen-backbone baseline, with further gains on zero-shot external transfers confirming that anatomy grounding improves both accuracy and generalization. Code is available at https://github.com/GenMI-Lab/GoD.git.
Read the full abstract

All authors Nichula Wasalathilaka, Abhijit Das, Imran Razzak, Dwarikanath Mahapatra

Computer Vision

LEViL: Label-Efficient Video Learning via Zero-Shot Distillation over VLM-Generated Pseudo-Label Spaces

Asl{\i} Çelik

Supervised video pretraining is a common transfer learning practice for improving downstream action recognition performance. However, it requires large-scale labeled source datasets, and the effectiveness of the learned initialization is influenced by the similarity between the source and target domains. Constructing such labeled pretraining datasets for different target domains is costly and difficult to scale. To address these limitations, this study proposes a label-efficient video learning framework that combines annotation-free video pretraining with target-label-set-aware fine-tuning. During pretraining, a vision-language model (VLM) generates textual descriptions of unlabeled videos, which are processed to construct an interpretable semantic pseudo-label space. A frozen video-language model then produces zero-shot soft target distributions over this space, allowing a student video encoder to learn semantically rich representations without manual source annotations. During downstream adaptation, target-label-set-aware fine-tuning combines supervised learning from labeled target videos with zero-shot distillation over the actual target label set, helping preserve VLM-derived semantic guidance while adapting the pretrained encoder to the target task. Experiments on UCF101 and HMDB51 show that the proposed framework outperforms the compared semi-supervised video action recognition methods across all evaluated limited-label regimes. Moreover, the annotation-free pretraining stage learns transferable representations that provide an effective initialization for full-data fine-tuning, despite relying on a comparatively modest unlabeled pretraining pool.
Read the full abstract

Computer Vision

WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlife

Vandita Shukla, Kilian Meier, Lucie Laporte-Devylder et al. (+6 more)

We introduce WildBox, a dataset and benchmark for monocular 3D detection of wildlife from drone video, comprising 237,505 3D bounding box annotations across seven African savanna species grouped into six benchmark classes. Annotations follow a KITTI/Omni3D-compatible format in a per-segment scale-normalised camera frame, with instance identities maintained across each segment. We evaluate two open-vocabulary monocular 3D architectures, OVMono3D-LIFT and DetAny3D, under zero-shot, ground-truth 2D box prompt, and supervised fine-tuning protocols. Open-vocabulary 2D foundation models provide usable zero-shot wildlife localisation (50.55 AP@50), but zero-shot 3D detection collapses to 0.00 AP across both architectures and every 2D-input condition tested, including ground-truth 2D box prompts, thus isolating the failure to the 3D stage. Fine-tuning on WildBox recovers performance to 8.68 +/- 0.47 AP-BEV@0.50 and 13.17 +/- 0.69 AP3D macro. Depth contributes 84% of normalised Hausdorff distance after fine-tuning and over 99% in zero-shot, identifying monocular aerial depth as the dominant open problem in this regime. A coarse-to-fine curriculum, i.e. pretraining on a merged zebra class before fine-tuning on the Grevy's/plains split, improves macro 3D performance with less total compute, with the largest gains on the two zebra subclasses. WildBox is released with video-level splits and baseline checkpoints to enable progress in 3D wildlife perception from drone video.
Read the full abstract

All authors Vandita Shukla, Kilian Meier, Lucie Laporte-Devylder, Camille Rondeau Saint-Jean, Jenna M. Kline, Blair R. Costelloe, Devis Tuia, Fabio Remondino, Benjamin Risse

Computer Vision

A Test-time Actor-Critic Approach to News Images Generation

Damianos Galanopoulos and Vasileios Mezaris

This paper introduces the CERTH-ITI solution for the MediaEval NewsImages 2026 challenge, which focuses on generating images related to news headlines. Inspired by the Actor-Critic paradigm in reinforcement learning, we present a test-time, model-agnostic Actor-Critic Image Generation approach (ACIG). ACIG generates prompts for image creation, produces the images, evaluates the generated results, and if needed refines the image generation prompts accordingly in a feedback loop. ACIG achieved the best results in the NewsImages 2026 challenge, according to the challenge's leaderboard.
Read the full abstract

Computer Vision

SCOPE: Scale-Consistent One-Pass Estimation of 3D Geometry

Zheng Zhang, Lihe Yang, Tianyu Yang et al. (+6 more)

We present SCOPE (Scale-Consistent One-Pass Estimation of 3D Geometry), a novel approach for estimating 3D geometry from extended monocular video sequences, where existing methods struggle to maintain both geometric accuracy and temporal consistency across hundreds of frames. Our approach generates affine-invariant 3D point maps with shared parameters across entire sequences, enabling consistent scale-invariant representations. We introduce three key innovations: viewpoint-invariant geometry aligning multi-perspective points in a unified reference frame; appearance-invariant learning enforcing consistency across exponential timescales; and frequency-modulated positioning enabling extrapolation to sequences vastly exceeding training length. Experiments across diverse datasets demonstrate significant improvements, reducing relative point map error by 24.2% and temporal alignment error by 34.9% on ScanNet compared to state-of-the-art methods. Our approach handles challenging scenarios with complex camera trajectories and lighting variations while efficiently processing extended sequences in a single pass. Project page: https://scope3d.github.io/.
Read the full abstract

All authors Zheng Zhang, Lihe Yang, Tianyu Yang, Chaohui Yu, Yixing Lao, Xiaoyang Guo, Biao Gong, Fan Wang, Hengshuang Zhao

Computer Vision

Lightweight 3D Feature Pretraining by Bayesian Inversion of 2D Foundation Models

Marwane Hariat, Gianni Franchi, David Filliat et al. (+1 more)

We present Casper3D, a lightweight probabilistic framework for converting noisy multi-view 2D foundation-model embeddings into a latent 3D semantic representation. We model view-level semantic features as noisy observations of an underlying 3D semantic state and infer this state with a set-based variational model that incorporates relative pose during multi-view reasoning. Casper3D is trained by predicting held-out semantic observations from novel viewpoints, while remaining aligned with visual and text semantic spaces for open-vocabulary 3D understanding. The framework is backbone-agnostic and applies to both language-aligned and self-supervised embeddings. Experiments show that Casper3D produces more stable 3D semantics than simple multi-view pooling, especially in ambiguous and noisy settings.
Read the full abstract

All authors Marwane Hariat, Gianni Franchi, David Filliat, Antoine Manzanera

Computer Vision

NoduLoCC2026: Lung Nodule Localization and Classification Contest from Chest X-Ray Images

Adnan Mustafic, Halim Benhabiles, Adnane Cabani et al. (+19 more)

We propose NoduLoCC2026, a challenge on lung nodule detection and localization in chest X-ray images. We have provided a dataset for both tasks and received submissions from 5 international teams. The participating teams' solutions are presented in this work along with results on an external dataset used for testing. Proposed methods show good performance on the classification task. The best method shows a balanced accuracy score of 0.72 and AUC-ROC of 0.79. We highlight the limitations of current approaches for the localization task, with the best approach having predicted the correct number of nodules on 53% of the test images with a median distance of 12.83mm, showing that it is a more challenging task than the first one. The challenge website is available via https://gt-i2mdp.github.io/website/nodule_challenge.html.
Read the full abstract

All authors Adnan Mustafic, Halim Benhabiles, Adnane Cabani, Kristhian André Oliveira Aguilar, Romain Amigon, Clément Bardin, Chiara Bentifece, Marin Boehm, Kévin Bouchard, Laura Burattini, Diedre Carmo, Fahima Idiri, Matthis Lahargoue, Ilaria Marcantoni, Hicham Messaoudi, Cyril Meyer, Farid Meziane, Léon Morales, Letícia Rittner, Agnese Sbrollini, Léonard Zipper, Karim Hammoudi

Computer Vision

Unsupervised Domain Adaptation for Sim-to-Real Object Pose Estimation with Contrastive Alignment and Pseudo-Label Refinement

Nidhal Eddine Chenni, Arunkumar Rathinam and Djamila Aouada

Unsupervised domain adaptation (UDA) enables robust transfer of knowledge from simulated to real environments while exploiting a subset of unlabeled target data to improve real-world performance. Existing UDA methods for Object pose estimation often rely on global feature matching, multi-stage larger frameworks, or image translation pipelines, which tend to overlook the pose-specific information embedded in feature representations. To bridge this limitation, we introduce CAPLR that targets the adaptation of pose-sensitive features in localized regions, ensuring that domain alignment preserves the geometric cues essential for accurate pose estimation. CAPLR achieves UDA with three key components: (1) Efficient Cross-Domain Pairing strategy leveraging intermediate features to identify pose similar image pairs across domains without supervision; (2) Contrastive Alignment to perform feature alignment at localised regions in both intermediate and task-specific representations; and (3) Consistency-Based Pseudo-Label Refinement to improve reliability by encouraging stable target predictions. Extensive experiments demonstrate that CAPLR achieves state-of-the-art performance across multiple well-known object pose estimation benchmarks featuring diverse and challenging scenarios.
Read the full abstract

Computer Vision

Beyond Damage Assessment: Recyclable Material Detection in Aerial Disaster Imagery Using a Lightweight Patch-Based Framework

Mahmoud Hazem and Karim Hammoudi

Nowadays, more and more disasters of different natures are appearing. Several disaster assessment approaches have been developed in order to identify damaged areas from aerial images. These damaged areas contain rich material that could be recycled towards several ecological purposes. In this paper, we present a lightweight approach that permits the efficient detection of recyclable material. Experimental results show the potential of the proposed approach towards localizing recyclable materials. Accordingly, we provide a rare dataset of material images that we labeled towards supporting the development of recyclable material detectors. The dataset of labeled material images is publicly available at: anonymous.
Read the full abstract

Computer Vision · Cross-listed from physics.optics

Non-line-of-sight imaging with arbitrary relay surface geometries via 3D Gaussian Transient Rendering

Yi Wang, Ziyu Zhan, Yuran Wang et al. (+5 more)

Imaging objects hidden outside the direct line of sight expands the effective field of view and is critical for applications such as autonomous driving and robotic perception. Despite impressive progress in time-of-flight (ToF)-based non-line-of-sight (NLOS) imaging, real-world deployment remains challenging because practical measurements are often collected over spatially limited, arbitrarily shaped relay regions-conditions that violate the planar-wall and dense-sampling assumptions made by most existing methods. To address these limitations, we propose a LOS-guided NLOS imaging pipeline that imposes no geometric assumptions on the relay surface and naturally supports both confocal and non-confocal configurations. Our method represents the hidden scene using 3D Gaussian primitives and couples them with an efficient, differentiable transient rendering model, enabling end-to-end optimization directly from measured transients. We validate our approach on real-world measurements from both a public dataset and a custom-built capture system. Across settings, our method achieves state-of-the-art reconstruction fidelity under spatially limited, sparsely sampled conditions, and significantly outperforms existing methods on complex, arbitrary relay surface geometries.
Read the full abstract

All authors Yi Wang, Ziyu Zhan, Yuran Wang, Hao Wang, Qiang Liu, Zuoqiang Shi, Lingyun Qiu, Xing Fu