Computer Vision · Cross-listed from cs.LG
K-STEMIT: Knowledge-Informed Spatio-Temporal Efficient Multi-Branch Graph Neural Network for Subsurface Stratigraphy Thickness Estimation from Radar Data
Zesheng Liu and Maryam Rahnemoonfar
Subsurface stratigraphy contains important spatio-temporal information about accumulation, deformation, and layer formation in polar ice sheets. In particular, variations in internal ice layer thickness provide valuable constraints for snow mass balance estimation and projections of ice sheet change. Although radar sensors can capture these layered structures as depth-resolved radargrams, convolutional neural networks applied directly to radar images are often sensitive to speckle noise and acquisition artifacts. In addition, purely data-driven methods may underuse physical knowledge, leading to unrealistic thickness estimates under spatial or temporal extrapolation. To address these challenges, we develop K-STEMIT, a novel knowledge-informed, efficient, multi-branch spatio-temporal graph neural network that combines a
geometric framework for spatial learning with temporal convolution to capture temporal dynamics, and incorporates physical data synchronized from the Model Atmospheric Regional physical weather model. An adaptive feature fusion strategy is employed to dynamically combine features learned from different branches. Extensive experiments have been conducted to compare K-STEMIT against current state-of-the-art methods in both knowledge-informed and non-knowledge-informed settings, as well as other existing methods. Results show that K-STEMIT consistently achieves the highest accuracy while maintaining near-optimal efficiency. Most notably, incorporating adaptive feature fusion and physical priors reduces the root mean-squared error by 21.01% with negligible additional cost compared to its conventional multi-branch variants. Additionally, our proposed K-STEMIT achieves consistently lower per-year relative MAE, enabling reliable, continuous spatiotemporal assessment of snow accumulation variability across large spatial regions.
Read the full abstract ▾
Computer Vision
Does Your VFM Speak Plant? The Botanical Grammar of Vision Foundation Models for Object Detection
Lars Lundqvist, Earl Ranario, Hamid Kamangir et al. (+4 more)
Vision foundation models (VFMs) offer the promise of zero-shot object detection without task-specific training data, yet their performance in complex agricultural scenes remains highly sensitive to text
prompt construction. We present a systematic
prompt optimization framework evaluating four open-vocabulary detectors – YOLO World, SAM3, Grounding DINO, and OWLv2 – for cowpea flower and pod detection across synthetic and real field imagery. We decompose
prompts into eight axes and conduct one-factor-at-a-time analysis followed by combinatorial optimization, revealing that models respond divergently to
prompt structure: conditions that optimize one architecture can collapse another. Applying model-specific combinatorial
prompts yields substantial gains over a naive species-name baseline, including +0.357 mAP@0.5 for YOLO World and +0.362 mAP@0.5 for OWLv2 on synthetic cowpea flower data. To evaluate cross-task generalization, we use an LLM to translate the discovered axis structure to a morphologically distinct target – cowpea pods – and compare against prompting using the discovered optimal structures from synthetic flower data. Crucially,
prompt structures optimized exclusively on synthetic data transfer effectively to real-world fields: synthetic-pipeline
prompts match or exceed those discovered on labeled real data for the majority of model-object combinations (flower: 0.374 vs. 0.353 for YOLO World; pod: 0.429 vs. 0.371 for SAM3). Our findings demonstrate that
prompt engineering can substantially close the gap between zero-shot VFMs and supervised detectors without requiring manual annotation, and that optimal
prompts are model-specific, non-obvious, and transferable across domains.
Read the full abstract ▾
All authors Lars Lundqvist, Earl Ranario, Hamid Kamangir, Heesup Yun, Christine Diepenbrock, Brian N. Bailey, J. Mason Earles
Computer Vision
From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping
Yu Wu, Guangzeng Han, Ibra Niang Niang et al. (+6 more)
High-throughput, effective and comprehensive phenotyping is a critical prerequisite for crop improvement. While phenotypic analysis was traditionally performed manually, recent advances in
multimodal foundation models, especially in
vision-
language models, have offered new opportunities for automated plant phenotyping. However, plant science remains particularly challenging for general-purpose VLMs because it requires domain-specific knowledge, fine-grained visual interpretation, and complex agronomic
reasoning. We introduce PlantXpert, an evidence-grounded
multimodal reasoning benchmark for plant phenotyping across four major crops: soybean, cotton, rice, and corn. Our
benchmark provides a structured and reproducible framework for agronomic adaptation of VLMs, and enables controlled comparison between base models and their domain-adapted counterparts. We constructed a
dataset comprising 545 digital images and 4,351
benchmark samples spanning key agronomic domains including disease, pest control, weed management and yield. The
benchmark can assess diverse capabilities including visual expertise, quantitative
reasoning, and multi-step agronomic
reasoning. We trained and evaluated a total of 11 state-of-the-art VLMs. Fine-tuning on our
dataset improves overall accuracy for all models, with many achieving around 70% accuracy, and improvements on 9 of 11 remaining statistically significant after Holm correction. We also identify diminishing returns from model scaling, uneven performance across crop subsets, and persistent weaknesses in quantitative
reasoning and specialized image analysis. These limitations show that general
multimodal competence alone is insufficient for reliable agronomic
reasoning and demonstrate the value of domain-specific adaptation. We hope PlantXpert can serve as a foundation for advancing evidence-grounded
multimodal reasoning in plant science.
Read the full abstract ▾
All authors Yu Wu, Guangzeng Han, Ibra Niang Niang, Francia Ravelombola, Maiara Oliveira, Jason Davis, Dong Chen, Feng Lin, Xiaolei Huang
Computer Vision
PointSplat: Efficient Geometry-Driven Pruning and Transformer Refinement for 3D Gaussian Splatting
Anh Thuan Tran and Jana Kosecka
3D Gaussian Splatting (3DGS) has recently unlocked real-time, high-fidelity novel view synthesis by representing scenes using explicit 3D primitives. However, traditional methods often require millions of Gaussians to capture complex scenes, leading to significant
memory and storage demands. Recent approaches have addressed this issue through
pruning and per-scene fine-tuning of Gaussian parameters, thereby reducing the model size while maintaining visual quality. These strategies typically rely on 2D images to compute important scores followed by scene-specific optimization. In this work, we introduce PointSplat, 3D geometry-driven prune-and-refine framework that bridges previously disjoint directions of gaussian
pruning and
transformer refinement. Our method includes two key components: (1) an efficient geometry-driven strategy that ranks Gaussians based solely on their 3D attributes, removing reliance on 2D images during
pruning stage, and (2) a dual-branch encoder that separates, re-weights
geometric and appearance to avoid feature imbalance. Extensive experiments on ScanNet++ and Replica across varying sparsity levels demonstrate that PointSplat consistently achieves competitive rendering quality and superior efficiency without additional per-scene optimization.
Read the full abstract ▾
Computer Vision
Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception
Gautham Vinod, Bruce Coburn, Siddeshwar Raghavan et al. (+1 more)
Accurate volume estimation of objects from visual data is a long-standing challenge in computer
vision with significant applications in robotics, logistics, and smart health. Existing methods often rely on complex 3D reconstruction pipelines or struggle with the ambiguity inherent in single-view images. To address these limitations, we introduce a new method that fuses implicit 3D cues from stereo
vision with explicit prior knowledge from natural
language text. Our approach extracts deep features from a stereo image pair and a descriptive text
prompt that contains the object's class and an approximate volume, then integrates them using a simple yet effective projection layer into a unified, multi-modal representation for regression. We conduct extensive experiments on public
datasets demonstrating that our text-guided approach significantly outperforms
vision-only baselines. Our findings show that leveraging even simple textual priors can effectively guide the volume estimation task, paving the way for more context-aware visual measurement systems. Code: https://gitlab.com/viper-purdue/stereo-typical-
estimator.
Read the full abstract ▾
All authors Gautham Vinod, Bruce Coburn, Siddeshwar Raghavan, Fengqing Zhu
Computer Vision
Topo-ADV: Generating Topology-Driven Imperceptible Adversarial Point Clouds
Gayathry Chandramana Krishnan Nampoothiry, Raghuram Venkatapuram, Anirban Ghosh et al. (+1 more)
Deep neural networks for 3D point cloud understanding have achieved remarkable success in object classification and recognition, yet recent work shows that these models remain highly vulnerable to
adversarial perturbations. Existing 3D attacks predominantly manipulate
geometric properties such as point locations,
curvature, or surface structure, implicitly assuming that preserving global shape fidelity preserves semantic content. In this work, we challenge this assumption and introduce the first
topology-driven
adversarial attack for point cloud deep learning. Our key insight is that the homological structure of a 3D object constitutes a previously unexplored vulnerability surface. We propose Topo-ADV, an end-to-end differentiable framework that incorporates persistent homology as an explicit optimization objective, enabling gradient-based manipulation of topological features during
adversarial example generation. By
embedding persistence diagrams through differentiable topological representations, our method jointly optimizes (i) a
topology divergence loss that alters persistence, (ii) a misclassification objective, and (iii)
geometric imperceptibility constraints that preserve visual plausibility. Experiments demonstrate that subtle
topology-driven perturbations consistently achieve up to 100% attack success rates on
benchmark datasets such as ModelNet40, ShapeNet Part, and ScanObjectNN using PointNet and DGCNN classifiers, while remaining geometrically indistinguishable from the original point clouds, beating state-of-the-art methods on various perceptibility metrics.
Read the full abstract ▾
All authors Gayathry Chandramana Krishnan Nampoothiry, Raghuram Venkatapuram, Anirban Ghosh, Ayan Dutta
Computer Vision
Genie 4D: Semantic-Prior-Guided 4D Dynamic Scene Reconstruction
Yiru Yang, Zhuojie Wu, Nishant Kumar Singh et al. (+1 more)
At the intersection of computer
vision and robotic perception, 4D reconstruction of dynamic scenes connects low-level
geometric sensing with high-level semantic understanding. We present Genie 4D, a framework that turns hand-held phone capture into a semantically grounded, action-controllable 4D world model. Genie 4D couples a real-time visual-inertial Gaussian splatting front-end for metric geometry with a feed-forward 4D backbone regularized by frozen DINOv3 features acting as structural priors. The semantic priors suppress identity drift during dynamic tracking, while a short conditional
diffusion refiner recovers high-frequency surface detail that regression backbones smooth away. Finally, a lightweight latent-action head exposes the reconstructed 4D state to a Genie-style world model trained with a JEPA-style next-
embedding objective, so that the scene can be rolled forward under user actions. On the Point Odyssey and TUM-Dynamics
benchmarks, Genie 4D retains the linear time complexity O(T) of feed-forward baselines while improving 3D tracking accuracy (APD) and reconstruction completeness, and it runs interactively on a single consumer GPU (RTX 5090) from iPhone, Mac, Windows, and Linux capture clients. Genie 4D offers a practical, semantic-prior-guided path toward physically grounded world models.
Read the full abstract ▾
All authors Yiru Yang, Zhuojie Wu, Nishant Kumar Singh, Max Schulthess
Computer Vision · Cross-listed from cs.LG
Efficient Personalization of Generative User Interfaces
Yi-Hao Peng, Jeffrey P. Bigham and Jason Wu
Generative user interfaces (GenUIs) create new opportunities to adapt interfaces to individual users on demand. Yet personalization is difficult because it is not possible to provide settings for screens that have not yet been generated, making it necessary to learn preferences from users' feedback on generated interfaces. Such feedback is sparse, subjective, and often difficult to articulate. We study this problem through a new
dataset in which 20 participants each judge the same 600 pairs of GenUIs. Their judgments differ substantially (Krippendorff's alpha = 0.25), and written rationales show that even when participants consider similar UI properties, such as color tone or hierarchy, they often make different choices and prioritize them differently. We develop a sample-efficient personalization method that uses a few pairwise judgments to weight prior users' preferences rather than relying on a fixed rubric of UI attributes. Our method outperforms a pretrained UI evaluator and a larger
multimodal model offline. In an online study with 12 new users, participants preferred interfaces personalized by our method over all baselines, including shared guidelines and users' own written preferences.
Read the full abstract ▾
Computer Vision
PAS: Estimating the target accuracy before domain adaptation
Raphaella Diniz, Jackson de Faria and Martin Ester
The goal of domain adaptation is to make predictions for unlabeled samples from a target domain with the help of labeled samples from a different but related source domain. The performance of domain adaptation methods is highly influenced by the choice of source domain and pre-trained feature extractor. However, the selection of source data and pre-trained model is not trivial due to the absence of a labeled validation set for the target domain and the large number of available pre-trained models. In this work, we propose PAS, a novel score designed to estimate the transferability of a source domain set and a pre-trained feature extractor to a target classification task before actually performing domain adaptation. PAS leverages the generalization power of pre-trained models and assesses source-target compatibility based on the pre-trained feature
embeddings. We integrate PAS into a framework that indicates the most relevant pre-trained model and source domain among multiple candidates, thus improving target accuracy while reducing the computational overhead. Extensive experiments on image classification
benchmarks demonstrate that PAS correlates strongly with actual target accuracy and consistently guides the selection of the best-performing pre-trained model and source domain for adaptation.
Read the full abstract ▾
Computer Vision
FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views
Chaoyi Zhou, Run Wang, Feng Luo et al. (+4 more)
Recent advances in
vision foundation models have revolutionized geometry reconstruction and semantic understanding. Yet, most of the existing approaches treat these capabilities in isolation, leading to redundant pipelines and compounded errors. This paper introduces FF3R, a fully annotation-free feed-forward framework that unifies
geometric and semantic
reasoning from unconstrained multi-view image sequences. Unlike previous methods, FF3R does not require camera poses, depth maps, or semantic labels, relying solely on rendering supervision for RGB and feature maps, establishing a scalable paradigm for unified 3D
reasoning. In addition, we address two critical challenges in feedforward feature reconstruction pipelines, namely global semantic inconsistency and local structural inconsistency, through two key innovations: (i) a
Token-wise Fusion Module that enriches geometry
tokens with semantic context via cross-
attention, and (ii) a Semantic-Geometry Mutual Boosting mechanism combining geometry-guided feature warping for global consistency with semantic-aware voxelization for local coherence. Extensive experiments on ScanNet and DL3DV-10K demonstrate FF3R's superior performance in novel-view synthesis, open-vocabulary semantic
segmentation, and depth estimation, with strong generalization to in-the-wild scenarios, paving the way for
embodied intelligence systems that demand both spatial and semantic understanding.
Read the full abstract ▾
All authors Chaoyi Zhou, Run Wang, Feng Luo, Mert D. Pesé, Zhiwen Fan, Yiqi Zhong, Siyu Huang
Computer Vision
Do vision models perceive illusory motion in static images like humans?
Isabella Elaine Rosario, Fan L. Cheng, Zitang Sun et al. (+1 more)
Understanding human motion processing is essential for building reliable, human-centered computer
vision systems. Although deep neural networks (DNNs) achieve strong performance in optical flow estimation, they remain less robust than humans and rely on fundamentally different computational strategies. Visual motion illusions provide a powerful probe into these mechanisms, revealing how human and machine
vision align or diverge. While recent DNN-based motion models can reproduce dynamic illusions such as reverse-phi, it remains unclear whether they can perceive illusory motion in static images, exemplified by the Rotating Snakes illusion. We evaluate several representative optical flow models on Rotating Snakes and show that most fail to generate flow fields consistent with human perception. Under simulated conditions mimicking saccadic eye movements, only the human-inspired Dual-Channel model exhibits the expected rotational motion, with the closest correspondence emerging during the saccade simulation. Ablation analyses further reveal that both luminance-based and higher-order color–feature–based motion signals contribute to this behavior and that a recurrent
attention mechanism is critical for integrating local cues. Our results highlight a substantial gap between current optical-flow models and human visual motion processing, and offer insights for developing future motion-estimation systems with improved correspondence to human perception and human-centric AI.
Read the full abstract ▾
All authors Isabella Elaine Rosario, Fan L. Cheng, Zitang Sun, Nikolaus Kriegeskorte
Computer Vision
Training-Free Object-Background Compositional T2I via Dynamic Spatial Guidance and Multi-Path Pruning
Yang Deng, David Mould, Paul L. Rosin et al. (+1 more)
Existing text-to-image
diffusion models, while excelling at subject synthesis, exhibit a persistent foreground bias that treats the background as a passive and under-optimized byproduct. This imbalance compromises global scene coherence and constrains compositional control. To address the limitation, we propose a training-free framework that restructures
diffusion sampling to explicitly account for foreground-background interactions. Our approach consists of two key components. First, Dynamic Spatial Guidance introduces a soft, time step dependent gating mechanism that modulates foreground and background
attention during the
diffusion process, enabling spatially balanced generation. Second, Multi-Path
Pruning performs multi-path latent exploration and dynamically filters candidate trajectories using both internal
attention statistics and external semantic
alignment signals, retaining trajectories that better satisfy object-background constraints. We further develop a
benchmark specifically designed to evaluate object-background compositionality. Extensive evaluations across multiple
diffusion backbones demonstrate consistent improvements in background coherence and object-background compositional
alignment.
Read the full abstract ▾
All authors Yang Deng, David Mould, Paul L. Rosin, Yu-Kun Lai
Computer Vision
Is There Knowledge Left to Extract? Evidence of Fragility in Medically Fine-Tuned Vision-Language Models
Oliver McLaughlin, Daniel Shubin, Carsten Eickhoff et al. (+3 more)
Vision-
language models (VLMs) are increasingly adapted through domain-specific fine-tuning, yet it remains unclear whether this improves
reasoning beyond superficial visual cues, particularly in high-stakes domains like medicine. We evaluate four paired open-source VLMs (LLaVA vs. LLaVA-Med; Gemma vs. MedGemma) across four medical imaging tasks of increasing difficulty:
brain tumor, pneumonia, skin cancer, and histopathology classification. We find that performance degrades toward near-random levels as task difficulty increases, indicating limited clinical
reasoning. Medical fine-tuning provides no consistent advantage, and models are highly sensitive to
prompt formulation, with minor changes causing large swings in accuracy and refusal rates. To test whether closed-form VQA suppresses latent knowledge, we introduce a description-based pipeline where models generate image descriptions that a text-only model (GPT-5.1) uses for diagnosis. This recovers a limited additional signal but remains bounded by task difficulty. Analysis of
vision encoder
embeddings further shows that failures stem from both weak visual representations and downstream
reasoning. Overall, medical VLM performance is fragile,
prompt-dependent, and not reliably improved by domain-specific fine-tuning.
Read the full abstract ▾
All authors Oliver McLaughlin, Daniel Shubin, Carsten Eickhoff, Ritambhara Singh, William Rudman, Michal Golovanevsky
Computer Vision
Vector Field Synthesis with Sparse Streamlines Using Diffusion Model
Nguyen K. Phan, Ricardo Morales, Sebastian D. Espriella et al. (+1 more)
We present a novel
diffusion-based framework for synthesizing 2D vector fields from sparse, coherent inputs (i.e., streamlines) while maintaining physical plausibility. Our method employs a conditional denoising
diffusion probabilistic model with classifier-free guidance, enabling progressive reconstruction that preserves both
geometric and physical constraints. Experimental results demonstrate our method's ability to synthesize plausible vector fields that adhere to physical laws while maintaining fidelity to sparse input observations, outperforming traditional optimization-based approaches in terms of flexibility and physical consistency.
Read the full abstract ▾
All authors Nguyen K. Phan, Ricardo Morales, Sebastian D. Espriella, Guoning Chen
Computer Vision
F3G-Avatar : Face Focused Full-body Gaussian Avatar
Willem Menu, Erkut Akdag, Pedro Quesado et al. (+2 more)
Existing full-body Gaussian avatar methods primarily optimize global reconstruction quality and often fail to preserve fine-grained facial geometry and expression details. This challenge arises from limited facial representational capacity that causes difficulties in modeling high-frequency pose-dependent deformations. To address this, we propose F3G-Avatar, a full-body, face-aware avatar synthesis method that reconstructs animatable human representations from multi-view RGB
video and regressed pose/shape parameters. Starting from a clothed Momentum Human Rig (MHR) template, front/back positional maps are rendered and decoded into 3D Gaussians through a two-branch architecture: a body branch that captures pose-dependent non-rigid deformations and a face-focused deformation branch that refines head geometry and appearance. The predicted Gaussians are fused, posed with linear blend skinning (LBS), and rendered with differentiable Gaussian splatting. Training combines reconstruction and perceptual objectives with a face-specific
adversarial loss to enhance realism in close-up views. Experiments demonstrate strong rendering quality, with face-view performance reaching PSNR/SSIM/LPIPS of 26.243/0.964/0.084 on the AvatarReX
dataset. Ablations further highlight contributions of the MHR template and the face-focused deformation. F3G-Avatar provides a practical, high-quality pipeline for realistic, animatable full-body avatar synthesis.
Read the full abstract ▾
All authors Willem Menu, Erkut Akdag, Pedro Quesado, Yasaman Kashefbahrami, Egor Bondarev
Computer Vision · Cross-listed from cs.RO
Nastaran Darabi and Amit Ranjan Trivedi
Vision language action (VLA) models enable generalist robotic
agents but often exhibit
language ignorance, relying on visual shortcuts and remaining insensitive to instruction changes. We present Prospective Grounding and
Alignment VLA (ProGAL-VLA), which constructs a 3D entity-centric graph (GSM), uses a slow planner to produce symbolic sub-goals, and aligns them with grounded entities via a Grounding
Alignment Contrastive (GAC) loss. All actions are conditioned on a verified goal
embedding $g_t$, whose
attention entropy provides an intrinsic ambiguity signal. On LIBERO-Plus, ProGAL-VLA increases robustness under
robot perturbations from 30.3 to 71.5 percent, reduces
language ignorance by 3x-4x, and improves entity
retrieval from 0.41 to 0.71 Recall@1. On the Custom Ambiguity
Benchmark, it reaches AUROC 0.81 (vs. 0.52), AUPR 0.79, and raises clarification on ambiguous inputs from 0.09 to 0.81 without harming unambiguous success. The verification bottleneck increases mutual information of
language-actions, the GAC loss imposes an entity-level InfoNCE bound, and
attention entropy yields calibrated selective prediction, indicating that explicit verified grounding is an effective path toward instruction-sensitive, ambiguity-aware
agents.
Read the full abstract ▾
Computer Vision
ACCIDENT: A Benchmark Dataset for Vehicle Accident Detection from Traffic Surveillance Videos
Lukas Picek, Michal Čermák, Marek Hanzl et al. (+1 more)
We introduce ACCIDENT, a
benchmark dataset for traffic accident detection in CCTV footage, designed to evaluate models in supervised (IID and OOD) and zero-shot settings, reflecting both data-rich and data-scarce scenarios. The
benchmark consists of a curated set of 2,027 real and 2,211 synthetic clips annotated with the accident time, spatial location, and high-level collision type. We define three core tasks: (i) temporal localization of the accident, (ii) its spatial localization, and (iii) collision type classification. Each task is evaluated using custom metrics that account for the uncertainty and ambiguity inherent in CCTV footage. In addition to the
benchmark, we provide a diverse set of baselines, including heuristic, motion-aware, and
vision-
language approaches, and show that ACCIDENT is challenging. You can access the ACCIDENT at: https://accidentbench.github.io
Read the full abstract ▾
All authors Lukas Picek, Michal Čermák, Marek Hanzl, Vojtěch Čermák
Computer Vision
RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation
Jieru Li, Matthew Chen, Micky C. Nnamdi et al. (+3 more)
Medical image
segmentation models built on Segment Anything Model (SAM) achieve strong performance on clean
benchmarks, yet their reliability often degrades under realistic image corruptions such as noise, blur, motion artifacts, and modality-specific distortions. Existing approaches address either medical-domain adaptation or corruption robustness, but not both jointly. In SAM, we find that these capabilities are concentrated in complementary modules: the image encoder preserves medical priors, while the mask decoder governs corruption robustness. Motivated by this observation, we propose RobustMedSAM, which adopts module-wise checkpoint fusion by initializing the image encoder from MedSAM and the mask decoder from RobustSAM under a shared ViT-B architecture. We then fine-tune only the mask decoder on 35 medical
datasets from MedSegBench, spanning six imaging modalities and 12 corruption types, while freezing the remaining components to preserve pretrained medical representations. We additionally investigate an SVD-based parameter-efficient variant for limited encoder adaptation. Experiments on both in-distribution and out-of-distribution
benchmarks show that RobustMedSAM improves degraded-image Dice from 0.613 to 0.719 (+0.106) over SAM, demonstrating that structured fusion of complementary pretrained models is an effective and practical approach for robust medical image
segmentation.
Read the full abstract ▾
All authors Jieru Li, Matthew Chen, Micky C. Nnamdi, J. Ben Tamo, Benoit L. Marteau, May D. Wang
Computer Vision
Biomarker-Based Pretraining for Chagas Disease Screening in Electrocardiograms
Elias Stenhede and Arian Ranjbar
Chagas disease screening via ECGs is limited by scarce and noisy labels in existing
datasets. We propose a biomarker-based pretraining approach, where an ECG feature extractor is first trained to predict percentile-binned blood biomarkers from the MIMIC-IV-ECG
dataset. The pretrained model is then fine-tuned on Brazilian
datasets for Chagas detection. Our 5-model ensemble, developed by the Ahus AIM team, achieved a challenge score of 0.269 on the hidden test set, ranking 5th in Detection of Chagas Disease from the ECG: The George B. Moody PhysioNet Challenge 2025. Source code and the model are shared on GitHub: github.com/Ahus-AIM/physionet-challenge-2025
Read the full abstract ▾
Computer Vision
Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agents
Sangwon Baik, Gunhee Kim, Mingi Choi et al. (+1 more)
Vision-
Language Models (VLMs) exhibit strong visual
reasoning capabilities, yet they still struggle with 3D understanding. In particular, VLMs often fail to infer a text-consistent goal 6D pose of a target object in a 3D scene. However, we find that with some inference-time techniques and iterative
reasoning, VLMs can achieve dramatic performance gains. Concretely, given a 3D scene represented by an RGB-D image (or a compositional scene of 3D meshes) and a text instruction specifying a desired state change, we repeat the following loop: observe the current scene; evaluate whether it is faithful to the instruction; propose a pose update for the target object; apply the update; and render the updated scene. Through this closed-loop interaction, the VLM effectively acts as an
agent. We further introduce three inference-time techniques that are essential to this closed-loop process: (i) multi-view
reasoning with supporting view selection, (ii) object-centered coordinate system visualization, and (iii) single-axis rotation prediction. Without any additional fine-tuning or new modules, our approach surpasses prior methods at predicting the text-guided goal 6D pose of the target object. It works consistently across both closed-source and open-source VLMs. Moreover, when combining our 6D pose prediction with simple
robot motion planning, it enables more successful
robot manipulation than recent
Vision-
Language-Action models (VLAs). Finally, we conduct an ablation study to demonstrate the necessity of each proposed technique.
Read the full abstract ▾
All authors Sangwon Baik, Gunhee Kim, Mingi Choi, Hanbyul Joo
Computer Vision
MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
Suyang Xi, Songtao Hu, Yuxiang Lai et al. (+4 more)
Medical
vision–
language models (VLMs) have shown strong potential for medical visual question answering (VQA), yet their
reasoning remains largely text-centric: images are encoded once as static context, and subsequent inference is dominated by
language. This paradigm is fundamentally limited in clinical scenarios, where accurate answers often depend on subtle, localized visual evidence that cannot be reliably preserved in static
embeddings. We propose MedLVR, a latent visual
reasoning framework that introduces an explicit visual evidence state into autoregressive decoding. Instead of relying solely on text-based intermediate
reasoning, MedLVR interleaves a short latent
reasoning segment within the decoder by reusing hidden states as continuous latent steps, enabling iterative preservation and refinement of query-relevant visual evidence before answer generation. To support effective visual supervision, we adopt a two-stage training strategy: region of interest (ROI)-supervised fine-tuning aligns latent states with clinically relevant image evidence, and Visual-Latent Policy Optimization (VLPO) further optimizes latent
reasoning and answer generation under outcome-level rewards. Experiments on OmniMedVQA and five external medical VQA
benchmarks show that MedLVR consistently outperforms recent
reasoning baselines and improves the average score over the Qwen2.5-VL-7B backbone from 48.3% to 53.4%. These results show that latent visual
reasoning provides an effective mechanism for preserving diagnostically relevant visual evidence and improving the reliability of medical VQA.
Read the full abstract ▾
All authors Suyang Xi, Songtao Hu, Yuxiang Lai, Wangyun Dan, Yaqi Liu, Shansong Wang, Xiaofeng Yang
Computer Vision
Mohammad Anas Azeez, Ankan Deria, Zohaib Hasan Siddiqui et al. (+5 more)
Multimodal large
language models (MLLMs) frequently hallucinate objects that are absent from the visual input, often because
attention during decoding is disproportionately drawn to visually dominant or frequently occurring content. We observe that this inequity in
attention allocation is a root cause of object
hallucination: when rare, small, or contextually peripheral objects receive insufficient
attention, the model fails to ground its generation in the full visual scene. We argue that every object in an image, regardless of its size, frequency or visual salience, deserves equal representational opportunity during decoding. To this end, we propose DOP-OBC, a training-free and architecture-agnostic decoding strategy built on the principle of equitable
attention. Two complementary object-aware signals work in tandem: a Dominant Object Penalty (DOP) that softly suppresses
attention over-concentration on visually dominant regions, and an Outlier Boost Coefficient (OBC) that amplifies
attention toward rare yet confidently detected objects. These signals are injected as per-row logit modulations within the causal
attention mask, requiring no weight updates and preserving autoregressive decoding properties. Extensive experiments across image and
video MLLMs demonstrate consistent reductions in object
hallucination on CHAIR and POPE
benchmarks, alongside improvements in GPT-4o assessed captioning quality across correctness, consistency, detail, context and temporal dimensions. DOP-OBC establishes that
fairness in
attention allocation is not merely a design principle but a practical and effective path toward more faithful
multimodal generation.
Read the full abstract ▾
All authors Mohammad Anas Azeez, Ankan Deria, Zohaib Hasan Siddiqui, Adinath Madhavrao Dukre, Rafiq Ali, Sara Atito, Yutong Xie, Imran Razzak
Computer Vision · Cross-listed from eess.IV
Search-MIND: Training-Free Multi-Modal Medical Image Registration
Boya Wang, Ruizhe Li, Chao Chen et al. (+1 more)
Multi-modal image registration plays a critical role in precision medicine but faces challenges from non-linear intensity relationships and local optima. While deep learning models enable rapid inference, they often suffer from generalization collapse on unseen modalities. To address this, we propose Search-MIND, a training-free, iterative optimization framework for instance-specific registration. Our pipeline utilizes a coarse-to-fine strategy: a hierarchical coarse
alignment stage followed by deformable refinement. We introduce two novel loss functions: Variance-Weighted Mutual Information (VWMI), which prioritizes informative tissue regions to shield global
alignment from background noise and uniform regions, and Search-MIND (S-MIND), which broadens the convergence basin of structural descriptors by considering larger local search range. Evaluations on CARE Liver 2025 and CHAOS Challenge
datasets show that Search-MIND consistently outperforms classical baselines like ANTs and foundation model-based approaches like DINO-reg, offering superior stability across diverse modalities.
Read the full abstract ▾
All authors Boya Wang, Ruizhe Li, Chao Chen, Xin Chen
Computer Vision · Cross-listed from cs.LG
Efficient Matrix Implementation for Rotary Position Embedding
Chen Minqi, Zhongqi Yue, Shihao Zhang et al. (+5 more)
Rotary Position
Embedding (RoPE) has become a core component of modern
Transformer architectures across
language,
vision, and 3D domains. However, existing implementations rely on vector-level split and merge operations that introduce non-negligible computational overhead, often overlooked in
attention optimization. The problem is further amplified in multi-dimensional settings (e.g., 2D and 3D RoPE), where additional vector operations and uneven feature partitions degrade hardware utilization. To overcome these limitations, we propose RoME (Rotary
Matrix position
Embedding), a mathematically equivalent yet computationally efficient reformulation of RoPE that replaces vector operations with unified
matrix transformations. RoME eliminates dimension-specific operations, simplifies implementation, and enables fused parallel execution across Cube and Vector units on modern NPUs. Experiments show that RoME delivers substantial acceleration at both the operator and full-model levels. The implementation is available at https://gitcode.com/cann/ops-
transformer/blob/master/experimental/posembedding/rope_
matrix/README.md.
Read the full abstract ▾
All authors Chen Minqi, Zhongqi Yue, Shihao Zhang, Yun Xu, Peng Wu, kaixiang Xu, Zeyi Huang, Hanwang Zhang
Computer Vision
Tango: Taming Visual Signals for Efficient Video Large Language Models
Shukang Yin, Sirui Zhao, Hanchao Wang et al. (+4 more)
Token pruning has emerged as a mainstream approach for developing efficient
Video Large
Language Models (
Video LLMs). This work revisits and advances the two predominant
token-
pruning paradigms:
attention-based selection and similarity-based clustering. Our study reveals two critical limitations in existing methods: (1) conventional top-k selection strategies fail to fully account for the
attention distribution, which is often spatially multi-modal and long-tailed in magnitude; and (2) direct similarity-based clustering frequently generates fragmented clusters, resulting in distorted representations after pooling. To address these bottlenecks, we propose Tango, a novel framework designed to optimize the utilization of visual signals. Tango integrates a diversity-driven strategy to enhance
attention-based
token selection, and introduces Spatio-temporal Rotary Position
Embedding (ST-RoPE) to preserve
geometric structure via locality priors. Comprehensive experiments across various
Video LLMs and
video understanding
benchmarks demonstrate the effectiveness and generalizability of our approach. Notably, when retaining only 10% of the
video tokens, Tango preserves 98.9% of the original performance on LLaVA-OV while delivering a 1.88$\times$ inference speedup.
Read the full abstract ▾
All authors Shukang Yin, Sirui Zhao, Hanchao Wang, Baozhi Jia, Xianquan Wang, Chaoyou Fu, Enhong Chen
Computer Vision
EgoTL: Egocentric Think-Aloud Chains for Long-Horizon Tasks
Lulin Liu, Dayou Li, Yiqing Liang et al. (+8 more)
Large foundation models have made significant advances in
embodied intelligence, enabling synthesis and
reasoning over
egocentric input for household tasks. However, VLM-based auto-labeling is often noisy because the primary data sources lack accurate human action labels, chain-of-thought (CoT), and spatial annotations; these errors are amplified during long-horizon spatial instruction following. These issues stem from insufficient coverage of minute-long, daily household planning tasks and from inaccurate spatial grounding. As a result, VLM
reasoning chains and world-model synthesis can hallucinate objects, skip steps, or fail to respect real-world physical attributes. To address these gaps, we introduce EgoTL. EgoTL builds a think-aloud capture pipeline for
egocentric data. It uses a say-before-act protocol to record step-by-step goals and spoken
reasoning with word-level timestamps, then calibrates physical properties with metric-scale spatial
estimators, a
memory-bank walkthrough for scene context, and clip-level tags for
navigation instructions and detailed manipulation actions. With EgoTL, we are able to
benchmark VLMs and World Models on six task dimensions from three layers and long-horizon generation over minute-long sequences across over 100 daily household tasks. We find that foundation models still fall short as
egocentric assistants or open-world simulators. Finally, we finetune foundation models with human CoT aligned with metric labels on the training split of EgoTL, which improves long-horizon planning and
reasoning, step-wise
reasoning, instruction following, and spatial grounding.
Read the full abstract ▾
All authors Lulin Liu, Dayou Li, Yiqing Liang, Sicong Jiang, Hitesh Vijay, Hezhen Hu, Xuhai Xu, Zirui Liu, Srinivas Shakkottai, Manling Li, Zhiwen Fan
Computer Vision
Seeing is Believing: Robust Vision-Guided Cross-Modal Prompt Learning under Label Noise
Zibin Geng, Xuefeng Jiang, Jia Li et al. (+6 more)
Prompt learning is a parameter-efficient approach for
vision-
language models, yet its robustness under label noise is less investigated. Visual content contains richer and more reliable semantic information, which remains more robust under label noise. However, the
prompt itself is highly susceptible to label noise. Motivated by this intuition, we propose VisPrompt, a lightweight and robust
vision-guided
prompt learning framework for noisy-label settings. Specifically, we exploit a cross-modal
attention mechanism to reversely inject visual semantics into
prompt representations. This enables the
prompt tokens to selectively aggregate visual information relevant to the current sample, thereby improving robustness by anchoring
prompt learning to stable instance-level visual evidence and reducing the influence of noisy supervision. To address the instability caused by using the same way of injecting visual information for all samples, despite differences in the quality of their visual cues, we further introduce a lightweight conditional modulation mechanism to adaptively control the strength of visual information injection, which strikes a more robust balance between text-side semantic priors and image-side instance evidence. The proposed framework effectively suppresses the noise-induced disturbances, reduce instability in
prompt updates, and alleviate memorization of mislabeled samples. VisPrompt significantly improves robustness while keeping the pretrained VLM backbone frozen and introducing only a small amount of additional trainable parameters. Extensive experiments under synthetic and real-world label noise demonstrate that VisPrompt generally outperforms existing baselines on seven
benchmark datasets and achieves stronger robustness. Our code is publicly available at https://github.com/gezbww/Vis_
Prompt.
Read the full abstract ▾
All authors Zibin Geng, Xuefeng Jiang, Jia Li, Zheng Li, Tian Wen, Lvhua Wu, Sheng Sun, Yuwei Wang, Min Liu
Computer Vision
VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images
Guanyu Zhou, Yida Yin, Wenhao Chai et al. (+3 more)
Vision-
language models (VLMs) still struggle with visual perception tasks such as spatial understanding and viewpoint recognition, largely because natural image
datasets provide limited supervision for low-level visual skills. Can targeted synthetic supervision address these weaknesses without reference images or manual annotation? To investigate this, we introduce VisionFoundry, an automated pipeline that takes only a task name as input, uses LLMs to synthesize paired questions, answers, and text-to-image (T2I)
prompts, generates images with T2I models, and filters samples via
multimodal verification. With VisionFoundry, we construct VisionFoundry-10k, a synthetic VQA
dataset spanning 10 perception tasks. Finetuning on VisionFoundry-10k consistently improves perception
benchmarks across three open-source backbones (e.g., +6.7% on MMVP-pair and +10.5% on CV-Bench-3D for Qwen2.5-VL-3B-Instruct) while preserving broader capabilities and showing positive data scaling. The same synthetic supervision also yields consistent gains under reinforcement learning (RL) across all three backbones, and the framework remains effective under open-source synthesis and self-verification. Our findings demonstrate that automated synthetic supervision offers an effective and scalable path toward systematic VLM training.
Read the full abstract ▾
All authors Guanyu Zhou, Yida Yin, Wenhao Chai, Shengbang Tong, Xingyu Fu, Zhuang Liu
Computer Vision
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning
Wenyi Xiao, Xinchi Xu and Leilei Gan
Large
Vision Language Models (LVLMs) achieve strong
multimodal reasoning but frequently exhibit
hallucinations and incorrect responses with high certainty, which hinders their usage in high-stakes domains. Existing verbalized confidence calibration methods, largely developed for text-only LLMs, typically optimize a single holistic confidence score using binary answer-level correctness. This design is mismatched to LVLMs: an incorrect prediction may arise from perceptual failures or from
reasoning errors given correct perception, and a single confidence conflates these sources while visual uncertainty is often dominated by
language priors. To address these issues, we propose VL-Calibration, a reinforcement learning framework that explicitly decouples confidence into visual and
reasoning confidence. To supervise visual confidence without ground-truth perception labels, we introduce an intrinsic visual certainty estimation that combines (i) visual grounding measured by KL-divergence under image perturbations and (ii) internal certainty measured by
token entropy. We further propose
token-level advantage reweighting to focus optimization on
tokens based on visual certainty, suppressing ungrounded
hallucinations while preserving valid perception. Experiments on thirteen
benchmarks show that VL-Calibration effectively improves calibration while boosting visual
reasoning accuracy, and it generalizes to out-of-distribution
benchmarks across model scales and architectures.
Read the full abstract ▾
Computer Vision
Envisioning the Future, One Step at a Time
Stefan Andreas Baumann, Jannik Wiese, Tommaso Martorella et al. (+2 more)
Accurately anticipating how complex, diverse scenes will evolve requires models that represent uncertainty, simulate along extended interaction chains, and efficiently explore many plausible futures. Yet most existing approaches rely on dense
video or latent-space prediction, expending substantial capacity on dense appearance rather than on the underlying sparse trajectories of points in the scene. This makes large-scale exploration of future hypotheses costly and limits performance when long-horizon, multi-modal motion is essential. We address this by formulating the prediction of open-set future scene dynamics as step-wise inference over sparse point trajectories. Our autoregressive
diffusion model advances these trajectories through short, locally predictable transitions, explicitly modeling the growth of uncertainty over time. This dynamics-centric representation enables fast rollout of thousands of diverse futures from a single image, optionally guided by initial constraints on motion, while maintaining physical plausibility and long-range coherence. We further introduce OWM, a
benchmark for open-set motion prediction based on diverse in-the-wild
videos, to evaluate accuracy and variability of predicted trajectory distributions under real-world uncertainty. Our method matches or surpasses dense simulators in predictive accuracy while achieving orders-of-magnitude higher sampling speed, making open-set future prediction both scalable and practical. Project page: http://compvis.github.io/myriad.
Read the full abstract ▾
All authors Stefan Andreas Baumann, Jannik Wiese, Tommaso Martorella, Mahdi M. Kalayeh, Björn Ommer