Computer Vision
Spatial Action Review: A Visual Analytics Dashboard for Auditing Language-to-Action Hand-offs in Electron Microscopy
Samia Mohinta and Albert Cardona
Multimodal large
language models (MLLMs) are increasingly explored as interfaces for scientific image analysis, where a visual question-answering (VQA) response may be paired with a spatial output that guides a downstream stage. A supervisor reads the
language answer, while a downstream workflow such as
segmentation or region review consumes the point-set output. We call this transition from inspecting the answer to relying on its point action the
language-to-action hand-off. A silent failure occurs when the answer is correct while the paired action misses annotated objects needed downstream, so answer-based oversight clears a region whose action is unreliable. We introduce Spatial Action Review, a visual analytics dashboard for
auditing this failure mode in electron microscopy (EM) mitochondria analysis. It links paired answer-action records through an answer-action ledger, a task-by-
dataset risk map, and an image-region audit view, connecting aggregate patterns to image evidence while an adjustable action-reliability gate supports re-audit. The review ends in a human-AI hand-off, where a supervisor records whether the action is accepted, escalated, held under a stricter gate, or flagged for model revision. Across 541 image regions from an EM-adapted Qwen3-VL case-study run, point actions fail the gate in 54.4% of records with a correct VQA response, and 27.4% of all records are silent failures. A correct answer is associated with only a 5.8-percentage-point higher probability of a reliable action, with a bootstrap interval spanning zero; the point-biserial correlation between answer correctness and object coverage is 0.061. This weak coupling persists across five model conditions on 753 matched image regions. Spatial Action Review makes answer-action mismatches visible and ties them to image evidence and a recorded decision before MLLM outputs enter autonomous scientific workflows.
Read the full abstract ▾
Computer Vision
SGAM: Shared Gaussian Geometry with Implicit Amplitude Modeling for Scan-Specific 3D Multi-Contrast MRI Reconstruction
Jingran Xu, Dong Liang, Hairong Zheng et al. (+2 more)
Three-dimensional (3D) multi-contrast
magnetic resonance imaging (MCMRI) provides rich anatomical and quantitative information but requires long acquisition times, motivating k-space undersampling. However, reconstruction of large volumetric
datasets imposes substantial computational and
memory demands. To address this challenge, we propose SGAM, a
memory-efficient, scan-specific framework for joint full-volume 3D MCMRI reconstruction. SGAM is based on a shared-geometry Gaussian representation in which amplitudes are modeled by a multi-output implicit neural representation (INR) and contrast-specific phases by explicit variables. This design exploits common anatomical structure across contrasts while preserving contrast-specific signal variations. The representation is jointly optimized using only the acquired multi-coil k-space without external training data. Experiments showed that SGAM consistently outperformed the comparison methods across imaging tasks and acceleration factors, with greater improvements under stronger undersampling. SGAM also achieved a favorable balance between reconstruction quality and computational cost, demonstrating its effectiveness for 3D multi-contrast MRI reconstruction.
Read the full abstract ▾
All authors Jingran Xu, Dong Liang, Hairong Zheng, Yuanyuan Liu, Yanjie Zhu
Computer Vision
MECAIL: Communication-Aware Incremental Learning for Object Detection with 14.6 KB Spatiotemporal Experts
Matthias Neuwirth-Trapp, Maarten Bieshaar, Danda Paudel et al. (+3 more)
Intelligent transportation systems require Incremental Learning (IL) to continually improve their overall performance in dynamic environments. However, most edge devices lack the computational resources to support on-device IL, requiring updates to be transmitted from centralized servers. We propose using this setup to obtain dense, specialized module coverage that adapts a fixed base model to specific spatiotemporal contexts, such as parking lots, gas stations, ferries, or construction sites. However, in order to reliably transmit these modules to the edge device, using TCP, UDP, and BTP over V2X, Wi-Fi, and 2G-5G hardware, we establish a strict limit of 14.6 KB per module to fit within the first TCP window and to minimize UDP/BTP fragmentation. We further introduce Mixture-of-Experts for Communication-Aware Incremental Learning (MECAIL), the first method that meets this strict requirement, in which each new domain or environment is served by a small expert network that adapts the base model. We validate MECAIL on D-RICO and ODinW-13, where it largely matches the performance of parameter-heavy approaches while enabling practical, bandwidth-efficient large-scale deployment. This allows comprehensive coverage by experts for highly specific, focused, and temporary situations.
Read the full abstract ▾
All authors Matthias Neuwirth-Trapp, Maarten Bieshaar, Danda Paudel, Konrad Schindler, Luc Van Gool, Christos Sakaridis
Computer Vision
Do LiDAR Language Models Really Understand Spatio-temporal Relationships?
Runyi Yang, Murat Akkoyun, Di Wen et al. (+8 more)
Recent 4D
LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same option nearly matches the multiple-choice accuracy of two B4DL-derived configurations. We introduce
LiDAR-Hallu, a geometry-referenced
benchmark and diagnostic protocol with 10,000 questions across 150 nuScenes scenes. It covers object existence, ego-relative position, distance ordering, relative motion, and temporal localization, with explicit rules for selecting objects, comparing times, and determining reference answers. Our protocol combines fixed-answer and candidate-content controls, cross-scene pairs with identical
prompts but opposite reference answers, and relation-specific recall. Analysis of 100,000 recorded responses reveals failures hidden by aggregate accuracy. Candidate duration alone makes temporal answers predictable without observing
LiDAR. On paired questions, the models frequently give the same answer to scenes requiring opposite answers. Relation-specific analysis further shows that both configurations miss every positive lateral-motion case across all tested conditions. Temporal-shuffle contrastive decoding provides little net improvement, as repairs are largely offset by new errors and the main failures persist. These results show that evaluating spatio-temporal
reasoning requires testing whether models distinguish the queried physical relationships, rather than relying on individual-answer accuracy alone. The source code, checkpoints, and data are released at https://github.com/Awesome4D/4DMLLM_
Hallucination_Bench.
Read the full abstract ▾
All authors Runyi Yang, Murat Akkoyun, Di Wen, Ruiping Liu, Yufan Chen, Junwei Zheng, Xiaoye Wang, Kailun Yang, Danda Pani Paudel, Luc Van Gool, Kunyu Peng
Computer Vision
Estimating Accurate Hand Pose in Camera Space with Vision Transformer
Kaiwen Ren, Yiran Jiang, Yongjing Ye et al. (+1 more)
Monocular RGB-based hand pose estimation has emerged as a critical research frontier in computer
vision. The local hand pose estimation methods predict hand poses relative to the wrist, while global hand pose estimation also requires estimating the wrist's position in the camera coordinate system. However, this camera-space estimation confronts two fundamental challenges: (1) depth ambiguity in
monocular settings, and (2) the coupling effect of hand local poses and global wrist positions in the perspective projections. In particular, this coupling reflects that the projections are jointly determined by local hand poses, wrist positions, and camera intrinsics. To overcome these challenges, our framework proposes two key innovations: Transformation-Isomorphism Supervision for hand-depth information extraction and Perspective Information
Embedding for resolving above coupling effect of local pose and wrist position, both integrated within the mainstream encoder-decoder architecture. Besides, we propose a novel framerate-aware multi-
dataset training strategy for sequential pose refinement. Our fully integrated approach achieves at most 37.1% superiority in CS-MJE over SOTA on HO3D. Project page: https://github.com/Mine268/CS-ViT.
Read the full abstract ▾
All authors Kaiwen Ren, Yiran Jiang, Yongjing Ye, Shihong Xia
Computer Vision
DeCo: Efficient Decouple-to-Couple Learning for Multi-Task Visual Grounding
Xiaoqiang Lu, Licheng Jiao, Long Sun et al. (+5 more)
Multi-task visual grounding requires models to jointly understand linguistic semantics and perform accurate visual localization and
segmentation. Despite the success of
multimodal large
language models, effectively adapting them to multiple grounding objectives remains challenging. Existing methods commonly enforce task cooperation through shared representations, while overlooking the intrinsic conflict between task-oriented feature interests. In this paper, we introduce $\textbf{DeCo}$, an efficient $\textbf{De}$couple-to-$\textbf{Co}$uple learning framework that resolves this dilemma through a two-stage paradigm: task-specific representation decoupling followed by complementary prior coupling. Specifically, we first propose Task-aware Semantic Decoupling (TSD) to route shared visual cues into individual features under salient word-level guidance, alleviating representation interference between localization and
segmentation. Furthermore, we observe that
segmentation naturally provides informative localization priors due to dense supervision. Based on this insight, we introduce Hybrid Prior Coupling (HPC), which integrates sentence-level semantic prior with mask-derived spatial prior for enhanced grounding. Built upon a frozen
multimodal encoder, DeCo requires lightweight trainable parameters while achieving strong generalization across multiple grounding objectives. Extensive experiments on RefCOCO/+, G-Ref, ReferIt, Flickr, DIOR-RSVG, SARVG1.0, RRSIS-D, RIS-LAD, and RefDIOR demonstrate that DeCo achieves state-of-the-art performance on both natural and remote sensing
benchmarks. The code and models are available at https://github.com/xiaoqiang-lu/DeCo.
Read the full abstract ▾
All authors Xiaoqiang Lu, Licheng Jiao, Long Sun, Yuting Yang, Xu Liu, Lingling Li, Wenping Ma, Fang Liu
Computer Vision
Can Spiking Neural Networks play pinball? A neuromorphic motion detector for target tracking
Mazdak Fatahi, Šárka Pryjmaková, Pierre Boulet et al. (+1 more)
Biological visual systems achieve continuous, low-latency motion perception by processing sparse, asynchronous spiking signals, enabling real-time tracking under strict energy constraints. Event-based cameras, inspired by the mammalian retina, replicate this efficiency by capturing only local brightness changes as asynchronous events, offering a natural substrate for spiking neural networks (SNNs) to parallelise computation and adapt to fast-changing scenes. Pinball provides a controlled yet dynamic testbed, requiring precise motion estimation and fast reaction to a small, rapidly moving target. This work presents a fully spiking, real-time perception-to-action pipeline for closed-loop pinball gameplay. A dynamic
vision sensor observes a small, fast-moving ball, and a network of spiking Time-Difference Encoders on the SpiNNaker neuromorphic platform jointly estimates its position, speed, and direction. The system is characterised across receptive field size, accumulation window, and angular tuning width for real-time operation, and benchmarked in closed loop against human players across two flipper regimes of increasing physical realism. It achieves a hit rate of 56.1%, nearly double the human average, reacting within 21.7 ms (5 ms network latency) and consuming an estimated 148 {\mu}W using fewer than 25k
neurons, among the fastest and most energy-efficient event-based closed-loop demonstrators benchmarked. Under more realistic flipper dynamics, tuning a single interpretable policy parameter reproduces the full spectrum of human play styles, from cautious to aggressive, with no change to the perception pipeline. A physical demonstrator, tracking a real ball and actuating real flippers in closed loop, confirms the principle operates beyond simulation. Its fully spiking, learning-free design offers a compact, energy-efficient example of real-time neuromorphic perception-to-action.
Read the full abstract ▾
All authors Mazdak Fatahi, Šárka Pryjmaková, Pierre Boulet, Giulia D'Angelo
Computer Vision
A Lightweight Convolutional Neural Network for Real-Time Recognition of Hand-Drawn Geometric Shapes
Shahir Abdullah
Recognizing hand-drawn
geometric shapes is a foundational sub-problem of sketch recognition, with applications in education, human-computer interaction, and diagram digitization. This paper presents the design, implementation, and evaluation of a desktop application that recognizes four basic hand-drawn
geometric shapes, circle, square, rectangle, and triangle using a compact Convolutional Neural Network (CNN). A
dataset of 2,000 labeled 28x28-pixel shape images was collected independently and released publicly. The classifier consists of three convolutional blocks (16, 32, and 64 filters) with max-pooling, an in-model data-augmentation stage (random horizontal flip, rotation, and zoom), a dropout-regularized dense layer of 128 units, and a 4-way linear output layer, totaling 97{,}956 trainable parameters. The network is trained with the Adam optimizer on a sparse categorical cross-
entropy objective computed directly on logits. On an 80/20 train-validation split, the model achieves 94.80% training accuracy and 96.01% validation accuracy with a validation loss of 0.1437. A Tkinter-based graphical interface allows a user to draw a shape with the mouse and receive an immediate class prediction with a confidence score. We situate this system within the broader sketch and shape-recognition literature, compare its accuracy against related hand-drawn shape classification studies, and discuss the limitations inherent to a small, single-contributor
dataset. The complete source code, trained model, and per-class
datasets are released publicly to support reproducibility.
Read the full abstract ▾
Computer Vision
Topographic Training Concentrates Causal Circuits Without Improving Neuron Monosemanticity
Gautam Ranka, Shubham Santosh Pandere and Aiden Dsouza
Mechanistic interpretability of
vision transformers seeks to decompose model computation into human-readable units, but learned representations entangle many concepts in each
neuron. Feature superposition is widely treated as the central obstacle to this decomposition, yet most mitigations (sparse autoencoders, dictionary learning) are post-hoc and leave the underlying network unchanged. We ask whether a spatial-locality training loss (TopoLoss) can act as a lightweight, training-time prior that improves interpretability of standard mech-interp tools. Training ViT on ImageNet-100 across multiple TopoLoss weights $\alpha$, we measure causal sufficiency of topographic clusters via activation patching and feature geometry via sparse autoencoders fit to the same residual stream. At $\alpha=1.0$, topographic clusters are 2.79$\times$ more causally sufficient than random unit sets of the same size, with the effect increasing monotonically in $\alpha$. SAE L0 sparsity decreases by 11% and dead-feature fraction rises 19-fold, yet standard
neuron-level monosemanticity scores are unchanged, indicating that topographic pressure acts at
circuit level, concentrating causal mass into spatially local structures without disentangling individual
neurons. This dissociation suggests current
neuron-level monosemanticity metrics are insensitive to a class of real interpretability gains, and positions cheap architectural priors as a viable training-time complement to post-hoc tooling.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Prescriptive SVD-Inspired Attention via Spectral Energy Retention
Vasileios Arampatzakis, Vasileios Sevetlidis and George Pavlidis
Self-
attention is central to modern
Transformer architectures, but its dense dot-product formulation makes it difficult to identify which internal directions are structurally important and which can be modified without disrupting the model. SVD-Inspired
Attention (SVDA) addresses part of this problem by introducing a learned diagonal spectrum into the query-key score interaction, making latent
attention directions explicitly inspectable through indicators such as spectral
entropy, effective rank, sparsity,
alignment, selectivity, and perturbation response. This paper examines the transition from diagnostic interpretation to operational intervention. A diagnosis–intervention–verification framework is proposed, and one intervention is evaluated: spectral energy retention in the
attention-score pathway. Across FashionMNIST, CIFAR-10, CIFAR-100, and Food-101, the $\rho=0.90$ prescription removes 24.5–53.7% of score directions, reduces parameters by 2.6–4.3%, and reduces estimated MACs by 2.8–5.4%. The paired mean accuracy change of the dimension-reduced model ranges from $-0.03$ to $+0.05$ percentage points over three seeds. These results support SVDA as an intrinsically interpretable
attention mechanism whose learned spectrum exposes an operational coordinate system for deterministic and verifiable modification of
attention-score formation.
Read the full abstract ▾
Computer Vision
TReViS: Temporal Repetition Structure Aware Video Synthesis for Self-supervised Repetitive Action Counting
Fanqi Yu, Shengming Ma, Stefano Fiorini et al. (+4 more)
Fully supervised repetitive action counting (RAC) has achieved strong performance, but requires dense temporal annotations that are costly and difficult to scale. We propose TReViS, a self-supervised
video synthesis framework that enables training RAC models without any repetition labels. TReViS estimates the underlying temporal repetition structure of an unlabeled
video via a Temporal Self-Similarity
Matrix, infers its cycle statistics, and synthesizes new training sequences that preserve realistic repetition patterns while introducing controlled temporal variability. These synthesized
videos are paired with pseudo-labels and used to train existing RAC architectures from scratch. Across multiple
datasets and backbones, TReViS consistently outperforms prior self-supervised methods and achieves performance competitive with several supervised baselines, while remaining fully label-free, demonstrating the effectiveness of structure-aware
video synthesis for label-free RAC. The source code is available at https://github.com/yfqi/TReViS.
Read the full abstract ▾
All authors Fanqi Yu, Shengming Ma, Stefano Fiorini, Vito Paolo Pastore, Xuan Qi, Vittorio Murino, Cigdem Beyan
Computer Vision
Dissecting Agentic Forensics: The Role of Triage, Prompting, and Evidence Arbitration in Open-World Fake Image Detection
Xianlong Li, Pietro Bongini, Niccoló Pancino et al. (+3 more)
Image forensics is increasingly an open-world problem: manipulations range from fully synthetic images to localized edits, splicing and swapping, while most forensic detectors remain specialized to a single manipulation family. Agentic AI has recently emerged as a promising solution. In principle, such systems can assess the reliability of individual detectors, identify out-of-scope evidence, and arbitrate conflicting reports. However, it remains unclear which components actually drive performance and whether their benefits persist under distribution shift. To answer these questions, we study a training-free agentic framework built around specialist detectors, per-detector triage, and conflict-aware evidence arbitration. Using six configurations and three
multimodal large
language model backbones, we dissect the role of triage, prompting, and
reasoning quality on both in-distribution and out-of-distribution data. Our results show that naive detector fusion suffers from severe false-positive rates on authentic images. Triage and prompting consistently improve performance by filtering unreliable evidence and exposing detector limitations. However, the dominant factor is represented by
reasoning itself: A stronger judge substantially outperforms a weaker one, particularly under distribution shift. Most notably, manipulation recall is nearly saturated across all configurations, indicating that the main challenge of open-world image forensics is not detecting manipulations, but calibrating trust in specialized forensic tools and arbitrating conflicting evidence.
Read the full abstract ▾
All authors Xianlong Li, Pietro Bongini, Niccoló Pancino, Marco Blanchini, Benedetta Tondi, Mauro Barni
Computer Vision · Cross-listed from cs.RO
LIBERO-VPro: Benchmarking Closed-Loop Visual Robustness of Robotic Foundation Models
Huiqiong Li, Zhiting Mei, Anirudha Majumdar et al. (+3 more)
Robotic foundation models achieve impressive performance on standard manipulation
benchmarks, yet these evaluations typically assume clean, timely, and consistent visual observations throughout execution. We introduce LIBERO-VPro, a
benchmark for systematically evaluating the closed-loop visual robustness of robotic foundation models by perturbing the visual evidence available during execution. LIBERO-VPro covers four complementary dimensions, including Visual Evidence Degradation, Camera Staleness, Visual Source Consistency, and Task-Relevant Scene Variation, spanning 12 challenge categories, 96 experimental settings, and 3,296 task-condition cases. We evaluate three
vision-
language-action models and three world-action models over approximately 196,000 simulated episodes, complemented by 200 real-world rollouts on a Franka Research 3. Our results reveal that strong nominal performance can mask substantial weaknesses in visual grounding and adaptation. Models often remain successful despite severe object-level occlusion, yet degrade sharply when local interaction cues are disrupted or familiar spatial priors are violated. They are also highly sensitive to stale or missing observations and struggle when changed task preconditions require behavioral adaptation. Finally, VLAs and WAMs exhibit distinct robustness profiles, showing that visual robustness is multi-dimensional and architecture-dependent. LIBERO-VPro provides a systematic diagnostic framework for developing robotic foundation models that can more reliably ground and adapt their actions under challenging visual conditions.
Read the full abstract ▾
All authors Huiqiong Li, Zhiting Mei, Anirudha Majumdar, Jingjing Chen, Yu-Gang Jiang, Bin Zhu
Computer Vision
LiAuto-MindViT: A Hybrid Vision Backbone with Adaptive Bidirectional Mamba
Lifu Mu, Shuai Chen, Wen Zheng et al. (+7 more)
While Mamba-based models have shown strong potential for long sequence modeling, adapting them to
vision is challenging due to the requirement of local neighborhood correlations and multi-directional spatial contexts for visual understanding. In this paper, we present LiAuto-MindViT, a novel hybrid
vision backbone that synergizes the strengths of CNNs, Mamba, and
Transformers. The core of our design is the Adaptive Bidirectional Mamba (ABM), which eliminates the directional bias of unidirectional SSMs through bidirectional selective scanning with learnable alpha blending, enabling content-adaptive directional fusion without the overhead of exhaustive multi-path routing. To further accelerate inference, we propose a deployment-friendly Reparameterized ConvSE (RepConvSE) module that leverages structural reparameterization to reduce latency and
memory access overhead. Extensive experiments demonstrate that LiAuto-MindViT achieves state-of-the-art performance on image classification, object detection, and semantic
segmentation while enabling efficient inference through reparameterization.
Read the full abstract ▾
All authors Lifu Mu, Shuai Chen, Wen Zheng, Haoyi Sun, Xueyang Fu, Pengfei Yu, Ning Mao, Tao Wei, Zhou Pan, Kun Zhan
Computer Vision
AlignMorph: Tuning-Free Diffusion Image Morphing via Explicit Semantic Transport
Wuyi Liu, Xu Han, Yuren Chen et al. (+3 more)
Image morphing aims to produce a smooth and semantically consistent transition between two input images. Existing
diffusion-based morphing methods either require expensive per-pair optimization or rely on implicit spatial
alignment, which easily fails under large layout discrepancies. To address these limitations, we propose AlignMorph, a novel tuning-free
diffusion framework guided by the principle of transport-then-denoise. We explicitly decouple
geometric alignment from generative denoising to avoid structural
entanglement. Our framework consists of two core components. (1) Global Semantic Transport, which achieves
diffusion-compatible semantic
alignment via entropic optimal transport and reliability-aware latent warping; and (2) Coordinate-Aligned Generation, which uses a symmetric bi-phase
attention handoff to maintain consistent spatial coordinates throughout denoising. Without any tuning, AlignMorph effectively eliminates ghosting and achieves superior structural coherence and temporal smoothness on morphing
benchmarks. Code is available at https://github.com/51xOne/Alignmorph.
Read the full abstract ▾
All authors Wuyi Liu, Xu Han, Yuren Chen, Yige Mao, Zishuo Peng, Xianzhi Li
Computer Vision
NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Models
Jiajing Lin, Xin Zhang and Jianhua Sun
World models aim to capture environmental dynamics and predict future trajectories, showing growing potential for
embodied intelligence. Physics-informed 4D generation integrates physical simulation to predict 3D object interactions, offering a promising pathway toward world models. However, this paradigm relies on manually imposed dynamical assumptions rather than internalizing world dynamics, and thus still leaves a gap toward a true world model. To bridge this gap, we propose NeuIDO, a novel world dynamics modeling framework that learns a unified intrinsic dynamics representation from visual observations, advancing physics-informed 4D generation toward a world model. Specifically, we formulate world modeling as a neural operator learning problem and introduce a two-stage training strategy to learn a generalizable mapping from the visual observation distribution to the intrinsic dynamics distribution. Building on this observation-dynamics mapping, NeuIDO enables zero-shot dynamics inference directly from
videos and can be further aligned with complex real-world dynamics via few-shot adaptation. Extensive experiments demonstrate that NeuIDO effectively unifies the intrinsic dynamics underlying diverse visual observations into a shared representation and rapidly infers dynamics in novel scenes.
Read the full abstract ▾
Computer Vision
AnalogDepth: Multi-view Geometry from FPV drones under Analog Video Transmission
André Amorim and Pedro F. Proença
Analog
video transmission (VTX) remains widespread in FPV drones due to low latency, weight and low cost. However analog VTX suffers from complex spatially structured image degradation which differ fundamentally from digital image corruption (e.g. AWGN) used in standard training augmentation. This work shows that this type of noise severely degrades the accuracy of Depth Anything 3 (DA3), a state-of-the-art feed forward visual geometry foundation model. To address this gap, we present AnalogDepth, a parameter-efficient training pipeline that adapts DA3 to analog FPV imagery using student-teacher knowledge
distillation with Low-Rank Adaptation (LoRA) injected into the DINOv2 backbone. Rather than synthesizing noise analytically, we build a noise bank from static FPV recordings under diverse conditions and compare real-noise injection against PSD-matched Gaussian synthesis and AWGN as baselines. Experiments on six real FPV flight sequences across three indoor scenes show that training with our noise bank consistently reduces per-frame depth RMSE and 3D reconstruction Chamfer distance compared to the pretrained DA3 baseline and both Gaussian noise variants. These results demonstrate that replicating the spatial structure of real analog transmission noise is critical for effective adaptation.
Read the full abstract ▾
Computer Vision
HappyWorld-Bench
Zhiqi Bai, Junai Cai, Yixin Chen et al. (+33 more)
Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, interaction, and modification. We introduce HappyWorld-Bench, a comprehensive
benchmark that evaluates whether generated worlds remain reliable as
agents interact with them. Our design is built on a hierarchical capability framework of six world capabilities (W1-W6), from generative construction to unified world modeling, instantiated across three independent evaluation tracks:
video world models, spatial world models, and
embodied world models. HappyWorld-Bench comprises 1,138
video prompts, 300 spatial scenes, and 254
embodied test cases. Across all three tracks, we build and operate HappyWorld-Arena to organize human A/B comparisons and derive model-level Elo ratings, which complement newly designed automated metrics that capture behavioral correctness. We evaluate 14
video world models, 9 spatial systems, and 8
embodied candidates under this unified framework. Results reveal remaining reliability gaps across all three tracks:
video models exhibit reduced consistency during extended rollouts and revisits, spatial models achieve at best 70.14% placement accuracy and 73.33% edit execution, and
embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physical rules. These findings highlight the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.
Read the full abstract ▾
All authors Zhiqi Bai, Junai Cai, Yixin Chen, Jingrun Du, Tao Feng, Wei Gong, Siyuan Huang, Xiao Lin, Jiaheng Liu, Jun Luo, Yongzhe Lyu, Liya Ma, Zenan Meng, Lin Qu, Wenbo Su, Jiaming Wang, Qinghe Wang, Shaofei Wang, Yanghai Wang, Zequn Wang, Ziming Wang, Hu Wei, Jiangtao Wu, Ruiqi Wu, Jiaxin Xie, Yuchi Xu, Ze Xu, Chengting Yu, Liangyu Yuan, Gang Zeng, Yawen Zeng, Xingyao Zhang, Zizheng Zhang, Bo Zheng, Jiancheng Zhu, Song-Chun Zhu
Computer Vision · Cross-listed from astro-ph.IM
Scale-Vector Alignment: A Scale-Aware Framework for Spatially Resolved Morphological Similarity in Astronomical Images
Mengke Zhao, Guang-Xing Li, Keping Qiu et al. (+1 more)
Astronomical maps made with different tracers are not expected to have identical morphology. Excitation, optical depth, chemistry, radiation, and ISM phase alter the response of a tracer, and the resulting differences can depend on both position and spatial scale. We propose scale-vector
alignment, a scale-aware method based on Constrained
Diffusion Decomposition (CDD). CDD decomposes an image into localized scale components; at each position, their amplitudes define a scale vector that describes how the measured intensity is distributed over spatial scale. We define the pixel-wise similarity $\Spix(x,y)$ as the normalized
alignment of two local scale vectors. The normalization removes the overall amplitude, so $\Spix$ compares relative scale composition rather than absolute flux. We also define the scale-wise similarity $\Sscale(l)$ by comparing the two CDD component maps at each spatial scale. Spatial shifts are used to construct an empirical shifted reference distribution for $\Spix$. In Orion~A, the tracer with the highest similarity to the
dust-derived column-density map changes from $^{12}$CO to $^{13}$CO to C$^{18}$O toward higher column density. In NGC~6334I(N), the line–continuum similarity decreases locally around the brightest compact structures, where radiative-transfer effects can alter the observed line morphology. In NGC~3627, CO is most similar to 21~$\mu$m emission, and $\Sscale$ reaches its maximum at an intermediate sub-kpc scale. The method measures where two tracers have similar multiscale structure and at which scales their spatial distributions agree. The implementation is publicly available at https://github.com/meng-ke/Scale-Vector-
Alignment.
Read the full abstract ▾
All authors Mengke Zhao, Guang-Xing Li, Keping Qiu, Shanghuo Li
Computer Vision
Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control
Jishen Peng and Zheng Ma
Classifier-free guidance (CFG) improves conditional generation in Flow Matching, but strong guidance can distort the generated distribution and reduce diversity. We provide a
geometric account of this behavior by viewing Flow Matching as a time-varying gradient flow and characterizing how CFG reshapes its underlying potential. This view explains how stronger
alignment can be accompanied by mean displacement and trajectory concentration, and motivates controlling guidance through the model-implied terminal posterior mean. We therefore propose Posterior-Mean-Capped CFG (PMC-CFG), a training-free, per-sample method that adaptively retains the strongest feasible guidance without additional network evaluations. Experiments on synthetic and large-scale image-generation
benchmarks show that PMC-CFG limits guidance-induced distortion and concentration while improving the
alignment–diversity trade-off, with particularly strong benefits when nominal guidance is large.
Read the full abstract ▾
Computer Vision
Andro Erdelez, Pascal Mettes and Behzad Bozorgtabar
Hyperbolic vision-
language models (VLMs) represent image and text features in a geometry naturally suited to hierarchy, but their adaptation to downstream tasks has largely relied on fixed
prompts. Existing
prompt learning methods, meanwhile, treat class labels as a flat set and do not exploit available taxonomic structure. We address this gap with a hierarchical
prompt learning plug-in for frozen
hyperbolic VLMs. Given a fixed offline parent-class hierarchy, it augments a class
prompt learner with a separate parent
prompt learner, parent-level supervision,
hyperbolic entailment regularization, and parent-feedback logit fusion. We instantiate the method with CoOp, CoCoOp and MaPLe, yielding HyPLO, CoHyPLO and MaHyPLO. Across the standard 11-
dataset benchmark, all variants improve base-to-new generalization and cross-
dataset transfer, and remain comparable to their
prompt learning baselines under domain shift. Six hierarchical metrics and
embedding analyses show that the method produces more taxonomically consistent predictions and induces a hierarchy-consistent organization of parent, class, and image
embeddings in
hyperbolic space. Its gains are largest when novel classes must be placed within a fixed taxonomy, and smallest for fine-grained confusions among sibling classes or shifts affecting only the image distribution.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Unsupervised Brain Anomaly Detection as a Bayesian Inverse Problem with Diffusion Prior
Hugues Roy, Reuben Dorent and Ninon Burgos
Unsupervised anomaly detection (UAD) aims to localize abnormal regions in medical scans without pixel-level annotations. A typical strategy seeks to reconstruct a pseudo-healthy image that preserves subject-specific anatomy. Recently,
diffusion models have been proposed to perform UAD. However, these methods rely on heuristic noise schedules or synthetic corruptions to balance subject-specificity and anomaly removal. In this work, we propose an alternative formulation of UAD as a
Bayesian inverse problem under a
diffusion prior. First, we introduce a latent spatial anomaly mask that models pixel-wise consistency between a test image and its latent corresponding pseudo-healthy image. Then, we propose an approximation of the unknown generation process that links healthy anatomy, anomalies, and the observed image, enabling a well-defined likelihood within the
Bayesian framework. Building on recent advances in
diffusion-based inverse problem methods, we jointly infer the pseudo-healthy image and the anomaly mask via annealed posterior sampling. We evaluate our approach on FDG PET (ADNI) and FLAIR MRI (BraTS 2021), demonstrating improved anomaly localization performance compared to other
diffusion-based approaches and validating the contribution of our introduced model. Our code is available at https://github.com/HuguesRoy/UAD_DAPS.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
OpenFlyScan: A Quality-Guided Aerial Reconstruction System for Consumer Drones
Zhongrui You, Zhen Li, Junli Liu et al. (+2 more)
3D Gaussian Splatting (3DGS) provides high-fidelity scenes for large-scale
embodied simulation, but constructing large-scale urban assets remains constrained by expensive equipment and delayed quality feedback. Preset surveys can leave complex surfaces insufficiently observed, with defects discovered only after reconstruction, requiring return visits and repeated processing. We present OpenFlyScan, a quality-guided aerial reconstruction system for consumer drones that integrates a GS quality model, a reacquisition planner, and a custom-designed mobile app. The model learns from GS rendering errors to predict regional reconstruction quality. Based on these predictions, the planner then generates complementary reacquisition strips to be executed through the app, which also supports automated oblique surveys and data transfer without additional hardware on board. Across real aerial scenes, the model effectively identifies regions that are likely to be poorly reconstructed. In the Expo West field experiment, targeted reacquisition improves PSNR at additional views by 10.95 dB. With consumer drones, OpenFlyScan integrates capture, targeted reacquisition, and reconstruction to support rapid, low-cost urban asset creation. Code and models will be made publicly available at https://openflyscan.github.io/.
Read the full abstract ▾
All authors Zhongrui You, Zhen Li, Junli Liu, Zhigang Wang, Bin Zhao
Computer Vision · Cross-listed from cs.LG
Reinforcement Learning Inspired Black-box Adversarial Attacks for Computer Vision
Florian Krone, Elena Hoemann and Sven Hallerbach
Neural networks, both convolution or
transformer based, are essential for modern computer
vision systems. However, they are vulnerable to small perturbations, almost imperceptible to humans, which significantly alter the model's prediction. These
adversarial attacks are often considered to be a significant threat to the implementation of neural networks in safety-critical applications. Most attacks utilize the white-box threat model and therefore require full access to the target model, making them unrealistic to use in practice. We propose a novel approach under the more realistic black-box threat model that utilizes concepts from reinforcement learning to optimize perturbations with a non-differentiable target model. Reinforcement learning
algorithms have already been optimized to be query efficient, making them an ideal starting point when designing black-box
adversarial attacks. We show the success of our reinforcement learning inspired black-box
adversarial attack (RIBA) in generating
adversarial perturbations using only a small number of queries to the target model, by comparing it to state of the art attacks on different models on the Cifar10 and ImageNet data sets. RIBA takes $25.4\%$ fewer median queries to generate attacked images against a ResNet-18 on Cifar10 and $22.5\%$ fewer median queries to fool a Vit-B/16 model on ImageNet. Additionally, we demonstrate that RIBA can match the performance of white-box attacks on an adversarially trained model.
Read the full abstract ▾
Computer Vision
Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
Santi Ram Tiwari, Nihal Naik, Devbrat Pandey et al. (+1 more)
Multimodal large
language models (MLLMs) fail at fine-grained visual questions less because they cannot reason than because they never see the evidence: high-resolution images are downsampled before encoding, so the model answers from linguistic priors. The standard remedies are expensive: annotated answers (SFT), hand-engineered verifiers (RLVR), or a large external teacher (on-policy
distillation). We ask whether the visual evidence itself can supply the signal for free. We formalize the contrastive evidence gap, the per-
token log-likelihood ratio that a model assigns to its own output when conditioned on a question-relevant region versus an irrelevant one, and study it across Qwen2.5-VL-7B, Qwen3-VL-8B, and Qwen3-VL-30B-A3B on V*Bench. Our main positive result is training-free: selecting the candidate crop under which the model's answer distribution is most peaked, using a single-view, label-free criterion, discovers the answer-bearing region with no bounding boxes, training, or labels. It localizes the target 4.4 to 5.1 times better than chance and raises fine-grained accuracy from 70 percent to 85 percent at inference. We further show that the gap is complementary to the model's own confidence. Combining them predicts correctness better than either alone, with AUC up to 0.99, and flags confidently wrong answers, with AUC ranging from 0.97 to 1.00 within the high-confidence subset. All effects concentrate on perception-bottleneck questions and vanish on a global-context control. Finally, we report an honest negative result: converting the same signal into a training method, gated self-
distillation (SEG-Distill), does not outperform the base model at pilot scale across three gate designs, while more aggressive gating degrades accuracy. The signal is real, but converting it into training gains remains an open problem.
Read the full abstract ▾
All authors Santi Ram Tiwari, Nihal Naik, Devbrat Pandey, Nishant Sinha
Computer Vision
IMPLICIT-Bench: Stereotyped Defaults and Prompt Following in Text-to-Image Generation
Yue Dai, Ziyang Liu, Marc Cheong et al. (+1 more)
Text-to-image (T2I) models often produce stereotypical images when stereotype-relevant attributes are left unspecified. We ask whether explicit requests can overcome these defaults, especially when the requested attribute contradicts the stereotype. We introduce \mbox{IMPLICIT-Bench}, a challenge
benchmark of 1,831
prompt triplets built on a bias knowledge graph (KG) of stereotypes recorded in StereoSet and CrowS-Pairs. Each triplet describes one scene with a neutral
prompt, a stereotype
prompt, and an anti-stereotype
prompt. The neutral
prompt measures the model's default, and the two explicit
prompts measure whether the model can follow a request on either side of the stereotype. On this
benchmark, Qwen-Image depicts the stereotype in 52% of neutral images and the anti-stereotype in 5%. On both Qwen-Image and Stable
Diffusion~3, explicit requests that contradict this default succeed far less often than requests that agree with it. Where Qwen-Image's default is strongest, explicitly requesting the stereotype succeeds in 97% of cases, whereas requesting the anti-stereotype succeeds in only 54%. In these scenes, 31% of the images for anti-stereotype requests show the stereotypical attribute instead. We further test whether
prompt rewriting and
embedding steering can reduce the default on neutral
prompts. With the
prompt's own KG unit, the best configurations improve a combined bias and
alignment score by up to 9.5 points, and with a unit retrieved from the bias KG, by at most 2.9. Debiasing thus depends on knowing which stereotype a
prompt evokes.
Read the full abstract ▾
All authors Yue Dai, Ziyang Liu, Marc Cheong, Caren Han
Computer Vision
SRPR-Net: Semantic and Relational Prompt Refinement for Automated SAM-based Instance Segmentation
Lufei Liu, Guojie Li, Suncheng Xiang et al. (+1 more)
Instance
segmentation is a fundamental computer
vision task with diverse real-world applications. Recently,
prompt-driven foundation models have shown promising generalization. However, automated prompting remains limited by insufficient semantic guidance and inter-instance modeling. To address this challenge, we propose a novel architecture, named Semantic Relational
Prompt Refinement Network (SRPR-Net), for automated SAM-based instance
segmentation. A sequential
prompt refinement mechanism is introduced to enrich detector geometry with visual-
language semantics and then incorporate same-image instance dependencies, enabling context-aware box adjustment before SAM
segmentation. Experiments on multiple standard
benchmarks demonstrate that SRPR-Net achieves consistent improvements in
segmentation performance over existing state-of-the-art approaches. The code is publicly available at https://github.com/JeremyXSC/SRPR-Net.
Read the full abstract ▾
All authors Lufei Liu, Guojie Li, Suncheng Xiang, Fan Zhang
Computer Vision
DiaSeg: Diagonal Segment Extraction from DTW Paths for Interpretable Gait Analysis
Tresor Y. Koffi, Amel Hidouri, Corentin Legrand et al. (+1 more)
Dynamic Time Warping (DTW) is the dominant approach for measuring similarity between time series, yet standard practice discards the optimal warping path after computing a single distance value, losing local
alignment information most relevant to clinical diagnosis. We introduce DiaSeg, a framework that extracts diagonal segments from DTW paths with controlled breaks, characterizing each segment by five
geometric features (effective length, interruption count, cost variation, temporal position, and path context), and enabling unsupervised pattern discovery without domain-specific feature engineering. Validated on 91 subjects across six clinical conditions (healthy aging, Parkinson's, Huntington's, ALS,
brain tumor, and stroke), three findings emerge. First, diagonal segments form consistent unsupervised patterns (silhouette 0.33) aligned with biomechanical phase annotations, with label-based validation confirming near-perfect separation of healthy and pathological gait (ARI up to 0.986). Second, segments discriminate pathology at 69% (supervised) and 75% (patient-level clustering), with pathology manifesting through distributional shifts in segment length; combining segment and cycle-level features further improves classification to 91.7%. Third, while cycle-based methods achieve higher accuracy (91%), diagonal segments provide phase-specific interpretability unavailable in global representations, localizing where coordination breaks down within the gait cycle. DiaSeg thus transforms DTW from a black-box distance into a source of interpretable temporal features for neurodegenerative disease assessment.
Read the full abstract ▾
All authors Tresor Y. Koffi, Amel Hidouri, Corentin Legrand, Aurélie Bertaux
Computer Vision
Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion
Uday Allu, Abhivanth Sivaprakash, Pratik Singh et al. (+1 more)
Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats – PDFs, Word documents, presentations, and scans – whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, flatten tables, and lose heading hierarchy, while fully agentic chunking over extracted text incurs high
token costs and
hallucination risk. We present Document
Retrieval-Aware Chunking (D-RAC), an extension of our Web
Retrieval-Aware Chunking (W-RAC) framework to arbitrary document formats. D-RAC first normalizes any input document into PDF, exploiting the fact that virtually every format has a faithful, deterministic PDF rendering. A single
multimodal LLM pass then converts rendered pages into
retrieval-optimized Markdown – rewriting tables as self-contained prose statements and preserving heading hierarchy – after which chunking proceeds exactly as in W-RAC: deterministic parsing into ID-addressable units followed by lightweight LLM-based chunk planning over identifiers rather than text. Source text is never regenerated during chunking, preserving W-RAC's cost, determinism, and observability benefits while unlocking every renderable format as a first-class input. On the 236-document, 795-page PDF subset of the RAG-Multi-Corpus
benchmark spanning five enterprise domains, D-RAC converts and chunks the entire corpus in 72 minutes with zero errors, producing 1,748
retrieval-ready chunks. Compared to agentic chunking with frontier LLMs, D-RAC reduces chunking-stage output
tokens by 95.7%, cutting chunking cost by 77.8% (GPT-4.1 pricing) to 85.6% (Gemini 2.5 Pro pricing) and chunking time by 75%. D-RAC scales linearly to documents of 500+ pages.
Read the full abstract ▾
All authors Uday Allu, Abhivanth Sivaprakash, Pratik Singh, Aman Manocha
Computer Vision
Beyond Emotion Prompts: Fine-Grained Text-to-Image Generation Driven by Valence-Arousal-Dominance
Minglang Li, Yueyue Fang and Xieping Gao
Although text-to-image models can accurately depict subjects and scenes, creators still struggle to specify the fine-grained
emotions an image should convey without rewriting its content description. Natural
language can suggest
emotions, but it offers no control scale with stable meanings and ordered intensities. We propose EMOTRANS, which transforms psychologically grounded valence-arousal-dominance (VAD) coordinates into generation conditions that are independent of the content text and modulated across denoising stages, making emotional style a finely adjustable creative variable. To support this goal, we construct EMOVAD, an art-painting
dataset that pairs objective content descriptions with separately collected emotional ratings from multiple annotators. We also coordinate emotional expression and content preservation through dual-branch training with a shared model. Objective and human evaluations show that the framework improves the accuracy of three-dimensional
emotion control and produces perceptible, orderable continuous changes while maintaining competitive text
alignment and image quality. This work provides a practical
emotion-driven approach to image generation that extends objective content depiction to fine-grained emotional adjustment.
Read the full abstract ▾