Computer Vision
Ultra Diffusion Poser: Diffusion-Based Human Motion Tracking From Sparse Inertial Sensors and Ranging-Based Between-Sensor Distances
Dominik Hollidt, Tommaso Bendinelli and Christian Holz
Methods using inertial measurement units (IMUs) provide a wearable alternative to camera-based motion capture. To mitigate drift from inertial signals, recent sparse inertial pose
estimators integrate inter-sensor distances measured by ultra-wideband (UWB) ranging. So far, UWB distances have only been used as an additional input feature, ignoring the physical constraints they impose on sensor positions. However, these distances can also be used to reconstruct the underlying 3D sensor layout, which in turn provides more informative input for pose reconstruction. We propose Ultra
Diffusion Poser, a
diffusion model that explicitly models these
geometric constraints. It includes a Spatial Layout Module that analytically reconstructs the 3D sensor positions from UWB measurements. These sensor positions are used alongside IMU signals and UWB distances as a conditioning signal during
diffusion. Still, network predictions can violate inter-sensor distance measurements. To address this, we introduce UWB-
Diffusion Guidance, which encourages
alignment between predicted poses and measured distances during
diffusion sampling. Together, these contributions enable our model to achieve state-of-the-art performance, reducing joint position error by up to 22% over prior work.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Rethinking Evaluation Paradigms in IBP-based Certified Training
Konstantin Kaulen, Hadar Shavit and Holger H. Hoos
Deep neural networks achieve strong performance on many supervised learning tasks but remain vulnerable to
adversarial perturbations. Neural network verification provides mathematically rigorous robustness guarantees, yet at substantial computational cost. To mitigate this, certified training techniques optimise for verifiable robustness during training, typically inducing a trade-off between natural and certified accuracy controlled by method-specific hyperparameters. Because these metrics are inherently conflicting, the common practice of reporting a single configuration is problematic: it can mislead conclusions about overall performance and prevents unbiased assessments of the state of the art. We address this by evaluating certified training methods via Pareto front comparisons over the natural–certified accuracy trade-off. To enable fair, method-agnostic comparisons, we perform efficient automated multi-objective hyperparameter optimisation to identify a set of Pareto-optimal configurations for each method. This approach often uncovers substantial undertuning in previously reported configurations, yielding superior performance and establishing a new state of the art. Leveraging these fronts, we present the first comprehensive multi-objective comparison of certified training approaches, showing that prior advancements are less pronounced than assumed and revealing previously unreported performance complementarities.
Read the full abstract ▾
Computer Vision
Equilibrated Diffusion: Frequency-aware Textual Embedding for Equilibrated Image Customization
Liyuan Ma, Xueji Fang and Guo-Jun Qi
Image customization learns target subjects from reference concept images and generates conditioned images per text
prompts, mainly modifying styles or backgrounds. Prevailing methods adopt fine-tuning to pack diverse concept attributes into a unified latent
embedding, yet entangled attributes hinder elimination of irrelevant disturbances from style and background. To address this issue, we propose Equilibrated
Diffusion, a frequency-driven approach that disentangles tangled concept features for balanced customization and consistent text-visual matching. Unlike conventional methods learning full concepts with shared
embeddings and unified tuning, our work utilizes the inherent link between image frequency components and semantics: low frequencies represent subject content and high frequencies correspond to styles. We decompose concepts in frequency space and optimize each
embedding independently. This separate optimization enables the denoiser to capture style detached from subject identity and generalize better to unseen stylistic
prompts. Merging multi-frequency
embeddings preserves the model's original spatial customization ability. We further deploy mask-guided
diffusion to restrict irrelevant background changes and boost text
alignment. Residual Reference
Attention (RRA) is inserted into spatial
attention to retain subject structure and identity consistency. Experiments prove Equilibrated
Diffusion exceeds mainstream baselines on subject fidelity and text adherence, verifying our method's superiority.
Read the full abstract ▾
Computer Vision
Understanding-Enhanced Model Collaboration for Long-Tailed Egocentric Mistake Detection
Boyu Han, Qianqian Xu, Shilong Bao et al. (+3 more)
In this report, we address the problem of determining whether a user performs an action incorrectly from
egocentric video data. To this end, we propose an Understanding-Enhanced Model Collaboration Method (UE-MCM) that combines efficient coarse-grained
video understanding with accurate fine-grained action
reasoning. Specifically, UE-MCM contains a small model branch and a large model branch. The large model branch focuses on whether the fine-grained action itself is executed incorrectly, while the small model branch jointly takes the coarse-grained
video and fine-grained segment as input to identify actions that may be locally correct but inconsistent with the overall workflow. The small model branch is built on a CLIP4CLIP
video encoder initialized from a CLIP model enhanced by
Diffusion Contrastive Reconstruction, and the large model branch uses the Qwen3-VL
Embedding model to extract high-capacity representations from fine-grained action segments. The small-branch prediction and the large-branch prediction are then adaptively fused by a lightweight collaboration gate. To handle the long-tailed distribution of mistake instances, we optimize the classifiers with complementary objectives, including reweighted cross-
entropy, AUC-oriented learning, and label-aware adjustment. The resulting system balances speed and accuracy, making it effective for detecting subtle, rare, and ambiguous mistakes in
egocentric instructional
videos.
Read the full abstract ▾
All authors Boyu Han, Qianqian Xu, Shilong Bao, Zhiyong Yang, Ruochen Cui, Qingming Huang
Computer Vision
Choongwon Kang, Seungjong Sun, Hyunmin Jun et al. (+1 more)
As
multimodal large
language models (MLLMs) have advanced to process
video inputs, concerns have emerged about their potential for malicious misuse. Prior jailbreak studies have shown that safety
alignment in MLLMs can be bypassed through visual inputs, yet it remains unclear which properties of
video inputs induce this vulnerability. To address this gap, we introduce Multi-Clip
Video (MCV) SafetyBench, a
dataset of 2,920
videos designed to evaluate how the diversity of
video inputs affects the vulnerability of MLLMs. Each
video consists of multiple short clips depicting diverse contexts related to a harmful query. Experiments on eight representative
video MLLMs show that attack success consistently increases with the number of clips. Our results further indicate that the
video modality is (1) more vulnerable than the image modality, (2) more vulnerable to dynamic
videos than to static
videos, and (3) more vulnerable when
videos contain more diverse contexts. Building on these findings, we propose a defense strategy that leverages the relative robustness of the image modality.
Read the full abstract ▾
All authors Choongwon Kang, Seungjong Sun, Hyunmin Jun, Jang Hyun Kim
Computer Vision
Multimodal Action Diffusion for Robust End-to-End Autonomous Driving
Jorge Daniel Rodríguez-Vidal, Diego Porres, Gabriel Villalonga Pineda et al. (+1 more)
End-to-End Autonomous Driving (E2E-AD) systems have largely converged on predicting intermediate trajectory waypoints, delegating final control to hand-crafted controllers with GPS access. Direct control-signal prediction (outputting throttle, steer and brake in an end-to-end fashion) remains underexplored, and critically, the role of action multimodality in such systems is not well understood. We argue that moving beyond deterministic, single-action outputs is not merely a modelling choice, but a key driver of driving performance, representational quality, and training stability. To validate this, we introduce the Action
Diffusion Transformer (ADT), an anchor-free
diffusion transformer trained with a MSE objective that natively models the
multimodal distribution of plausible driving actions. Rather than committing to a single deterministic command, ADT generates K action candidates and selects the most suitable one at inference via Nearest Neighbour Matching (NNM). Beyond strong
benchmark numbers, we show that action multimodality yields measurable benefits in learned representations and behavioral consistency, effects that deterministic architectures cannot replicate. ADT surpasses previous state-of-the-art on the challenging closed-loop Bench2Drive
benchmark while achieving ten times lower latency, demonstrating that expressive,
multimodal action modelling is both practically efficient and conceptually essential for robust end-to-end driving.
Read the full abstract ▾
All authors Jorge Daniel Rodríguez-Vidal, Diego Porres, Gabriel Villalonga Pineda, Antonio M. López Peña
Computer Vision
WebSpline: Structure-Informed Splines for Real-Time 3D Gaussians from Monocular Videos
Jongmin Park, Jeonghwan Yun, Minh-Quan Viet Bui et al. (+1 more)
Dynamic scene reconstruction from
monocular videos remains highly challenging, as existing methods often struggle to balance global structural coherence and local fine-grained details under limited multi-view cues. To address this challenge, we propose WebSpline, a novel dynamic 3D Gaussian framework that enables structurally coherent and high-fidelity reconstruction from
monocular videos with fast rendering. The core of WebSpline is the Structure-Informed Spline (SIS) representation, which models each dynamic Gaussian trajectory using a learnable cubic Hermite spline whose motion is structurally organized with an auxiliary Structural Proxy Graph (SPG). The proposed framework is optimized in two stages: (i) in the first stage, the SPG is initialized from 2D point tracks and refined with temporal
rigidity regularization to establish structural coherence for moving objects across the sequence; and (ii) in the second stage, the SIS representation is initialized from the refined SPG and optimized under both spatial and structural neighborhood constraints. At inference, Gaussian motion is obtained solely by evaluating the learned SIS, enabling fast rendering. Extensive experiments on the challenging
monocular dynamic scene
benchmarks, iPhone and NVIDIA, demonstrate that our WebSpline achieves state-of-the-art rendering quality while rendering over 10 times faster than WorldTree, the second-best method on the iPhone
dataset.
Read the full abstract ▾
All authors Jongmin Park, Jeonghwan Yun, Minh-Quan Viet Bui, Munchurl Kim
Computer Vision · Cross-listed from eess.IV
LALE: Lightweight-Transformer Architecture for Land-Cover Estimation
Ümit Mert Çağlar and Alptekin Temizel
Semantic
segmentation of remote sensing imagery requires models that capture both global context and local detail under tight computational budgets. Prior work typically optimizes for one of these axes:
attention for global context, convolution for local detail, or compactness for efficiency. While hybrid approaches aim to capture both, they require architectural changes and encoder backbones with computational overhead, limiting efficiency and performance. We present LALE (Lightweight-
transformer Architecture for Land-cover Estimation), an end-to-end remote sensing image
segmentation architecture, that bifurcates its encoder by resolution: lightweight ConvMixer stages handle high-resolution local features, while
transformer stages handle low-resolution global context, confining the quadratic cost of self-
attention to deep, downsampled feature maps. An all-MLP multi-scale decoder, together with RMSNorm and StarReLU throughout, further reduces compute and parameter count. On the large-scale ARAS400k remote-sensing
segmentation benchmark, LALE establishes a strong efficiency-performance trade-off against CNN,
transformer, and hybrid baselines. Our smallest variant, (just 1.6M parameters), reaches within 2.6 F1 points of the best baseline (UPerNet) while using 4.5x fewer parameters, 7x less storage, 17x fewer GMACs, and delivering 1.8x higher throughput. The codebase for LALE is publicly available at https://github.com/caglarmert/LALE.
Read the full abstract ▾
Computer Vision
FocusDiT: Masking Queries in Diffusion Transformers for Fine-grained Image Generation
Xueji Fang, Liyuan Ma, Jianhao Zeng et al. (+3 more)
Diffusion transformer (DiT) has been widely adopted in the generative
diffusion field, advancing the denoising of query
tokens through
attention and Feed-Forward (\text{FFN}) layers. FFN actually acts as the key-value vocabulary for decoding visual contents where the value embeds the visual semantical knowledge. We present that focusing on critical query
tokens corresponding to more complex details and encouraging the model to improve these
tokens is essential for fine-grained visual generation. To this end, we propose FocusDiT, which applies a Masking scheme to focus on critical query
tokens that are exclusively fed into FFN. The masked queries can retrieve visual
tokens from the FFN vocabularies, and use them to decode their visual details. Extensive text-to-image experiments validate the effectiveness of
token masking in enhancing generative performance.
Read the full abstract ▾
All authors Xueji Fang, Liyuan Ma, Jianhao Zeng, Jinjin Cao, Mingyuan Zhou, Guo-Jun Qi
Computer Vision · Cross-listed from cs.MA
Agentic-J: An AI Agent for Biological Microscopy Image Analysis
Lukas Johanns, Marilin Moor, Davide Panzeri et al. (+9 more)
Biological image analysis increasingly demands integration across heterogeneous tools, programming environments, and domain knowledge that few researchers can command simultaneously. We present Agentic-J, a containerised, multi-
agent AI assistant, primarily for ImageJ/Fiji that enables biologists to specify analysis tasks in natural
language, from
nuclei segmentation and
cell tracking to multi-condition quantification. The
agent generates executable scripts organised into a documented project structure, so every analysis decision is traceable and the workflow can be reproduced or shared. The specialised sub-
agents handle plugin management, code generation, debugging, quality assurance, and statistical reporting. In this paper we introduce the system's design, demonstrate real biological microscopy image analysis workflows, and detailed the technical implementation.
Read the full abstract ▾
All authors Lukas Johanns, Marilin Moor, Davide Panzeri, Yu Zhou, Xinyi Chen, Nora F. K. Pauly, Zixuan Pan, Matthias Gunzer, Andreas Müller, Yiyu Shi, Hedi Peterson, Jianxu Chen
Computer Vision
FACT: A Simple and Efficient Framework for Active Finetuning
Wenshuai Xu, You Song, Yuzhuo Cui et al. (+3 more)
The main goal of active finetuning is to improve a pretrained model's performance on a specific task or domain by finetuning it with carefully selected informative or challenging data. Previous research has predominantly focused on the active aspect (i.e., data selection) while uniformly employing full finetuning for model adaptation, which inevitably distorts pretrained features due to distribution shift. This issue becomes particularly pronounced when the model size is large relative to the finetuning data quantity, leading to heightened overfitting risks. To address this critical gap, we formally outline the FiAF task that emphasizes systematic exploration of finetuning methodologies in active learning. We propose FACT, a three-phase hierarchical finetuning framework featuring both efficiency and simplicity, specifically designed for active finetuning scenarios. Our comprehensive experiments span: (1) Three major
dataset categories encompassing classic (CIFAR10, CIFAR100, ImageNet-1k), imbalanced (CIFAR10-LT, CIFAR100-LT), and fine-grained (StanfordCars, FGVCAircraft) image classification
datasets, each evaluated under 3-5 distinct sampling ratios; (2) Diverse pretrained architectures including Convolutional Neural Network (ConvNeXt),
Vision Transformer (ViT), and
Vision LSTM (ViL) networks; (3) A systematic investigation of frozen feature augmentation (FroFA) strategies. (4) A comprehensive and rigorous analysis of efficiency and generalizability. The results demonstrate significant improvements with strong generalization and robustness. Notably, under low sampling ratios, our framework achieves remarkable performance gains of over 20% on the ViT model for CIFAR10, CIFAR100, and ImageNet-1k
benchmarks. This systematic approach establishes new state-of-the-art performance while maintaining parameter efficiency, proving particularly effective when labeled data is scarce.
Read the full abstract ▾
All authors Wenshuai Xu, You Song, Yuzhuo Cui, Minjie Ren, Qingjie Liu, Zhenghui Hu
Computer Vision
Fast and Lightweight Novel View Synthesis with Differentiable Multiplane Image
Kaidi Zhang and Guanxu Zhu
Recently, novel view synthesis has witnessed remarkable progress, with mainstream methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) delivering impressive results. However, these approaches often struggle to balance rendering speed and model size, and their optimization-based training can be highly time-consuming. Furthermore, they typically rely on dense observations, often failing to produce satisfactory results under sparse-view conditions. Although feed-forward reconstruction significantly reduces the optimization time of 3DGS, its pixel-aligned formulation generates millions of Gaussians from a single image, severely limiting its practical deployment on mobile devices. To address these limitations, we revisit the Multiplane Image(MPI) representation, which represents scenes using a compact set of planar layers for efficient novel view synthesis. Leveraging recent advances in visual foundation models, we utilize predicted point maps for reliable
geometric initialization, followed by differentiable optimization. To address the issues of holes and artifacts in sparsely initialized MPI, we introduce one-step
diffusion, which participates in both the differentiable optimization of MPI and the postprocessing of rendering results. Compared with a representative GS-based method, our approach is 30.7% faster and uses only 14.8% of its model size, while achieving competitive synthesis quality on front-view scenarios
Read the full abstract ▾
Computer Vision
TIDES: Time-Derivative Event Simulation via Deformable Reconstruction
Christopher Thirgood, Dipon Kumar Ghosh and Simon Hadfield
Event cameras emit asynchronous events in response to environmental appearance changes. The scarcity of real-world event
datasets makes simulation essential. However, most simulators infer event timestamps from frame sequences, forcing many threshold crossings to share a small set of discrete times; a failure mode we term timestamp batching that worsens under fast motion and occlusion. We present TIDES, a continuous-time event simulator built on dynamic Gaussian splatting. Because TIDES operates on an explicit 3D scene representation with learnt geometry and motion, it can derive per-pixel intensity dynamics directly from the scene, rather than by differencing rendered frames. This enables accurate threshold-crossing prediction, including multiple crossings per rendering step, without temporal upsampling or frame interpolation. The same 3D scene model reveals where objects partially occlude one another; TIDES uses this to guide adaptive time stepping, concentrating computation only in regions where occlusion dynamics make simple models of brightness change unreliable. Finally, we model finite sensor bandwidth using a tile-level arbiter whose throughput, jitter, and event drops reproduce realistic sensor artifacts. Across paired RGB-event
benchmarks, TIDES attains state-of-the-art event-stream fidelity. We also show that events simulated by TIDES transfer more effectively to real downstream tasks than competitors'.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Topological texture analysis of microscopy images of dynamic casein gelation and its relation to rheological properties
Zahra Tabatabaei, Diana Soto Aguilar, Jose C. Bonilla et al. (+2 more)
We propose a novel computational toolbox that integrates Topological Data Analysis (TDA), Differential Box Counting (DBC), Multifractal Partition (MFP), and Local Binary Patterns (LBP), applied to time-lapse super-resolution STED microscopy images of sodium caseinate gelation induced by glucono-delta-lactone (GDL) at 30 {\deg}C and 40 {\deg}C and two GDL concentrations (1.8% and 3.5% w/v). TDA tracked topological loops, closed ring-like structures reflecting
protein network interconnectivity, via max-Betti-1 curves, which revealed a lag phase of dispersed aggregates, a sharp decay coinciding with network percolation and the rheologically observed sol-gel transition, and a post-gelation increase corresponding to network rearrangements. These topological transitions were corroborated by DBC and MFP as these methods were able to resolve changes in structural complexity and spatial heterogeneity. The toolbox was validated on simulated fractal images prior to experimental application. Together, these descriptors provided sensitivity to subtle microstructural transitions that bulk rheology captured as averaged bulk mechanical responses. This integrated approach provides a robust quantitative tool for characterizing complex microstructure in food and material science with evolving microstructural dynamics. Code is available at https://github.com/Zahratabatabaei/Delifood_CV_paper.git
Read the full abstract ▾
All authors Zahra Tabatabaei, Diana Soto Aguilar, Jose C. Bonilla, Mathias P. Clausen, Jon Sporring
Computer Vision
Quantifying and Mitigating Domain Shift in Peach Leaf Damage Classification: Attention Mechanisms and Fine-Tuning Strategies
Adrián Cánovas-Rodriguez, Miguel A. González-Illán, Maria Fernanda García-Cruz et al. (+5 more)
Deep learning models for crop damage assessment are typically trained and validated on curated public imagery, yet their behaviour when deployed in real orchards remains poorly quantified. This work measures and mitigates that gap for peach leaf damage classification, where
climate-driven abiotic and biotic stresses produce visually similar foliar symptoms. A
benchmark of 1366 manually annotated peach leaves covering six damage types was assembled from public sources, and a second, independently acquired
dataset of 180 field images across four classes was collected in a commercial orchard as an unseen target domain. Eleven convolutional backbones and three
attention-enhanced variants were compared; CBAM-EfficientNetB5 achieved the best source-domain performance (93.3% accuracy, 0.849 macro F1). Applied directly to the target domain, source-trained models lost on average 0.21 macro F1 points (26.5% relative), with 12 of 14 architectures degrading, confirming that
benchmark performance substantially overestimates field behaviour. Three fine-tuning strategies were then evaluated as mitigation: feature extraction proved insufficient in nearly all cases, whereas full fine-tuning recovered performance, with CBAM-EfficientNetB3 reaching 0.9459 accuracy and 0.9297 macro F1 on the local domain.
Attention mechanisms improved minority-class recall and adaptation efficiency, but did not by themselves confer robustness to domain shift. The results establish a transferability baseline for peach leaf diagnosis and quantify the adaptation cost of moving from public
benchmarks to operational orchards.
Read the full abstract ▾
All authors Adrián Cánovas-Rodriguez, Miguel A. González-Illán, Maria Fernanda García-Cruz, Pedro Nortes Tortosa, José Salvador Rubio-Asensio, Miguel A. Zamora Izquierdo, Juan Antonio Martínez Navarro, Antonio F. Skarmeta
Computer Vision
Normality-Preserving Continual Industrial Anomaly Detection via Orthogonal LoRA Banks
Weibai Fang, Haijun Che, Feiyang Ren et al. (+1 more)
Continual industrial anomaly detection with
diffusion models suffers from historical normality prior drift and catastrophic forgetting. Existing continual
diffusion methods preserve previous knowledge through replay or constrained optimization, but they lack an explicit mechanism for isolating and protecting category-specific normality priors during sequential adaptation. Although low-rank adaptation provides modular residual updates, standard LoRA neither freezes historical normality subspaces nor prevents new adapters from interfering with previous ones. To address this issue, we propose a normality-preserving continual anomaly detection framework based on two modules: History Frozen Orthogonal LoRA Bank (HF-OLB) and Hierarchical Novelty Adaptive Bank Growth module (HNABG). HF-OLB freezes both the pre-trained U-Net backbone and the learned LoRA banks, and constrains new task-specific normality residuals to the orthogonal complement of historical LoRA subspaces. HNABG further allocates layer-dependent residual capacity and expands the bank only when the residual normality novelty exceeds the expressive capacity of existing banks. Extensive experiments on MVTec and VisA demonstrate the effectiveness of the proposed method. On the challenging VisA 2x6 setting, our method achieves 83.6/91.8 image and pixel level A-AUROC with 3.8/3.9 FM, improving pixel level A-AUROC over the state of the art by 3.2 points while reducing pixel level FM by 1.3. These results show that our method effectively preserves historical normality priors in long horizon continual category sequences.
Read the full abstract ▾
All authors Weibai Fang, Haijun Che, Feiyang Ren, Qiancheng Lao
Computer Vision · Cross-listed from cs.LG
OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents
Rui Yang, Qianhui Wu, Yuxi Chen et al. (+7 more)
Building capable visual web
agents requires long-horizon
reasoning, precise grounding, and robust interaction with dynamic real-world websites. Despite rapid progress, the strongest systems remain largely proprietary, while open
agents still depend heavily on supervised post-training over large collections of curated web trajectories. This dependence creates a major scalability bottleneck: high-quality demonstrations are expensive to collect, and static
datasets offer limited coverage of the diverse, ever-changing open web. Although online RL has shown promise for text-based
agents, its potential for training visual web
agents directly on live websites remains largely underexplored. In this paper, we introduce OpenWebRL, an open framework for training visual web
agents with online multi-turn RL on real websites. OpenWebRL covers the full training pipeline, including scalable live-browser infrastructure, supervised initialization,
multimodal context management, trajectory-level success judging, and efficient multi-turn policy optimization. Using this framework, we train OpenWebRL-4B, which establishes a new open-source state of the art on challenging live-web
benchmarks. With only 0.4K initialization trajectories and 2.2K open-ended RL training tasks, OpenWebRL-4B achieves 67.0% success on Online-Mind2Web and 64.0% on DeepShop, outperforming prior open
agents of similar or larger scale and remaining competitive with proprietary systems including OpenAI CUA and Gemini CUA. Beyond strong
benchmark performance, we systematically study the key design choices that make online RL effective for visual web
agents, and analyze how RL improves agentic
reasoning. Overall, our work offers a practical path toward building more capable, reproducible, and cost-efficient open web
agents. We will release our training data, models, and code to support future research.
Read the full abstract ▾
All authors Rui Yang, Qianhui Wu, Yuxi Chen, Hao Bai, Wenlin Yao, Hao Cheng, Baolin Peng, Huan Zhang, Tong Zhang, Jianfeng Gao
Computer Vision
Ranking vs. Assignment: The Metric Mismatch in Multi-View Object Association
Matvei Shelukhan, Timur Mamedov, Aleksandr Chukhrov et al. (+1 more)
Multi-view object association is an important computer
vision problem that underlies many multi-camera perception tasks. While this task is naturally formulated as a constrained one-to-one matching problem, recent works heavily rely on pairwise ranking metrics like AP and FPR-95 for model evaluation. We highlight a fundamental mismatch between these metrics and the actual assignment objective. Theoretically, we show that AP and FPR-95 can be imperfect even when the assignment is already correct, and that Sinkhorn-based normalization can make them perfect. Conversely, optimal pairwise ranking can still lead to incorrect assignments. We validate this mismatch in practice by using our Sinkhorn-based normalization as a controlled post-processing stress test. We show that optimizing just a few post-processing parameters significantly boosts AP and FPR-95 without corresponding improvements in assignment-level metrics such as ACC and IPAA.
Read the full abstract ▾
All authors Matvei Shelukhan, Timur Mamedov, Aleksandr Chukhrov, Karina Kvanchiani
Computer Vision
PerBite: A Curated Diagnostic Workflow for Bite-Aware Food Volume Estimation
Ahmad AlMughrabi, Farid Al-Areqi, David Fernández Gómez et al. (+4 more)
Can a visually plausible food mesh be trusted to estimate the volume of consumed food? \method investigates this question using selected paired before- and after-consumption states from the MetaFood CVPR 2026 Continuous 3D Reconstruction While Eating Challenge. The submitted workflow follows a curated reconstruction protocol: SAM~3 segments the food and plate regions; Hunyuan3D/SAM~3D generates a dimensionless food mesh; the plate diameter provides the metric scale; the plate geometry is removed in Blender; and the remaining mesh is hole-filled, made watertight, and integrated to estimate volume. MoGe-2 is used only as an auxiliary cue for initial dish-diameter estimation when direct plate measurement is uncertain; it is not the primary scale source for the reported challenge result. \method ranks first, with an average Chamfer distance of 8.31 across 34 meshes using rigid ICP without scale correction. On 17 before- and after-pairs, it achieves 33.87% state-level volume MAPE and zero monotonicity violations, while consumed-volume MAPE remains 53.74%. The results show that surface reconstruction, metric scale, controlled mesh cleanup, watertight volume integration, and physical depletion consistency should be evaluated separately for dietary assessment. Source code and evaluation scripts will be available at \href{https://github.com/GCVCG/PerBite-CVPR-MetaFood-2026}{github.com/GCVCG/PerBite-CVPR-MetaFood-2026}.
Read the full abstract ▾
All authors Ahmad AlMughrabi, Farid Al-Areqi, David Fernández Gómez, Umair Haroon, Marc Bolaños, Ricardo Marques, Petia Radeva
Computer Vision
Parameter-Efficient Adaptation of a Multi-Stream Vision-Language Framework for Blind Image Quality Assessment
Bishr Omer Adam and Xu Li
Blind image quality assessment (BIQA) predicts perceived image quality without access to a pristine reference and is fundamental to applications such as image compression, transmission, and restoration. Recent BIQA methods increasingly rely on large
vision-
language models (VLMs). Although frozen VLMs provide an efficient alternative to computationally expensive full fine-tuning, it remains unclear how much performance is sacrificed by not adapting the backbone and, more importantly, under what conditions such adaptation is truly beneficial. Answering this question, however, is complicated by the widespread use of image-level splitting on synthetic-distortion
benchmarks, where distorted versions of the same reference image can appear in both training and test partitions. This content overlap artificially inflates the apparent performance of frozen representations, masking their true generalization ability and potentially leading to incorrect conclusions about the value of backbone adaptation. We therefore address these two issues jointly. We develop an efficient BIQA framework that fuses a natural-scene-statistics descriptor with frozen SigLIP and CLIP-H
embeddings through a lightweight regression head, and then apply parameter-efficient Low-Rank Adaptation (LoRA) to the SigLIP backbone, training only $0.23\%$ of its parameters. Evaluating both frozen and adapted models across six
datasets under image-level and reference-level protocols, we find that image-level splitting inflates frozen-feature SROCC by up to $0.44$ and masks wide variation in true difficulty, which reference-level evaluation reveals. Under this content-independent protocol, LoRA adaptation recovers performance in proportion to the exposed difficulty, with the largest gains where frozen features generalize poorly (up to $+0.357$ SROCC on TID2013) and little benefit where they are already strong.
Read the full abstract ▾
Computer Vision
Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenization
Jingyun Liang, Min Wei, Shikai Li et al. (+5 more)
Diffusion models have shown remarkable success in
video generation. However, whether such models are truly aware of the 3D structure underlying visual observations, rather than simply reproducing plausible 2D projections, remains an open question. In this work, we investigate this question through human motion control, a task that requires precise modelling of 3D human geometry, motion, camera viewpoint, and scene context. Unlike prior methods that rely on rendered 2D motion guidance
videos, we propose a render-free framework that conditions
video generation directly on compressed 3D human mesh
tokens. This representation preserves full 3D
geometric information while enabling a unified
token-based generation pipeline that processes
video tokens jointly with motion
tokens in a DiT-based architecture. This design requires the model to reason jointly about appearance, 3D structure, and camera viewpoint during
video generation. Experimental results demonstrate strong performance on human motion control
benchmarks, while reducing artifacts induced by view-dependent 2D guidance and trajectory-pose mismatches during editing. These findings suggest that
video diffusion models, when equipped with mesh tokenization, can better capture complex 3D human structures and their interactions with the surrounding environment.
Read the full abstract ▾
All authors Jingyun Liang, Min Wei, Shikai Li, Yizeng Han, Hangjie Yuan, Lei Sun, Weihua Chen, Fan Wang
Computer Vision
A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision
Stefano Samele, Eugenio Lomurno, Teodora Jovanovic et al. (+3 more)
Industrial anomaly detection has historically been a unimodal task. Recent
multimodal vision-
language models have produced systems that admit textual input alongside the image and are presented as enabling text-guided zero- and few-shot inspection. Yet these methods are evaluated with protocols inherited from unimodal
benchmarks that hold the textual condition constant and therefore cannot measure whether
language conditions the decision; whether reported gains reflect text guidance or strong pretrained visual features remains open. We introduce Text-Guided Anomaly Detection (TGAD), a structured
benchmark that progressively increases the functional role of
language across three scenarios: a controlled
prompt-sensitivity setting on MVTec AD; a component-tagged extension of MVTec AD that requires the model to restrict its assessment to an instructed part; and the new Assembled Panel
Dataset (APD), a realistic industrial setting that requires both defect-type and component-location knowledge. We evaluate one representative model per paradigm: generative large
vision-
language, training-free discriminative, and
embedding-adaptive discriminative. In all three, the textual interface conditions the decision only superficially:
prompt content is absorbed unless the object noun is removed (the generative model's I-AUROC drops from 97.4 to 82.6); component-level instructions do not constrain the decision once defects outside the instructed part are admitted as normal (from 90.3 to 66.3); and when both combine on APD, image-level discrimination collapses below the MVTec level, in one case below chance (71.2, 50.5, 31.5). These results suggest that standard
benchmarks overstate the text-guided capabilities of current
multimodal anomaly detection systems, and that a protocol of this kind is a prerequisite for models that can be reliably controlled through
language for industrial deployment.
Read the full abstract ▾
All authors Stefano Samele, Eugenio Lomurno, Teodora Jovanovic, Sanjay Shivakumar Manohar, Alberto Crivellaro, Matteo Matteucci
Computer Vision
MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching
Jiahui Huang, Yasi Zhang, Tianyu Chen et al. (+6 more)
Recent breakthroughs in instruction-based image editing have captured significant
attention, as models are now capable of handling real-world editing demands with the practicality required by everyday users. However, editing models trained primarily for single-turn edits often break down in multi-turn editing–the natural interactive setting where a user iteratively refines an image based on the model's own previous outputs. This failure stems from the all-or-nothing requirement, where a single failed turn compromises the entire sequence, and error propagation, where exposure bias leads to compounding editing errors. To address these challenges, we introduce MT-EditFlow, a flow-matching reinforcement learning framework designed to optimize reward signals for sequential image editing. MT-EditFlow integrates a multi-turn perspective with a multi-reward formulation to provide a unified structure applicable to both GRPO and NFT-based reinforcement learning methods. We systematically analyze and optimize the reward signal by investigating effective scoring strategies for turn-level aggregation, VLM
reasoning modes to trade off reward bias and variance, and advantage fusion levels to prevent reward hacking. Our findings reveal that broadcasting the aggregated advantage across the entire editing trajectory effectively bridges the gap between local planning and global multi-turn task success. Extensive experiments demonstrate that MT-EditFlow significantly improves performance across diverse base models. Notably, it boosts FLUX.1-Kontext-dev by 6.85 points in turn-3 overall performance, surpassing state-of-the-art open-source models such as Qwen-Image-Edit. By maintaining high marginal success rates and reducing exposure bias, MT-EditFlow provides a foundation for more reliable and natural human-AI collaboration in visual content creation.
Read the full abstract ▾
All authors Jiahui Huang, Yasi Zhang, Tianyu Chen, Shu Wang, Jianwen Xie, Oscar Leong, Mingyuan Zhou, Nanzhu Wang, Ying Nian Wu
Computer Vision
Generalization Limits in Vehicle Re-Identification
Anis Yassine Ben Mabrouk, Antoine Tadros, Rafael Grompone von Gioi et al. (+3 more)
Vehicle re-identification focuses on retrieving images of the same vehicle from a gallery given a query image. Upon closer inspection of commonly used
datasets, we observe that vehicles with few visual differences-e.g., the same make, model, and color-appear in both the training and test sets. As a result, methods that effectively memorize the training data tend to perform well on these test sets but struggle to generalize to other
datasets. In this paper, we address this issue by proposing a novel evaluation approach that more effectively measures generalization capability to unseen vehicle types. To further study generalization performance, we also propose splitting the evaluation based on view, allowing us to differentiate the effect of viewpoint robustness from that of same-view re-identification. Our findings reveal that most state-of-the-art methods struggle with unseen vehicle types, and that their robustness to viewpoint changes and
attention to detail are limited to vehicle types seen during training.
Read the full abstract ▾
All authors Anis Yassine Ben Mabrouk, Antoine Tadros, Rafael Grompone von Gioi, Gabriele Facciolo, Axel Davy, Rodrigo Verschae
Computer Vision · Cross-listed from cs.LG
A Closer Look at In-Distribution vs. Out-of-Distribution Accuracy for Open-Set Test-time Adaptation
Zefeng Li and Evan Shelhamer
Open-set test-time adaptation (TTA) updates models on new data in the presence of input shifts and unknown output classes. While recent methods have made progress on improving in-distribution (InD) accuracy for known classes, their ability to accurately detect out-of-distribution (OOD) unknown classes remains underexplored. We
benchmark robust and open-set TTA methods (SAR, OSTTA, UniEnt, and SoTTA) on the standard corruption
benchmarks of CIFAR-10-C at the small scale and ImageNet-C at the large scale. For CIFAR-10-C, we use OOD data from SVHN and CIFAR-100 in their respective corrupted forms of SVHN-C and CIFAR-100-C. For ImageNet-C, we use OOD data from ImageNet-O and Textures in their respective corrupted forms of ImageNet-O-C and Textures-C. ImageNet-O is nearer to ImageNet, as unknown but related object classes (like ”garlic bread” vs. ”hot dog” for food, or ”highway” vs. ”dam” for infrastructure), while Textures is farther from ImageNet, as non-object patterns (like ”cracked” mud, ”porous” sponge, ”veined” leaves). We evaluate the accuracy and confidence of TTA methods for InD vs. OOD recognition on CIFAR-10-C and ImageNet-C. We verify the accuracy of each method's own OOD detection technique on CIFAR-10-C. We also evaluate on ImageNet-C and report both accuracy and standard OOD detection metrics. We further examine more realistic settings, in which the proportions and rates of OOD data can vary. To explore the trade-off between InD recognition and OOD rejection, we propose a new baseline that replaces softmax/multi-class output with sigmoid/multi-label output. Our analysis shows for the first time that current open-set TTA methods struggle to balance InD and OOD accuracy and that they only imperfectly filter OOD data for their own adaptation updates.
Read the full abstract ▾
Computer Vision
Contrastive Augmented Transformer with Domain-specific Enhancement for Robust Multi-scenario Metal Surface Defect Detection
Yiyao Liu, Wenxiao He, Liyuan Ren et al. (+1 more)
Metal surface defect detection is critical for maintaining product quality in industrial manufacturing. However, it faces significant challenges, including limited annotated data, difficulty in identifying subtle multi-scale defects, and poor generalization across diverse scenarios. To address these issues, this paper proposes a novel Contrastive Augmented
Transformer (CAT) framework for robust defect detection. CAT employs a hierarchical Swin
Transformer backbone and redesigns the feature pyramid network to effectively fuse low-level textures with high-level semantics, enabling precise modeling of subtle and multi-scale defect patterns. To enhance robustness under real-world noise conditions, we propose a domain-specific droplet augmentation
algorithm. Furthermore, we incorporate a hard negative mining strategy into the contrastive loss to strengthen the model's discrimination ability in ambiguous defect regions. Experimental results on the KolektorSDD2
dataset demonstrate that CAT achieves a pixel-level AUROC of 99.54%, outperforming existing methods. In addition, CAT exhibits superior generalization and robustness on three unseen
datasets, including KSDD1, MTD for tile defects, and MSDD for rail surface defects, demonstrating its potential for wide-scale industrial deployment.
Read the full abstract ▾
All authors Yiyao Liu, Wenxiao He, Liyuan Ren, Huan Wang
Computer Vision · Cross-listed from cs.RO
WALL-WM: Carving World Action Modeling at the Event Joints
Shalfun Li, Victor Yao, Charles Yang et al. (+29 more)
WALL-WM is a World Action Model that shifts
video-action learning from chunk-centric optimization to event-grounded
Vision-
Language-Action pretraining, using semantically coherent action events as the atomic unit of learning. Existing WAMs commonly initialize from
multimodal or
video foundation models and then optimize fixed-length action chunks conditioned directly on the current observation and instruction. Although convenient, this chunk-centric formulation creates a fundamental granularity mismatch.
Language describes semantic goals and events,
vision evolves through continuous scene dynamics, and actions operate at control-level timescales; forcing all three into the same fixed-length prediction window turns VLA training into short-horizon correlation fitting. WALL-WM addresses this mismatch by organizing both supervision and data around semantic events. Specifically, it pairs event-grounded VLA pretraining with a data ecosystem built from event-level captions and cluster-balanced sampling, enabling scalable learning over diverse behaviors, scenes, and task structures. From the same event-pretrained backbone, WALL-WM supports two complementary inference modes. The event mode consumes next-event descriptions and enables variable-length execution chunks, while the unified mode uses a VLM with Staircase Decoding to condition conventional fixed-length chunk inference while preserving a gradient-continuous VLA path. Together with
Muon-optimizer-based large-scale pretraining infrastructure, WALL-WM provides a practical scale-up recipe for general-purpose WAMs. Experiments show that WALL-WM generalizes broadly across
language, scenes, and tasks, achieving state-of-the-art performance in large-scale real-world generalization evaluation.
Read the full abstract ▾
All authors Shalfun Li, Victor Yao, Charles Yang, Truth Qu, Regis Cheng, Ryan Yu, Howard Lu, Newton Von, Vincent Chen, Yohann Tang, Maeve Zhang, Ellie Ma, Gody Li, Starrick Liu, Sage Yang, Lorien Shu, J.W. Gao, Ethan Chen, Colin Ye, Yu Sun, Elise Mon, PS Zhang, Neo Li, Lily Li, James Wang, Ping Yang, Chris Pan, Lucy Liang, Hang Su, Roy Gan, Hao Wang, Qian Wang
Computer Vision · Cross-listed from cs.RO
Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects
Jens U. Kreber, Lukas Mack and Joerg Stueckler
World models enable intelligent
agents to predict the consequences of their actions on the environment. In this paper, we propose Multi Rigid Object Gaussian World Model (MRO-GWM), a novel model that learns action-conditional dynamics of rigid objects in 3D. Using object-centric Gaussians, we can represent arbitrary object shapes and multi-object scenes. We develop a novel temporal-spatial
transformer architecture that predicts future rigid body motion from a history of object Gaussians and future actions. Objects are represented by Gaussians in an object-centric frame, which allows to describe object motion as a rigid body transformation. Our model is trained on reconstructions from multiple viewpoints, which requires the model to also handle partial observations of objects due to occlusions. We analyze prediction performance of our approach on synthetic
datasets composed of typical household objects with multi-object dynamics and interactions by a
robot end effector. We also evaluate our model in model-predictive control for non-prehensile manipulation in simulation and for image-based tracking and prediction of object motion in simulated and real-world scenes.
Read the full abstract ▾
Computer Vision
Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks
Nermeen Abou Baker, David Rohrschneider and Uwe Handmann
Research and applications in artificial intelligence have recently shifted with the rise of large pretrained models, which deliver state-of-the-art results across numerous tasks. However, the substantial increase in parameters introduces a need for parameter-efficient training strategies. Despite significant advancements, limited research has explored parameter-efficient fine-tuning (PEFT) methods in the context of
transformer-based models for instance
segmentation. Addressing this gap, this study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA), applied to two models across four
benchmark datasets. Integrating sequentially arranged adapter modules and applying LoRA to deformable
attention–explored here for the first time–achieves competitive performance while fine-tuning only about 1-6% of model parameters, a marked improvement over the 40-55% required in traditional fine-tuning. Key findings indicate that using 2-3 adapters per
transformer block offers an optimal balance of performance and efficiency. Furthermore, LoRA, exhibits strong parameter efficiency when applied to deformable
attention, and in certain cases surpasses adapter configurations. These results show that the impact of PEFT techniques varies based on
dataset complexity and model architecture, underscoring the importance of context-specific tuning. Overall, this work demonstrates the potential of PEFT to enable scalable, customizable, and computationally efficient transfer learning for instance
segmentation tasks.
Read the full abstract ▾
Computer Vision
Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorization
Yumiao Zhao, Bo Jiang, Beibei Wang et al. (+3 more)
Visual Prompting (VP) has emerged as an efficient paradigm for adapting large-scale pre-trained
vision models to downstream tasks by incorporating learnable
prompts at the input level. However, existing VP methods typically employ dense pixel-level
prompts, which often suffer from redundant perturbations, limited generalization and energy inefficiency. To overcome these limitations, we propose to integrate
brain-inspired spiking learning into visual
prompt learning tasks. As we know that spiking
neuron can perform inexpensive information processing by transmitting the input data into discrete spike trains and return sparse outputs. Inspired by this, we propose Low-Rank visual Spike Prompting (LoRSP), a novel framework that learns dynamic low-rank sparse visual
prompts naturally via a Spiking
neuron learning mechanism. The core idea of LoRSP is to exploit the
brain-inspired sparse firing mechanism of spiking
neurons to generate pixel-level sparse
prompt for each instance. To be specific, we first construct a series of
prompt factors via low-rank factorization to capture distinct
prompt subspaces. These
prompt factors are then fed into an SNN architecture, which performs the integrate-and-fire process to emit spikes. As a result, our LoRSP generates a sparse visual
prompt while maintaining the low-rank constraint. This design enables instance-specific selective prompting, leading to more compact and robust adaptation across diverse downstream tasks. Extensive experiments on five heterogeneous
vision backbones and multiple
benchmarks demonstrate that LoRSP achieves competitive performance while requiring fewer tunable parameters compared to existing VP methods.
Read the full abstract ▾
All authors Yumiao Zhao, Bo Jiang, Beibei Wang, Xixi Wan, Xiao Wang, Jin Tang