Computer Vision
Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding
Jiaqi Li, Shuntian Zheng, Yixian Shen et al. (+4 more)
Video Temporal Grounding (VTG) localizes the temporal boundaries of query-relevant moments in long, untrimmed
videos, making
video-
language-model prohibitively expensive. While recent training-free
token pruning has shown success in
video question answering, naively applying these objectives to VTG causes drastic degradation, as VTG crucially depends on boundary-sensitive evidence and cross-frame
reasoning chains. We therefore identify two VTG-specific
pruning principles: evidence retention, which keeps query-critical patches especially around event boundaries, and connectivity strength, which preserves cross-frame connectivity for long-range evidence aggregation. Building on these insights, we propose SemVID, a training-free
pruning framework that constructs a compact yet coherent
token subset with complementary semantic roles. SemVID first allocates per-frame budgets by balancing query relevance and inter-frame variation to avoid over-pruned segments, and then selects three types of
tokens: object
tokens for diverse query-critical evidence, motion
tokens to capture meaningful transitions and serve as cross-frame relays, and context
tokens for scene continuity. Extensive experiments show that SemVID achieves a strong accuracy-efficiency trade-off, retaining up to 95.4% mIoU with only 12.5% visual
tokens and delivering up to a 5.8x prefill speedup, consistently outperforming prior methods under the same budgets. Our code is available at https://github.com/JiaqiLi404/SemVID
Read the full abstract ▾
All authors Jiaqi Li, Shuntian Zheng, Yixian Shen, Jia-Hong Huang, Xiaoman Lu, Minzhe Ni, Yu Guan
Computer Vision
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna et al. (+1 more)
Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers. But many real-world tasks admit multiple valid outputs and lack the single ideal answer that rubric generation depends on. We identify this reference-free setting as a gap in current post-training methods and propose Implicit Error Counting (IEC) to fill it. Instead of checking what a response gets right against a rubric, IEC enumerates what it gets wrong, applying severity-weighted scores across task-relevant axes and converting them into calibrated per-aspect rewards. We show that naïve explicit enumeration is too noisy for stable optimization, and that two design choices: implicit score emission and group calibration are necessary to make error counting a reliable reward. As a case study, we validate IEC on virtual try-on (VTO), a domain that is simultaneously too constrained for holistic scoring and too permissive for rubric-based evaluation: subtle garment errors are unacceptable, yet many output variations are correct. We introduce Cascaded Error Counting (CEC) as an evaluation metric, which tracks human preferences well (60% top-1 vs. 30% others), and curate Mismatch-DressCode (MDressBench), a
benchmark with maximal attribute mismatch to stress-test reward designs. On MDressBench, IEC outperforms RaR across all metrics (CEC: 5.31 vs. 5.60 on flat references; 5.20 vs. 5.53 on non-flat). On VITON-HD and DressCode, IEC matches or surpasses six baselines on 6 of 8 perceptual metrics. These results suggest that when ideal answers are unavailable, counting errors provide a stronger signal than constructing rubrics.
Read the full abstract ▾
All authors Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna, Karim Bouyarmane
Computer Vision
Making Reconstruction FID Predictive of Diffusion Generation FID
Tongda Xu, Mingwei He, Shady Abu-Hussein et al. (+6 more)
It is well known that the reconstruction FID (rFID) of a VAE is poorly correlated with the generation FID (gFID) of a latent
diffusion model. We propose interpolated FID (iFID), a simple variant of rFID that exhibits a strong correlation with gFID. Specifically, for each
dataset element, we retrieve its nearest neighbor in latent space, interpolate between their latent representations, decode the interpolated latent, and compute the FID between the decoded samples and the original
dataset. We provide an intuitive explanation for why iFID correlates well with gFID, and why reconstruction metrics can be negatively correlated with gFID, by connecting iFID to recent results on
diffusion generalization and
hallucination. Theoretically, we show that iFID evaluates decoded interpolations aligned with the ridge set around which
diffusion samples concentrate, thereby measuring a quantity closely related to
diffusion sample quality. Empirically, iFID is the first metric shown to strongly correlate with
diffusion gFID across diverse VAEs, achieving Pearson and Spearman correlations of approximately $0.85$. The project page is available at https://tongdaxu.github.io/pages/ifid.html.
Read the full abstract ▾
All authors Tongda Xu, Mingwei He, Shady Abu-Hussein, Jose Miguel Hernandez-Lobato, Chunhang Zheng, Kai Zhao, Chao Zhou, Ya-Qin Zhang, Yan Wang
Computer Vision
Rethinking Concept Bottleneck Models: From Pitfalls to Solutions
Merve Tapli, Quentin Bouniot, Wolfgang Stammer et al. (+2 more)
Concept Bottleneck Models (CBMs) ground predictions in human-understandable concepts but face fundamental limitations: the absence of a metric to pre-evaluate concept relevance, the "linearity problem" causing recent CBMs to bypass the concept bottleneck entirely, an accuracy gap compared to opaque models, and finally the lack of systematic study on the impact of different visual backbones and VLMs. We introduce CBM-Suite, a methodological framework to systematically addresses these challenges. First, we propose an
entropy-based metric to quantify the intrinsic suitability of a concept set for a given
dataset. Second, we resolve the linearity problem by inserting a non-linear layer between concept activations and the classifier, which ensures that model accuracy faithfully reflects concept relevance. Third, we narrow the accuracy gap by leveraging a
distillation loss guided by a linear teacher probe. Finally, we provide comprehensive analyses on how different
vision encoders,
vision-
language models, and concept sets interact to influence accuracy and interpretability in CBMs. Extensive evaluations show that CBM-Suite yields more accurate models and provides insights for improving concept-based interpretability.
Read the full abstract ▾
All authors Merve Tapli, Quentin Bouniot, Wolfgang Stammer, Zeynep Akata, Emre Akbas
Computer Vision
Post Fusion Bird's Eye View Feature Stabilization for Robust Multimodal 3D Detection
Trung Tien Dong, Dev Thakkar, Arman Sargolzaei et al. (+1 more)
Camera-
LiDAR fusion is widely used in autonomous driving to enable accurate 3D object detection. However, bird's-eye view (BEV) fusion detectors can degrade significantly under domain shift and sensor failures, limiting reliability in real-world deployment. Existing robustness approaches often require modifying the fusion architecture or retraining specialized models, making them difficult to integrate into already deployed systems. We propose a Post Fusion Stabilizer (PFS), a lightweight module that operates on intermediate BEV representations of existing detectors and produces a refined feature map for the original detection head. The design stabilizes feature statistics under domain shift, suppresses spatial regions affected by sensor degradation, and adaptively restores weakened cues through residual correction. Designed as a near-identity transformation, PFS preserves performance while improving robustness under diverse camera and
LiDAR corruptions. Evaluations on the nuScenes
benchmark demonstrate that PFS achieves state-of-the-art results in several failure modes, notably improving camera dropout robustness by +1.2% and low-light performance by +4.4% mAP while maintaining a lightweight footprint of only 3.3 M parameters.
Read the full abstract ▾
All authors Trung Tien Dong, Dev Thakkar, Arman Sargolzaei, Xiaomin Lin
Computer Vision
Adversarial Batch Representation Augmentation for Batch Correction in High-Content Cellular Screening
Lei Tong, Xujing Yao, Adam Corrigan et al. (+6 more)
High-Content Screening routinely generates massive volumes of
cell painting images for phenotypic profiling. However, technical variations across experimental executions inevitably induce biological batch (bio-batch) effects. These cause
covariate shifts and degrade the generalization of deep learning models on unseen data. Existing batch correction methods typically rely on additional prior knowledge (e.g., treatment or
cell culture information) or struggle to generalize to unseen bio-batches. In this work, we frame bio-batch mitigation as a Domain Generalization (DG) problem and propose
Adversarial Batch Representation Augmentation (ABRA). ABRA explicitly models batch-wise statistical fluctuations by parameterizing feature statistics as structured uncertainties. Through a min-max optimization framework, it actively synthesizes worst-case bio-batch perturbations in the representation space, guided by a strict angular
geometric margin to preserve fine-grained class discriminability. To prevent representation collapse during this
adversarial exploration, we introduce a synergistic distribution
alignment objective. Extensive evaluations on the large-scale RxRx1 and RxRx1-WILDS
benchmarks demonstrate that ABRA establishes a new state-of-the-art for siRNA perturbation classification.
Read the full abstract ▾
All authors Lei Tong, Xujing Yao, Adam Corrigan, Long Chen, Navin Rathna Kumar, Kerry Hallbrook, Jonathan Orme, Yinhai Wang, Huiyu Zhou
Computer Vision
DreamCAD: Scaling Multi-modal CAD Generation using Differentiable Parametric Surfaces
Mohammad Sadil Khan, Muhammad Usama, Rolandos Alexandros Potamias et al. (+4 more)
Computer-Aided Design (CAD) relies on structured and editable
geometric representations, yet existing generative methods are constrained by small annotated
datasets with explicit design histories or boundary representation (BRep) labels. Meanwhile, millions of unannotated 3D meshes remain untapped, limiting progress in scalable CAD generation. To address this, we propose DreamCAD, a multi-modal generative framework that directly produces editable BReps from point-level supervision, without CAD-specific annotations. DreamCAD represents each BRep as a set of parametric patches (e.g., Bézier surfaces) and uses a differentiable tessellation method to generate meshes. This enables large-scale training on 3D
datasets while reconstructing connected and editable surfaces. Furthermore, we introduce CADCap-1M, the largest CAD captioning
dataset to date, with 1M+ descriptions generated using GPT-5 for advancing text-to-CAD research. DreamCAD achieves state-of-the-art performance on ABC and Objaverse
benchmarks across text, image, and point modalities, improving
geometric fidelity and surpassing 75% user preference. Code and
dataset will be publicly available.
Read the full abstract ▾
All authors Mohammad Sadil Khan, Muhammad Usama, Rolandos Alexandros Potamias, Didier Stricker, Muhammad Zeshan Afzal, Jiankang Deng, Ismail Elezi
Computer Vision
From Decoupled to Coupled: Robustness Verification for Learning-based Keypoint Detection with Joint Specifications
Xusheng Luo and Changliu Liu
Keypoint detection underpins many
vision tasks, including pose estimation, viewpoint recovery, and 3D reconstruction, yet modern neural models remain vulnerable to small input perturbations. Despite its importance, formal robustness verification for keypoint detectors is largely unexplored due to high-dimensional inputs and continuous coordinate outputs. We propose the first coupled robustness verification framework for heatmap-based keypoint detectors that bounds the joint deviation across all keypoints, capturing their interdependencies and downstream task requirements. Unlike prior decoupled, classification-style approaches that verify each keypoint independently and yield conservative guarantees, our method verifies collective behavior. We formulate verification as a falsification problem using a mixed-integer linear program (MILP) that combines reachable heatmap sets with a polytope encoding joint deviation constraints. Infeasibility certifies robustness, while feasibility provides counterexamples, and we prove the method is sound: if it certifies the model as robust, then the keypoint detection model is guaranteed to be robust. Experiments show that our coupled approach achieves high verified rates and remains effective under strict error thresholds where decoupled methods fail.
Read the full abstract ▾
Computer Vision
Thinking with Spatial Code for Physical-World Video Reasoning
Jieneng Chen, Wenxin Ma, Ruisheng Yuan et al. (+3 more)
We introduce Thinking with Spatial Code, a framework that transforms RGB
video into explicit, temporally coherent 3D representations for physical-world visual question answering. We highlight the empirical finding that our proposed spatial encoder can parse
videos into structured spatial code with explicit 3D oriented bounding boxes and semantic labels, enabling large
language models (LLMs) to reason directly over explicit spatial variables. Specifically, we propose the spatial encoder that encodes image and
geometric features by unifying 6D object parsing and tracking backbones with
geometric prediction, and we further finetuning LLMs with reinforcement learning using a spatial rubric reward that encourages perspective-aware, geometrically grounded inference. As a result, our model outperforms proprietary
vision-
language models on VSI-Bench, setting a new state-of-the-art. Code is available at https://github.com/Beckschen/spatialcode.
Read the full abstract ▾
All authors Jieneng Chen, Wenxin Ma, Ruisheng Yuan, Yunzhi Zhang, Jiajun Wu, Alan Yuille
Computer Vision · Cross-listed from cs.LG
Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models
Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva et al. (+2 more)
The issue of algorithmic biases in deep learning has led to the development of various debiasing techniques, many of which perform complex training procedures or
dataset manipulation. However, an intriguing question arises: is it possible to extract fair and bias-agnostic subnetworks from standard vanilla-trained models without relying on additional data, such as unbiased training set? In this work, we introduce Bias-Invariant Subnetwork Extraction (BISE), a learning strategy that identifies and isolates "bias-free" subnetworks that already exist within conventionally trained models, without retraining or finetuning the original parameters. Our approach demonstrates that such subnetworks can be extracted via
pruning and can operate without modification, effectively relying less on biased features and maintaining robust performance. Our findings contribute towards efficient bias mitigation through structural adaptation of pre-trained neural networks via parameter removal, as opposed to costly strategies that are either data-centric or involve (re)training all model parameters. Extensive experiments on common
benchmarks show the advantages of our approach in terms of the performance and computational efficiency of the resulting debiased model.
Read the full abstract ▾
All authors Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva, Vito Paolo Pastore, Enzo Tartaglione
Computer Vision · Cross-listed from cs.IR
AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interaction
Jiashu Yang, Chi Zhang, Abudukelimu Wuerkaixi et al. (+5 more)
Multimodal document question answering requires retrieving dispersed evidence from visually rich long documents and performing reliable
reasoning over heterogeneous information. Existing
multimodal RAG systems remain limited by two bottlenecks: static
retrieval that ignores query complexity, and end-to-end
Vision-
Language Models (VLMs) that couple visual perception with logical
reasoning, leading to inefficient computation and unstable answer generation. We propose AutoThinkRAG, a complexity-aware inference architecture for
multimodal document QA. It has two components: (1) a Query Complexity Router that analyzes query difficulty and structure to adaptively select
retrieval and
reasoning paths; and (2) a Perception–
Reasoning Decoupling architecture that uses a lightweight VLM as a high-fidelity visual interpreter to convert query-relevant visual cues into textual representations, which are then passed to an LLM for logical
reasoning and answer synthesis. This design improves both efficiency and robustness, especially on long-document and unanswerable queries. Experiments on DocBench and MMLongBench show that AutoThinkRAG achieves 82.13% and 51.29% overall accuracy, respectively, while reducing per-query
token consumption by 18.9% and monetary cost by 18.2%. Further analyses show that the gains are most pronounced on complex queries requiring adaptive
retrieval and multi-step
reasoning.
Read the full abstract ▾
All authors Jiashu Yang, Chi Zhang, Abudukelimu Wuerkaixi, Xuxin Cheng, Cao Liu, Ke Zeng, Xu Jia, Xunliang Cai
Computer Vision
Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setups
Leif Van Holland, Domenic Zingsheim, Mana Takhsha et al. (+4 more)
High-quality 3D streaming from multiple cameras is crucial for immersive experiences in many AR/VR applications. The limited number of views - often due to real-time constraints - leads to missing information and incomplete surfaces in the rendered images. Existing approaches typically rely on simple heuristics for the hole filling, which can result in inconsistencies or visual artifacts. We propose to complete the missing textures using a novel, application-targeted inpainting method independent of the underlying representation as an image-based post-processing step after the novel view rendering. The method is designed as a standalone module compatible with any calibrated multi-camera system. For this we introduce a multi-view aware,
transformer-based network architecture using spatio-temporal
embeddings to ensure consistency across frames while preserving fine details. Additionally, our resolution-independent design allows adaptation to different camera setups, while an adaptive patch selection strategy balances inference speed and quality, allowing real-time performance. We evaluate our approach against state-of-the-art inpainting techniques under the same real-time constraints and demonstrate that our model achieves the best trade-off between quality and speed, outperforming competitors in both image and
video-based metrics.
Read the full abstract ▾
All authors Leif Van Holland, Domenic Zingsheim, Mana Takhsha, Hannah Dröge, Patrick Stotko, Markus Plack, Reinhard Klein
Computer Vision
FaceCam: Portrait Video Camera Control via Scale-Aware Conditioning
Weijie Lyu, Ming-Hsuan Yang and Zhixin Shu
We introduce FaceCam, a system that generates
video under customizable camera trajectories for
monocular human portrait
video input. Recent camera control approaches based on large
video-generation models have shown promising progress but often exhibit
geometric distortions and visual artifacts on portrait
videos due to scale-ambiguous camera representations or 3D reconstruction errors. To overcome these limitations, we propose a face-tailored scale-aware representation for camera transformations that provides deterministic conditioning without relying on 3D priors. We train a
video generation model on both multi-view studio captures and in-the-wild
monocular videos, and introduce two camera-control data generation strategies: synthetic camera motion and multi-shot stitching, to exploit stationary training cameras while generalizing to dynamic, continuous camera trajectories at inference time. Experiments on Ava-256
dataset and diverse in-the-wild
videos demonstrate that FaceCam achieves superior performance in camera controllability, visual quality, identity and motion preservation.
Read the full abstract ▾
Computer Vision
Accelerating Text-to-Video Generation with Calibrated Sparse Attention
Shai Yehezkel, Shahar Yadin, Noam Elata et al. (+2 more)
Recent
diffusion models enable high-quality
video generation, but suffer from slow runtimes. The large
transformer-based backbones used in these models are bottlenecked by spatiotemporal
attention. In this paper, we identify that a significant fraction of
token-to-
token connections consistently yield negligible scores across various inputs, and their patterns often repeat across queries. Thus, the
attention computation in these cases can be skipped with little to no effect on the result. This observation continues to hold for connections among local
token blocks. Motivated by this, we introduce CalibAtt, a training-free method that accelerates
video generation via calibrated sparse
attention. CalibAtt performs an offline calibration pass that identifies block-level sparsity and repetition patterns that are stable across inputs, and compiles these patterns into optimized
attention operations for each layer, head, and
diffusion timestep. At inference time, we compute the selected input-dependent connections densely, and skip the unselected ones in a hardware-efficient manner. Extensive experiments on Wan 2.1 14B, Mochi 1, and few-step distilled models at various resolutions show that CalibAtt achieves up to 1.58x end-to-end speedup, outperforming existing training-free methods while maintaining
video generation quality and text-
video alignment.
Read the full abstract ▾
All authors Shai Yehezkel, Shahar Yadin, Noam Elata, Yaron Ostrovsky-Berman, Bahjat Kawar
Computer Vision
Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baseline
Guo Chen, Lidong Lu, Yicheng Liu et al. (+17 more)
While
datasets for
video understanding have scaled to hour-long durations, they typically consist of densely concatenated clips that differ from natural, unscripted daily life. To bridge this gap, we introduce MM-Lifelong, a
dataset designed for
Multimodal Lifelong Understanding. Comprising 181.1 hours of footage, it is structured across Day, Week, and Month scales to capture varying temporal densities. Extensive evaluations reveal two critical failure modes in current paradigms: end-to-end MLLMs suffer from a Working
Memory Bottleneck due to context saturation, while representative agentic baselines experience Global Localization Collapse when navigating sparse, month-long timelines. To address this, we propose the Recursive
Multimodal Agent (ReMA), which employs dynamic
memory management to iteratively update a recursive belief state, significantly outperforming existing methods. Finally, we establish
dataset splits designed to isolate temporal and domain biases, providing a rigorous foundation for future research in supervised learning and out-of-distribution generalization.
Read the full abstract ▾
All authors Guo Chen, Lidong Lu, Yicheng Liu, Liangrui Dong, Lidong Zou, Jixin Lv, Zhenquan Li, Xinyi Mao, Baoqi Pei, Shihao Wang, Zhiqi Li, Karan Sapra, Fuxiao Liu, Yin-Dong Zheng, Yifei Huang, Limin Wang, Zhiding Yu, Andrew Tao, Guilin Liu, Tong Lu
Computer Vision
Towards 3D Scene Understanding of Gas Plumes in LWIR Hyperspectral Images Using Neural Radiance Fields
Scout Jarman, Zigfried Hampel-Arias, Adra Carr et al. (+1 more)
Hyperspectral images (HSI) have many applications, ranging from environmental monitoring to national security, and can be used for material detection and identification. Longwave infrared (LWIR) HSI can be used for gas plume detection and analysis. Oftentimes, only a few images of a scene of interest are available and are analyzed individually. The ability to combine information from multiple images into a single, cohesive representation could enhance analysis by providing more context on the scene's geometry and spectral properties. Neural radiance fields (NeRFs) create a latent neural representation of volumetric scene properties that enable novel-view rendering and geometry reconstruction, offering a promising avenue for hyperspectral 3D scene reconstruction. We explore the possibility of using NeRFs to create 3D scene reconstructions from LWIR HSI and demonstrate that the model can be used for the basic downstream analysis task of gas plume detection. The physics-based DIRSIG software suite was used to generate a synthetic multi-view LWIR HSI
dataset of a simple facility with a strong sulfur hexafluoride gas plume. Our method, built on the standard Mip-NeRF architecture, combines state-of-the-art methods for hyperspectral NeRFs and sparse-view NeRFs, along with a novel adaptive weighted MSE loss. Our final NeRF method requires around 50% fewer training images than the standard Mip-NeRF and achieves an average PSNR of 39.8 dB with as few as 30 training images. Gas plume detection applied to NeRF-rendered test images using the adaptive coherence
estimator achieves an average AUC of 0.821 when compared with detection masks generated from ground-truth test images.
Read the full abstract ▾
All authors Scout Jarman, Zigfried Hampel-Arias, Adra Carr, Kevin R. Moon
Computer Vision
Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun et al. (+1 more)
Hallucinations remain a persistent challenge for
vision-
language models (VLMs), which often describe nonexistent objects or fabricate facts. Existing detection methods typically operate after text generation, making intervention both costly and untimely. We investigate whether
hallucination risk can instead be predicted before any
token is generated by probing a model's internal representations in a single forward pass. Across a diverse set of
vision-
language tasks and eight modern VLMs, including Llama-3.2-
Vision, Gemma-3, Phi-4-VL, and Qwen2.5-VL, we examine three families of internal representations: (i) visual-only features without
multimodal fusion, (ii)
vision-
token representations within the text decoder, and (iii) query-
token representations that integrate visual and textual information before generation. Probes trained on these representations achieve strong
hallucination-detection performance without decoding, reaching up to 0.93 AUROC on Gemma-3-12B, Phi-4-VL 5.6B, and Molmo 7B. Late query-
token states are the most predictive for most models, while visual or mid-layer features dominate in a few architectures (e.g., ~0.79 AUROC for Qwen2.5-VL-7B using visual-only features). These results demonstrate that (1)
hallucination risk is detectable pre-generation, (2) the most informative layer and modality vary across architectures, and (3) lightweight probes have the potential to enable early abstention, selective routing, and adaptive decoding to improve both safety and efficiency.
Read the full abstract ▾
All authors Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou
Computer Vision
EdgeDAM: Real-time Object Tracking for Mobile Devices
Syed Muhammad Raza, Syed Murtaza Hussain Abidi, Khawar Islam et al. (+2 more)
Single-object tracking (SOT) on edge devices is a critical computer
vision task, requiring accurate and continuous target localization across
video frames under occlusion, distractor interference, and fast motion. However, recent state-of-the-art distractor-aware
memory mechanisms are largely built on
segmentation-based trackers and rely on mask prediction and
attention-driven
memory updates, which introduce substantial computational overhead and limit real-time deployment on resource-constrained hardware; meanwhile, lightweight trackers sustain high throughput but are prone to drift when visually similar distractors appear. To address these challenges, we propose EdgeDAM, a lightweight detection-guided tracking framework that reformulates distractor-aware
memory for bounding-box tracking under strict edge constraints. EdgeDAM introduces two key strategies: (1) Dual-Buffer Distractor-Aware
Memory (DAM), which integrates a Recent-Aware
Memory to preserve temporally consistent target hypotheses and a Distractor-Resolving
Memory to explicitly store hard negative candidates and penalize their re-selection during recovery; and (2) Confidence-Driven Switching with Held-Box Stabilization, where tracker reliability and temporal consistency criteria adaptively activate detection and
memory-guided re-identification during occlusion, while a held-box mechanism temporarily freezes and expands the estimate to suppress distractor contamination. Extensive experiments on five
benchmarks, including the distractor-focused DiDi
dataset, demonstrate improved robustness under occlusion and fast motion while maintaining real-time performance on mobile devices, achieving 88.2% accuracy on DiDi and 25 FPS on an iPhone 15. Code will be released.
Read the full abstract ▾
All authors Syed Muhammad Raza, Syed Murtaza Hussain Abidi, Khawar Islam, Muhammad Ibrahim, Ajmal Saeed Mian
Computer Vision
Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixes
Pengxiang Li, Joey Tsai, Hongwei Xue et al. (+2 more)
Diffusion Language Models (DLMs) promise highly parallel text generation, yet their practical inference speed is often bottlenecked by suboptimal decoding schedulers. Standard approaches rely on 'scattered acceptance'-committing high confidence
tokens at disjoint positions throughout the sequence. This approach inadvertently fractures the Key-Value (KV) cache, destroys
memory locality, and forces the model into costly, repeated repairs across unstable
token boundaries. To resolve this, we present the Longest Stable Prefix (LSP) scheduler, a training-free and model-agnostic inference paradigm based on monolithic prefix absorption. In each denoising step, LSP evaluates
token stability via a single forward pass, dynamically identifies a contiguous left-aligned block of stable predictions, and snaps its boundary to natural linguistic or structural delimiters before an atomic commitment. This prefix-first
topology yields dual benefits: systemically, it converts fragmented KV cache updates into efficient, contiguous appends; algorithmically, it preserves bidirectional lookahead over a geometrically shrinking active suffix, drastically reducing
token flip rates and denoiser calls. Extensive evaluations on LLaDA-8B and Dream-7B demonstrate that LSP accelerates inference by up to 3.4x across rigorous
benchmarks including mathematical
reasoning, code generation,
multilingual (CJK) tasks, and creative writing while matching or slightly improving output quality. By fundamentally restructuring the commitment
topology, LSP bridges the gap between the theoretical parallelism of DLMs and practical hardware efficiency.
Read the full abstract ▾
All authors Pengxiang Li, Joey Tsai, Hongwei Xue, Kunyu Shi, Shilin Yan
Computer Vision
RealWonder: Real-Time Physical Action-Conditioned Video Generation
Wei Liu, Ziyu Chen, Zizhang Li et al. (+3 more)
Current
video generation models cannot simulate physical consequences of 3D actions like forces and robotic manipulations, as they lack structural understanding of how actions affect 3D scenes. We present RealWonder, the first real-time system for action-conditioned
video generation from a single image. Our key insight is using physics simulation as an intermediate bridge: instead of directly encoding continuous actions, we translate them through physics simulation into visual representations (optical flow and RGB) that
video models can process. RealWonder integrates three components: 3D reconstruction from single images, physics simulation, and a distilled
video generator requiring only 4
diffusion steps. Our system achieves 13.2 FPS at 480x832 resolution, enabling interactive exploration of forces,
robot actions, and camera controls on rigid objects, deformable bodies, fluids, and granular materials. We envision RealWonder opens new opportunities to apply
video models in immersive experiences, AR/VR, and
robot learning. Our code and model weights are publicly available in our project website: https://liuwei283.github.io/RealWonder/
Read the full abstract ▾
All authors Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, Jiajun Wu
Computer Vision
NaiLIA: Multimodal Nail Design Retrieval Based on Dense Intent Descriptions and Palette Queries
Kanon Amemiya, Daichi Yashima, Kei Katsumata et al. (+4 more)
We focus on the task of retrieving nail design images based on dense intent descriptions, which represent multi-layered user intent for nail designs. This is challenging because such descriptions specify unconstrained painted elements and pre-manufactured embellishments as well as visual characteristics, themes, and overall impressions. In addition to these descriptions, we assume that users provide palette queries by specifying zero or more colors via a color picker, enabling the expression of subtle and continuous color nuances. Existing
vision-
language foundation models often struggle to incorporate such descriptions and palettes. To address this, we propose NaiLIA, a
multimodal retrieval method for nail design images, which comprehensively aligns with dense intent descriptions and palette queries during
retrieval. Our approach introduces a relaxed loss based on confidence scores for unlabeled images that can align with the descriptions. To evaluate NaiLIA, we constructed a
benchmark consisting of 10,625 images collected from people with diverse cultural backgrounds. The images were annotated with long and dense intent descriptions given by over 200 annotators. Experimental results demonstrate that NaiLIA outperforms standard methods.
Read the full abstract ▾
All authors Kanon Amemiya, Daichi Yashima, Kei Katsumata, Takumi Komatsu, Ryosuke Korekata, Seitaro Otsuki, Komei Sugiura
Computer Vision
Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Model
Dongwon Kim, Gawon Seo, Jinsung Lee et al. (+2 more)
World models provide a powerful framework for simulating environment dynamics conditioned on actions or instructions, enabling downstream tasks such as action planning or policy learning. Recent approaches leverage world models as learned simulators, but its application to decision-time planning remains computationally prohibitive for real-time control. A key bottleneck lies in latent representations: conventional tokenizers encode each observation into hundreds of
tokens, making planning both slow and resource-intensive. To address this, we propose CompACT, a discrete tokenizer that compresses each observation into as few as 8
tokens, drastically reducing computational cost while preserving essential information for planning. An action-conditioned world model that occupies CompACT tokenizer achieves competitive planning performance with orders-of-magnitude faster planning, offering a practical step toward real-world deployment of world models.
Read the full abstract ▾
All authors Dongwon Kim, Gawon Seo, Jinsung Lee, Minsu Cho, Suha Kwak
Computer Vision
SAIL: Similarity-Aware Guidance and Inter-Caption Augmentation-based Learning for Weakly-Supervised Dense Video Captioning
Ye-Chan Kim, SeungJu Cha, Si-Woo Kim et al. (+3 more)
Weakly-Supervised Dense
Video Captioning aims to localize and describe events in
videos trained only on caption annotations, without temporal boundaries. Prior work introduced an implicit supervision paradigm based on Gaussian masking and complementary captioning. However, existing method focuses merely on generating non-overlapping masks without considering their semantic relationship to corresponding events, resulting in simplistic, uniformly distributed masks that fail to capture semantically meaningful regions. Moreover, relying solely on ground-truth captions leads to sub-optimal performance due to the inherent sparsity of existing
datasets. In this work, we propose SAIL, which constructs semantically-aware masks through cross-modal
alignment. Our similarity aware training objective guides masks to emphasize
video regions with high similarity to their corresponding event captions. Furthermore, to guide more accurate mask generation under sparse annotation settings, we introduce an LLM-based augmentation strategy that generates synthetic captions to provide additional
alignment signals. These synthetic captions are incorporated through an inter-mask mechanism, providing auxiliary guidance for precise temporal localization without degrading the main objective. Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance on both captioning and localization metrics.
Read the full abstract ▾
All authors Ye-Chan Kim, SeungJu Cha, Si-Woo Kim, Minju Jeon, Hyungee Kim, Dong-Jin Kim
Computer Vision
RelaxFlow: Text-Driven Amodal 3D Generation
Jiayin Zhu, Guoji Fu, Xiaolu Liu et al. (+3 more)
Image-to-3D generation faces inherent semantic ambiguity under occlusion, where partial observation alone is often insufficient to determine object category. In this work, we formalize text-driven amodal 3D generation, where text
prompts steer the completion of unseen regions while strictly preserving input observation. Crucially, we identify that these objectives demand distinct control granularities: rigid control for the observation versus relaxed structural control for the
prompt. To this end, we propose RelaxFlow, a training-free dual-branch framework that decouples control granularity via a Multi-Prior Consensus Module and a Relaxation Mechanism. Theoretically, we prove that our relaxation is equivalent to applying a low-pass filter on the generative vector field, which suppresses high-frequency instance details to isolate
geometric structure that accommodates the observation. To facilitate evaluation, we introduce two diagnostic
benchmarks, ExtremeOcc-3D and AmbiSem-3D. Extensive experiments demonstrate that RelaxFlow successfully steers the generation of unseen regions to match the
prompt intent without compromising visual fidelity.
Read the full abstract ▾
All authors Jiayin Zhu, Guoji Fu, Xiaolu Liu, Qiyuan He, Yicong Li, Angela Yao
Computer Vision
DARK: Diagonal-Anchored Repulsive Knowledge Distillation for Vision-Language Models under Extreme Compression
Numan Saeed, Asif Hanif, Fadillah Adamsyah Maani et al. (+2 more)
Compressing
vision-
language models for on-device deployment is increasingly important in clinical settings, but knowledge
distillation (KD) degrades sharply when the teacher-student capacity gap spans an order of magnitude or more. We argue that, under such gaps, strict imitation of the teacher is a poor objective: much of the teacher's pairwise similarity structure reflects its own architectural biases rather than information a compact student can efficiently represent. We propose Diagonal-Anchored Repulsive Knowledge
Distillation (DARK), a contrastive KD framework that decomposes the
distillation loss into a diagonal term (matched image-text pairs) and an off-diagonal term (non-target similarities). The diagonal term anchors matched-pair
alignment throughout training; the off-diagonal term is annealed from positive to negative weighting, transitioning the student from imitating to repelling the teacher's non-target similarity structure. We instantiate DARK by distilling FetalCLIP, a 427M-parameter fetal ultrasound
vision-
language model, into MobileFetalCLIP, a 75M-parameter student model with a $26\times$ smaller visual encoder, running in 1.6\,ms on an iPhone~16~Pro. The student matches or exceeds its teacher on three zero-shot
benchmarks, including HC18 biometry validity (88.6% vs.\ 83.5%) and
brain sub-plane F1 (0.784 vs.\ 0.702).
Embedding-geometry and logit analyses show that DARK induces structured decorrelation: the student preserves teacher-aligned per-image confidence while diverging from inherited inter-class confusion, suggesting that controlled repulsion can be more efficient than imitation under extreme compression.
Read the full abstract ▾
All authors Numan Saeed, Asif Hanif, Fadillah Adamsyah Maani, Hussain Alasmawi, Mohammad Yaqub
Computer Vision
Video-based Locomotion Analysis for Fish Health Monitoring
Timon Palm, Clemens Seibold, Anna Hilsmann et al. (+1 more)
Monitoring the health conditions of fish is essential, as it enables the early detection of disease, safeguards animal welfare, and contributes to sustainable aquaculture practices. Physiological and pathological conditions of cultivated fish can be inferred by analyzing locomotion activities. In this paper, we present a system that estimates the locomotion activities from
videos using multi object tracking. The core of our approach is a YOLOv11 detector embedded in a tracking-by-detection framework. We investigate various configurations of the YOLOv11-architecture as well as extensions that incorporate multiple frames to improve detection accuracy. Our system is evaluated on a manually annotated
dataset of Sulawesi ricefish recorded in a home-aquarium-like setup, demonstrating its ability to reliably measure swimming direction and speed for fish health monitoring. The
dataset will be made publicly available upon publication.
Read the full abstract ▾
All authors Timon Palm, Clemens Seibold, Anna Hilsmann, Peter Eisert
Computer Vision · Cross-listed from cs.RO
Loop Closure via Maximal Cliques in 3D LiDAR-Based SLAM
Javier Laserna, Saurabh Gupta, Oscar Martinez Mozos et al. (+2 more)
Reliable loop closure detection remains a critical challenge in 3D
LiDAR-based SLAM, especially under sensor noise, environmental ambiguity, and viewpoint variation conditions. RANSAC is often used in the context of loop closures for
geometric model fitting in the presence of outliers. However, this approach may fail, leading to map inconsistency. We introduce a novel deterministic
algorithm, CliReg, for loop closure validation that replaces RANSAC verification with a maximal clique search over a compatibility graph of feature correspondences. This formulation avoids random sampling and increases robustness in the presence of noise and outliers. We integrated our approach into a real- time pipeline employing binary 3D descriptors and a Hamming distance
embedding binary search tree-based matching. We evaluated it on multiple real-world
datasets featuring diverse
LiDAR sensors. The results demonstrate that our proposed technique consistently achieves a lower pose error and more reliable loop closures than RANSAC, especially in sparse or ambiguous conditions. Additional experiments on 2D projection-based maps confirm its generality across spatial domains, making our approach a robust and efficient alternative for loop closure detection.
Read the full abstract ▾
All authors Javier Laserna, Saurabh Gupta, Oscar Martinez Mozos, Cyrill Stachniss, Pablo San Segundo
Computer Vision
Fusion-CAM: Integrating Gradient and Region-Based Class Activation Maps for Robust Visual Explanations
Hajar Dekdegue, Moncef Garouani, Josiane Mothe et al. (+1 more)
Interpreting the decision-making process of deep convolutional neural networks remains a central challenge in achieving trustworthy and transparent artificial intelligence. Explainable AI (XAI) techniques, particularly Class Activation Map (CAM) methods, are widely adopted to visualize the input regions influencing model predictions. Gradient-based approaches (e.g. Grad-CAM) provide highly discriminative, fine-grained details by computing gradients of class activations but often yield noisy and incomplete maps that emphasize only the most salient regions rather than the complete objects. Region-based approaches (e.g. Score-CAM) aggregate information over larger areas, capturing broader object coverage at the cost of over-smoothing and reduced sensitivity to subtle features. We introduce Fusion-CAM, a novel framework that bridges this explanatory gap by unifying both paradigms through a dedicated fusion mechanism to produce robust and highly discriminative visual explanations. Our method first denoises gradient-based maps, yielding cleaner and more focused activations. It then combines the refined gradient map with region-based maps using contribution weights to enhance class coverage. Finally, we propose an adaptive similarity-based pixel-level fusion that evaluates the agreement between both paradigms and dynamically adjusts the fusion strength. This adaptive mechanism reinforces consistent activations while softly blending conflicting regions, resulting in richer, context-aware, and input-adaptive visual explanations. Extensive experiments on standard
benchmarks show that Fusion-CAM consistently outperforms existing CAM variants in both qualitative visualization and quantitative evaluation, providing a robust and flexible tool for interpreting deep neural networks.
Read the full abstract ▾
All authors Hajar Dekdegue, Moncef Garouani, Josiane Mothe, Jordan Bernigaud
Computer Vision
ORMOT: A Dataset and Framework for Omnidirectional Referring Multi-Object Tracking
Zihan Zhou, Sijia Chen, Yanqiu Yu et al. (+2 more)
Omnidirectional cameras provide 360{\deg} spatial coverage, making them increasingly valuable in applications such as autonomous driving and
video surveillance. While Multi-Object Tracking (MOT) and its
language-guided extension, Referring Multi-Object Tracking (RMOT), have achieved notable progress, existing methods rely on conventional cameras with limited fields of view, causing critical contextual cues to be lost when targets move outside the frame. This fundamentally limits the model's ability to interpret long-horizon
language descriptions involving sequential actions, spatial relations, and group behaviors. In this work, we propose Omnidirectional Referring Multi-Object Tracking (ORMOT), a novel task extending RMOT to omnidirectional imagery, where 360{\deg} coverage ensures complete scene context for accurate
language-guided tracking. To advance this task, we construct ORSet, a
dataset comprising 27 omnidirectional scenes, 848
language descriptions, and 3,401 annotated objects. Furthermore, we propose ORTrack, an LVLM-driven framework that enables zero-shot
language-guided detection and robust cross-frame association in complex 360{\deg} environments. Experiments on ORSet demonstrate that ORTrack achieves state-of-the-art performance, providing a strong baseline for future research. The
dataset and code will be open-sourced at https://github.com/chen-si-jia/ORMOT.
Read the full abstract ▾
All authors Zihan Zhou, Sijia Chen, Yanqiu Yu, En Yu, Wenbing Tao
Computer Vision · Cross-listed from cs.RO
OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys et al. (+1 more)
Open-world
navigation requires
robots to make decisions in complex everyday environments while adapting to flexible task requirements. Conventional
navigation approaches often rely on dense 3D reconstruction and hand-crafted goal metrics, which limits their generalization across tasks and environments. Recent advances in
vision-
language navigation (VLN) and
vision-
language-action (VLA) models enable end-to-end policies conditioned on natural
language, but typically require interactive training, large-scale data collection, or task-specific fine-tuning with a mobile
agent. We formulate
navigation as a sparse subgoal identification and reaching problem and observe that providing visual anchoring targets for high-level semantic priors enables highly efficient goal-conditioned
navigation. Based on this insight, we select visual frontiers as semantic anchors and propose OpenFrontier, a
navigation framework that requires no task-specific training or fine-tuning and seamlessly integrates diverse
vision-
language prior models. OpenFrontier enables efficient
navigation with a lightweight system design, without dense 3D semantic mapping, task-specific policy training, or model fine-tuning. We evaluate OpenFrontier across multiple
navigation benchmarks and demonstrate strong zero-shot performance, as well as effective real-world deployment on a mobile
robot.
Read the full abstract ▾
All authors Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys, Hermann Blum