Computer Vision
Event-based Selective Attention for Multi-resolution Fast Region of Interest (ROI) Detection
Luca Peres, Giulia D'Angelo, Chiara Bartolozzi et al. (+1 more)
Neuromorphic
vision systems operate under strict constraints on bandwidth,
memory, and energy, particularly at the edge, motivating early mechanisms for data reduction and selective processing. In this work, we investigate a multi-scale training-free, saliency-based, bottom-up visual
attention model that operates directly on low-resolution event-based input and selects Regions of Interest (ROI) from the visual scene. The model is evaluated across multiple downscaling factors applied to the incoming event stream, with input resolutions reduced by up to 256x relative to full resolution. Performance is assessed on the Prophesee Automotive
dataset, the largest publicly available event-based
dataset, demonstrating robust ROI selection across different scales on a real-world use-case. The proposed approach is capable of detecting ROIs belonging to multiple object classes, including various vehicle types, pedestrians, traffic lights, and traffic signs, with accuracy up to 70.8%, while operating at millisecond temporal resolution, 16x finer than the temporal resolution provided by the
dataset ground truth. These results highlight the potential of combining early event downscaling with saliency-based
attention as an effective front-end for efficient edge neuromorphic
vision systems.
Read the full abstract ▾
All authors Luca Peres, Giulia D'Angelo, Chiara Bartolozzi, Oliver Rhodes
Computer Vision
Predicting Human Disagreement for Calibrated Dynamic Facial Expression Recognition
Yiming Wang, Frederick W. B. Li and Jingyun Wang
Dynamic facial expression recognition (DFER)
benchmarks such as DFEW provide multiple annotator votes per clip, yet most models collapse them to a majority label and cannot represent human disagreement at inference time. We propose a disagreement-aware DFER framework that trains directly on the raw annotator count vector using a Dirichlet-Multinomial likelihood. Unlike mean-only soft-label objectives, the proposed likelihood provides scale-sensitive supervision for the Dirichlet concentration while preserving the predictive mean. A separate ambiguity head predicts annotation
entropy for unseen clips, and a monotone Chow-style reject rule combines predicted ambiguity, vacuity, temporal instability, and input quality for selective prediction. On DFEW, the method preserves recognition accuracy while reducing ECE by 30% and AURC by 15%, and predicted ambiguity reaches a Spearman correlation of 0.52 with the annotation
entropy of test clips. The calibration and selective-prediction gains transfer to FERV39k and remain under identity- and movie-disjoint DFEW splits.
Read the full abstract ▾
Computer Vision
Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models
Rwiddhi Chakraborty, Yinong Oliver Wang, Cheng Zhang et al. (+6 more)
Large
video models have exhibited impressive performance on a wide range of visual question answering tasks, owing to the rise of powerful, pretrained text and
vision encoders. The usefulness of such models have also been demonstrated on a wide range of
benchmarks, with an important caveat - the dominant approach in these
benchmarks evaluates multiple choice
reasoning via text options. This is a natural way to test text-based
reasoning in these models, and has led to significant insights regarding model behavior in the community. In this work, we ask a different question - what happens when the evaluation modality is visual, rather than text? We introduce three new
vision-centric evaluation
benchmarks in temporal frame
retrieval,
video future prediction, and causal
memory distortion, all designed around evaluating visual understanding capabilities in large
video models. Our approach complements the existing approaches to evaluate
video understanding in frontier models. We show that current frontier models exhibit significant weakness when attempting to reason through visual queries, rather than text. We conclude with an extended analysis section that provides pointers for future improvements in visual understanding for large
video models.
Read the full abstract ▾
All authors Rwiddhi Chakraborty, Yinong Oliver Wang, Cheng Zhang, Fan Bai, Zhuoran Yu, Michael Kampffmeyer, Yong Jae Lee, Fernando De la Torre, Robert Jenssen
Computer Vision
GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos
Yifan Xie, Hekun Tian, Jinkun Liu et al. (+3 more)
Human
videos provide rich manipulation experience, but extracting action representations that preserve useful motion remains challenging. Visual reconstruction alone can entangle manipulation-related motion with appearance changes and camera movement. We present GeoLAM, a framework for learning geometry-grounded latent actions from action-free human
videos. GeoLAM combines future-frame reconstruction through a frozen
geometric feature hierarchy with motion supervision from a training-only 4D geometry teacher. The
geometric representation provides a structural prior, while the teacher's predictions yield spatially pooled targets capturing 3D displacement, residual image-plane motion, and surface-orientation changes. Visibility and confidence weighting reduces the contribution of unreliable estimates, encouraging continuous latent actions to retain
geometric motion without explicit hand-pose or hand-trajectory annotations. After
video pretraining without action labels, the learned representation provides transition targets for a world-action model trained on action-labeled
robot demonstrations. The model jointly denoises latent actions and executable action chunks, with future-
video prediction used only as an auxiliary training task. Deployment therefore requires neither the geometry teacher nor future-
video generation. Evaluations on a latent-action
benchmark and robotic manipulation tasks demonstrate the strong performance of GeoLAM.
Read the full abstract ▾
All authors Yifan Xie, Hekun Tian, Jinkun Liu, YuAn Wang, Qiao Sun, Wenbo Ding
Computer Vision
Hub-Spectral Activation of Latent Multimodal Knowledge
Ying Guo, Haidong Chen, Linrui Xu et al. (+7 more)
Multimodal representation learning seeks shared representations for cross-modal
retrieval and knowledge transfer. Hub-based binding reduces pairwise supervision costs, but separate hub connections cannot guarantee reliable
alignment between modalities without direct joint training. We introduce Hub-Spectral Activation (HSA), a closed-form method for recovering and activating the hub-readable component of latent
multimodal knowledge in frozen representations. We formalize this knowledge as source-induced cross-modal dependence and characterize the component determined by the second-order statistics of two trained hub edges. Under a second-order source model, we establish conditions for exact recovery of the complete source-induced relation and bound the dimension of its hub-readable component by the hub covariance rank. HSA composes and standardizes hub-edge statistics, extracts paired spectral directions, and combines reliability-weighted matching evidence with source-gated candidate resolution for bidirectional
retrieval and prototype classification. HSA requires no target-pair supervision, gradient optimization, or backbone updates. Across 19
retrieval and 11 prototype-classification relations on ImageBind and LanguageBind, HSA raises mean bidirectional Recall@10 from 18.27% to 31.15% and mean macro Top-1 accuracy from 29.01% to 52.43%, respectively. Controlled analyses further identify valid hub-edge correspondence and leading spectral directions as key sources of
retrieval gains, demonstrating the utility of latent
multimodal knowledge beyond native similarity scores. Code and models are publicly available at https://github.com/Luo1Yan/HSA.
Read the full abstract ▾
All authors Ying Guo, Haidong Chen, Linrui Xu, Xiaohao Liu, Chuancheng Shi, Canran Xiao, Dan Zhang, Fei Shen, Li Shen, Tat-Seng Chua
Computer Vision
Beyond In-Distribution Metrics: A Systematic Out-of-Distribution Evaluation of Congenital Heart Disease Segmentation
Aniketh Vijesh, Shrisharanyan Vasu, Abhijit Ramesh et al. (+5 more)
Congenital heart disease (CHD) diagnosis and
surgical planning often require patient-specific 3D anatomical models, but manual
segmentation is labor-intensive, particularly in complex anatomies. Although deep-learning methods can automate this process, they are typically evaluated in-distribution, despite clinically relevant shifts in scanner, protocol, institution, population, and imaging modality. We present, to our knowledge, the first systematic evaluation of out-of-distribution (OOD) generalization in CHD
segmentation, using ImageCHD as a held-out target cohort. We compare representative
segmentation architectures under combined CT and CMR training, CT-only training, self-supervised pretraining, and limited target-domain adaptation. In-distribution performance proves to be a poor indicator of cross-cohort robustness: nnU-Net achieves the highest validation Dice (0.77) but falls to 0.51 on ImageCHD, while SwinUNETR generalizes substantially better, reaching 0.67 Dice. MAE and JEPA pretraining provide only modest additional benefit, suggesting that architecture contributes more to robustness than the tested pretraining strategies in this setting. When limited target-domain supervision is introduced, all SwinUNETR variants exceed 0.76 Dice with only 11 labeled ImageCHD cases. These findings demonstrate that conventional in-distribution evaluation can obscure clinically important generalization failures and support explicit cross-
dataset testing as a key component of CHD
segmentation evaluation.
Read the full abstract ▾
All authors Aniketh Vijesh, Shrisharanyan Vasu, Abhijit Ramesh, Clare Pomeroy-Ward, Harikrishnan Anil Maya, Sarin Xavier, Mahesh Kappanayil, Gilad Gressel
Computer Vision · Cross-listed from cs.AI
Neuro-Symbolic Hierarchical Intention Anticipation in Human Behavior
Farnaz Soleimani, Abdelghani Chibani, Yacine Amirat et al. (+1 more)
Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inference from a partially observed
multimodal episode together with structured prediction of the remaining behavior, rather than exact motor
forecasting. A compact Hierarchical Planning Decoder (HPD) is attached to a frozen neuro-symbolic recognition encoder and predicts, at four ontological levels, the next actions, the remaining activities and low-level intentions, and the episode high-level intention(HLI). The decoder is trained with soft neuro-symbolic regularization combining transition-coherence and hierarchical continuity losses, and is decoded with hard reachability masks that enforce ontological validity at inference. On a compositional four-level
benchmark of 15,002
multimodal episodes built over NTU RGB+D 120 features, three headline properties are observed together. The advantage over the strongest sequential baseline grows with the anticipation horizon, from +1.7 points at step 1 to +7.3 points at step 3 (top-5). Under compositional generalization, where one parent association per multi-parent low level intention is held out, this advantage widens to +4.9 points at step 1. At the episode level, 96.8% of anticipated trajectories satisfy the joint logic constraints, above the 88.1% strongest-baseline value and the 73.9% ground-truth floor; soft logic terms alone account for a 59.8 to 71.1% relative reduction of HLI-reachability violations, and the hard masks then eliminate them entirely. Neural generation supplies predictive ranking, symbolic constraints supply onto logical validity, and their combination yields coherent hierarchical anticipation while exposing remaining challenges in compositional goal generalization and unordered set prediction.
Read the full abstract ▾
All authors Farnaz Soleimani, Abdelghani Chibani, Yacine Amirat, Ghazaleh Khodabandelou
Computer Vision
Bi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinement
Yuetong Wang, Jinsheng Quan, Yi Yang et al. (+1 more)
Sparse-view 3D scene reconstruction with 3D Gaussian Splatting (3DGS) is inherently underconstrained. Plausible renderings can also coexist with erroneous Gaussian geometry, as errors in positions or depths may be concealed by opacity, scale, and appearance; we term this failure mode Geometry Cheating. Existing regularization methods constrain geometry but remain limited to observed views, while
video-
diffusion-based methods complete unseen views yet mainly use them as RGB pseudo-supervision, underusing motion and temporal priors and lacking explicit geometry supervision. We present Bi-FlowGS, which uses optical flow to bridge generative view completion and Gaussian geometry regularization. Our plug-and-play
Video-to-Geometry Flow
Distillation (V2G) distills temporal correspondence priors from restored
videos into Gaussian geometry to alleviate Geometry Cheating. Conversely, Geometry-to-
Video Flow-Guided Restoration (G2V) uses the current 3DGS geometry to guide temporally consistent
video restoration, providing more reliable generative supervision. Together, V2G and G2V form an implicit bidirectional co-refinement process, enabling restored
videos and the optimized 3DGS scene to iteratively improve each other. Experiments demonstrate improved rendering quality and
geometric consistency across wide-baseline and unbounded 360{\deg}
benchmarks.
Read the full abstract ▾
All authors Yuetong Wang, Jinsheng Quan, Yi Yang, Yawei Luo
Computer Vision · Cross-listed from cs.LG
CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework
Yunxiang Fu, Meng Lou, Zicheng Liao et al. (+1 more)
Continual learning must balance the learning of new knowledge with the retention of previously learned knowledge to incrementally learn tasks from a data stream without catastrophic forgetting. While leveraging pretrained models has significantly advanced continual learning, existing methods exhibit a scalability bottleneck when trained sequentially on many tasks, suffering from performance degradation due to inter-task interference and loss of plasticity. Inspired by evidence that sparse fine-tuning achieves performance comparable to full fine-tuning, this paper presents a novel sparsity-driven continual learning framework. Our continual learning method, termed CLARE, operates in two stages: it first identifies a sparse, task-critical parameter mask via a sparsity-inducing objective, then performs mask-constrained fine-tuning by only optimizing parameters selected by the mask. This two-stage sparse adapter mechanism enables all tasks to be accumulated within a shared adapter space while reducing destructive interference across tasks. Extensive experiments demonstrate the scalability of CLARE. On the long task-sequence
benchmark Omnibenchmark-1k, CLARE outperforms strong baselines in final accuracy by a large margin, e.g, improving EASE by 4.64% and 13.34% after learning 100 tasks, respectively.
Read the full abstract ▾
All authors Yunxiang Fu, Meng Lou, Zicheng Liao, Yizhou Yu
Computer Vision
sensVLA: Spatially-Grounded Vision-Language-Action Model for Autonomous Wheel Loader
Gopi Krishna Erabati, Bjarne Johannsen, Angus Stewart et al. (+1 more)
Autonomous wheel-loader control requires joint
reasoning over task semantics,
egocentric vision, proprioception, and 3D scene geometry. We present sensVLA, a
Vision-
Language-Action (VLA) architecture that combines a Qwen3-2B
Vision-
Language Model (VLM) with a fully trainable
transformer action expert trained by flow-matching velocity regression. sensVLA routes Bird's-Eye-View (BEV) features, extracted from fused front and rear
lidar, directly to the action expert through a dedicated cross-
attention pathway, while the VLM consumes front and rear RGB views to provide task-conditioned semantic context. This design decouples spatial grounding from linguistic
reasoning while preserving interaction between both streams at decision time. The expert predicts six action dimensions: longitudinal velocity, steering, body-frame displacement, arm rate, and bucket rate. On a real-world
dataset from a wheel loader, sensVLA reaches aggregate per-step parity with a strong camera-only baseline and reduces longitudinal velocity RMSE by 28% and displacement error by 9% on loading centric scenarios. It also degrades 29% less when the camera stream is corrupted or removed, evidencing that explicit spatial grounding improves accuracy and fault-tolerance for heavy equipment autonomy.
Read the full abstract ▾
All authors Gopi Krishna Erabati, Bjarne Johannsen, Angus Stewart, Vardeep Singh Sandhu
Computer Vision
Symmetry-Aware Likelihood-Orbit Aggregation for Selective Left-Right Claim Verification
Zhouzhi Xiong, Chuxi Zhang, Weizhen He et al. (+3 more)
Frozen
vision-
language models (VLMs) remain unreliable on fine-grained left-right claims, and raw claim likelihoods need not reliably rank verification errors. After a horizontal-reflection intervention is fixed, how should its induced likelihood measurements be combined into a selective verification signal? We introduce Relation-Orbit, a closed-form contrast with no learned fusion parameters that assigns eight normalized likelihoods to query-supporting and counterfactual roles determined by reflection, inverse relation, and entity exchange. A claim is asserted only when the signed contrast exceeds a threshold selected on held-out data using pointwise Clopper-Pearson upper confidence bounds. On VSR and GQA across four frozen VLMs, Relation-Orbit yields higher mean test coverage at a 10% selective-risk calibration target than an all-eight Orbit-Max baseline in all eight
dataset-backbone settings; gains over a nearly abstain-all one-sided intervention score are reported separately. A separate LLaVA-1.5/COCO evaluation, reduced-orbit controls, and a two-sided partition diagnostic further characterize the structural advantage.
Read the full abstract ▾
All authors Zhouzhi Xiong, Chuxi Zhang, Weizhen He, Yi Chen, Qi Li, Donglian Qi
Computer Vision
High-Fidelity Video Quality Assessment with VQA-Specific Saliency
Hakan Emre Gedik, Shashank Gupta and Alan Bovik
No-reference
video quality assessment (NR VQA) has recently seen promising progress with deep learning. However,
video data is inherently large, and processing them with deep models incurs high computational cost. This challenge is particularly acute in VQA, where preserving original-resolution cues and dense temporal information is critical for accuracy. Existing efficiency-driven preprocessing strategies, such as fragmenting, reduce computation but alter the input data distribution, limiting effective reuse of pretrained
video foundation models (ViFMs). To address these challenges, we propose High-Fidelity
Video Quality Assessment (HFVQA), a framework built on fixed-size spatio-temporal (ST) patches that is fully compatible with pretrained ViFMs. HFVQA samples ST patches across multiple scales, including the original resolution, with minimal temporal subsampling to preserve low-level quality cues and semantic context. To limit computation, HFVQA introduces a lightweight auxiliary network trained end-to-end with the ViFM encoder to learn VQA-specific saliency. Distilled directly from quality supervision, this saliency captures task-specific importance patterns, reflecting that
video quality perception is dominated by a small subset of spatio-temporal regions. By combining high-fidelity spatio-temporal cues with learned, task-specific saliency, HFVQA achieves SOTA performance on standard NR VQA
benchmarks while processing as little as 12% of candidate ST patches, making high-fidelity ViFM-based VQA computationally tractable.
Read the full abstract ▾
Computer Vision
MedPCFM-TED: One-Step Point Cloud Flow Matching for Implant Generation via Teacher-Guided Endpoint Distillation
Kamil Kwarciak and Marek Wodzinski
Cranial implant generation is an important task in medical imaging. Recent point cloud based generative methods, particularly flow matching, offer strong reconstruction quality and efficient sampling, but still require multiple neural function evaluations during inference. This limits rapid generation of multiple plausible implant candidates. We propose Teacher-guided Endpoint
Distillation (TED), a simple one-step
distillation framework for conditional cranial implant generation on point clouds. TED trains a one-step student using teacher-guided endpoint supervision and
geometric matching losses, while avoiding explicit path straightening. We evaluate TED on the SkullFix and SkullBreak
benchmarks. TED achieves the best overall performance on the SkullBreak
dataset, remains competitive on SkullFix, and provides the strongest Chamfer distance performance among the compared one-step methods. In addition, TED generates implants in approximately 0.04s per sample. These results show that one-step
distillation can substantially accelerate conditional point cloud implant generation without sacrificing reconstruction quality.
Read the full abstract ▾
Computer Vision
Evaluating Mesh Reconstruction Methods for Crop Phenotyping
Karanvir Singh, Theo Morales, Binh-Son Hua et al. (+1 more)
Phenotyping an agricultural crop is crucial for studying its entire life cycle, as it provides vital insights to improve yield and, ultimately, food production. Doing the same for crops grown on remote sites is a challenge for the specialists who cannot be available on-site. 3D reconstruction techniques offer a promising solution to this problem by enabling crop digitization, allowing specialists to access the resulting 3D crop models from anywhere at any time. In this work, we evaluate recent 3D reconstruction pipelines for crop phenotyping. We focus on 7 mesh reconstruction pipelines and measure the fidelity and consistency of their outputs qualitatively and quantitatively. Our results suggest that the meshes produced by the GGGS, PGSR, and 2DGS are preferable to the other pipelines, owing to their quantitative metrics and visually pleasing outputs. The GGGS pipeline is better than the second-best pipeline (2DGS) by about 27% on the radar chart with 5 dimensions, namely, User ratings, Chamfer distance, LPIPS, PSNR, and SSIM.
Read the full abstract ▾
All authors Karanvir Singh, Theo Morales, Binh-Son Hua, Mukesh Saini
Computer Vision
NeuroSymbEAD: A Large Scale Neuro-Symbolic Caption Dataset for Omni-Directional Embodied Autonomous Driving
Muhammad Ahmed Ullah Khan, Mohammed Elamine, Sheikh Talha Uddin et al. (+3 more)
This paper introduces NeuroSymbEAD, a large-scale neuro-symbolic caption
dataset featuring an ego-centric knowledge graph (KG) of static and dynamic objects annotated with classes, categories, heading directions, orientations, and distances from the ego-vehicle. These annotations are used on the KITTI-360
dataset to generate multilevel textual captions representing a lightweight version of an ego-centric scene map. Outdoor scene-map reconstruction, visual recognition, and object grounding establish baselines for driving common sense and traffic/scene understanding. For these purposes, natural
language-based grounded captioning of objects and their complex relationships is a widely adopted contextual representation for indoor scene tasks. Neuro-symbolic representations have proven effective in handling structured information for various computer
vision and
language applications. Our data annotation pipeline allows the generation of varied map segments, populating simulated or real objects within the bounding boxes predicted by any 3D object detection network, and building hierarchical text captions. We
benchmark our neuro-symbolic and ontological caption generation using pre-trained grounding and learned auto-regressive captioning networks. By converting 3D driving scenes into structured ego-centric
language, NeuroSymbEAD provides a
benchmark for
vision-
language and foundation models for traffic-scene explanation, 3D
reasoning, and interpretable autonomous-driving perception.
Read the full abstract ▾
All authors Muhammad Ahmed Ullah Khan, Mohammed Elamine, Sheikh Talha Uddin, Didier Stricker, Sk Aziz Ali, Muhammad Zeshan Afzal
Computer Vision
PiPS: Post-Hoc Prototypical Explanations for Interpretable Semantic Segmentation
Miłosz Adamczyk, Tymoteusz Zapala, Piotr Borycki et al. (+1 more)
With the increasing deployment of deep neural networks in critical systems, such as medical diagnostics and autonomous vehicles, ensuring their interpretability is crucial to building trust in decision-making systems. In the field of explainable artificial intelligence, prototype-based
reasoning has gained particular popularity, as it mimics human cognitive processes by explaining model decisions based on visual similarity under the looks like this paradigm. While this paradigm has been thoroughly investigated in the context of global image classification, the interpretability of dense predictions, particularly semantic
segmentation, remains largely unexplored despite its immense importance in tasks requiring precise object localization. Existing prototype-based interpretable
segmentation models rely on ante-hoc architectures, which entails significant limitations because they require costly training from scratch and modifications to the network structure, ultimately leading to a noticeable drop in predictive performance compared to standard black-box models. To address this issue, we propose PiPS (Post-hoc interpretable Prototypical
Segmentation), the first fully post-hoc solution for generating prototypical explanations for semantic
segmentation models. Our method enables the extraction of intuitive, spatially localized explanations from any pre-trained network without modification or fine-tuning, thereby preserving 100% of the model's original predictive performance. This approach opens a new avenue for the safe and cost-effective deployment of transparent systems in advanced computer
vision tasks. Codebase available at https://github.com/gmum/PIPS.
Read the full abstract ▾
All authors Miłosz Adamczyk, Tymoteusz Zapala, Piotr Borycki, Przemysław Spurek
Computer Vision
Temporally Consistent Graph Extraction and Matching for Longitudinal Angiographic Images
Linus Kreitner, Laurin Lux, Carmen Baumann et al. (+2 more)
Recent advances in angiographic imaging have enabled longitudinal visualization of the microvasculature. Image processing pipelines based on vessel graphs are able to resolve subtle temporal changes at the level of individual blood vessels. However, current strategies for graph extraction, refinement, and matching are highly sensitive, with even minuscule differences in the underlying
segmentation map resulting in substantially different vessel graphs. These artifacts severely inhibit the ability to accurately match sequential vessel graphs of the same subject over time. To address this problem, we propose a strategy that matches graphs before jointly refining them. Specifically, we perform an early matching after basic graph extraction before removing spurious bulges and merging junctions in both graphs using joint information. In experiments with complex retinal vessel graphs, we demonstrate that this strategy results in a higher matched area without graph fragmentation compared to separate or no refinement, respectively.
Read the full abstract ▾
All authors Linus Kreitner, Laurin Lux, Carmen Baumann, Daniel Rueckert, Martin J. Menten
Computer Vision
VOR-Bench: A Human Perception-Driven Benchmark for Video Object Removal
Haonan Huang, Tianrui Qiu, Xianghao Zang et al. (+10 more)
Despite its crucial role in
video object removal (VOR), existing evaluation paradigms face two critical limitations: questionable references and a misalignment between tradi- tional metrics and human preference. To address these challenges, we introduce VOR- Bench, which advances VOR evaluation through three integrated components. First, we present the VOR
Dataset (VORD), the first
benchmark dataset providing both paired edited
videos and graffiti masks. Its unique strength lies in a diverse data spectrum, which encompasses model-generated, tool-rendered, and camera-captured data, ensuring robust assessment across real-world scenarios. Second, we develop rMPAF, a realistic Motion- capable Paired-
video Acquisition Framework. By combining the strengths of image- based object removal and fine-tuned
video generation models, rMPAF automatically generates realistic, motion-coherent paired
videos. Finally, we propose three evaluation dimensions and introduce VOR-MDSM, the first perception-driven VLM-based scoring model specifically designed for mask-guided VOR. It bridges the gap between arithmetic metrics and human perception by covering the essential visual attributes and matching nuanced human judgment. Extensive experiments demonstrate that VOR-Bench yields evaluation results that align closely with human perception, achieving a remarkable cor- relation (h̊o} > 0.9) with subjective assessments. We will release VOR-Bench along with its documentation to ensure full reproducibility.
Read the full abstract ▾
All authors Haonan Huang, Tianrui Qiu, Xianghao Zang, Yinan Du, Zhixiang He, Chi Zhang, Hao Sun, Zhongjiang He, Tianwei Cao, Xuchong Zhang, Hongbin Sun, Kongming Liang, Zhanyu Ma
Computer Vision
Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility
Jaeseok Byun, Gukyeong Kwon, Han-Kai Hsu et al. (+4 more)
Upgrading
embedding models typically requires expensive database re-indexing, as new query
embeddings are incompatible with existing database
embeddings. While Backward Compatible Training (BCT) mitigates this by enforcing compatibility during training, existing approaches often require updating the backbone model. This is impractical because of significant training cost, the risk of performance regression, and limited access to proprietary model weights. We introduce Multi-modal Knowledge Preserving Adapter (MKP-Adapter), the first adapter-only BCT approach for Multi-modal Large
Language Models (MLLMs) that requires no backbone updates. We identified that the primary challenge in adapter-only BCT is preserving the knowledge of the new
embeddings while enforcing backward compatibility. Hence, we propose a multi-level preservation loss that maintains the
geometric structure of the
embedding spaces throughout BCT. Furthermore, a focal re-weighting strategy is integrated to prioritize learning from challenging samples. Experiments demonstrate that our method achieves strong backward compatibility across diverse multi-modal
benchmarks (image, text, visual document, and
video retrieval tasks) and model types. Notably, MKP-Adapter is trained solely on pre-extracted
embeddings and requires only negligible additional latency relative to the original backbone forward pass, highlighting its efficiency.
Read the full abstract ▾
All authors Jaeseok Byun, Gukyeong Kwon, Han-Kai Hsu, Meher Gitika Karumuri, Zhikang Zhang, Hao Yang, Davide Modolo
Computer Vision
Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation
Carmelo Scribano, Filippo Muzzini, Nedyalko Prisadnikov et al. (+6 more)
Beyond model inference, the decoding stage, which converts raw network outputs into task-level representations, constitutes a significant portion of the execution cost. Despite its practical impact, prediction decoding has received comparatively little
attention and is often implemented using generic CPU routines or inefficient GPU kernels, limiting the benefits of advances in model efficiency. In this work, we investigate the decoding overhead associated with a recent sinusoidal centroid encoding for Instance
Segmentation, in which each pixel regresses a positional
embedding of its instance centroid. This approach allows flexible
segmentation without predefined proposals, but extracting instance masks from dense
embeddings incurs a high computational cost. We present an optimized CUDA-based implementation of the decoding
algorithm tailored to this encoding, explicitly addressing challenges related to parallelization, synchronization, and
memory access on modern GPUs. Our solution significantly reduces decoding overhead and improves End-to-End inference latency, outperforming both CPU-based approaches and naive GPU implementations. The results demonstrate that efficient decoding is essential to fully exploit the advantages of advanced output representations and highlight the importance of jointly designing encoding schemes and their decoding
algorithms for real-time computer
vision systems.
Read the full abstract ▾
All authors Carmelo Scribano, Filippo Muzzini, Nedyalko Prisadnikov, Mohammad Mahdi, Yuqian Fu, Giorgia Franchini, Danda Pani Paudel, Marko Bertogna, Luc Van Gool
Computer Vision
NeuroTS-Net: Multi-Class Semantic Segmentation of Pediatric Brain Tumors in Multi-Modal MRI
Darius Peteleaza, Razvan-Gabriel Dumitru, Bogdan Neamtu et al. (+3 more)
Pediatric
brain tumors are a leading cause of cancer-related mortality in children, and their small, rare, and often low-contrast subregions make accurate manual delineation challenging. Reliable automated
segmentation is therefore needed to support diagnosis, treatment planning, and response assessment. Accordingly, we introduce NeuroTS-Net, a three-dimensional encoder-decoder convolutional neural network architecture for multi-class semantic
segmentation that incorporates a dual-scale raw-detail stream, adaptive low-resolution context selection, and detail-preserving multipath downsampling. These components preserve fine intensity and boundary information while efficiently modeling broader tumor context. NeuroTS-Net was trained on the BraTS 2026 pediatric
dataset without external data or pretrained weights and evaluated against nnU-Net and MedNeXt under the same experimental protocol. NeuroTS-Net outperformed the baseline methods, achieving whole-tumor and tumor-core Dice scores of 0.938 and 0.937 on the internal validation set and 0.927 and 0.926 on the official challenge validation set. The code is open-sourced at: https://github.com/maenstru56/NeuroTS.
Read the full abstract ▾
All authors Darius Peteleaza, Razvan-Gabriel Dumitru, Bogdan Neamtu, Arpad Gellert, Mariana Sandu, Claudiu Matei
Computer Vision
GRACE: Geometry- and Ray-Aware Camera-Efficient Multi-View Pedestrian Tracking
Taigo Sakai, Hiroki Kouno, Naoki Kato et al. (+1 more)
Reducing the number of cameras reduces the deployment cost but removes views that correct BEV responses stretched away from true pedestrian positions by projection and short score drops that can split tracks} in Bird's-Eye View (BEV) tracking. We introduce GRACE, a camera-efficient multi-view tracker with three components. Volumetric-Guided Fusion combines homography-based BEV features with features lifted through 3D space. Ray Conditioning exposes each camera's viewing direction to the fusion network. Its tracking component, BEV Track Recovery (BTR), uses low-confidence detections only to continue existing tracks. The same detections cannot start new tracks. With two WildTrack cameras, GRACE improves MOTA from 83.54 for TrackTacular, our baseline, to 91.07.
Read the full abstract ▾
All authors Taigo Sakai, Hiroki Kouno, Naoki Kato, Kazuhiro Hotta
Computer Vision
SPEAR NeXT Causal Latent Forecasting Across Multiple Horizons for Spectral Temporal Earth Representation Learning
Rajiv Ranjan, Udaiveer Singh, Shashank Tamaskar et al. (+1 more)
Earth observation is inherently dynamic, yet temporal information in many foundation models is learned through reconstruction, invariance, or retrospective sequence summarization. SPEAR NeXT is introduced as a compact pixel-wise
multimodal spectral temporal foundation model in which temporal self supervision is formulated as past only, multi horizon latent Earth state prediction. Instantaneous states are first encoded by the pretrained SPEAR model from optical, radar, and environmental observations into compact 32 dimensional
embeddings. Their temporal
evolution is then modeled by a causally masked Trans former that predicts multiple future latent states from pre ceding observations. Relative temporal order is represented using Rotary Position
Embeddings, while month and year
embeddings encode seasonal phase and interannual con text.
Read the full abstract ▾
All authors Rajiv Ranjan, Udaiveer Singh, Shashank Tamaskar, Dharmendra Saraswat
Computer Vision
tcnerv:dual-domain temporal context modeling for implicit neural video compression
Xuezhi Xiang, Yixin Zhao, Heqi Xiang et al. (+2 more)
Video compression aims to minimize reconstruction distor tion under a constrained bit rate. Existing
video implicit neural representations (INRs) often decode frames independently, leaving intermediate features unconditioned on previous reconstructions and content
embeddings without explicit temporal prediction. We propose TCNeRV, which exploits reconstructed context in both feature and
embedding domains. Its multi-scale temporal-context fusion (MTCF) module injects gated historical features at multiple decoder scales, while temporal
embedding-residual coding (TERC) predicts each content
embedding and codes only its residual. With approximately 3M parameters, TCNeRV achieves an average PSNR of 36.08 dB on the UVG
dataset, outperforming HNeRV-Boost by 2.20 dB. It reduces BD-rate by 22.06%, 66.73%, and 29.85% relative to HM, DCVC, and HiNeRV, respectively, demonstrating competitive rate-distortion performance with limited model capacity.
Read the full abstract ▾
All authors Xuezhi Xiang, Yixin Zhao, Heqi Xiang, Jiayao Liu, Shanjun Zhang
Computer Vision · Cross-listed from cs.RO
TEMPO: Learning Temporal Context for Dynamic Robot Manipulation
Zhenyang Feng, Jimin Heo, Erik B. Sudderth et al. (+1 more)
Vision-
language-action (VLA) models have achieved impressive performance in quasi-static manipulation, but struggle in dynamic manipulation tasks because they operate on a single observation at inference time. We identify two representational failures that underlie this limitation. The first is motion ambiguity, where a single observation does not include scene dynamics and therefore cannot anticipate the future state of moving objects. The second is state aliasing, where visually similar observations from different points in a task require different actions. We argue that these failures persist regardless of model scale and inference latency, showing that the bottleneck is missing temporal context rather than model capacity. Based on this insight, we propose TEMPO, which augments a pretrained VLA with two temporal inputs: a motion summary extracted from a frozen
video foundation model to resolve motion ambiguity and a compact proprioceptive history to resolve state aliasing. TEMPO requires no modification to the backbone and adds minimal compute overhead at training or deployment. Across four dynamic manipulation tasks, it improves Bottle Handover success from 44% to 74% and is the only method that solves state aliasing. Probing and ablation studies confirm that each temporal signal independently addresses its corresponding failure. We further release TEMPO-Bench, a
benchmark of over 50k annotated frames for evaluating motion-aware
robot perception in both regression and multiple-choice formats. Project Website: https://tempo-
robot.github.io/
Read the full abstract ▾
All authors Zhenyang Feng, Jimin Heo, Erik B. Sudderth, Unnat Jain
Computer Vision
Measuring Annotation Efficiency for Handwritten Devanagari Recognition: Sample-Complexity Curves for Four Pretraining Regimes
Manglesh Kumar Pandey and Sumit Kumar Banshal
To train handwritten text recognition systems we need word images and their corresponding transcriptions, and these transcriptions are produced manually. For a script that can be read by only a small number of specialists, this manual transcription is a limitation, because the trained models are supposed to save the time of those same specialists. A relevant question therefore arises: how many transcriptions are needed before a recogniser becomes useful, and how much of that cost can pretraining remove? In this study the answer is measured directly for handwritten Devanagari. We keep the recogniser, optimiser and evaluation protocol the same and change only the number of real transcribed words used for fine-tuning across nine budgets from 10 to 4,000 and four initialisation regimes, with six seeds at every point. The resulting curves are then converted into annotation-equivalent terms. A CER of 0.50 is reached by supervised synthetic pretraining using only 81 transcribed words, whereas random initialisation requires 355, which gives a label multiplier of 4.40 [3.56, 4.99]. There is a zero-shot reference point as well: with no real transcribed words at all, this pretraining is worth about 136 of them. This advantage gets smaller as the target accuracy improves, and at the most demanding target we measure, it cannot be distinguished from no saving at all. A fourth arm in which only the encoder is transferred separates the effect of the pretraining method from that of transfer scope, and masked image modelling is observed to transfer negatively over a bounded range of budgets. We emphasise that the scarcity in this study is constructed by subsampling a large corpus.
Read the full abstract ▾
Computer Vision
TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models
Zeyi Shao, Haowen Hua, Jiaxin Zhang et al. (+3 more)
Prompt learning adapts
vision-
language models, such as CLIP, by adjusting a small set of context
tokens. However, under few-shot supervision, even moderate label noise can disrupt
prompt optimization. To address this issue, we propose TecoPrompt, a closed-loop robust
prompt-learning framework that revisits optimal transport (OT) pseudo-labeling from a temporal perspective. TecoPrompt employs an entropic OT plan in the CLIP semantic space to obtain globally consistent label candidates. It verifies the reliability of these candidates by examining trajectory stability: a noisy label is only rewritten if the OT candidate remains unchanged within a K-epoch temporal stability window and passes a confidence gate based on Exponential Moving Average (EMA). This approach helps reduce confirmation bias. The rewritten labels are then integrated back into
prompt training using a tri-group objective that includes three loss functions aligned with clean, mid, and noisy subsets. Experiments on seven
datasets with synthetic symmetric and asymmetric noise, as well as Food101N, demonstrate significant performance improvements. For example, on the OxfordPets
dataset, with 50% asymmetric noise, TecoPrompt achieves an accuracy of 0.843, up from 0.775.
Read the full abstract ▾
All authors Zeyi Shao, Haowen Hua, Jiaxin Zhang, John See, Zeyd Boukhers, Cong Yang
Computer Vision
RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models
Xun Liang, Honghui Yang, Weihang Pan et al. (+6 more)
Region-level
retrieval aims to align user-specified image regions with relevant regions or textual descriptions, playing a crucial role in realworld applications such as e-commerce product search and RAG. Although recent Large
Multimodal Models (LMMs) have made significant strides in
multimodal retrieval, they primarily focus on global-level tasks and struggle to capture effective region-level representations. To bridge this gap, we present RegRet, an LMM-based Region-level
Retrieval framework that enhances the regional representations without compromising overall global
retrieval performance. At its core, RegRet integrates a Region-Aware Encoder to capture detailed regional features while balancing them with the global background context. To further enhance the fine-grained understanding and discriminability of representations, we design a multi-stage training pipeline that includes detailed localized captioning and regional contrastive learning tasks. In addition, considering the absence of region-level contrastive training data and the limited diversity of evaluation tasks in current
benchmarks, we introduce the REGMB
benchmark. It comprises 225k contrastive pairs, covering four
multimodal retrieval tasks. Extensive experiments validate the effectiveness of our approach. RegRet outperforms strong baselines in the zero-shot setting. Further training with contrastive learning leads to an average improvement of more than 20% on both REGMB and public
benchmarks, while achieving comparable or better results on global-level
retrieval tasks.
Read the full abstract ▾
All authors Xun Liang, Honghui Yang, Weihang Pan, Ruisi Zhao, Boyuan Pan, Yao Hu, Wenxiao Wang, Binbin Lin, Deng Cai
Computer Vision
FAHCD-Net: Frequency-Adaptive Heatmap-Conditional Diffusion Networks for Robust Facial Landmark Detection
Jun Wan, Jiwei Hu, Shengkai Hu et al. (+1 more)
Facial Landmark Detection(FLD) is a crucial task in various applications and has achieved significant advancements in recent years. However, current FLD methods still struggle under challenging conditions, where facial structural variations, information loss, and noise interference severely compromise the integrity and accuracy of learned facial features. To address these issues, we propose Frequency-Adaptive Heatmap-Conditional
Diffusion Network (FAHCD-Net), which integrates a Frequency-Adaptive Heatmap-Conditional
Diffusion (FAHCD) model with a Smoothness Regularization (SR) loss in a cascaded framework. Specifically, the FAHCD model incorporates a Hierarchical Frequency Adaptation (HFA) module designed to suppress redundant high-frequency noise through multi-layer frequency decomposition and adaptive reconstruction, thereby preserving essential facial structures. Additionally, the SR loss is proposed to further mitigate the interference of high-frequency noise and enhance the smoothness of the generated landmark heatmaps. By cascading the FAHCD model with the SR loss, FAHCD-Net effectively leverages both statistical and frequency-based distribution characteristics of the data to progressively generate more accurate landmark heatmaps from noisy inputs. Extensive experiments on popular
benchmarks demonstrate the effectiveness and robustness of the proposed method, achieving state-of-the-art performance in FLD tasks under challenging scenarios. The source code is available at https://github.com/HJWKryptonite/FAHCD-Net.
Read the full abstract ▾
All authors Jun Wan, Jiwei Hu, Shengkai Hu, Qilu Zhu
Computer Vision
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Zhenbin Wang, Lei Zhang, Lituan Wang et al. (+3 more)
Increasing image resolution produces ever-longer visual-
token sequences in
vision-
language models (VLMs), substantially raising their inference cost. To reduce this overhead without retraining, existing methods select compact
token subsets that prioritize query relevance, visual coverage, or a fixed trade-off between them. The appropriate balance, however, varies across queries and
token budgets: localized questions favor relevance, whereas holistic questions demand broader visual coverage. We introduce StackTok, a training-free selector that treats query relevance as the objective and visual coverage as budget-calibrated support. StackTok builds a size-indexed coverage reference from a coverage-only greedy sequence and adjusts its support target using query–
vision affinity
entropy. A reference-gated interleaved selection policy then switches between relevance- and coverage-oriented additions according to the current subset's support deficit. For high-resolution inputs, StackTok allocates one shared
token budget across crops according to the combined marginal gain of locally nominated
tokens. Evaluated with five VLMs over ten distinct image-understanding
benchmarks, StackTok ranks first among training-free selectors in every tested model–budget setting. On high-resolution LLaVA-NeXT-7B, it retains 95.26% of full-
token performance with only 160 of 2{,}880 (5.6%) visual
tokens.
Read the full abstract ▾
All authors Zhenbin Wang, Lei Zhang, Lituan Wang, Wei Huang, Yan Wang, Zhenwei Zhang