Computer Vision · Cross-listed from cs.LG
Vision-Language Models Unlock Task-Centric Latent Actions
Alexander Nikulin, Ilya Zisman, Albina Klepach et al. (+5 more)
Latent Action Models (LAMs) have rapidly gained traction as an important component in the pre-training pipelines of leading
Vision-
Language-Action models. However, they fail when observations contain action-correlated distractors, often encoding noise instead of meaningful latent actions. Humans, on the other hand, can effortlessly distinguish task-relevant motions from irrelevant details in any
video given only a brief task description. In this work, we propose to utilize the common-sense
reasoning abilities of
Vision-
Language Models (VLMs) to provide promptable representations, effectively separating controllable changes from the noise in unsupervised way. We use these representations as targets during LAM training and
benchmark a wide variety of popular VLMs, revealing substantial variation in the quality of promptable representations as well as their robustness to different
prompts and hyperparameters. Interestingly, we find that more recent VLMs may perform worse than older ones. Finally, we show that simply asking VLMs to ignore distractors can substantially improve latent action quality, yielding up to a six-fold increase in downstream success rates on Distracting MetaWorld.
Read the full abstract ▾
All authors Alexander Nikulin, Ilya Zisman, Albina Klepach, Denis Tarasov, Alexander Derevyagin, Andrei Polubarov, Lyubaykin Nikita, Vladislav Kurenkov
Computer Vision · Cross-listed from cs.LG
SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks
Matteo Gambella, Fabrizio Pittorino, Giuliano Casale et al. (+1 more)
Early-exit neural networks have become popular for reducing inference latency by allowing intermediate predictions when sufficient confidence is achieved. However, standard approaches typically rely on single-model confidence thresholds, which are frequently unreliable due to inherent calibration issues. To address this, we introduce SQUAD (Scalable Quorum Adaptive Decisions), the first inference scheme that integrates early-exit mechanisms with distributed ensemble learning, improving uncertainty estimation while reducing the inference time. Unlike traditional methods that depend on individual confidence scores, SQUAD employs a quorum-based stopping criterion on early-exit learners by collecting intermediate predictions incrementally in order of computational complexity until a consensus is reached and halting the computation at that exit if the consensus is statistically significant. To maximize the efficacy of this voting mechanism, we also introduce QUEST (Quorum Search Technique), a Neural Architecture Search method to select early-exit learners with optimized hierarchical diversity, ensuring learners are complementary at every intermediate layer. This consensus-driven approach yields statistically robust early exits, improving the test accuracy up to 5.95% compared to state-of-the-art dynamic solutions with a comparable computational cost and reducing the inference latency up to 70.60% compared to static ensembles while maintaining a good accuracy.
Read the full abstract ▾
All authors Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri
Computer Vision
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Yanlong Chen, Amirhossein Habibian, Luca Benini et al. (+1 more)
Vision-
Language Models (VLMs) achieve strong
multimodal performance but are costly to deploy, and post-training quantization often causes significant accuracy loss. Despite its potential, quantization-aware training for VLMs remains underexplored. We propose GRACE, a framework unifying knowledge
distillation and QAT under the Information Bottleneck principle: quantization constrains information capacity while
distillation guides what to preserve within this budget. Treating the teacher as a proxy for task-relevant information, we introduce confidence-gated decoupled
distillation to filter unreliable supervision, relational centered kernel
alignment to transfer visual
token structures, and an adaptive controller via Lagrangian relaxation to balance fidelity against capacity constraints. Across extensive
benchmarks on LLaVA and Qwen families, our INT4 models consistently outperform FP16 baselines (e.g., LLaVA-1.5-7B: 70.1 vs. 66.8 on SQA; Qwen2-VL-2B: 76.9 vs. 72.6 on MMBench), nearly matching teacher performance. Using real INT4 kernel, we achieve 3$\times$ throughput with 54%
memory reduction. This principled framework significantly outperforms existing quantization methods, making GRACE a compelling solution for resource-constrained deployment. Code and data are available at: https://github.com/ForeverBlue816/GRACE.
Read the full abstract ▾
All authors Yanlong Chen, Amirhossein Habibian, Luca Benini, Yawei Li
Computer Vision
DAVIS: OOD Detection via Dominant Activations and Variance for Increased Separation
Abid Hassan, Tuan Ngo, Saad Shafiq et al. (+1 more)
Detecting out-of-distribution (OOD) inputs is a critical safeguard for deploying machine learning models in the real world. However, most post-hoc detection methods operate on penultimate feature representations derived from global average pooling (GAP) – a lossy operation that discards valuable distributional statistics from activation maps prior to global average pooling. We contend that these overlooked statistics, particularly channel-wise variance and dominant (maximum) activations, are highly discriminative for OOD detection. We introduce DAVIS, a simple and broadly applicable post-hoc technique that enriches feature vectors by incorporating these crucial statistics, directly addressing the information loss from GAP. Extensive evaluations show DAVIS sets a new
benchmark across diverse architectures, including ResNet, DenseNet, and EfficientNet. It achieves significant reductions in the false positive rate (FPR95), with improvements of 48.26% on CIFAR-10 using ResNet-18, 38.13% on CIFAR-100 using ResNet-34, and 26.83% on ImageNet-1k
benchmarks using MobileNet-v2. Our analysis reveals the underlying mechanism for this improvement, providing a principled basis for moving beyond the mean in OOD detection.
Read the full abstract ▾
All authors Abid Hassan, Tuan Ngo, Saad Shafiq, Nenad Medvidovic
Computer Vision
Bi-MCQ: Reformulating Vision-Language Alignment for Negation Understanding
Tae Hun Kim and Hyun Gyu Lee
Recent
vision-
language models (VLMs) achieve strong zero-shot performance via large-scale image-text pretraining and have been widely adopted in medical image analysis. However, existing VLMs remain notably weak at understanding negated clinical statements, largely due to contrastive
alignment objectives that treat negation as a minor linguistic variation rather than a meaning-inverting operator. In multi-label settings,
prompt-based InfoNCE fine-tuning further reinforces easy-positive image-
prompt alignments, limiting effective learning of disease absence. To overcome these limitations, we reformulate
vision-
language alignment as a conditional semantic comparison problem, which is instantiated through a bi-directional multiple-choice learning framework(Bi-MCQ). By jointly training Image-to-Text and Text-to-Image MCQ tasks with affirmative, negative, and mixed
prompts, our method implements fine-tuning as conditional semantic comparison instead of global similarity maximization. We further introduce direction-specific Cross-
Attention fusion modules to address asymmetric cues required by bi-directional
reasoning and reduce
alignment interference. Experiments on ChestXray14, Open-I, CheXpert, and PadChest show that Bi-MCQ improves negation understanding by up to 0.47 AUC over the zero-shot performance of the state-of-the-art CARZero model, while achieving up to a 0.08 absolute gain on positive-negative combined (PNC) evaluation. Additionally, Bi-MCQ reduces the affirmative-negative AUC gap by an average of 0.12 compared to InfoNCE-based fine-tuning, demonstrating that objective reformulation can substantially enhance negation understanding in medical VLMs.
Read the full abstract ▾
Computer Vision
PEAR: Pixel-aligned Expressive humAn mesh Recovery
Jiahao Wu, Yunfei Liu, Lijian Lin et al. (+4 more)
Reconstructing detailed 3D human meshes from a single in-the-wild image remains a fundamental challenge in computer
vision. Existing SMPLX-based methods often suffer from slow inference, produce only coarse body poses, and exhibit misalignments or unnatural artifacts in fine-grained regions such as the face and hands. These issues make current approaches difficult to apply to downstream tasks. To address these challenges, we propose PEAR-a fast and robust framework for pixel-aligned expressive human mesh recovery. PEAR explicitly tackles three major limitations of existing methods: slow inference, inaccurate localization of fine-grained human pose details, and insufficient facial expression capture. Specifically, to enable real-time SMPLX parameter inference, we depart from prior designs that rely on high resolution inputs or multi-branch architectures. Instead, we adopt a clean and unified ViT-based model capable of recovering coarse 3D human geometry. To compensate for the loss of fine-grained details caused by this simplified architecture, we introduce pixel-level supervision to optimize the geometry, significantly improving the reconstruction accuracy of fine-grained human details. To make this approach practical, we further propose a modular data annotation strategy that enriches the training data and enhances the robustness of the model. Overall, PEAR is a preprocessing-free framework that can simultaneously infer EHM-s (SMPLX and scaled-FLAME) parameters at over 100 FPS. Extensive experiments on multiple
benchmark datasets demonstrate that our method achieves substantial improvements in pose estimation accuracy compared to previous SMPLX-based approaches. Project page: https://wujh2001.github.io/PEAR
Read the full abstract ▾
All authors Jiahao Wu, Yunfei Liu, Lijian Lin, Ye Zhu, Lei Zhu, Jingyi Li, Yu Li
Computer Vision
OOVDet: Low-Density Prior Learning for Zero-Shot Out-of-Vocabulary Object Detection
Binyi Su, Chenghao Huang and Haiyong Chen
Zero-shot out-of-vocabulary detection (ZS-OOVD) aims to accurately recognize objects of in-vocabulary (IV) categories provided at zero-shot inference, while simultaneously rejecting undefined ones (out-of-vocabulary, OOV) that lack corresponding category
prompts. However, previous methods are prone to overfitting the IV classes, leading to the OOV or undefined classes being misclassified as IV ones with a high confidence score. To address this issue, this paper proposes a zero-shot OOV detector (OOVDet), a novel framework that effectively detects predefined classes while reliably rejecting undefined ones in zero-shot scenes. Specifically, due to the model's lack of prior knowledge about the distribution of OOV data, we synthesize region-level OOV
prompts by sampling from the low-likelihood regions of the class-conditional Gaussian distributions in the hidden space, motivated by the assumption that unknown semantics are more likely to emerge in low-density areas of the latent space. For OOV images, we further propose a Dirichlet-based gradient attribution mechanism to mine pseudo-OOV image samples, where the attribution gradients are interpreted as Dirichlet evidence to estimate prediction uncertainty, and samples with high uncertainty are selected as pseudo-OOV images. Building on these synthesized OOV
prompts and pseudo-OOV images, we construct the OOV decision boundary through a low-density prior constraint, which regularizes the optimization of OOV classes using Gaussian kernel density estimation in accordance with the above assumption. Experimental results show that our method significantly improves the OOV detection performance in zero-shot scenes. The code is available at https://github.com/binyisu/OOV-detector.
Read the full abstract ▾
Computer Vision
Visual Personalization Turing Test
Rameen Abdal, James Burgess, Sergey Tulyakov et al. (+1 more)
We introduce the Visual Personalization Turing Test (VPTT), a new paradigm for evaluating contextual visual personalization based on perceptual indistinguishability, rather than identity replication. A model passes the VPTT if its output (image,
video, 3D asset, etc.) is indistinguishable to a human or calibrated VLM judge from content a given person might plausibly create or share. To operationalize VPTT, we present the VPTT Framework, integrating a 10k-persona
benchmark (VPTT-Bench), a visual
retrieval-augmented generator (VPRAG), and the VPTT Score, a text-only metric calibrated against human and VLM judgments. We show high correlation across human, VLM, and VPTT evaluations, validating the VPTT Score as a reliable perceptual proxy. Experiments demonstrate that VPRAG achieves the best
alignment-originality balance, offering a scalable and
privacy-safe foundation for personalized generative AI.
Read the full abstract ▾
All authors Rameen Abdal, James Burgess, Sergey Tulyakov, Kuan-Chieh Jackson Wang
Computer Vision · Cross-listed from cs.LG
Stabilizing Consistency Training: A Flow Map Analysis and Self-Distillation
Youngjoong Kim, Duhoe Kim, Woosung Kim et al. (+1 more)
Consistency models have been proposed for fast generative modeling, achieving results competitive with
diffusion and flow models. However, these methods exhibit instability and limited reproducibility when training from scratch, motivating subsequent work to explain and stabilize these issues. While these efforts have provided valuable insights, the explanations remain fragmented, and the theoretical relationships remain unclear. In this work, we provide a theoretical examination of consistency models by analyzing them from a flow map-based perspective. This joint analysis clarifies how training and convergence behavior can give rise to degenerate solutions. Building on these insights, we revisit self-
distillation as a practical remedy for certain forms of suboptimal convergence. To avoid excessive gradient norms during its optimization, we reformulate its objective empirically, yielding improved Self-
Distillation (iSD). We further extend iSD with training-time classifier-free guidance, yielding iSD-T, which improves performance and reproducibility in the studied settings. We demonstrate that iSD extends beyond image generation to
diffusion-based policy learning, without reliance on pretrained
diffusion models for initialization.
Read the full abstract ▾
All authors Youngjoong Kim, Duhoe Kim, Woosung Kim, Jaesik Park
Computer Vision
Fire on Motion: Optimizing Video Pass-bands for Efficient Spiking Action Recognition
Shuhan Ye, Yuanbin Qian, Yi Yu et al. (+5 more)
Spiking neural networks (SNNs) have gained traction in
vision due to their energy efficiency, bio-plausibility, and inherent temporal processing. Yet, despite this temporal capacity, most progress concentrates on static image
benchmarks, and SNNs still underperform on dynamic
video tasks compared to artificial neural networks (ANNs). In this work, we diagnose a fundamental pass-band mismatch: Standard spiking dynamics behave as a temporal low pass that emphasizes static content while attenuating motion bearing bands, where task relevant information concentrates in dynamic tasks. This phenomenon explains why SNNs can approach ANNs on static tasks yet fall behind on tasks that demand richer temporal understanding.To remedy this, we propose the Pass-Bands Optimizer (PBO), a plug-and-play module that optimizes the temporal pass-band toward task-relevant motion bands. PBO introduces only two learnable parameters, and a lightweight consistency constraint that preserves semantics and boundaries, incurring negligible computational overhead and requires no architectural changes. PBO deliberately suppresses static components that contribute little to discrimination, effectively high passing the stream so that spiking activity concentrates on motion bearing content. On UCF101, PBO yields over ten percentage points improvement. On more complex multi-modal action recognition and weakly supervised
video anomaly detection, PBO delivers consistent and significant gains, offering a new perspective for SNN based
video processing and understanding.
Read the full abstract ▾
All authors Shuhan Ye, Yuanbin Qian, Yi Yu, Chong Wang, Yuqi Xie, Jiazhen Xu, Kun Wang, Xudong Jiang
Computer Vision
VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration
Hanxun Yu, Wentong Li, Xuan Qu et al. (+3 more)
Multimodal large
language models (MLLMs) suffer from high computational costs due to excessive visual
tokens, particularly in high-resolution and
video-based scenarios. Existing
token reduction methods typically focus on isolated pipeline components and often neglect textual
alignment, leading to performance degradation. In this paper, we propose VisionTrim, a unified framework for training-free MLLM acceleration, integrating two effective plug-and-play modules: 1) the Dominant
Vision Token Selection (DVTS) module, which preserves essential visual
tokens via a global-local view, and 2) the Text-Guided
Vision Complement (TGVC) module, which facilitates context-aware
token merging guided by textual cues. Extensive experiments across diverse image and
video multimodal benchmarks demonstrate the performance superiority of our VisionTrim, advancing practical MLLM deployment in real-world applications. The code is available at: https://github.com/hanxunyu/VisionTrim.
Read the full abstract ▾
All authors Hanxun Yu, Wentong Li, Xuan Qu, Song Wang, Junbo Chen, Jianke Zhu
Computer Vision
ExpAlign: Expectation-Guided Vision-Language Alignment for Open-Vocabulary Grounding
Junyi Hu, Tian Bai, Fengyi Wu et al. (+3 more)
Open-vocabulary grounding requires accurate
vision-
language alignment under weak supervision, yet existing methods either rely on global sentence
embeddings that lack fine-grained expressiveness or introduce
token-level
alignment with explicit supervision or heavy cross-
attention designs. We propose ExpAlign, a theoretically grounded
vision-
language alignment framework built on a principled multiple instance learning formulation. ExpAlign introduces an Expectation
Alignment Head that performs
attention-based soft MIL pooling over
token-region similarities, enabling implicit
token and instance selection without additional annotations. To further stabilize
alignment learning, we develop an energy-based multi-scale consistency regularization scheme, including a Top-K multi-positive contrastive objective and a Geometry-Aware Consistency Objective derived from a Lagrangian-constrained free-energy minimization. Extensive experiments show that ExpAlign consistently improves open-vocabulary detection and zero-shot instance
segmentation, particularly on long-tail categories. Most notably, it achieves 36.2 AP$_r$ on the LVIS minival split, outperforming other state-of-the-art methods at comparable model scale, while remaining lightweight and inference-efficient.
Read the full abstract ▾
All authors Junyi Hu, Tian Bai, Fengyi Wu, Wenyan Li, Zhenming Peng, Yi Zhang
Computer Vision
Unsupervised Synthetic Image Attribution: Alignment and Disentanglement
Zongfang Liu, Guangyi Chen, Boyang Sun et al. (+2 more)
As the quality of synthetic images improves, identifying the underlying concepts of model-generated images is becoming increasingly crucial for copyright protection and ensuring model transparency. Existing methods achieve this attribution goal by training models using annotated pairs of synthetic images and their original training sources. However, obtaining such paired supervision is challenging, as it requires either well-designed synthetic concepts or precise annotations from millions of training sources. To eliminate the need for costly paired annotations, in this paper, we explore the possibility of unsupervised synthetic image attribution. We propose a simple yet effective unsupervised method called
Alignment and Disentanglement. Specifically, we begin by performing basic concept
alignment using contrastive self-supervised learning. Next, we enhance the model's attribution ability by promoting representation disentanglement with the Infomax loss. This approach is motivated by an interesting observation: contrastive self-supervised models, such as MoCo and DINO, inherently exhibit the ability to perform simple cross-domain
alignment. By formulating this observation as a theoretical assumption on cross-covariance, we provide a theoretical explanation of how
alignment and disentanglement can approximate the concept-matching process through a decomposition of the canonical correlation analysis objective. On the real-world
benchmarks, AbC, we show that our unsupervised method surprisingly outperforms the supervised methods. As a starting point, we expect our intuitive insights and experimental findings to provide a fresh perspective on this challenging task.
Read the full abstract ▾
All authors Zongfang Liu, Guangyi Chen, Boyang Sun, Tongliang Liu, Kun Zhang
Computer Vision · Cross-listed from eess.IV
Training Beyond Convergence: Grokking nnU-Net for Glioma Segmentation in Sub-Saharan MRI
Mohtady Barakat, Omar Salah, Ahmed Yasser et al. (+8 more)
Gliomas are placing an increasingly clinical burden on Sub-Saharan Africa (SSA). In the region, the median survival for patients remains under two years, and access to diagnostic imaging is extremely limited. These constraints highlight an urgent need for automated tools that can extract the maximum possible information from each available scan, tools that are specifically trained on local data, rather than adapted from high-income settings where conditions are vastly different. We utilize the
Brain Tumor
Segmentation (BraTS) Africa 2025 Challenge
dataset, an expert annotated collection of glioma MRIs. Our objectives are: (i) establish a strong baseline with nnUNet on this
dataset, and (ii) explore whether the celebrated "grokking" phenomenon an abrupt, late training jump from memorization to superior generalization can be triggered to push performance without extra labels. We evaluate two training regimes. The first is a fast, budget-conscious approach that limits optimization to just a few epochs, reflecting the constrained GPU resources typically available in African institutions. Despite this limitation, nnUNet achieves strong Dice scores: 92.3% for whole tumor (WH), 86.6% for tumor core (TC), and 86.3% for enhancing tumor (ET). The second regime extends training well beyond the point of convergence, aiming to trigger a grokking-driven performance leap. With this approach, we were able to achieve grokking and enhanced our results to higher Dice scores: 92.2% for whole tumor (WH), 90.1% for tumor core (TC), and 90.2% for enhancing tumor (ET).
Read the full abstract ▾
All authors Mohtady Barakat, Omar Salah, Ahmed Yasser, Mostafa Ahmed, Zahirul Arief, Waleed Khan, Dong Zhang, Aondona Iorumbur, Confidence Raymond, Mohannad Barakat, Noha Magdy
Computer Vision
What can Computer Vision learn from Ranganathan?
Mayukh Bagchi and Fausto Giunchiglia
The Semantic Gap Problem (SGP) in Computer
Vision (CV) arises from the misalignment between visual and lexical semantics leading to flawed CV
dataset design and CV
benchmarks. This paper proposes that classification principles of S.R. Ranganathan can offer a principled starting point to address SGP and design high-quality CV
datasets. We elucidate how these principles, suitably adapted, underpin the vTelos CV annotation methodology. The paper also briefly presents experimental evidence showing improvements in CV annotation and accuracy, thereby, validating vTelos.
Read the full abstract ▾
Computer Vision
LINA: Linear Autoregressive Image Generative Models with Continuous Tokens
Jiahao Wang, Ting Pan, Haoge Deng et al. (+4 more)
Autoregressive models with continuous
tokens form a promising paradigm for visual generation, especially for text-to-image (T2I) synthesis, but they suffer from high computational cost. We study how to design compute-efficient linear
attention within this framework. Specifically, we conduct a systematic empirical analysis of scaling behavior with respect to parameter counts under different design choices, focusing on (1) normalization paradigms in linear
attention (division-based vs. subtraction-based) and (2) depthwise convolution for locality augmentation. Our results show that although subtraction-based normalization is effective for image classification, division-based normalization scales better for linear generative
transformers. In addition, incorporating convolution for locality modeling plays a crucial role in autoregressive generation, consistent with findings in
diffusion models. We further extend gating mechanisms, commonly used in causal linear
attention, to the bidirectional setting and propose a KV gate. By introducing data-independent learnable parameters to the key and value states, the KV gate assigns
token-wise
memory weights, enabling flexible
memory management similar to forget gates in
language models. Based on these findings, we present LINA, a simple and compute-efficient T2I model built entirely on linear
attention, capable of generating high-fidelity 1024x1024 images from user instructions. LINA achieves competitive performance on both class-conditional and T2I
benchmarks, obtaining 2.18 FID on ImageNet (about 1.4B parameters) and 0.74 on GenEval (about 1.5B parameters). A single linear
attention module reduces FLOPs by about 61 percent compared to softmax
attention. Code and models are available at: https://github.com/techmonsterwang/LINA.
Read the full abstract ▾
All authors Jiahao Wang, Ting Pan, Haoge Deng, Dongchen Han, Taiqiang Wu, Xinlong Wang, Ping Luo
Computer Vision
UniGeo: A Unified 3D Indoor Object Detection Framework Integrating Geometry-Aware Learning and Dynamic Channel Gating
Xing Yi, Jinyang Huang, Feng-Qi Cui et al. (+4 more)
The growing adoption of robotics and augmented reality in real-world applications has driven considerable research interest in 3D object detection based on point clouds. While previous methods address unified training across multiple
datasets, they fail to model
geometric relationships in sparse point cloud scenes and ignore the feature distribution in significant areas, which ultimately restricts their performance. To deal with this issue, a unified 3D indoor detection framework, called UniGeo, is proposed. To model
geometric relations in scenes, we first propose a geometry-aware learning module that establishes a learnable mapping from spatial relationships to feature weights, which enabes explicit
geometric feature enhancement. Then, to further enhance point cloud feature representation, we propose a dynamic channel gating mechanism that leverages learnable channel-wise weighting. This mechanism adaptively optimizes features generated by the sparse 3D U-Net network, significantly enhancing key
geometric information. Extensive experiments on six different indoor scene
datasets clearly validate the superior performance of our method.
Read the full abstract ▾
All authors Xing Yi, Jinyang Huang, Feng-Qi Cui, Anyang Tong, Ruimin Wang, Liu Liu, Dan Guo
Computer Vision
TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction
Zhijie Zheng, Xinhao Xiang and Jiawei Zhang
Streaming recurrent models enable efficient 3D reconstruction by maintaining persistent state representations. However, they suffer from catastrophic forgetting over long sequences due to balancing historical information with new observations. Recent methods alleviate this by deriving adaptive signals from the
attention perspective, but they operate on single dimensions without considering temporal and spatial consistency. To this end, we propose a training-free framework termed TTSA3R that leverages both temporal state
evolution and spatial observation quality for adaptive state updates in 3D reconstruction. In particular, we devise a Temporal Adaptive Update Module that regulates update magnitude by analyzing temporal state
evolution patterns. Then, a Spatial Context Update Module is introduced to localize spatial regions that require updates through observation-state
alignment and observation changes. These complementary signals are finally fused to determine the state updating strategies. Extensive experiments show that TTSA3R achieves competitive performance on standard short-sequence
benchmarks and provides substantially stronger robustness on extended sequences. On NRGBD, as sequences extend from 50 to 250 frames, TTSA3R exhibits only a 1.33$\times$ error increase, compared with 3.8$\times$ degradation for CUT3R. This highlights the practical value of temporal-spatial adaptive updates for long-term reconstruction stability. Our code is available at \url{https://github.com/anonus2357/ttsa3r}.
Read the full abstract ▾
Computer Vision
FOTBCD: A Large-Scale Building Change Detection Benchmark from French Orthophotos and Topographic Data
Abdelrrahman Moubane
We introduce FOTBCD, a large-scale building change detection
dataset derived from authoritative French orthophotos and topographic building data provided by IGN France. Unlike existing
benchmarks that are geographically constrained to single cities or limited regions, FOTBCD spans 28 departments across mainland France, with 25 used for training and three geographically disjoint departments held out for evaluation. The
dataset covers diverse urban, suburban, and rural environments at 0.2m/pixel resolution. We publicly release FOTBCD-Binary, a
dataset comprising approximately 28,000 before/after image pairs with pixel-wise binary building change masks, each associated with patch-level spatial metadata. The
dataset is designed for large-scale benchmarking and evaluation under geographic domain shift, with validation and test samples drawn from held-out departments and manually verified to ensure label quality. In addition, we publicly release FOTBCD-Instances, a publicly available instance-level annotated subset comprising several thousand image pairs, which illustrates the complete annotation schema used in the full instance-level version of FOTBCD. Using a fixed reference baseline, we
benchmark FOTBCD-Binary against LEVIR-CD+ and WHU-CD, providing strong empirical evidence that geographic diversity at the
dataset level is associated with improved cross-domain generalization in building change detection.
Read the full abstract ▾
Computer Vision
Cross-Domain Few-Shot Learning for Hyperspectral Image Classification Based on Mixup Foundation Model
Naeem Paeedeh, Mahardhika Pratama, Ary Shiddiqi et al. (+3 more)
Although cross-domain few-shot learning (CDFSL) for hyper-spectral image (HSI) classification has attracted significant research interest, existing works often rely on an unrealistic data augmentation procedure in the form of external noise to enlarge the sample size, thus greatly simplifying the issue of data scarcity. They involve a large number of parameters for model updates, being prone to the overfitting problem. To the best of our knowledge, none has explored the strength of the foundation model, having strong generalization power to be quickly adapted to downstream tasks. This paper proposes the MIxup FOundation MOdel (MIFOMO) for CDFSL of HSI classifications. MIFOMO is built upon the concept of a remote sensing (RS) foundation model, pre-trained across a large scale of RS problems, thus featuring generalizable features. The notion of coalescent projection (CP) is introduced to quickly adapt the foundation model to downstream tasks while freezing the backbone network. The concept of mixup domain adaptation (MDM) is proposed to address the extreme domain discrepancy problem. Last but not least, the label smoothing concept is implemented to cope with noisy pseudo-label problems. Our rigorous experiments demonstrate the advantage of MIFOMO, where it beats prior arts with up to 14% margin. The source code of MIFOMO is open-sourced at https://github.com/Naeem-Paeedeh/MIFOMO for reproducibility and convenient further study.
Read the full abstract ▾
All authors Naeem Paeedeh, Mahardhika Pratama, Ary Shiddiqi, Zehong Cao, Mukesh Prasad, Wisnu Jatmiko
Computer Vision · Cross-listed from eess.IV
Bonnet: Ultra-fast whole-body bone segmentation from CT scans
Hanjiang Zhu, Pedro Martelleto Rezende, Zhang Yang et al. (+5 more)
This work proposes Bonnet, an ultra-fast sparse-volume pipeline for whole-body bone
segmentation from CT scans. Accurate bone
segmentation is important for
surgical planning and anatomical analysis, but existing 3D voxel-based models such as nnU-Net and STU-Net require heavy computation and often take several minutes per scan, which limits time-critical use. The proposed Bonnet addresses this by integrating a series of novel framework components including HU-based bone thresholding, patch-wise inference with a sparse spconv-based U-Net, and multi-window fusion into a full-volume prediction. Trained on TotalSegmentator and evaluated without additional tuning on RibSeg, CT-Pelvic1K, and CT-Spine1K, Bonnet achieves high Dice across ribs, pelvis, and spine while running in only 2.69 seconds per scan on an RTX A6000. Compared to strong voxel baselines, Bonnet attains a similar accuracy but reduces inference time by roughly 25x on the same hardware and tiling setup. The toolkit and pre-trained models will be released at https://github.com/HINTLab/Bonnet.
Read the full abstract ▾
All authors Hanjiang Zhu, Pedro Martelleto Rezende, Zhang Yang, Tong Ye, Bruce Z. Gao, Feng Luo, Siyu Huang, Jiancheng Yang
Computer Vision
PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarios
Xudong Lu, Huankang Guan, Yang Bo et al. (+10 more)
Multimodal Large
Language Models excel at offline audio-visual understanding, but their ability to serve as mobile assistants in continuous real-world streams remains underexplored. In daily phone use, mobile assistants must track streaming audio-visual inputs and respond at the right time, yet existing
benchmarks are often restricted to multiple-choice questions or use shorter
videos. In this paper, we introduce PhoStream, the first mobile-centric streaming
benchmark that unifies on-screen and off-screen scenarios to evaluate
video, audio, and temporal
reasoning. PhoStream contains 5,572 open-ended QA pairs from 578
videos across 4 scenarios and 10 capabilities. We build it with an Automated Generative Pipeline backed by rigorous human verification, and evaluate models using a realistic Online Inference Pipeline and LLM-as-a-Judge evaluation for open-ended responses. Experiments reveal a temporal asymmetry in LLM-judged scores (0-100): models perform well on Instant and Backward tasks (Gemini 3 Pro exceeds 80), but drop sharply on Forward tasks (16.40), largely due to early responses before the required visual and audio cues appear. This highlights a fundamental limitation: current MLLMs struggle to decide when to speak, not just what to say. Code and
datasets used in this work will be made publicly accessible at https://github.com/Lucky-Lance/PhoStream.
Read the full abstract ▾
All authors Xudong Lu, Huankang Guan, Yang Bo, Jinpeng Chen, Xintong Guo, Shuhan Li, Fang Liu, Peiwen Sun, Xueying Li, Wei Zhang, Xue Yang, Rui Liu, Hongsheng Li
Computer Vision
Enhancing Video Representations with Spatiotemporal-Semantic Residual to Mitigate Hallucinations in Video Large Multimodal Models
Yuansheng Gao, Jinman Zhao, Tong Zhang et al. (+5 more)
Although
Video Large
Multimodal Models have achieved strong performance in
video understanding, they still suffer from
hallucination. Existing inference-time intervention methods usually modify
videos under the contrastive decoding framework, but their heuristic designs bring limited improvements and increase inference latency. To address these issues, we propose ViSSRes, an inference-time intervention method that enhances
video representations through a lightweight MLP-style network. Specifically, we use a contrastive random walk approach to characterize the spatiotemporal consistency of
video representations, and introduce conditional mutual information to associate
video representations with the model's semantic understanding. With the model backbone kept frozen, ViSSRes learns residuals for
video representations and optimizes them from both spatiotemporal and semantic consistency perspectives. During inference, ViSSRes requires only a single forward pass and introduces no substantial additional inference cost. Experiments show that ViSSRes reduces the
hallucination rate of LLaVA-NeXT-
Video on EventHallusion by 40.69% and improves
video understanding on MMVU by 18.36% under the CoT setting, demonstrating its effectiveness in mitigating
hallucinations.
Read the full abstract ▾
All authors Yuansheng Gao, Jinman Zhao, Tong Zhang, Xingguo Xu, Wenbin Xing, Han Bao, Zonghui Wang, Wenzhi Chen
Computer Vision
DELNet: Continuous All-in-One Weather Removal via Dynamic Expert Library
Shihong Liu, Kun Zuo and Hanguang Xiao
All-in-one weather image restoration methods are valuable in practice but depend on pre-collected data and require retraining for unseen degradations, leading to high cost. We propose DELNet, a continual learning framework for weather image restoration. DELNet integrates a judging valve that measures task similarity to distinguish new from known tasks, and a dynamic expert library that stores experts trained on different degradations. For new tasks, the valve selects top-k experts for knowledge transfer while adding new experts to capture task-specific features; for known tasks, the corresponding experts are directly reused. This design enables continuous optimization without retraining existing models. Experiments on OTS, Rain100H, and Snow100K demonstrate that DELNet surpasses state-of-the-art continual learning methods, achieving PSNR gains of 16%, 11%, and 12%, respectively. These results highlight the effectiveness, robustness, and efficiency of DELNet, which reduces retraining cost and enables practical deployment in real-world scenarios.
Read the full abstract ▾
Computer Vision
Leveraging Data to Say No: Memory Augmented Plug-and-Play Selective Prediction
Aditya Sarkar, Yi Li, Jiacheng Cheng et al. (+2 more)
Selective prediction aims to endow predictors with a reject option, to avoid low confidence predictions. However, existing literature has primarily focused on closed-set tasks, such as visual question answering with predefined options or fixed-category classification. This paper considers selective prediction for visual
language foundation models, addressing a taxonomy of tasks ranging from closed to open set and from finite to unbounded vocabularies, as in image captioning. We seek training-free approaches of low-complexity, applicable to any foundation model and consider methods based on external
vision-
language model
embeddings, like CLIP. This is denoted as Plug-and-Play Selective Prediction (PaPSP). We identify two key challenges: (1) instability of the visual-
language representations, leading to high variance in image-text
embeddings, and (2) poor calibration of similarity scores. To address these issues, we propose a
memory augmented PaPSP (MA-PaPSP) model, which augments PaPSP with a
retrieval dataset of image-text pairs. This is leveraged to reduce
embedding variance by averaging retrieved nearest-neighbor pairs and is complemented by the use of contrastive normalization to improve score calibration. Through extensive experiments on multiple
datasets, we show that MA-PaPSP outperforms PaPSP and other selective prediction baselines for selective captioning, image-text matching, and fine-grained classification. Code is publicly available at https://github.com/kingston-aditya/MA-PaPSP.
Read the full abstract ▾
All authors Aditya Sarkar, Yi Li, Jiacheng Cheng, Shlok Mishra, Nuno Vasconcelos
Computer Vision
Hybrid Cross-Device Localization via Neural Metric Learning and Feature Fusion
Meixia Lin, Mingkai Liu, Shuxue Peng et al. (+5 more)
We present a hybrid cross-device localization pipeline developed for the CroCoDL 2025 Challenge. Our approach integrates a shared
retrieval encoder and two complementary localization branches: a classical
geometric branch using feature fusion and PnP, and a neural feed-forward branch (MapAnything) for metric localization conditioned on
geometric inputs. A neural-guided candidate
pruning strategy further filters unreliable map frames based on translation consistency, while depth-conditioned localization refines metric scale and translation precision on Spot scenes. These components jointly lead to significant improvements in recall and accuracy across both HYDRO and SUCCU
benchmarks. Our method achieved a final score of 92.62 (R@0.5m, 5{\deg}) during the challenge.
Read the full abstract ▾
All authors Meixia Lin, Mingkai Liu, Shuxue Peng, Dikai Fan, Shengyu Gu, Xianliang Huang, Haoyang Ye, Xiao Liu
Computer Vision · Cross-listed from eess.IV
EndoCaver: Handling Fog, Blur and Glare in Endoscopic Images via Joint Deblurring-Segmentation
Zhuoyu Wu, Wenhui Ou, Pei-Sze Tan et al. (+4 more)
Endoscopic image analysis is vital for colorectal cancer screening, yet real-world conditions often suffer from lens fogging, motion blur, and specular highlights, which severely compromise automated polyp detection. We propose EndoCaver, a lightweight
transformer with a unidirectional-guided dual-decoder architecture, enabling joint multi-task capability for image deblurring and
segmentation while significantly reducing computational complexity and model parameters. Specifically, it integrates a Global
Attention Module (GAM) for cross-scale aggregation, a Deblurring-
Segmentation Aligner (DSA) to transfer restoration cues, and a cosine-based scheduler (LoCoS) for stable multi-task optimisation. Experiments on the Kvasir-SEG
dataset show that EndoCaver achieves 0.922 Dice on clean data and 0.889 under severe image degradation, surpassing state-of-the-art methods while reducing model parameters by 90%. These results demonstrate its efficiency and robustness, making it well-suited for on-device clinical deployment. Code is available at https://github.com/ReaganWu/EndoCaver.
Read the full abstract ▾
All authors Zhuoyu Wu, Wenhui Ou, Pei-Sze Tan, Jiayan Yang, Wenqi Fang, Zheng Wang, Raphaël C.-W. Phan
Computer Vision
SHED Light on Segmentation for Dense Prediction
Seung Hyun Lee, Sangwoo Mo and Stella X. Yu
Dense prediction infers per-pixel values from a single image and is fundamental to 3D perception and robotics. Although real-world scenes exhibit strong structure, existing methods treat it as an independent pixel-wise prediction, often resulting in structural inconsistencies. We propose SHED, a novel encoder-decoder architecture that enforces
geometric prior explicitly by incorporating
segmentation into dense prediction. By bidirectional hierarchical
reasoning, segment
tokens are hierarchically pooled in the encoder and unpooled in the decoder to reverse the hierarchy. The model is supervised only at the final output, allowing the segment hierarchy to emerge without explicit
segmentation supervision. SHED improves depth boundary sharpness and segment coherence, while demonstrating strong cross-domain generalization from synthetic to the real-world environments. Its hierarchy-aware decoder better captures global 3D scene layouts, leading to improved semantic
segmentation performance. Moreover, SHED enhances 3D reconstruction quality and reveals interpretable part-level structures that are often missed by conventional pixel-wise methods.
Read the full abstract ▾
Computer Vision
Can 3D point cloud data improve automated body condition score prediction in dairy cattle?
Zhou Tang, Jin Wang, Angelo De Castro et al. (+7 more)
Body condition score (BCS) is a widely used indicator of body energy status and is closely associated with metabolic status, reproductive performance, and health in dairy cattle; however, conventional visual scoring is subjective and labor-intensive. Computer
vision approaches have been applied to BCS prediction, with depth images widely used because they capture
geometric information independent of coat color and texture. More recently, three-dimensional point cloud data have attracted increasing interest due to their ability to represent richer
geometric characteristics of animal morphology, but direct head-to-head comparisons with depth image-based approaches remain limited. In this study, we compared top-view depth image and point cloud data for BCS prediction under four settings: 1) unsegmented raw data, 2) segmented full-body data, 3) segmented hindquarter data, and 4) handcrafted feature data. Prediction models were evaluated using data from 1,020 dairy cows collected on a commercial farm, with cow-level cross-validation to prevent data leakage. Depth image-based models consistently achieved higher accuracy than point cloud-based models when unsegmented raw data and segmented full-body data were used, whereas comparable performance was observed when segmented hindquarter data were used. Both depth image and point cloud approaches showed reduced accuracy when handcrafted feature data were employed compared with the other settings. Overall, point cloud-based predictions were more sensitive to noise and model architecture than depth image-based predictions. Taken together, these results indicate that three-dimensional point clouds do not provide a consistent advantage over depth images for BCS prediction in dairy cattle under the evaluated conditions.
Read the full abstract ▾
All authors Zhou Tang, Jin Wang, Angelo De Castro, Yuxi Zhang, Victoria Bastos Primo, Ana Beatriz Montevecchio Bernardino, Gota Morota, Xu Wang, Ricardo C Chebel, Haipeng Yu
Computer Vision
DNA: Uncovering Universal Latent Forgery Knowledge
Jingtong Dou, Chuancheng Shi, Yemin Wang et al. (+6 more)
As generative AI achieves hyper-realism, superficial artifact detection has become obsolete. While prevailing methods rely on resource-intensive fine-tuning of black-box backbones, we propose that forgery detection capability is already encoded within pre-trained models rather than requiring end-to-end retraining. To elicit this intrinsic capability, we propose the discriminative neural anchors (DNA) framework, which employs a coarse-to-fine excavation mechanism. First, by analyzing feature decoupling and
attention distribution shifts, we pinpoint critical intermediate layers where the focus of the model logically transitions from global semantics to local anomalies. Subsequently, we introduce a triadic fusion scoring metric paired with a
curvature-truncation strategy to strip away semantic redundancy, precisely isolating the forgery-discriminative units (FDUs) inherently imprinted with sensitivity to forgery traces. Moreover, we introduce HIFI-Gen, a high-fidelity synthetic
benchmark built upon the very latest models, to address the lag in existing
datasets. Experiments demonstrate that by solely relying on these anchors, DNA achieves superior detection performance even under few-shot conditions. Furthermore, it exhibits remarkable robustness across diverse architectures and against unseen generative models, validating that waking up latent
neurons is more effective than extensive fine-tuning.
Read the full abstract ▾
All authors Jingtong Dou, Chuancheng Shi, Yemin Wang, Shiming Guo, Anqi Yi, Wenhua Wu, Li Zhang, Fei Shen, Tat-Seng Chua