Computer Vision
Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering
Soumyadeep Jana, Pulkit Mittal and Sanasam Ranbir Singh
Large
vision-
language models (LVLMs) often hallucinate objects that are not present in the input image, largely because visual grounding weakens as decoding progresses. Existing inference-time mitigation methods modify logits or hidden states throughout generation, but they suffer from three key limitations: they lack an explicit grounding objective, intervene even when the model is already well-grounded, and use fixed correction strengths that do not adapt to the severity of grounding failure. We propose BRACS (Barrier-Regulated Adaptive Closed-form Steering), a training-free steering framework that addresses these issues through barrier-regulated adaptive closed-form steering. BRACS monitors the model's own
attention to measure visual grounding and applies corrections to the hidden states only when grounding deteriorates. The corrective update is computed analytically in closed form, requiring no training of auxiliary networks or model retraining. Experiments on LLaVA-1.5-7B and Qwen-VL-Chat show that BRACS consistently outperforms prior methods on
hallucination benchmarks, reducing CHAIR$_s$ by 9.4 points and improving POPE F1 by 2.7 points, while matching or improving performance on four general
multimodal benchmarks. BRACS also remains efficient, operating at 80% of greedy decoding throughput and achieving 1.3 times higher speed on average than the baselines.
Read the full abstract ▾
Computer Vision
DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding
Luzhou Ge, Xiangyu Zhu, Jinyan Liu et al. (+1 more)
Integrating open-vocabulary semantic information into dynamic 3D scene representations is essential for long-term
embodied scene understanding. However, existing methods often suffer from fragile instance association due to incomplete cross-view cues, while their limited ability to handle object-level topological changes restricts long-term robotic task execution. Moreover, current 3D scene understanding methods either rely on simple feature matching without explicit spatial
reasoning or assume offline ground-truth 3D geometry. To address these challenges, we present DGSG-Mind, a hybrid instance-aware 3D Gaussian dynamic scene graph system with an
embodied reasoning agent. Our system couples a probabilistic voxel grid with explicit 3D Gaussians to enable robust cross-modal instance fusion and incremental semantic mapping. It handles dynamic changes through Gaussian-based visual relocalization and localized masked refinement guided by
geometric-semantic consistency. Built on the instance Gaussian map, DGSG-Mind further constructs a hierarchical scene graph and develops the 3D Gaussian Mind, which integrates structural relations, spatial-semantic information, and visually annotated RoI Gaussian renderings for
multimodal reasoning. Extensive experiments show that DGSG-Mind achieves the best zero-shot 3DVG performance among methods operating on self-reconstructed maps, while also delivering strong performance in 3D open-vocabulary semantic
segmentation and scene reconstruction. We further deploy DGSG-Mind on real-world
robots to demonstrate its target-oriented
reasoning and dynamic update capabilities. The project page of DGSG-Mind is available at https://icr-lab.github.io/DGSG-Mind
Read the full abstract ▾
All authors Luzhou Ge, Xiangyu Zhu, Jinyan Liu, Xuesong Li
Computer Vision · Cross-listed from cs.CR
Ciphera: A Decentralised Biometric Identity Framework
Ankit Kanaiyalal Prajapati, Shahzad Memon, Mohammed Mahir Rahman et al. (+1 more)
Centralised biometric identity systems expose users to single points of failure, opaque verification processes, and irreversible biometric compromise. Decentralised Identifiers (DIDs) and Verifiable Credentials (VCs) offer stronger
privacy guarantees, yet their integration with biometric authentication and distributed verification remains insufficiently explored. This paper presents Ciphera, a decentralised biometric identity framework combining
privacy-preserving facial recognition, multi-node verification, IPFS-based credential metadata storage, and blockchain-anchored revocation. Evaluated across functional, performance, security, and distributed consistency dimensions, Ciphera achieved an 81% functional success rate, with stable enrolment and authentication but measurable revocation propagation delays and occasional audit-log inconsistencies. Performance testing demonstrated sub-second p95 verification latency of approximately 820ms under concurrent multi-node conditions. Security analysis confirmed strong confidentiality and integrity guarantees, though incomplete liveness detection leaves susceptibility to deepfake and replay attacks. The results demonstrate the feasibility of decentralised biometric identity while identifying key engineering challenges for production-grade deployment.
Read the full abstract ▾
All authors Ankit Kanaiyalal Prajapati, Shahzad Memon, Mohammed Mahir Rahman, Ameer Al-Nemrat
Computer Vision
Masked Diffusion Vision-Language Models for Temporal Action Localization
Fengshun Wang, Zhengbo Zhang and Zhigang Tu
Temporal action localization (TAL) requires recognizing the target event and localizing its start and end times precisely in untrimmed
videos. Recent
vision-
language formulations improve semantic
reasoning and support
language-conditioned outputs, but their autoregressive decoders still generate
tokens from left to right, preventing later semantic evidence from revising earlier timestamp predictions. We adapt masked
diffusion vision-
language models (MDVLMs) to TAL so that semantic
tokens and boundary
tokens remain editable throughout iterative denoising with bidirectional
attention, allowing temporal boundaries and semantic content to be refined jointly. Direct adaptation, however, creates two TAL-specific mismatches: standard masked
diffusion training corrupts all positions uniformly at random, but the time
tokens are more reliable when enough semantic context is available; and
token-level cross-
entropy does not reflect temporal IoU. To address these mismatches, we introduce a Planned Training Objective that uses boundary-aware masking and step-weighted reconstruction to rehearse the late recovery of time
tokens, together with a Step-Level IoU Reward that provides overlap-aware supervision during denoising. A standard sequence-level cross-
entropy term provides the base reconstruction signal. Experiments on ActivityNet-RTL, ActivityNet-1.3, and THUMOS-14 show that MDVLM-TAL improves both temporal
reasoning and boundary localization over autoregressive
vision-
language baselines, with especially strong gains under stricter temporal IoU criteria. The source code is available at https://github.com/ycwfs/MDVLM-TAL .
Read the full abstract ▾
Computer Vision
Building and Road Recognition in Dense Urban Informal Settlements: A Dataset and Benchmark
Hongyu Long, Jiaxuan Liu and Rui Cao
As a widespread form of informal settlements, urban villages present significant challenges for sustainable urban development and governance. Precise mapping of their infrastructure is essential, however, existing remote sensing
datasets primarily focus on formal urban environments, lacking fine-grained annotated data for the high-density building patterns and narrow road networks typical of urban villages. To address this gap, we introduce the DenseUIS
dataset, the first high-resolution remote sensing
dataset specifically designed for building and road extraction in extremely dense urban informal settlements, covering 126 urban villages across Shenzhen and Guangzhou in China. Furthermore, we conduct a comprehensive evaluation of state-of-the-art deep learning models on this
dataset. Experimental results reveal the limitations of existing methods in handling the unique morphological patterns of dense informal settlements, underscoring the need for specialized approaches. DenseUIS therefore provides a robust
benchmark for advancing fine-grained urban mapping in complex and high-density informal environments. The
dataset is publicly available at https://github.com/rui-research/DenseUIS.
Read the full abstract ▾
Computer Vision
Parameter-Efficient Subspace Decoupling ViT for Mitigating Multi-Task Negative Transfer in Histological Scoring
Youhan Huang, Jiajun Li, Yilin Fang et al. (+2 more)
Histological scoring is essential for diagnosing Non-Alcoholic Fatty Liver Disease (NAFLD), yet its automation remains challenging due to the high annotation cost and negative transfer among the strongly correlated NAFLD Activity Score (NAS) indicators in multi-task learning. To address this issue, we propose a subspace-decoupled multi-task
Vision Transformer (ViT) that integrates lightweight task-specific Adapters with orthogonality-based constraints. This design constructs independent feature subspaces for steatosis, ballooning, and inflammation, effectively reducing task interference while retaining shared representations. We further construct a curated multi-task mouse NAFLD histology
dataset with expert annotations for all NAS components. Experimental results demonstrate that the proposed method improves multi-task stability and generalization with substantially reduced computational cost compared to training separate single-task models. The code and the curated
dataset have been prepared and will be made publicly available upon acceptance to support reproducibility.
Read the full abstract ▾
All authors Youhan Huang, Jiajun Li, Yilin Fang, Shuai Wang, Chuheng Li
Computer Vision
Fairness Beyond Demographics: Optimizing Performance Across Appearance-Based Hidden Cohorts in Medical Imaging
Milad Masroor, Cuong Nguyen, Kevin Wells et al. (+1 more)
Medical image analysis models can exhibit performance disparities across patient subgroups, threatening clinical safety and
fairness. Existing methods typically address this issue by optimizing accuracy and
fairness metrics for visible demographic attributes (e.g., sex or age) considered in isolation. This strategy not only overlooks potentially more informative latent stratifications, which may reveal deeper sources of model error and inequity, but also fails to scale when multiple demographic attributes are considered simultaneously due to the resulting sparsity of training data within each subgroup. We deal with these issues by introducing the label-free hidden-cohort
fairness (LHCF) training paradigm that instead of maximizing
fairness over visible demographic attributes, it optimizes
fairness across latent subpopulations discovered from image appearance. By clustering images into K appearance-based cohorts and applying
fairness optimization over them, LHCF uncovers underlying sources of model error and avoids the combinatorial sparsity of multi-demographic attributes, reducing disparities across both single and multiple demographic attributes. We demonstrate on our proposed
fairness benchmark, HIDFairBench, that LHCF provides state-of-the-art
fairness results on single and multiple demographic attributes, despite never using demographic labels for training. Our results position hidden-cohort
fairness as a practical, scalable, and robust alternative to demographic-based
fairness optimization for trustworthy medical image analysis.
Read the full abstract ▾
All authors Milad Masroor, Cuong Nguyen, Kevin Wells, Gustavo Carneiro
Computer Vision
Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language
Xiang Fang, Wanlong Fang, Daizong Liu et al. (+8 more)
Video Moment
Retrieval (VMR) targets to retrieve the specific moment corresponding to a sentence query from an untrimmed
video. Although recent works have made remarkable progress in this task, they implicitly are rooted in the closed-set assumption that all the given queries as
video-relevant\footnote{In this paper, we treat “
video-relevant query” as “in-distribution (ID) query” and “
video-irrelevant query” as “out-of-distribution (OOD) query”.}. Given an OOD query in open-set scenarios, they still utilize it for wrong
retrieval, which might lead to irrecoverable losses in high-risk scenarios, e.g., criminal activity detection. To this end, we creatively explore a brand-new VMR setting termed Open-Set
Video Moment
Retrieval (OS-VMR), where we should not only retrieve the precise moments based on ID query, but also reject OOD queries. In this paper, we make the first attempt to step toward OS-VMR and propose a novel model OpenVMR, which first distinguishes ID and OOD queries based on the normalizing flow technology, and then conducts moment
retrieval based on ID queries. Specifically, we first learn the ID distribution by constructing a normalizing flow, and assume the ID query distribution obeys the multi-variate Gaussian distribution. Then, we introduce an uncertainty score to search the ID-OOD separating boundary. After that, we refine the ID-OOD boundary by pulling together ID query features. Besides,
video-query matching and frame-query matching are designed for coarse-grained and fine-grained cross-modal interaction, respectively. Finally, a positive-unlabeled learning module is introduced for moment
retrieval. Experimental results on three VMR
datasets show the effectiveness of our OpenVMR.
Read the full abstract ▾
All authors Xiang Fang, Wanlong Fang, Daizong Liu, Xiaoye Qu, Jianfeng Dong, Pan Zhou, Renfu Li, Zichuan Xu, Lixing Chen, Panpan Zheng, Yu Cheng
Computer Vision · Cross-listed from cs.CR
Cert-LAS: Toward Certified Model Ownership Verification for Text-to-Image Diffusion Models via Layer-Adaptive Smoothing
Leyi Qi, Yiming Li, Siyuan Liang et al. (+2 more)
Large-scale text-to-image (T2I)
diffusion models have enabled unprecedented creative applications, but their unauthorized use has raised serious intellectual property concerns, making model ownership verification (MOV) increasingly critical. We find that existing
backdoor-based
diffusion watermarking methods often (implicitly) assume a "faithful" verification process, namely, that the verifier can query a suspicious model and obtain the faithful watermark response to complete MOV. However, in practice, adversaries may intentionally or unintentionally damage potential watermark signals, significantly degrading verification reliability. To address this issue, we propose Cert-LAS, the first certified MOV method for T2I models based on layer-adaptive smoothing. In general, Cert-LAS embeds specified watermarks using
diffusion classifiers and an LFS-guided layer-adaptive noise, and verifies ownership by examining whether the suspected model exhibits significantly stronger watermark responses compared to unwatermarked references through hypothesis testing. We further prove that, under certain conditions, our Cert-LAS can still achieve reliable verification even in the presence of malicious removal attacks. Extensive experiments validate the effectiveness of Cert-LAS and its resistance to adaptive attacks. Our code is available at https://github.com/Leyi-Qi/Cert-LAS.
Read the full abstract ▾
All authors Leyi Qi, Yiming Li, Siyuan Liang, Zhengzhong Tu, Dacheng Tao
Computer Vision · Cross-listed from cs.AI
AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security
Dongrui Liu, Yu Li, Zhonghao Yang et al. (+47 more)
Modern open-world
agents such as OpenClaw exhibit powerful cross-environment execution capabilities yet introduce broad new safety risk sources. Meanwhile, advanced frontier AI models drastically lower attack barriers, rendering current
agent alignment frameworks inadequate for real-world deployment. To tackle these emerging threats, we propose a lightweight and scalable
agent safety
alignment framework. Specifically, we update the
agent safety taxonomy to accommodate emergent risks from Codex and OpenClaw execution scenarios. We further build a taxonomy-guided data engine with influence-function purification to train lightweight AgentDoG 1.5 variants (0.8B, 2B, 4B, and 8B parameters) using only around 1k samples, achieving comparable performance with leading closed-source models (e.g., GPT-5.4). Based on AgentDoG 1.5, we construct a highly efficient agentic safety SFT and RL training environment, which reduces deployment overhead in Docker-level environments by two orders of magnitude. Finally, we deploy AgentDoG 1.5 as a training-free online guardrail for real-time safety moderation. Extensive experimental results indicate that AgentDoG 1.5 achieves state-of-the-art performance in diverse and complex interactive agentic scenarios. All models and
datasets are openly released.
Read the full abstract ▾
All authors Dongrui Liu, Yu Li, Zhonghao Yang, Peng Wang, Guanxu Chen, Yuejin Xie, Qinghua Mao, Wanying Qu, Yanxu Zhu, Tianyi Zhou, Leitao Yuan, Zhijie Zheng, Qihao Lin, Yimin Wang, Haoyu Luo, Shuai Shao, Chen Qian, Qingyu Liu, Ling Tang, Ruiyang Qin, Qihan Ren, Junxiao Yang, Kun Wang, Zhiheng Xi, Linfeng Zhang, Ranjie Duan, Bo Zhang, Wenjie Wang, Wen Shen, Qiaosheng Zhang, Yan Teng, Chaochao Lu, Rui Mei, Man Li, Jialing Tao, Xi Lin, Tianhang Zheng, Yong Liu, Quanshi Zhang, Lei Zhu, Xingjun Ma, Junhua Liu, Hui Xue, Xiaoxiang Zuo, Xiangnan He, Chao Shen, Xianglong Liu, Minlie Huang, Jing Shao, Xia Hu
Computer Vision
Low-Magnification SEM May Suffice: Interpretable Deep Learning for Multi-Scale Fracture-Cause Classification in Zirconia-Toughened Alumina
Julian Schmid, Pawel Astankow, Tom Vater et al. (+3 more)
Reliable identification of fracture origins in alumina
matrix composite hip and knee implants is critical for quality assurance and patient safety, yet current fractographic workflows are time-consuming, partly subjective, and reliant on high-magnification scanning electron microscopy (SEM). We present an interpretable
vision-
transformer (ViT) workflow for automated classification of fracture causes in an alumina
matrix composite (BIOLOX delta, CeramTec GmbH) widely used in total joint replacements. A
dataset of 8,493 SEM images (50x-10,000x) was curated from five years of in-production burst and proof tests and annotated into three defect categories defined along the manufacturing chain: green body, hard machining, and material defects. Under severe class imbalance, the fine-tuned ViT reached an accuracy of 0.907 and a macro-F1 of 0.888 in stratified five-fold cross-validation, with a two-stage perceptual-hash/SSIM leakage audit confirming negligible specimen overlap. Notably, performance at low magnification (50x) was comparable to that at high magnification (1k-10kx), indicating that macro-scale features - mirror geometry and hackle line fields - already encode sufficient diagnostic signal. Grad-CAM attributions consistently localised on canonical fractographic cues (mirrors, hackles, pores, machining marks), aligning with established fractographic criteria. Together, these results position interpretable ViTs as a complementary tool for ceramic-implant quality assurance, enabling low-magnification pre-screening and reducing reliance on time-intensive high-magnification inspection.
Read the full abstract ▾
All authors Julian Schmid, Pawel Astankow, Tom Vater, Julius Beck, Robert Cichon, Danny Krautz
Computer Vision
Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language
Xiang Fang, Daizong Liu, Wanlong Fang et al. (+5 more)
Given an untrimmed
video and a sentence query,
video moment
retrieval using
language (VMR) aims to locate a target query-relevant moment. Since the untrimmed
video is overlong, almost all existing VMR methods first sparsely down-sample each untrimmed
video into multiple fixed-length
video clips and then conduct multi-modal interactions with the query feature and expensive clip features for
reasoning, which is infeasible for long real-world
videos that span hours. Since the
video is downsampled into fixed-length clips, some query-related frames may be filtered out, which will blur the specific boundary of the target moment, take the adjacent irrelevant frames as new boundaries, easily leading to cross-modal misalignment and introducing both boundary-bias and
reasoning-bias. To this end, in this paper, we propose an efficient approach, SpotVMR, to trim the query-relevant clip. Besides, our proposed SpotVMR can serve as plug-and-play module, which achieves efficiency for state-of-the-art VMR methods while maintaining good
retrieval performance. Especially, we first design a novel clip search model that learns to identify promising
video regions to search conditioned on the
language query. Then, we introduce a set of low-cost semantic indexing features to capture the context of objects and interactions that suggest where to search the query-relevant moment. Also, the
distillation loss is utilized to address the optimization issues arising from end-to-end joint training of the clip selector and VMR model. Extensive experiments on three challenging
datasets demonstrate its effectiveness.
Read the full abstract ▾
All authors Xiang Fang, Daizong Liu, Wanlong Fang, Pan Zhou, Zichuan Xu, Wenzheng Xu, Junyang Chen, Renfu Li
Computer Vision
Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning
Shuai Yi, Yixiong Zou, Yuhua Li et al. (+1 more)
Vision-
Language Models (VLMs) such as CLIP demonstrate strong zero-shot generalization, but their performance significantly degrades in cross-domain scenarios with scarce target-domain training data (Cross-Domain Few-Shot Learning, CDFSL). In this paper, we focus on the target-domain few-shot finetuning in the CLIP-based CDFSL task. Prevailing finetuning paradigms uniformly align all image patch
tokens with their corresponding textual
embeddings. However, we find a counterintuitive phenomenon: actively pushing away certain low-similarity image
tokens, termed "tail
tokens", from their textual
embeddings consistently improves target-domain performance. We delve into this phenomenon and provide a novel interpretation: under great domain shifts and scarce training data, the model can hardly extract semantic information from visual inputs; therefore, the common belief of
alignment is valid only for
tokens already containing sufficient semantic information; for tail
tokens, forcing the
alignment would lead to excessive overfitting to the scarce training, while breaking the
alignment is more useful. Motivated by this, we propose Adaptive Tail-Head
Alignment (ATHA), a novel fine-tuning strategy for CLIP that transforms the conventional uniform
alignment paradigm to an adaptive
alignment paradigm, with both
alignment strengthening and weakening. Extensive experiments on four challenging CDFSL
benchmarks validate our state-of-the-art performance. Our code is available at https://github.com/shuaiyi308/ATHA.
Read the full abstract ▾
All authors Shuai Yi, Yixiong Zou, Yuhua Li, Ruixuan Li
Computer Vision
Energy-Aware NECO for Single-Pass Pixel-wise Out-of-Distribution Detection in Semantic Segmentation
Boyuan Zhang, Huanshan Huang and Yifei Cao
Reliable semantic
segmentation for mobile
robots requires both accurate dense prediction and robust uncertainty estimation under distribution shift. Strong uncertainty baselines such as Monte Carlo Dropout often require repeated
stochastic forward passes and are difficult to deploy on edge platforms. We propose Energy-Aware NECO, a single-pass pixel-wise out-of-distribution (OOD) detector for semantic
segmentation. The method combines a centered NECO-style
geometric ratio computed from decoder features with a logit-based Energy score. Both components are standardized using statistics fitted on a pure in-distribution validation split and fused through a convex combination. We evaluate the method on the miniMUAD subset using true pixel-level OOD labels. The proposed hybrid score achieves an AUROC of 0.8539, outperforming NECO-only (0.8280), Energy-only (0.8171), and an ensemble predictive-
entropy baseline (0.8124). Additional qualitative and operating-point analyses show that the hybrid detector improves overall ranking performance while preserving the efficiency advantages of a single-pass design. Code is available at https://github.com/boyuan-zhangx/Energy-Aware_NECO
Read the full abstract ▾
Computer Vision
GeoMag: Geometric-Aware Video Motion Magnification via State Space Model
Kecheng Han, Yuchen Zhang, Bingqing Liu et al. (+3 more)
Video Motion Magnification (VMM) reveals imperceptible dynamics but often suffers from structural inconsistencies under complex
geometric transformations. Existing learning-based methods generally face a trade-off between the limited global context of CNNs and the high computational cost of
Transformers. In addition, current training protocols, largely dominated by simple linear motion, fail to capture the
geometric and imaging complexities encountered in real-world
videos. To address these issues, we propose GeoMag, a
geometric-aware VMM framework built upon State Space Models to achieve globally consistent motion amplification with linear complexity. We further construct Geo-200K, a large-scale synthetic
dataset that introduces rich
geometric transformations together with sensor-realistic degradations, improving the diversity and realism of training signals. Extensive experiments on synthetic and real-world
benchmarks show that GeoMag consistently outperforms prior methods in visual fidelity and computational efficiency, while producing fewer artifacts and better structural consistency.
Read the full abstract ▾
All authors Kecheng Han, Yuchen Zhang, Bingqing Liu, Boqiang Guo, Wenbin Zheng, Shiyuan Pei
Computer Vision
S2MDF: A Plug-And-Play Layer for Intersection-Free Multi-Object Signed Distance Fields
Deniz Sayin Mercadier, Federico Stella, Aurel Bizeau et al. (+2 more)
Compositional implicit surface representations model scenes as collections of objects, each encoded by a Signed Distance Field (SDF). A fundamental limitation of this approach is that multiple SDFs can produce geometries that interpenetrate, violating physical plausibility. Existing mitigation strategies rely on soft penalty terms that reduce but do not eliminate intersections, and require careful loss weighting. To truly prevent interpenetration, we propose a hard constraint on vector-valued SDFs and introduce S2MDF, a lightweight plug-and-play module that enforces the constraint on any object-compositional SDF representation without architectural modifications. It introduces negligible computational overhead and is compatible with linearly-interpolated standard meshing
algorithms such as Marching Cubes. It can be applied during training or as a post-processing step. Experiments on multiple state-of-the-art compositional methods show that S2MDF reduces intersections to numerical precision while preserving reconstruction quality, outperforming existing mitigation strategies.
Read the full abstract ▾
All authors Deniz Sayin Mercadier, Federico Stella, Aurel Bizeau, Nicolas Talabot, Pascal Fua
Computer Vision
SLAD : Shared LoRA Adapters for Task Specific Distillation
Reda Bensaid, Yassir Bendou, Vincent Gripon et al. (+1 more)
In the context of resource-constrained environments such as embedded systems, adapting reduced-size foundation models to downstream tasks has become increasingly popular. This has recently motivated the emerging setting of task-specific
distillation, where a larger and a smaller version of the same foundation model are both adapted to the same downstream task, with the goal of transferring knowledge from the former to the latter. Recent work has demonstrated the benefits of using a larger version of the same foundation model to assist the adaptation of a smaller one. Typically, the larger model (teacher) is first adapted via fine-tuning or linear probing before its knowledge is distilled into the smaller model (student). While fine-tuning the teacher often increases its performance, recent work showed that probing it leads to better knowledge
distillation to the student. Our findings show that this is mainly due to a mis-
alignment in feature representation between the teacher and the student which occurs during the teacher's fine-tuning. Inspired by existing efforts to preserve previously learned knowledge, we first propose leveraging low-rank adaptation, resulting in better feature
alignment and therefore better knowledge transfer. Drawing from this insight, we further enhance the feature
alignment through a parameter-sharing strategy of the adapters between the two encoders during joint training. Our proposed method, SLAD, shows better feature
alignment between the teacher and student, which results in increased performance for not only the student but also the teacher model, while being 2x faster to train than fine-tuning. Through extensive experiments on multiple classification and
segmentation datasets, we demonstrate the improved accuracy and transfer efficiency of our method, achieving state-of-the-art performance in the task-specific
distillation framework.
Read the full abstract ▾
All authors Reda Bensaid, Yassir Bendou, Vincent Gripon, François Leduc-Primeau
Computer Vision
Efficient, Validation-Free Intrinsic Quality Estimation for Large-Scale Face Recognition Datasets
Zhichao Chen, Yongle Zhao, Kaicheng Yang et al. (+3 more)
We propose Intrinsic Quality (IQ), a validation-free metric designed to estimate the inherent potential of face recognition (FR)
datasets to produce high-performance models without the need for full-scale training. IQ integrates two components: (i) a Neighbor-Consistency Score that quantifies local identity label agreement via nearest neighbors, and (ii) Global Representation Subspace Complexity (Effective Rank, ER), which captures the underlying
embedding geometry and
dataset diversity. IQ allows for rapid evaluation using lightweight proxy models or data subsets, facilitating
dataset diagnosis and curation prior to resource-intensive full-scale training. We describe an experimental protocol tailored to clean, noisy, and mixed-quality FR
datasets, and outline evaluation methodologies to validate IQ's predictive power for downstream performance.
Read the full abstract ▾
All authors Zhichao Chen, Yongle Zhao, Kaicheng Yang, Meng Yang, Yin Xie, Ziyong Feng
Computer Vision · Cross-listed from q-bio.NC
Subcortical Shape Variations and Their Associations with Cognition Across the 8th Decade of Life. A Study in the Lothian Birth Cohort 1936
Maria del C. Valdes-Hernandez, Wonjung Park, Joanna Moodie et al. (+7 more)
The study of
brain morphology changes in normal individuals may capture aspects of functionally-relevant
brain aging not fully indicated by gross volumetry. Despite the important role of subcortical
brain structures in cognition, the associations between their morphological trajectories and cognitive changes in aging have not been documented. We use neuroimaging, demographic, and cognitive data from a large longitudinal study of cognitive aging, the Lothian Birth Cohort 1936, to explore shape changes in subcortical
brain structures of community-dwelling individuals across their 8th decade of life. We investigate the association of these changes with cognitive aging using ANCOVA and mixed linear model analyses. Subcortical shape changes were heterogeneous, with varied atrophy patterns across whole period. The hippocampus and the ventral DC experienced varied morphological deformations (from its baseline point) different in left and right hemispheres, while the thalami and globus pallidi shapes, for example, experienced a more uniform volume contraction, nearly symmetrical throughout different timelines. Changes in general cognition were mainly associated with inwards and outwards vertex displacements between the time-points.
Read the full abstract ▾
All authors Maria del C. Valdes-Hernandez, Wonjung Park, Joanna Moodie, Susana Muñoz Maniega, Janie Corley, Fraser N. Sneden, Mark E. Bastin, Joanna M. Wardlaw, Simon R. Cox, Jinah Park
Computer Vision
Unsupervised Semantic Segmentation Facilitates Model Understanding
Xiaoyan Yu, Lisa Mais, Jannik Franzen et al. (+4 more)
Self-supervised learning (SSL) has produced a diverse landscape of
vision transformers (ViTs) whose pretrained representations support a wide range of downstream tasks. Towards a better understanding of these models, a body of work has assessed the mechanics of their self-
attention as well as the types of information captured across their representations, revealing, for example, stark differences between models trained with contrastive learning (CL) and masked image modeling (MIM). However, the total of these advances on model understanding has to date not yet fully permeated a larger community, where, e.g., insights that are specific to CL models are still at times generalized to MIM models. To make model understanding straightforward and intuitive for a broad community, we propose a simple and easily interpretable visualization protocol. Our protocol is based on visualizing unsupervised semantic
segmentation results, yet by no means do we focus on top
segmentation performance. Instead, our protocol allows us to easily convey model behavior that consistently emerges across images. Benchmarked on a diverse set of SSL models across layers and representations, our protocol allows us to gain novel insights into distinct positional biases and scaling behaviors, including, e.g., strong boundary artifacts in DINOv3-Large model
tokens. These novel insights come on top of more easily conveying a range of previous findings. Our protocol further allows us to clearly visually convey and distinguish between positional effects and the closely related but distinct locality bias, the latter being much more extensively studied in the literature so far. Our protocol is publicly available, serving to catalyze further model understanding for a broad community.
Read the full abstract ▾
All authors Xiaoyan Yu, Lisa Mais, Jannik Franzen, Peter Hirsch, Nick Lechtenbörger, Andreas Mardt, Dagmar Kainmüller
Computer Vision · Cross-listed from cs.LG
A Geometric View of SRC: Learning Representations for Stable Residual Inference
Vangelis P. Oikonomou
Reconstruction-based inference assigns a class by comparing class-wise reconstruction residuals; Sparse Representation Classification (SRC) is a canonical instance whose reliability depends on the geometry of the learned representation. We adopt a strict training-inference separation: SRC is used only as a fixed test-time rule and is never differentiated, unrolled, or optimized during training. In a span-level idealization based on class-conditional spans and their associated projection residuals, we formalize residual-ordering stability through a residual margin and characterize
geometric obstructions – span overlap, dominance, and near-overlap via small principal angles – that can collapse this margin in worst-case directions. This span-level theory is primary: it specifies when the idealized residual family is well-separated, and it provides a conditional solver-level interpretation for practical residual approximations (e.g., OMP) insofar as they remain close to the span-level residual ordering. Under explicit coverage and separation assumptions, we derive a quantitative lower bound on the (idealized) residual margin. Guided by these targets, we propose geometry-shaping objectives that promote masked within-class self-expressiveness, discourage cross-class reconstruction pathways and inter-class span
alignment, and prevent collapse – without invoking SRC residuals or predictions during training. Experiments on images (COIL-100), text (TREC), and
EEG connectivity evaluate all representations under identical fixed SRC/OMP inference and report residual margins and
geometric diagnostics; cross-
entropy is included only as a reference geometry under the same evaluation protocol.
Read the full abstract ▾
Computer Vision
SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation
Shilin Ma, Chubin Zhang, Changyuan Wang et al. (+6 more)
Real-time inference of
vision-
language-action (VLA) models is essential for robotic control. While visual
token pruning has shown strong potential for accelerating inference, most existing methods mainly base
pruning decisions on shallow-layer cues and risk discarding visual information required by deep layers. To address this issue, we propose SAFE-Pruner, a plug-and-play
pruning framework that incorporates
attention cues of future layers into
pruning decisions. Specifically, we identify semantic
attention consistency, the tendency that VLA models concentrate their
attention probability mass on the same semantic entity across control timesteps. Based on this observation, we design a forward-looking strategy to forecast the
token saliency in deep layers, which prevents the premature removal of critical
tokens and leads to more stable acceleration. We further introduce a reference timestep refresh strategy that triggers updates upon
attention shifts, thereby improving
forecasting accuracy and
pruning reliability. Extensive experiments across diverse evaluation settings demonstrate that our method achieves up to 1.89x speedup with a minimal degradation in success rate of less than 1.5%, while outperforming state-of-the-art methods by up to 1.9%.
Read the full abstract ▾
All authors Shilin Ma, Chubin Zhang, Changyuan Wang, Yuji Wang, Yue Wu, Zixuan Wang, Jingqi Tian, Zheng Zhu, Yansong Tang
Computer Vision
Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning
Yiyao Ma, Kai Chen, Zhongxiang Zhou et al. (+5 more)
Monocular 3D shape recovery is fundamental to
geometric understanding, yet achieving robust generalization across arbitrary viewpoints and unseen object categories remains a significant challenge. In this paper, we present a generalizable deformation learning framework that reconstructs 3D objects by explicitly deforming a category-level shape template to match the target observation. To address complex shape variations between the template and the target, we introduce a geometry-guided feature modeling mechanism. This process first enriches foundation features with template
topology to yield a geometry-aware representation, which is then explicitly correlated with the target observation to guide precise deformation. Furthermore, to bridge the disparity between the fixed template and arbitrary target views, we propose a view-adaptive feature aggregation module. This module leverages multi-view template features and their corresponding camera poses to enrich the canonical template representation, ensuring robust feature
alignment regardless of the target's perspective. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods in handling large shape variations and diverse viewpoints, exhibiting strong generalization to novel categories and effectively supporting downstream real-world dexterous robotic manipulation tasks. Project homepage: https://GODeform.github.io/
Read the full abstract ▾
All authors Yiyao Ma, Kai Chen, Zhongxiang Zhou, Zhuheng Song, Dongsheng Xie, Zelong Tan, Rong Xiong, Qi Dou
Computer Vision
OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
Geng Li, Guohao Chen, Ting Chen et al. (+6 more)
Vision-
language models (VLMs) rely on long visual
token sequences for visual understanding, making the prefill stage expensive in both computation and
memory. Most existing
pruning methods follow an absolute-ranking paradigm, assigning importance scores to visual
tokens and retaining a fixed top-K subset. In this work, we argue that this paradigm is fundamentally brittle:
attention sinks distort
token importance rankings, while image redundancy and query-dependent visual evidence make fixed
token budgets unreliable across inputs. We propose OccamToken, a training-free framework that replaces absolute
token ranking with register-anchored relative evidence testing. Instead of asking which
tokens are globally important, OccamToken evaluates whether a visual
token provides information beyond a register-based reference. Our key insight is that register
tokens naturally absorb low-information
attention patterns, making them a stable reference for identifying genuinely informative visual evidence. Based on this principle, OccamToken performs both image-adaptive redundancy
pruning and query-adaptive relevance
pruning through dynamic thresholds derived from register
attention. Across LLaVA-NeXT, LLaVA-v1.5, and Qwen3-VL, OccamToken consistently improves the accuracy-efficiency trade-off without additional training. Notably, on LLaVA-NeXT, it reduces 2,880 visual
tokens to approximately 40 while preserving over 93% of the original accuracy, enabling stable visual
token compression even in the extreme 1.4% retention regime.
Read the full abstract ▾
All authors Geng Li, Guohao Chen, Ting Chen, Shilin Shan, Kuangji Zuo, Bofan Lyu, Tuo An, Gen Li, Jianfei Yang
Computer Vision
SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation
Yuan Li, Congyi Zhang, Xifeng Gao et al. (+1 more)
Autoregressive
multimodal large
language models (MLLMs) enable 3D generation but struggle to scale to high-resolution shapes due to inadequate 3D tokenizations. Compact set-based representations discard deterministic spatial ordering, leading to ambiguous sequence prediction, while uniform or octree-based voxel grids preserve ordering at the cost of severe redundancy and excessively long sequences. This structural trade-off limits stable and efficient autoregressive 3D generation. We present SuperVoxelGPT, a representation-first framework that resolves this tension through adaptive and deterministically ordered supervoxel tokenization. Given a
prompt, we first predict a coarse
geometric saliency distribution and construct a shape-adaptive supervoxel partition using saliency-guided centroidal Voronoi tessellation, allocating fine-grained
cells to complex regions and larger
cells to smooth regions. Conditioned on this
prompt and ordered supervoxel layout, we introduce a SuperVoxelVAE and fine-tune a pretrained MLLM to autoregressively generate supervoxel
tokens. Experiments using Trellis-500K data show that SuperVoxelGPT reduces
token sequence length to 12.8% of uniform voxel tokenization while achieving state-of-the-art generation quality and an average 10x speedup over prior methods.
Read the full abstract ▾
All authors Yuan Li, Congyi Zhang, Xifeng Gao, Xiaohu Guo
Computer Vision
MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data
Dario Pisanti and Georgios Georgakis
Aerial
navigation on Mars requires
vision-based pipelines that are robust to the diverse illumination conditions and terrain morphology of the Martian surface. A key bottleneck for training and evaluating such methods is the scarcity of large-scale, annotated aerial
datasets. We present MARTIAN, an open-source Blender-based rendering framework that leverages real HiRISE orbital map products to synthesize realistic aerial views of the Martian terrain under controllable lighting conditions and at varying altitudes. MARTIAN generates observations with accurate pose annotations, directly addressing the scarcity of training data for
vision-based
navigation on Mars. The framework has been validated through its deployment in concurrent work on map-based localization systems for Ingenuity and future Mars rotorcraft, where synthetically trained deep image matchers were successfully evaluated on real Mars imagery. MARTIAN is publicly available at: https://github.com/nasa-jpl/martian.
Read the full abstract ▾
Computer Vision
AgentCVR: Active Multi-Agent Cross-Video Reasoning via Script-Simulated Reinforcement Learning
Yilun Qiu, Jiahe Wang, Cilin Yan et al. (+4 more)
Cross-
Video Reasoning (CVR) has emerged as a critical frontier in
multimodal intelligence, requiring models to retrieve, align, and aggregate evidence distributed across multiple
videos. Current
Multimodal Large
Language Models (MLLMs) often struggle with CVR, as simple single-pass strategies encode multiple
videos into a shared compressed context, potentially obscuring rare but critical evidence. In this paper, we propose AgentCVR, a multi-
agent framework that treats CVR as an active evidence-acquisition task. AgentCVR employs a Master
Agent to iteratively coordinate specialized Visual and Audio
Agents for targeted evidence extraction. To ensure efficient training, we introduce Script-Simulated RL, which optimizes the
agent's policy with LLM-generated semantic scripts and a lightweight text-based simulator, bypassing costly
multimodal inference during online exploration. Experimental results on a comprehensive CVR
benchmark show that AgentCVR outperforms single-pass baselines and achieves comparable performance to state-of-the-art closed-source systems, particularly in complex cross-
video alignment and localization. To ensure reproducibility, our code is available at https://github.com/wang-jh24/AgentCVR.
Read the full abstract ▾
All authors Yilun Qiu, Jiahe Wang, Cilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, Chun Yuan
Computer Vision
DiffSpot: Can VLMs Spot Fine-Grained Visual Differences in Web Interfaces?
Linhao Zhang, Aiwei Liu, Yuan Liu et al. (+1 more)
Vision-
language models (VLMs) have made strong progress on high-level image-text
alignment, yet their ability to perceive subtle visual differences remains limited. We study this problem in rendered web interfaces, where localized visual changes are both a diagnostic test of fine-grained perception and a practical requirement for GUI
agents and design tools. We introduce DiffSpot, a code-driven
benchmark for open-ended spot-the-difference on web interfaces. DiffSpot constructs controlled image pairs by mutating a single CSS property of a target element in self-contained HTML, re-rendering the page, and recording the changed property, element, and mutation magnitude. A grounding gate retains only pairs whose rendered pixel difference is confined to the target element. The
benchmark contains 4{,}400 pairs, including 3{,}900 has-diff pairs balanced across 13 CSS-property operators and three difficulty tiers, plus 500 no-diff pairs for
hallucination control. Evaluating 13 frontier VLMs zero-shot, we find that even the best model identifies only $40.7\%$ of true changes, with Hard-tier Recall below $23\%$ for every model. DiffSpot further shows that difficulty is strongly property-dependent: across CSS operators, neither pixel magnitude nor CLIP distance reliably predicts Recall.
Read the full abstract ▾
All authors Linhao Zhang, Aiwei Liu, Yuan Liu, Xiao Zhou
Computer Vision
Learning Context-Conditioned Predicate Semantics via Prototype Feedback
NamGyu Jung and Chang Choi
In scene graph generation, a central challenge is modeling polysemous predicates whose meanings shift across contexts. Prior approaches address this issue by decomposing predicates into multiple static prototypes or retrieving semantically similar exemplars. However, these strategies keep predicate representations static and cannot reorganize semantics to reflect image-specific evidence, leading to systematic confusions in ambiguous contexts. We propose AlignG, which learns context-conditioned predicate semantics via prototype feedback. AlignG infers context-conditioned predicate semantics from the relation candidates within each image and feeds the adapted semantics back to recalibrate relation representations. The learning objective anchors this adaptation to global semantic centers, preventing semantic drift while still allowing selective reorganization when the scene provides consistent relational cues. Experiments on VG-150 and GQA-200 show consistent improvements over state-of-the-art baselines, with F@100 improvements of +1.4 on VG-150 and +2.7 on GQA-200 under SGDet. We further visualize per-image prototype similarity shifts and observe coherent context-dependent reorganization where prototypes selectively merge or separate predicates according to scene evidence. The code is available at https://github.com/Namgyu97/AlignG-SGG.pytorch.
Read the full abstract ▾
Computer Vision
CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning
Xiang Fang, Wanlong Fang and Changshuo Wang
Multi-modal
Retrieval-Augmented Generation (MMRAG) has emerged as a powerful paradigm for enhancing
Multimodal Large
Language Models in knowledge-intensive question answering by integrating external visual, textual, and structural knowledge. However, existing MMRAG frameworks suffer from critical limitations, including noisy and irrelevant
retrieval, cross-modal semantic misalignment, lack of adaptive
reasoning, and incoherent generation across local and global contexts. We introduce CogniVerse, a novel MMRAG framework that addresses these challenges through a cognitive-inspired, mathematically rigorous approach. Drawing from human-like
reasoning, CogniVerse integrates three synergistic components: (1) a Cognitive Reflection Module that dynamically assesses
retrieval necessity and filters relevant multi-modal content, reducing noise and computational overhead; (2) a Multi-modal
Retrieval Module that aligns
embeddings in a Riemannian
manifold using information geometry and refines knowledge graphs via spectral graph theory, ensuring precise and coherent
retrieval; and (3) a Hierarchical Generation Module that employs an optimal transport-based loss to balance
token-level accuracy and global semantic coherence. Extensive experiments demonstrate that CogniVerse significantly outperforms state-of-the-art systems in both accuracy and coherence, while reducing
retrieval latency.
Read the full abstract ▾