Computer Vision
FairEnc: A Fair Vision-Language Model with Fair Vision and Text Encoders for Glaucoma Detection
Mohamed Elhabebe, Ayman El-Baz and Qing Liu
Automated glaucoma detection is critical for preventing irreversible
vision loss and reducing the burden on healthcare systems. However, ensuring
fairness across diverse patient populations remains a significant challenge. In this paper, we propose FairEnc, a fair pretraining method for
vision-
language models (VLMs) that enables simultaneous debiasing across multiple sensitive attributes. FairEnc jointly mitigates biases in both textual and visual modalities with respect to multiple sensitive attributes, including race, gender, ethnicity, and
language. Specifically, for the textual encoder, we leverage a large
language model to generate synthetic clinical descriptions with varied sensitive attributes while preserving disease semantics, and employ a contrastive
alignment objective to encourage demographic-invariant representations. For the visual encoder, we propose a dual-level
fairness strategy that combines mutual information regularization to reduce statistical dependence between learned features and demographic groups, with multi-discriminator
adversarial debiasing. Comprehensive experiments on the publicly available Harvard-FairVLMed
dataset demonstrate that FairEnc effectively reduces demographic disparity as measured by DPD and DEOdds while achieving strong diagnostic performance under both zero-shot and linear probing evaluations. Additional experiments on the private FairFundus
dataset show that FairEnc consistently preserves
fairness advantages under cross-domain and cross-modality settings and maintains diagnostic performance within a competitive range. These results highlight FairEnc's ability to generalize
fairness under distribution shifts, supporting its potential for more equitable deployment in real-world clinical settings. Our codebase and synthetic clinical notes are available at https://github.com/Mohamed-Elhabebe/FairEnc
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models
Huatian Zhang, Zhendong Mao, Lei Zhang et al. (+1 more)
Direct Preference Optimization (DPO) has proven to be an effective solution for mitigating
hallucination in
Multimodal Large
Language Models (MLLMs) by learning from preference pairs. One of its key challenges lies in how to transfer the sequence-level preference into fine-grained supervision on visual fidelity. To safeguard
vision-related
tokens that are prone to
hallucination, existing methods typically allocate training emphasis according to the model's self-assessed visual sensitivity signals. However, such sensitivity, estimated by a model still under training, introduces self-referential bias: reinforcing already well-learned visual cues while neglecting hard-to-perceive but critical details, thereby limiting deeper
alignment. In this work, we propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for MLLMs, which enables the model to uncover its cognitive deficiencies and actively explore for self-correction, guided by
token-level epistemic uncertainty. Specifically, we first quantify the uncertainty from the model's failure to ground
token predictions in the given image. Then, based on an uncertainty-aware exploration intensity, we encourage more learning pressure on visually deficient
tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples. Further, we provide a theoretical justification for our method, and extensive experiments demonstrate its effectiveness and robustness.
Read the full abstract ▾
All authors Huatian Zhang, Zhendong Mao, Lei Zhang, Yongdong Zhang
Computer Vision
VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA
Haibin He, Maoyuan Ye, Jing Zhang et al. (+2 more)
Video text-based visual question answering (
Video TextVQA) aims to answer questions by
reasoning over visual textual content appearing in
videos. Despite the strong
multimodal video understanding capabilities of recent
Video-LLMs, their performance on existing
Video TextVQA
benchmarks remains limited. To better understand this gap, we conduct an upper-bound analysis through frame-wise question answering, counting a sample as correct if any frame yields the right answer, which significantly outperforms direct
video-based inference and reveals a substantial performance gap. The results suggest that the primary bottleneck lies in the localization of key question-relevant evidence, rather than in
reasoning capacity itself. Building on this insight, we propose a question-guided
agent framework that explicitly anchors the relevant keyframes before answering. The approach operates effectively in a training-free setting and consistently surpasses direct
video inference. With additional supervised fine-tuning (SFT) and reinforcement learning (RL), it achieves an average improvement of +12.12 in accuracy and +11.15 in ANLS across
benchmarks, establishing new state-of-the-art results. Our study underscores the critical role of explicit keyframe anchoring for advancing
Video TextVQA. The code will be publicly released.
Read the full abstract ▾
All authors Haibin He, Maoyuan Ye, Jing Zhang, Juhua Liu, Bo Du
Computer Vision · Cross-listed from cs.CL
Assessing Cognitive Effort in L2 Idiomatic Processing: An Eye-Tracking Dataset
Eduardo Santos, Juliana Carvalho and César Rennó-Costa
This paper presents the development and validation of an eye-tracking
dataset designed to investigate how second-
language (L2) learners process idiomatic expressions. While native speakers often rely on direct
retrieval of figurative meanings, L2 speakers frequently adopt a literal-first approach, which incurs measurable cognitive costs. This resource captures these costs through ocular metrics recorded from Portuguese L1 speakers of English across all CEFR proficiency levels (A1-C2). Although the study uses entry-level 60 Hz hardware (Tobii Pro Spark), we demonstrate that this sampling rate provides sufficient data density to detect macro-cognitive events such as fixations and regressions in reading. Preliminary analysis validates the
dataset by revealing a strong inverse correlation between
language proficiency and regressive eye movements. Integrated into the MIA (Modeling Idiomaticity in Human and Artificial
Language Processing) initiative, this
dataset serves as a cognitively grounded
benchmark for evaluating both human processing models and the
alignment of large
language models with human-like figurative understanding.
Read the full abstract ▾
Computer Vision
3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network for Real-Time Operator Guidance
Sapna Sachan and Amulya Kumar Mahto
Computed
tomography (CT) is indispensable for clinical diagnosis and image-guided interventions but exposes patients to ionizing radiation, motivating the development of safer imaging alternatives. Ultrasound (US) is non-ionizing and widely accessible; however, it is highly operator dependent and lacks quantitative tissue characterization, often leading to diagnostic uncertainty and unnecessary CT examinations. This work presents a 3D ultrasound-derived pseudo-CT (UD-pCT) framework that generates CT-like anatomical reference volumes inferred from US, without aiming to reproduce physically accurate Hounsfield Units. Paired 3D kidney US and CT volumes from the TRUSTED
dataset are first spatially aligned using a landmark-based
multimodal registration pipeline, creating high-quality paired inputs for supervised training of an
adversarial framework. The proposed Bottleneck
Transformer Residual U-Net3D (BT-ResUNet3D) model employs a 3D residual encoder-decoder generator augmented with a
transformer bottleneck, enabling effective modeling of fine-grained local anatomical structures as well as long-range volumetric dependencies, while a 3D Conditional PatchGAN discriminator enforces local structural realism in the synthesized pseudo-CT volumes. Quantitative evaluation using PSNR and SSIM demonstrates that the proposed method outperforms established baselines in structural fidelity and perceptual image quality. The UD-pCT volumes provide real-time anatomical reference for operator guidance, potentially reducing acquisition variability and unnecessary CT use. A limitation of this study is the relatively small paired
dataset, which may limit the generalizability of the proposed model.
Read the full abstract ▾
Computer Vision
QuadBox: Accelerating 3D Gaussian Splatting with Geometry-Aware Boxes
Xinze Li, Bohan Yang, Pengxu Chen et al. (+4 more)
3D Gaussian Splatting (3DGS) has emerged as an advanced technique for real-time novel view synthesis by representing scene geometry and appearance using differentiable Gaussian primitives. However, efficiently computing precise Gaussian-tile intersections remains a critical task in the rasterization pipeline. To this end, we propose QuadBox, a method that leverages four axis-aligned bounding boxes to tightly encapsulate projected Gaussians in a discrete manner. First, we derive a geometry-aware stretching factor that enables the construction of a tile-aligned QuadBox, which covers the elliptical projection and largely excludes irrelevant tiles. Second, we introduce QPass, a single-pass tile traversal
algorithm that exhaustively exploits the discrete nature of QuadBox, ensuring that the tile intersection check is performed with simple interval tests. Experiments on public
datasets show that our method accelerates the rendering speed of 3DGS by 1.85$\times$. Code is available at \href{https://github.com/Powertony102/QuadBox}{https://github.com/Powertony102/QuadBox}.
Read the full abstract ▾
All authors Xinze Li, Bohan Yang, Pengxu Chen, Yiyuan Wang, Hongcheng Luo, Wentao Cheng, Weifeng Su
Computer Vision
MIRAGE: Retrieval and Generation of Multimodal Images and Texts for Medical Education
Miguel Diaz Benito, Cecilia Diana Albelda, Alvaro Garcia Martin et al. (+3 more)
Access to diverse, well-annotated medical images with interactive learning tools is fundamental for training practitioners in medicine and related fields to improve their diagnostic skills and understanding of anatomical structures. While medical atlases are valuable, they are often impractical due to their size and lack of interactivity, whereas online image search may provide mislabeled or incomplete material. To address this, we propose MIRAGE, a
multimodal medical text and image
retrieval and generation system that allows users to find and generate clinically relevant images from trustworthy sources by mapping both text and images to a shared latent space, enabling semantically meaningful queries. The system is based on a fine-tuned medical version of CLIP (MedICaT-ROCO), trained with the ROCO
dataset, obtained from PubMed Central. MIRAGE allows users to give
prompts to retrieve images, generate synthetic ones through a medical
diffusion model (
Prompt2MedImage) and receive enriched descriptions from a large
language model (Dolly-v2-3b). It also supports a dual search option, enabling the visual comparison of different medical conditions. A key advantage of the system is that it relies entirely on publicly available pretrained models, ensuring reproducibility and accessibility. Our goal is to provide a free, transparent and easy-to-use didactic tool for medical students, especially those without programming skills. The system features an interface that enables interactive and personalized visual learning through medical image
retrieval and generation. The system is accessible to medical students worldwide without requiring local computational resources or technical expertise, and is currently deployed on Kaggle: http://www-vpu.eps.uam.es/mirage
Read the full abstract ▾
All authors Miguel Diaz Benito, Cecilia Diana Albelda, Alvaro Garcia Martin, Jesus Bescos Cano, Marcos Escudero-Vinolo, Juan C. SanMiguel
Computer Vision
Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction
Berk Sezer, Ali Görkem Küçük, Erol Şahin et al. (+1 more)
While zero-shot appearance-based 3D gaze estimation offers significant cost-efficiency by directly mapping RGB images to gaze vectors, its reliability in Human-
Robot Interaction (HRI) settings remains uncertain. Existing
benchmarks frequently overlook fundamental HRI conditions, such as dynamic camera viewpoints and moving targets in
video. Furthermore, current cross-
dataset evaluations often suffer from a complexity gap, where methods trained on diverse
datasets are tested on significantly smaller and less varied sets, failing to assess true robustness. To bridge these gaps, we introduce Gaze4HRI, a large-scale
dataset (50+ subjects, 3,000+
videos, 600,000+ frames) designed to evaluate state-of-the-art performance against critical HRI variables: illumination, head-gaze conflict, as well as the motion of camera and gaze target in
video. Our
benchmark reveals that all evaluated methods fail in at least one condition, identifying steeply-downward gaze as a universal failure point. Notably, PureGaze trained on the ETH-X-Gaze
dataset uniquely maintains resilience across all other conditions. These results challenge the recent focus in the literature on complex spatial-temporal modeling and
Transformer-based architectures. Instead, our findings suggest that extensive data diversity, as exemplified by the ETH-X-Gaze
dataset, serves as the primary driver of zero-shot robustness in unconstrained environments, while resilience-enhancing frameworks, such as PureGaze's self-
adversarial loss for gaze feature purification, provide a substantial further improvement. Ultimately, this study establishes a rigorous
benchmark that provides practical guidelines for practitioners as well as reshaping future research. The
dataset and codes are available at https://gazeforhri.github.io.
Read the full abstract ▾
All authors Berk Sezer, Ali Görkem Küçük, Erol Şahin, Sinan Kalkan
Computer Vision
Lightweight Cross-Spectral Face Recognition via Contrastive Alignment and Distillation
Anjith George and Sebastien Marcel
Heterogeneous Face Recognition (HFR) aims at matching face images captured across different sensing modalities, such as thermal-to-visible or near-infrared-to-visible, enhancing the usability of face recognition systems in challenging real-world conditions. Although recent HFR methods have achieved significant improvements in performance, many rely on computationally expensive models, making them impractical for deployment on resource-limited edge devices. In this work, we introduce a lightweight yet effective HFR framework by adapting a hybrid CNN-
Transformer model originally developed for RGB homogeneous face recognition. Our approach enables efficient end-to-end training with only a small amount of paired heterogeneous data, while still maintaining strong performance on standard RGB face recognition
benchmarks. This makes it suitable for both homogeneous and heterogeneous settings. Comprehensive experiments on several challenging HFR and face recognition
benchmarks show that our method achieves state-of-the-art or competitive performance while keeping computational requirements low.
Read the full abstract ▾
Computer Vision
Hybrid Congestion Classification Framework Using Flow-Guided Attention and Empirical Mode Decomposition
Eugene Kofi Okrah Denteh, Blessing Agyei Kyem, Joshua Kofi Asamoah et al. (+1 more)
Accurate traffic congestion classification requires models that jointly capture roadway scene context and non-stationary traffic motion, yet most prior work treats these requirements in isolation.
Vision-based methods often depend on appearance cues with standard temporal pooling, which can bias predictions toward static infrastructure, whereas signal-based approaches characterize temporal dynamics but lack the spatial context needed for scene-level localization. These complementary limitations motivate a unified framework that links motion evidence to spatial feature selection while preserving data-adaptive temporal characterization. This study therefore proposes FLO-EMD, a hybrid approach that couples motion-guided
attention with empirical, data-driven temporal decomposition. Dense optical flow guides channel and spatial
attention so that RGB features are refined toward motion-relevant regions. In parallel, aggregated flow statistics form compact motion traces that are decomposed using Empirical Mode Decomposition (EMD) to extract intrinsic temporal components. The resulting EMD
embedding is fused with learned spatiotemporal representations to classify light, medium, and heavy congestion. Experiments on 1,050 five-second clips from four surveillance networks show that FLO-EMD achieves 97.5% overall test accuracy (weighted F1 = 0.9742), outperforming established baselines and remaining robust across diverse environmental conditions; ablation and sensitivity analyses further quantify the contributions of EMD, the number of intrinsic mode functions, and the selected motion descriptors.
Read the full abstract ▾
All authors Eugene Kofi Okrah Denteh, Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah
Computer Vision
VC-FeS: Viewpoint-Conditioned Feature Selection for Vehicle Re-identification in Thermal Vision
Yasod Ginige, Ransika Gunasekara, Darsha Hewavitharana et al. (+3 more)
Identification of less-articulated objects using single-channel images, such as thermal images, is important in many applications, such as surveillance. However, in this domain, existing methods show poor performance due to high similarity among objects of the same category in the absence of color information (overlooking shape information) and de-emphasized texture information. Furthermore, variability in viewpoint adds more complexity as the features vary from side to side. We address these issues by constructing viewpoint-conditioned feature vectors and area-specific feature comparisons in separate feature spaces. These interventions enable leveraging the advancements of existing RGB-pre-trained ViT feature extractors while effectively adapting them to address the challenges specific to the thermal domain. We test our system with RGBNT100 (IR) vehicle
dataset and a thermal maritime
dataset acquired by us. Our results surpass the state-of-the-art methods by 19.7% and 12.8% for the above
datasets in mAP scores, respectively. We also plan to make our thermal
dataset available, the first of its kind for maritime vessel identification.
Read the full abstract ▾
All authors Yasod Ginige, Ransika Gunasekara, Darsha Hewavitharana, Manjula Ariyarathne, Peshala Jayasekara, Ranga Rodrigo
Computer Vision
Morphology-Guided Cross-Task Coupling for Joint Building Height and Footprint Estimation
Jinzhen Han, JinByeong Lee, Jisung Kim et al. (+1 more)
Building height (BH) and building footprint (BF) jointly describe the vertical and horizontal extent of the built environment and are required inputs for urban
climate, disaster-risk, and population-mapping models. The two parameters are coupled through floor-area-ratio (FAR) constraints, yet remote-sensing approaches typically treat them as independent regression targets. We argue that explicitly encoding this cross-task coupling is more impactful than further refining individual encoders, and propose MorphoFormer, a joint BH/BF estimation framework built around two complementary mechanisms: (i) a BF-Guided Task Decoder (BGTD) that gates the height branch via cross-
attention on a footprint-derived morphology context, and (ii) a Morphology Consistency Loss (MCL) that supervises a height-from-footprint surrogate against the ground-truth BH, indirectly forcing the BF feature to encode height-correlated structure. The encoder is a single-stage Swin backbone fed by Sentinel-1 SAR, Sentinel-2 multispectral, and DEM inputs, trained and evaluated on a geo-blocked split of 54 cities. Against a Swin-MTL baseline at identical receptive field, MorphoFormer reduces BH test RMSE from 3.39 to 3.15 m (R^2 improves 0.62 -> 0.67) with BF R^2 stable at 0.80. Controlled ablations at identical capacity attribute most of this 0.24 m improvement to the two proposed mechanisms: removing BGTD raises BH RMSE by 0.11 m and removing MCL raises it by 0.11 m, with the residual approximately 0.02 m falling within the noise floor of encoder-side variations. Because both mechanisms act on cross-task representations rather than pixels, the design carries no intrinsic dependence on input resolution.
Read the full abstract ▾
All authors Jinzhen Han, JinByeong Lee, Jisung Kim, HongSik Yun
Computer Vision
ULF-Loc: Unbiased Landmark Feature for Robust Visual Localization with 3D Gaussian Splatting
Yingdong Gu, Shaocheng Yan, Zhenjun Zhao et al. (+4 more)
Visual localization is a core technology for augmented reality and autonomous
navigation. Recent methods combine the efficient rendering of 3D Gaussian Splatting (3DGS) with feature-based localization. These methods rely on direct matching between 2D query features and the 3D Gaussian feature field, but this often results in mismatches due to an inherent bias in the learned Gaussian feature. We theoretically analyze the feature learning process in 3DGS, revealing that the widely adopted $\alpha$-blending optimization inherently introduces bias into 3D point features. This bias stems from the
entanglement between individual Gaussians and their neighboring Gaussians, making the learned features unsuitable for precise matching tasks. Motivated by these findings, we propose ULF-Loc, an unbiased landmark feature framework that replaces biased feature optimization with geometry-weighted feature fusion. We further introduce keypoint-consensus landmark sampling to select reliable Gaussians and local
geometric consistency verification to reject mismatches caused by rendering artifacts. On the Cambridge Landmarks
dataset, ULF-Loc reduces the mean median translation error by 17% compared to the state-of-the-art, while achieving superior efficiency with only 1/10 the training time and 1/6 the GPU
memory of STDLoc.
Read the full abstract ▾
All authors Yingdong Gu, Shaocheng Yan, Zhenjun Zhao, Yuan Kou, Jianxin Luo, Pengcheng Shi, Jiayuan Li
Computer Vision
Anny-Fit: All-Age Human Mesh Recovery
Laura Bravo-Sánchez, Matthieu Armando, Romain Brégier et al. (+3 more)
Recovering 3D human pose and shape from a single image remains a cornerstone of human-centric
vision, yet most methods assume adult subjects and optimize each person independently. These assumptions fail in real-world, all-age scenes, where body proportions and depth must be resolved jointly. We introduce Anny-Fit, a multi-person, camera-space optimization framework for all-age 3D human mesh recovery (HMR). Unlike existing per-person fitting methods, Anny-Fit jointly optimizes all individuals directly in the camera coordinate system, enforcing global spatial consistency. At the core of our approach is the use of multiple forms of expert knowledge – including metric depth maps, instance
segmentation, 2D keypoints, and, VLM-derived semantic attributes such as age and gender – each obtained from dedicated off-the-shelf networks. These complementary signals jointly guide the optimization, constraining the depth-scale ambiguity characteristic of all-age scenes. Across diverse
datasets, Anny-Fit consistently improves 2D reprojection accuracy (+13 to 16), relative depth ordering (+6 to 7), 3D estimation error (-9 to -29) and shape estimation (+25 to +82), producing more coherent scenes. Finally, we show that VLM-based semantic knowledge can be distilled into an HMR model via the pseudo-ground-truth annotations produced by Anny-Fit on training data, enabling it to learn semantically meaningful shape parameters while improving HMR performance. Our approach bridges adult-only and all-age modeling by enabling zero-shot adaptation of adult-trained HMR pipelines to the full age spectrum without retraining. Code is publicly available at https://github.com/naver/anny-fit.
Read the full abstract ▾
All authors Laura Bravo-Sánchez, Matthieu Armando, Romain Brégier, Grégory Rogez, Serena Yeung-Levy, Fabien Baradel
Computer Vision
Not Every Subject Should Stay: Machine Unlearning for Noisy Engagement Recognition
Alexander Vedernikov
Engagement recognition
datasets are typically subject-indexed and often contain noisy, subjective supervision, making post-hoc
dataset revision a practical problem. Existing noisy-label and data-cleaning methods largely operate at the sample level before or during training, but do not directly address a different question: once a model has already been trained, can the influence of an entire problematic subject be removed without full retraining? We study this setting through subject-level machine unlearning as a post-hoc sanitization mechanism for engagement recognition. Starting from a baseline trained on all subjects, we rank candidate harmful subjects using a model-dependent proxy, apply a lightweight approximate unlearning update, and compare the result against an oracle model retrained from scratch on the retained subjects only. We instantiate this protocol on DAiSEE and EngageNet using Tensor-Convolution and Convolution-
Transformer Network (TCCT-Net) as a fixed platform and evaluate three matched model states under the same removal scenario: baseline, unlearned, and oracle. In representative K=3 forget-set settings, the unlearned model recovers 89.3% and 92.5% of the oracle gain on EngageNet and DAiSEE, respectively, at roughly one quarter of retraining cost. Across the tested small-audit regimes, effectiveness is strongest at an intermediate forget-set size, indicating that approximate subject-level unlearning is a useful low-cost correction mechanism, but one whose benefit depends on subject selection quality and removal regime.
Read the full abstract ▾
Computer Vision
FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation
Yuanzhi Wang, Xuhua Ren, Jiaxiang Cheng et al. (+7 more)
Identity-preserving text-to-
video generation (IPT2V) empowers users to produce diverse and imaginative
videos with consistent human facial identity. Despite recent progress, existing methods often suffer from significant identity distortion under large facial pose variations or facial occlusions. In this paper, we propose FaithfulFaces, a pose-faithful facial identity preservation learning framework to improve IPT2V in complex dynamic scenes. The key of FaithfulFaces is a pose-shared identity aligner that refines and aligns facial poses across distinct views via a pose-shared dictionary and a pose variation-identity invariance constraint. By mapping single-view inputs into a global facial pose representation with explicit Euler angle
embeddings, FaithfulFaces provides a pose-faithful facial prior that guides generative foundations toward robust identity-preserving generation. In particular, we develop a specialized pipeline to curate a high-quality
video dataset featuring substantial facial pose diversity. Extensive experiments demonstrate that FaithfulFaces achieves state-of-the-art performance, maintaining superior identity consistency and structural clarity even as pose changes and occlusions occur. Project Page: https://mdswyz.github.io/FaithfulFaces/
Read the full abstract ▾
All authors Yuanzhi Wang, Xuhua Ren, Jiaxiang Cheng, Bing Ma, Kai Yu, Sen Liang, Wenyue Li, Tianxiang Zheng, Qinglin Lu, Zhen Cui
Computer Vision · Cross-listed from cs.LG
HEXST: Hexagonal Shifted-Window Transformer for Spatial Transcriptomics Gene Expression Prediction
Keunho Byeon and Jin Tae Kwak
Spatial transcriptomics offers spatially resolved gene expression profiling within tissue sections, but its cost and limited throughput hinder large-scale deployment. To extend this capability to routine practice, recent computational methods aim to infer spatial gene expression directly from ubiquitous hematoxylin and eosin-stained histology slides. However, most existing models assume Cartesian or geometry-agnostic locality, despite the hexagonal sampling of widely used spot-array platforms, and point-wise regression objectives often yield over-smoothed gene expression profiles, obscuring gene-specific spatial heterogeneity. To address these, we propose HEXST, a geometry-aligned
Transformer for spatial gene expression prediction from histology. HEXST operates directly on hexagonal spot coordinates to enable efficient local-to-global contextual modeling via tailored shifted-window
attention mechanism and hexagonal rotary positional encoding. To enhance gene-wise spatial contrast, HEXST complements point-wise regression with a contrast-sensitive differential objective and transcriptomic priors from a pretrained single-
cell foundation model during training. Across seven spatial transcriptomics
datasets, HEXST consistently outperforms state-of-the-art models, providing accurate and robust spatial gene expression predictions while preserving gene-wise contrast and spatial heterogeneity.
Read the full abstract ▾
Computer Vision
Multi-Level Bidirectional Biomimetic Learning for EEG-Based Visual Decoding
Jingtao Liu, Peiliang Gong, Chuhang Zheng et al. (+2 more)
EEG-based visual neural decoding aims to align neural responses with visual stimuli for tasks such as image
retrieval. However, limited paired data and a fundamental mismatch between high-fidelity digital images and biological visual perception - distorted by retinotopic mapping and subject-specific neuroanatomy - severely impede cross-modal
alignment. To address this, we propose MB2L, a Multi-Level Bidirectional Biomimetic Learning framework that incorporates structured physiological inductive biases into representation learning. Specifically, we propose Adaptive Blur with Visual Priors to mitigate perceptual-structural mismatch by reweighting visual inputs according to retinotopic priors. We further propose Biomimetic Visual Feature Extraction to learn multi-level visual representations consistent with hierarchical cortical processing, enhancing subject-invariant encoding. These modules are jointly optimized via Multi-level Bidirectional Contrastive Learning, which aligns
EEG and visual features in a shared semantic space through bidirectional contrastive objectives. Experiments show MB2L achieves 80.5% Top-1 and 97.6% Top-5 accuracy on zero-shot
EEG-to-image
retrieval, significantly outperforming prior methods and demonstrating strong generalization across subjects and experimental settings.
Read the full abstract ▾
All authors Jingtao Liu, Peiliang Gong, Chuhang Zheng, Yiheng Liu, Qi Zhu
Computer Vision · Cross-listed from cs.RO
From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models
Yihan Lin, Haoyang Li, Yang Li et al. (+4 more)
Latent actions serve as an intermediate representation that enables consistent modeling of
vision-
language-action (VLA) models across heterogeneous
datasets. However, approaches to supervising VLAs with latent actions are fragmented and lack a systematic comparison. This work structures the study of latent action supervision from two perspectives: (i) regularizing the trajectory via image-based latent actions, and (ii) unifying the target space with action-based latent actions. Under a unified VLA baseline, we instantiate and compare four representative integration strategies. Our results reveal a formulation-task correspondence: image-based latent actions benefit long-horizon
reasoning and scene-level generalization, whereas action-based latent actions excel at complex motor coordination. Furthermore, we find that directly supervising the VLM with discrete latent action
tokens yields the most effective performance. Finally, our experiments offer initial insights into the benefits of latent action supervision in mixed-data, suggesting a promising direction for VLA training. Code is available at https://github.com/RUCKBReasoning/From_Pixels_to_
Tokens.
Read the full abstract ▾
All authors Yihan Lin, Haoyang Li, Yang Li, Haitao Shen, Yihan Zhao, Chao Shao, Jing Zhang
Computer Vision
Physical Adversarial Clothing Evades Visible-Thermal Detectors via Non-Overlapping RGB-T Pattern
Xiaopei Zhu, Guanning Zeng, Zhanhao Hu et al. (+2 more)
Visible-thermal (RGB-T) object detection is a crucial technology for applications such as autonomous driving, where
multimodal fusion enhances performance in challenging conditions like low light. However, the security of RGB-T detectors, particularly in the physical world, has been largely overlooked. This paper proposes a novel approach to RGB-T physical attacks using
adversarial clothing with a non-overlapping RGB-T pattern (NORP). To simulate full-view (0$^{\circ}$–360$^{\circ}$) RGB-T attacks, we construct 3D RGB-T models for human and
adversarial clothing. NORP is a new
adversarial pattern design using distinct visible and thermal materials without overlap, avoiding the light reduction in overlapping RGB-T patterns (ORP). To optimize the NORP on
adversarial clothing, we propose a spatial discrete-continuous optimization (SDCO) method. We systematically evaluated our method on RGB-T detectors with different fusion architectures, demonstrating high attack success rates both in the digital and physical worlds. Additionally, we introduce a fusion-stage ensemble method that enhances the transferability of
adversarial attacks across unseen RGB-T detectors with different fusion architectures.
Read the full abstract ▾
All authors Xiaopei Zhu, Guanning Zeng, Zhanhao Hu, Jun Zhu, Xiaolin Hu
Computer Vision
Contact Matrix: Enhancing Dance Motion Synthesis with Precise Interaction Modeling
Xuhai Chen, Zhi Cen, Huaijin Pi et al. (+3 more)
Generating realistic reactive motions, in which one person reacts to the fixed motions of others, is challenging due to strict interaction constraints and a limited feasible solution space. This paper focuses on a typical scenario: duet dance, where high-quality data is scarce, motion patterns are complex, and the details of human interactions are both intricate and abundant. To tackle these challenges, we propose a novel two-stage framework. In the first stage, we introduce a motion VQ-VAE with separate body-part encoders and a joint decoder, enabling specialized codebooks to enhance representation capacity while dynamically modeling dependencies across body parts during decoding, thereby preventing inconsistencies in the generated motions. In the second stage, we propose a contact-aware
diffusion model for reactive motion generation that jointly generates motion and a contact
matrix between individuals, enabling explicit interaction modeling and providing guidance toward more precise and constrained interaction dynamics during sampling. Experiments show that our method outperforms Duolando with lower $\text{FID}_k$ (8.89 vs. 25.30) and $\text{FID}_{cd}$ (8.01 vs. 9.97), as well as a higher BED (0.4606 vs. 0.2858), indicating improved interaction fidelity and rhythmic synchronization.
Read the full abstract ▾
All authors Xuhai Chen, Zhi Cen, Huaijin Pi, Sida Peng, Xiaowei Zhou, Yong Liu
Computer Vision
CAST: Mitigating Object Hallucination in Large Vision-Language Models via Caption-Guided Visual Attention Steering
Qiming Li, Zekai Ye, Xiaocheng Feng et al. (+9 more)
Although Large
Vision-
Language Models (LVLMs) have demonstrated remarkable performance on downstream tasks, they frequently produce contents that deviate from visual information, leading to object
hallucination. To tackle this, recent works mostly depend on expensive manual annotations and training cost, or decoding strategies which significantly increase inference time. In this work, we observe that LVLMs'
attention to visual information is significantly enhanced when answering caption queries compared to non-caption queries. Inspired by this phenomenon, we propose Caption-guided Visual
Attention Steering (CAST), a training-free, plug-and-play
hallucination mitigation method that leverages the
attention activation pattern corresponding to caption queries to enhance LVLMs' visual perception capability. Specifically, we use probing techniques to identify
attention heads that are highly sensitive to caption queries and estimate optimized steering directions for their outputs. This steering strengthens LVLM's fine-grained visual perception capabilities, thereby effectively mitigating object
hallucination. CAST reduced object
hallucination by an average of 6.03% across five widely used LVLMs and five
benchmarks including both discriminative and generative tasks, demonstrating state-of-the-art performance while adding little inference cost and preserving other foundational capabilities.
Read the full abstract ▾
All authors Qiming Li, Zekai Ye, Xiaocheng Feng, Weihong Zhong, Libo Qin, Ruihan Chen, Lei Huang, Baohang Li, Kui Jiang, Yaowei Wang, Ting Liu, Bing Qin
Computer Vision
UniPCB: A Generation-Assisted Vision-Based Measurement Framework for PCB Defect Inspection
Huan Zhang, Lianghong Tan, Yichu Xu et al. (+5 more)
Automated optical inspection (AOI) is a
vision-based instrumentation process that detects and localizes physical defects on printed
circuit boards (PCBs) from optically acquired images. Its reliability is often limited by scarce and class-imbalanced defect observations and by insufficient representation of small, low-contrast defects against dense
circuit patterns. We propose UniPCB, a generation-assisted PCB inspection framework that combines controlled defect synthesis with task-specific detection. The generation branch derives heterogeneous edge, relative pseudo-depth, and text conditions from PCB images. A ScaleEncoder embeds the conditions at four U-Net resolutions, while a Condition Modulation block performs FiLM-style spatially adaptive fusion. The detection branch introduces an Inverted Residual Shift
Attention block for joint global-local modeling and a Cross-level Complementary Fusion block for selective hierarchical feature integration. The generated images augment the detector training data and improve defect coverage without changing the online inspection path. Experiments using the corrected annotations of DsPCBSD+ yield an mAP@0.5 of 98.0% and an mAP@0.5:0.95 of 61.8%. Under the same generation-augmented training setting, these values exceed those of the RT-DETR baseline by 2.1 and 1.7 percentage points, respectively. The generation branch obtains an FID of 129.61 and an SSIM of 0.619. These results indicate the potential of structured synthetic data and PCB-oriented feature modeling for improving
vision-based defect inspection. Code is available at https://github.com/House-yuyu/UniPCB.
Read the full abstract ▾
All authors Huan Zhang, Lianghong Tan, Yichu Xu, Zishan Su, Jiangzhong Cao, Huanqi Wu, Linwei Zhu, Xu Zhang
Computer Vision
Temporal Structure Matters for Efficient Test-Time Adaptation in Wearable Human Activity Recognition
Zishu Zhou, Zaipeng Xie and Xuanyao Jie
Wearable human activity recognition (WHAR) models often suffer from performance degradation under real-world cross-user distribution shifts. Test-time adaptation (TTA) mitigates this degradation by adapting models online using unlabeled test streams, yet existing methods largely inherit assumptions from
vision tasks and underexploit the inherent inter-window temporal structure in WHAR streams. In this paper, we revisit such temporal structure as a feature-conditioned inference signal rather than merely an output-space smoothing prior. We derive the insight that temporal continuity and observation-induced feature deviations provide complementary cues for determining when to preserve or release temporal inertia and where to route prediction refinement during likely transitions. Building upon this insight, we propose SIGHT, a lightweight and backpropagation-free TTA framework for WHAR, enabling real-time edge deployment. SIGHT estimates predictive surprise by comparing the current feature with a prototype-based expected state, and then uses the resulting feature deviation to guide geometry-aware transition routing based on prototype
alignment and stream-level marginal habit tracking. Evaluations on real-world
datasets confirm that SIGHT outperforms existing TTA baselines while reducing computational and
memory costs.
Read the full abstract ▾
Computer Vision
Advancing Aesthetic Image Generation via Composition Transfer
Kai Zou, Zhiwei Zhao, Bin Liu et al. (+1 more)
Composition is a cornerstone of visual aesthetics, influencing the appeal of an image. While its principles operate independently of specific content, in practice, composition is often coupled with semantics. As a result, existing methods often enhance composition either through implicit learning or by semantics-based layout control, rather than explicitly modeling composition itself. To address this gap, we introduce Composer, a framework rooted in aesthetic theory, designed to model composition in a semantic-agnostic manner. First, it supports composition transfer by extracting key composition-aware representations from a reference image and leveraging a tailored conditional guidance module to control composition based on pre-trained
diffusion models. Second, when users specify only text themes without a composition reference, Composer supports theme-driven composition
retrieval by leveraging the in-context learning capabilities of Large
Vision-
Language Models (LVLMs), achieving explicit composition planning. To enhance composition in a reference-free mode, we conduct text-to-composition fine-tuning on the trained control module to enable implicit composition planning. Furthermore, we curated a high-quality
dataset comprising 2 million image-text pairs using state-of-the-art generative models to support model training. Experimental results demonstrate that Composer significantly enhances aesthetic quality in text-to-image tasks and facilitates personalized composition control and transfer, offering users precision and flexibility in the creative process.
Read the full abstract ▾
All authors Kai Zou, Zhiwei Zhao, Bin Liu, Nenghai Yu
Computer Vision
Reference-based Category Discovery: Unsupervised Object Detection with Category Awareness
Yichen Li, Qiankun Liu and Ying Fu
Traditional one-shot detection methods have addressed the closed-set problem in object detection, but the high cost of data annotation remains a critical challenge. General unsupervised methods generate pseudo boxes without category labels, thus failing to achieve category-aware classification. To overcome these limitations, we propose Reference-based Category Discovery (RefCD), an unsupervised detector that enables category-aware\footnotemark[1] detection without any manually annotated labels. It leverages feature similarity between predicted objects and unlabeled reference images. Unlike previous unsupervised methods that lack category guidance and one-shot methods which require labeled data, RefCD introduces a carefully designed feature similarity loss to explicitly guide the learning of potential category-specific features. Additionally, RefCD supports category-agnostic detection without reference images, serving as a unified framework. Comprehensive quantitative and qualitative analysis of category-aware and category-agnostic detection results demonstrates its effectiveness, and RefCD can learn category information in an unsupervised paradigm even without category labels.
Read the full abstract ▾
Computer Vision
DiCLIP: Diffusion Model Enhances CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentation
Zhiwei Yang, Pengfei Song, Yucong Meng et al. (+3 more)
Weakly Supervised Semantic
Segmentation (WSSS) with image-level labels typically leverages Class Activation Maps (CAMs) to achieve pixel-level predictions. Recently, Contrastive
Language-Image Pre-training (CLIP) has been introduced to generate CAMs in WSSS. However, previous WSSS methods solely adopt CLIP's
vision-
language paired property for dense localization, neglecting its inherently limited dense knowledge across both visual and text modalities, which renders CAM generation suboptimal. In this work, we propose DiCLIP, a novel WSSS framework that leverages the generative
diffusion model to enhance CLIP's dense knowledge across two modalities. Specifically, Visual Correlation Enhancement (VCE) and Text Semantic Augmentation (TSA) modules are proposed for dense prediction enhancement. To improve the spatial awareness of visual features, our VCE module utilizes
diffusion's reliable spatial consistency to mitigate the over-smoothing issue in CLIP's
attention. It designs the
Attention Clustering Refinement (ACR) module to reliably extract diverse correlation maps from the
diffusion model. The correlation maps act as a diversity bias for CLIP's self-
attention, recursively pushing its visual features towards a more discriminative dense distribution. To augment the semantics of text
embeddings, our TSA module argues that a single text modality is insufficient to encompass the variability of visual categories. Thus, we leverage
diffusion's generative power to maintain a dynamic key-value cache model, shifting CAM generation from a patch-text matching mechanism to a novel visual knowledge
retrieval paradigm. With these enhancements, DiCLIP not only outperforms state-of-the-art methods on PASCAL VOC and MS COCO but also significantly reduces training costs. Code is publicly available at https://github.com/zwyang6/DiCLIP.
Read the full abstract ▾
All authors Zhiwei Yang, Pengfei Song, Yucong Meng, Kexue Fu, Shuo Wang, Zhijian Song
Computer Vision
From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation
Zishen Qu, Xuesong Li, Haijian Gu et al. (+5 more)
Text-based image
segmentation aims to delineate object boundaries within an image from text
prompts, offering higher flexibility and broader application scope compared to traditional fixed-category
segmentation tasks. Recent studies have shown that
diffusion models (e.g., Stable
Diffusion) can provide rich
multimodal semantic features, leading to studies of using
diffusion models as feature extractors for
segmentation tasks. Such methods, however, inherit the generative natures of
diffusion models that are harmful to discriminative
segmentation tasks. In response, we propose RLFSeg, a novel framework that leverages Rectified Flow to learn direct mapping from the image to the
segmentation mask within the latent space. The model is thus freed from the noise-denoise process and the need to optimize the time step of
diffusion models, resulting in substantially better performance than previous
diffusion-based methods, especially on zero-shot scenarios. By introducing label refinement and an Adaptive One-Step Sampling strategy, the model achieves higher accuracy even on a single inference step. The framework redirects a pretrained generative model to the discriminative
segmentation task with zero modification to model structure, thus reveals promising application potential and significant research value.
Read the full abstract ▾
All authors Zishen Qu, Xuesong Li, Haijian Gu, Hongwei Kang, Quan Meng, Tianrui Niu, Xin Yang, Ruidong Pan
Computer Vision
GTF: Omnidirectional EPI Transformer for Light Field Super-Resolution
Kunyu Li, Fei Wang, Lichao Zhang et al. (+2 more)
Light field (LF) image super-resolution benefits from Epipolar Plane Images (EPIs), whose line slopes explicitly encode disparity. However, existing
Transformer-based LF SR methods mainly attend to horizontal and vertical EPIs, leaving diagonal epipolar geometry underexplored. We present GTF, an omnidirectional EPI
Transformer that explicitly models horizontal, vertical, 45-degree, and 135-degree EPIs within a unified reconstruction framework. GTF combines directional EPI processing, MacPI-based prior injection, adaptive directional fusion, and a
topology-preserving feed-forward network to better exploit LF geometry. For the NTIRE 2026 fidelity tracks, we use GTF as the main model, while a lightweight GTF-Tiny variant targets the efficiency track. On five standard LF SR
benchmarks covering both real-captured and synthetic scenes, GTF reaches 32.78 dB without inference-time enhancement, and stronger inference settings with EPSW and test-time augmentation further improve performance. Under the NTIRE 2026 efficiency constraint, GTF-Tiny attains 32.57 dB with only 0.915M parameters and 19.81 GFLOPs. In the NTIRE 2026 Light Field Image Super-Resolution Challenge, our submissions rank 3rd on Track 1 and Track 3 and 4th on Track 2. Architecture-
evolution, channel-width, and inference analyses further support the effectiveness of diagonal EPI modeling, directional fusion, and the lightweight design.
Read the full abstract ▾
All authors Kunyu Li, Fei Wang, Lichao Zhang, Junjie Liu, Bihong Li
Computer Vision
VL-UniTrack: A Unified Framework with Visual-Language Prompts for UAV-Ground Visual Tracking
Boyue Xu, Ruichao Hou, Tongwei Ren et al. (+1 more)
UAV-ground visual tracking (UGVT) aims to simultaneously track the same object from both the
UAV and the ground view. However, existing two-stream methods suffer from isolated feature extraction and rely heavily on implicit appearance matching, which struggles to establish reliable correspondence under drastic view differences, leading to tracking unreliability. To address these limitations, we propose VL-UniTrack, a fully unified framework enhanced by visual-
language prompts. By encoding features from both views within a single shared encoder, our method breaks the barrier of feature isolation to facilitate sufficient cross-view interaction. To overcome the ambiguity caused by relying solely on appearance matching, we design visual-
language geometric prompting module, which fuses
language descriptions with visual features to generate learnable
prompts. These
prompts are then fed into our
prompt-guided cross-view adapter module to enable sufficient cross-view feature interaction and to guide the learning of view-specific feature representations. Furthermore, a confidence-modulated mutual
distillation loss is proposed to regularize the training by mitigating noise propagation. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the latest
benchmark. The code can be downloaded in https://github.com/xuboyue1999/VL-UniTrack.git
Read the full abstract ▾
All authors Boyue Xu, Ruichao Hou, Tongwei Ren, Gangshan Wu