Computer Vision
Gaussian-Constrained LeJEPA Representations for Unsupervised Scene Discovery and Pose Consistency
Mohsen Mostafa
Unsupervised 3D scene reconstruction from unstructured image collections remains a fundamental challenge in computer
vision, particularly when images originate from multiple unrelated scenes and contain significant visual ambiguity. The Image Matching Challenge 2025 (IMC2025) highlights these difficulties by requiring both scene discovery and camera pose estimation under real-world conditions, including outliers and mixed content. This paper investigates the application of Gaussian-constrained representations inspired by LeJEPA (Joint
Embedding Predictive Architecture) to address these challenges. We present three progressively refined pipelines, culminating in a LeJEPA-inspired approach that enforces isotropic Gaussian constraints on learned image
embeddings. Rather than introducing new theoretical guarantees, our work empirically evaluates how these constraints influence clustering consistency and pose estimation robustness in practice. Experimental results on IMC2025 demonstrate that Gaussian-constrained
embeddings can improve scene separation and pose plausibility compared to heuristic-driven baselines, particularly in visually ambiguous settings. These findings suggest that theoretically motivated representation constraints offer a promising direction for bridging self-supervised learning principles and practical structure-from-motion pipelines.
Read the full abstract ▾
Computer Vision
Robust and Real-Time Bangladeshi Currency Recognition: A Dual-Stream MobileNet and EfficientNet Approach
Subreena, Mohammad Amzad Hossain, Mirza Raquib et al. (+4 more)
Accurate currency recognition is essential for assistive technologies, particularly for visually impaired individuals who rely on others to identify banknotes. This dependency puts them at risk of fraud and exploitation. To address these challenges, we first build a new Bangladeshi banknote
dataset that includes both controlled and real-world scenarios, ensuring a more comprehensive and diverse representation. Next, to enhance the
dataset's robustness, we incorporate four additional
datasets, including public
benchmarks, to cover various complexities and improve the model's generalization. To overcome the limitations of current recognition models, we propose a novel hybrid CNN architecture that combines MobileNetV3-Large and EfficientNetB0 for efficient feature extraction. This is followed by an effective multilayer perceptron (MLP) classifier to improve performance while keeping computational costs low, making the system suitable for resource-constrained devices. The experimental results show that the proposed model achieves 97.95% accuracy on controlled
datasets, 92.84% on complex backgrounds, and 94.98% accuracy when combining all
datasets. The model's performance is thoroughly evaluated using five-fold cross-validation and seven metrics: accuracy, precision, recall, F1-score, Cohen's Kappa, MCC, and AUC. Additionally, explainable AI methods like LIME and SHAP are incorporated to enhance transparency and interpretability.
Read the full abstract ▾
All authors Subreena, Mohammad Amzad Hossain, Mirza Raquib, Saydul Akbar Murad, Farida Siddiqi Prity, Muhammad Hanif, Nick Rahimi
Computer Vision
Vectra: A New Metric, Dataset, and Model for Visual Quality Assessment in E-Commerce In-Image Machine Translation
Qingyu Wu, Yuxuan Han, Haijun Li et al. (+5 more)
In-Image Machine Translation (IIMT) powers cross-border e-commerce product listings; existing research focuses on machine translation evaluation, while visual rendering quality is critical for user engagement. When facing context-dense product imagery and
multimodal defects, current reference-based methods (e.g., SSIM, FID) lack explainability, while model-as-judge approaches lack domain-grounded, fine-grained reward signals. To bridge this gap, we introduce Vectra, to the best of our knowledge, the first reference-free, MLLM-driven visual quality assessment framework for e-commerce IIMT. Vectra comprises three components: (1) Vectra Score, a multidimensional quality metric system that decomposes visual quality into 14 interpretable dimensions, with spatially-aware Defect Area Ratio (DAR) quantification to reduce annotation ambiguity; (2) Vectra
Dataset, constructed from 1.1M real-world product images via diversity-aware sampling, comprising a 2K
benchmark for system evaluation, 30K
reasoning-based annotations for instruction tuning, and 3.5K expert-labeled preferences for
alignment and evaluation; and (3) Vectra Model, a 4B-parameter MLLM that generates both quantitative scores and diagnostic
reasoning. Experiments demonstrate that Vectra achieves state-of-the-art correlation with human rankings, and our model outperforms leading MLLMs, including GPT-5 and Gemini-3, in scoring performance. The
dataset and model will be released upon acceptance.
Read the full abstract ▾
All authors Qingyu Wu, Yuxuan Han, Haijun Li, Zhao Xu, Jianshan Zhao, Xu Jin, Longyue Wang, Weihua Luo
Computer Vision
Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Models
Jiaxi Yang, Shicheng Liu, Yuchen Yang et al. (+1 more)
With the rapid advancement of
Vision Language Models (VLMs), refusal mechanisms have become a critical component for ensuring responsible and safe model behavior. However, existing refusal strategies are largely one-size-fits-all and fail to adapt to diverse user needs and contextual constraints, leading to either under-refusal or over-refusal. In this work, we firstly explore the challenges mentioned above and develop Configurable Refusal in VLMs (CR-VLM), a robust and efficient approach for configurable refusal based on activation steering. CR-VLM consists of three integrated components: (1) extracting a configurable refusal vector via a teacher-forced mechanism to amplify the refusal signal; (2) introducing a gating mechanism that mitigates over-refusal by preserving acceptance for in-scope queries; and (3) designing a counterfactual
vision enhancement module that aligns visual representations with refusal requirements. Comprehensive experiments across multiple
datasets and various VLMs demonstrate that CR-VLM achieves effective, efficient, and robust configurable refusals, offering a scalable path toward user-adaptive safety
alignment in VLMs.
Read the full abstract ▾
All authors Jiaxi Yang, Shicheng Liu, Yuchen Yang, Dongwon Lee
Computer Vision
A General Model for Retinal Segmentation and Quantification
Zhonghua Wang, Lie Ju, Sijia Li et al. (+12 more)
Retinal imaging is fast, non-invasive, and widely available, offering quantifiable structural and vascular signals for ophthalmic and systemic health assessment. This accessibility creates an opportunity to study how quantitative retinal phenotypes relate to ocular and systemic diseases. However, such analyses remain difficult at scale due to the limited availability of public multi-label
datasets and the lack of a unified
segmentation-to-quantification pipeline. We present RetSAM, a general retinal
segmentation and quantification framework for fundus imaging. It delivers robust multi-target
segmentation and standardized biomarker extraction, supporting downstream ophthalmologic studies and oculomics correlation analyses. Trained on over 200,000 fundus images, RetSAM supports three task categories and segments five anatomical structures, four retinal phenotypic patterns, and more than 20 distinct lesion types. It converts these
segmentation results into over 30 standardized biomarkers that capture structural morphology, vascular geometry, and degenerative changes. Trained with a multi-stage strategy using both private and public fundus data, RetSAM achieves superior
segmentation performance on 17 public
datasets. It improves on prior best methods by 3.9 percentage points in DSC on average, with up to 15 percentage points on challenging multi-task
benchmarks, and generalizes well across diverse populations, imaging devices, and clinical settings. The resulting biomarkers enable systematic correlation analyses across major ophthalmic diseases, including diabetic retinopathy, age-related macular degeneration, glaucoma, and pathologic myopia. Together, RetSAM transforms fundus images into standardized, interpretable quantitative phenotypes, enabling large-scale ophthalmic research and translation.
Read the full abstract ▾
All authors Zhonghua Wang, Lie Ju, Sijia Li, Wei Feng, Sijin Zhou, Ming Hu, Jianhao Xiong, Xiaoying Tang, Yifan Peng, Mingquan Lin, Yaodong Ding, Yong Zeng, Wenbin Wei, Li Dong, Zongyuan Ge
Computer Vision
MAU-GPT: Enhancing Multi-type Industrial Anomaly Understanding via Anomaly-aware and Generalist Experts Adaptation
Zhuonan Wang, Zhenxuan Fan, Siwen Tan et al. (+8 more)
As industrial manufacturing scales, automating fine-grained product image analysis has become critical for quality control. However, existing approaches are hindered by limited
dataset coverage and poor model generalization across diverse and complex anomaly patterns. To address these challenges, we introduce MAU-Set, a comprehensive
dataset for Multi-type industrial Anomaly Understanding. It spans multiple industrial domains and features a hierarchical task structure, ranging from binary classification to complex
reasoning. Alongside this
dataset, we establish a rigorous evaluation protocol to facilitate fair and comprehensive model assessment. Building upon this foundation, we further present MAU-GPT, a domain-adapted
multimodal large model specifically designed for industrial anomaly understanding. It incorporates a novel AMoE-LoRA mechanism that unifies anomaly-aware and generalist experts adaptation, enhancing both understanding and
reasoning across diverse defect classes. Extensive experiments show that MAU-GPT consistently outperforms prior state-of-the-art methods across all domains, demonstrating strong potential for scalable and automated industrial inspection.
Read the full abstract ▾
All authors Zhuonan Wang, Zhenxuan Fan, Siwen Tan, Yu Zhong, Yuqian Yuan, Haoyuan Li, Hao Jiang, Wenqiao Zhang, Feifei Shao, Hongwei Wang, Jun Xiao
Computer Vision · Cross-listed from eess.IV
DINO-AD: Unsupervised Anomaly Detection with Frozen DINO-V3 Features
Jiayu Huo, Jingyuan Hong and Liyun Chen
Unsupervised anomaly detection (AD) in medical images aims to identify abnormal regions without relying on pixel-level annotations, which is crucial for scalable and label-efficient diagnostic systems. In this paper, we propose a novel anomaly detection framework based on DINO-V3 representations, termed DINO-AD, which leverages self-supervised visual features for precise and interpretable anomaly localization. Specifically, we introduce an
embedding similarity matching strategy to select a semantically aligned support image and a foreground-aware K-means clustering module to model the distribution of normal features. Anomaly maps are then computed by comparing the query features with clustered normal
embeddings through cosine similarity. Experimental results on both the
Brain and Liver
datasets demonstrate that our method achieves superior quantitative performance compared with state-of-the-art approaches, achieving AUROC scores of up to 98.71. Qualitative results further confirm that our framework produces clearer and more accurate anomaly localization. Extensive ablation studies validate the effectiveness of each proposed component, highlighting the robustness and generalizability of our approach.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Trajectory Consistency for One-Step Generation on Euler Mean Flows
Zhiqi Li, Yuchen Sun, Duowen Chen et al. (+2 more)
We propose Euler Mean Flows (EMF), a flow-based generative framework for one-step and few-step generation that enforces long-range trajectory consistency with minimal sampling cost. The key idea of EMF is to replace the trajectory consistency constraint, which is difficult to supervise and optimize over long time scales, with a principled linear surrogate that enables direct data supervision for long-horizon flow-map compositions. We derive this approximation from the
semigroup formulation of flow-based models and show that, under mild regularity assumptions, it faithfully approximates the original consistency objective while being substantially easier to optimize. This formulation leads to a unified, JVP-free training framework that supports both $u$-prediction and $x_1$-prediction variants, avoiding explicit Jacobian computations and significantly reducing
memory and computational overhead. Experiments on image synthesis, particle-based geometry generation, and functional generation demonstrate improved optimization stability and sample quality under fixed sampling budgets, together with approximately $50\%$ reductions in training time and
memory consumption compared to existing one-step methods for image generation.
Read the full abstract ▾
All authors Zhiqi Li, Yuchen Sun, Duowen Chen, Jinjin He, Bo Zhu
Computer Vision · Cross-listed from cs.RO
CLAMP: Contrastive Learning for 3D Multi-View Action-Conditioned Robotic Manipulation Pretraining
I-Chun Arthur Liu, Krzysztof Choromanski, Sandy Huang et al. (+1 more)
Leveraging pre-trained 2D image representations in behavior cloning policies has achieved great success and has become a standard approach for robotic manipulation. However, such representations fail to capture the 3D spatial information about objects and scenes that is essential for precise manipulation. In this work, we introduce Contrastive Learning for 3D Multi-View Action-Conditioned Robotic Manipulation Pretraining (CLAMP), a novel 3D pre-training framework that utilizes point clouds and
robot actions. From the merged point cloud computed from RGB-D images and camera extrinsics, we re-render multi-view four-channel image observations with depth and 3D coordinates, including dynamic wrist views, to provide clearer views of target objects for high-precision manipulation tasks. The pre-trained encoders learn to associate the 3D
geometric and positional information of objects with
robot action patterns via contrastive learning on large-scale simulated
robot trajectories. During encoder pre-training, we pre-train a
Diffusion Policy to initialize the policy weights for fine-tuning, which is essential for improving fine-tuning sample efficiency and performance. After pre-training, we fine-tune the policy on a limited amount of task demonstrations using the learned image and action representations. We demonstrate that this pre-training and fine-tuning design substantially improves learning efficiency and policy performance on unseen tasks. Furthermore, we show that CLAMP outperforms state-of-the-art baselines across six simulated tasks and five real-world tasks. The project website and
videos can be found at https://clamp3d.github.io/CLAMP/.
Read the full abstract ▾
All authors I-Chun Arthur Liu, Krzysztof Choromanski, Sandy Huang, Connor Schenck
Computer Vision
OCTOPUS: Enhancing the Spatial-Awareness of Vision SSMs with Multi-Dimensional Scans and Traversal Selection
Kunal Mahatha, Ali Bahri, Pierre Marza et al. (+5 more)
State space models (SSMs) have recently emerged as an alternative to
transformers due to their unique ability of modeling global relationships in text with linear complexity. However, their success in
vision tasks has been limited due to their causal formulation, which is suitable for sequential text but detrimental in the spatial domain where causality breaks the inherent spatial relationships among pixels or patches. As a result, standard SSMs fail to capture local spatial coherence, often linking non-adjacent patches while ignoring neighboring ones that are visually correlated. To address these limitations, we introduce OCTOPUS , a novel architecture that preserves both global context and local spatial structure within images, while maintaining the linear complexity of SSMs. OCTOPUS performs discrete reoccurrence along eight principal orientations, going forward or backward in the horizontal, vertical, and diagonal directions, allowing effective information exchange across all spatially connected regions while maintaining independence among unrelated patches. This design enables multi-directional recurrence, capturing both global context and local spatial structure with SSM-level efficiency. In our classification and
segmentation benchmarks, OCTOPUS demonstrates notable improvements in boundary preservation and region consistency, as evident from the
segmentation results, while maintaining relatively better classification accuracy compared to existing V-SSM based models. These results suggest that OCTOPUS appears as a foundation method for multi-directional recurrence as a scalable and effective mechanism for building spatially aware and computationally efficient
vision architectures.
Read the full abstract ▾
All authors Kunal Mahatha, Ali Bahri, Pierre Marza, Sahar Dastani, Maria Vakalopoulou, Stergios Christodoulidis, Jose Dolz, Christian Desrosiers
Computer Vision
DIAMOND: Directed Inference for Artifact Mitigation in Flow Matching Models
Alicja Polowczyk, Agnieszka Polowczyk, Piotr Borycki et al. (+3 more)
Despite impressive results from recent text-to-image models like FLUX, visual and anatomical artifacts remain a significant hurdle for practical and professional use. Existing methods for artifact reduction, typically work in a post-hoc manner, consequently failing to intervene effectively during the core image formation process. Notably, current techniques require problematic and invasive modifications to the model weights, or depend on a computationally expensive and time-consuming process of regional refinement. To address these limitations, we propose DIAMOND, a training-free method that applies trajectory correction to mitigate artifacts during inference. By reconstructing an estimate of the clean sample at every step of the generative trajectory, DIAMOND actively steers the generation process away from latent states that lead to artifacts. Furthermore, we extend the proposed method to standard
Diffusion Models, demonstrating that DIAMOND provides a robust, zero-shot path to high-fidelity, artifact-free image synthesis without the need for additional training or weight modifications in modern generative architectures. Code is available at https://gmum.github.io/DIAMOND/
Read the full abstract ▾
All authors Alicja Polowczyk, Agnieszka Polowczyk, Piotr Borycki, Joanna Waczyńska, Jacek Tabor, Przemysław Spurek
Computer Vision
Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardware
Brandon Leblanc and Charalambos Poullis
While multi-view 3D reconstruction has shifted toward large-scale foundation models capable of inferring globally consistent geometry, their reliance on massive computational clusters for training has created a significant barrier to entry for most academic laboratories. To bridge this compute divide, we introduce Distill3R, a framework designed to distill the
geometric reasoning of 3D foundation models into compact students fully trainable on a single workstation. Our methodology centers on two primary innovations: (1) an offline caching pipeline that decouples heavy teacher inference from the training loop through compressed supervision signals, and (2) a confidence-aware
distillation loss that leverages teacher uncertainty to enable training on commodity hardware. We propose a 72M-parameter student model which achieves a 9x reduction in parameters and a 5x inference speedup compared to its 650M-parameter teacher. The student is fully trainable in under 3 days on a single workstation, whereas its teacher requires massive GPU clusters for up to a week. We demonstrate that the student preserves the structural consistency and qualitative
geometric understanding required for functional 3D awareness. By providing a reproducible, single-workstation training recipe, Distill3R serves as an exploratory entry point for democratized 3D
vision research and efficient edge deployment. This work is not intended to compete with state-of-the-art foundation models, but to provide an accessible research baseline for laboratories without access to large-scale compute to train and specialize models on their own domain-specific data at minimal cost.
Read the full abstract ▾
Computer Vision
Beyond First-Order: Learning Riemannian Geometries for Invariant Visual Place Recognition
Jintao Cheng, Weibin Li, Zhijian He et al. (+3 more)
Visual Place Recognition (VPR) demands representations robust to drastic environmental and viewpoint shifts. Existing aggregation paradigms either depend on extensive supervised training or rely on first-order pooling, often struggling to preserve structural correlations under extreme shifts or incurring high adaptation costs. In this work, we propose Riemannian Invariant Aggregation (RIA), a unified
geometric framework that explicitly models second-order scene structure on the Symmetric Positive Definite (SPD)
manifold. By treating perturbations as tractable congruence transformations, RIA leverages geometry-aware Riemannian mappings to project covariance descriptors into a linearized Euclidean space, effectively preserving invariant structural components while suppressing noise. Extensive evaluations demonstrate that RIA achieves zero-shot performance comparable to supervised methods, and establishes state-of-the-art accuracy with simple fine-tuning, particularly in unstructured environments. The source code will be released.
Read the full abstract ▾
All authors Jintao Cheng, Weibin Li, Zhijian He, Jin Wu, Chi Man Vong, Wei Zhang
Computer Vision
TransNormal: Dense Visual Semantics for Diffusion-based Transparent Object Normal Estimation
Mingwei Li, Hehe Fan and Yi Yang
Monocular normal estimation for transparent objects is critical for laboratory automation, yet it remains challenging due to complex light refraction and reflection. These optical properties often lead to catastrophic failures in conventional depth and normal sensors, hindering the deployment of
embodied AI in scientific environments. We propose TransNormal, a novel framework that adapts pre-trained
diffusion priors for single-step normal regression. To handle the lack of texture in transparent surfaces, TransNormal integrates dense visual semantics from DINOv3 via a cross-
attention mechanism, providing strong
geometric cues. Furthermore, we employ a multi-task learning objective and wavelet-based regularization to ensure the preservation of fine-grained structural details. To support this task, we introduce TransNormal-Synthetic, a physics-based
dataset with high-fidelity normal maps for transparent labware. Extensive experiments demonstrate that TransNormal significantly outperforms state-of-the-art methods: on the ClearGrasp
benchmark, it reduces mean error by 24.4% and improves 11.25{\deg} accuracy by 22.8%; on ClearPose, it achieves a 15.2% reduction in mean error. The code and
dataset will be made publicly available at https://longxiang-ai.github.io/TransNormal.
Read the full abstract ▾
Computer Vision
Zero-Shot Generative De-identification: Inversion-Free Flow for Privacy-Preserving Skin Image Analysis
Konstantinos Moutselos and Ilias Maglogiannis
The secure analysis of dermatological images in clinical environments is fundamentally restricted by the critical trade-off between patient
privacy and the preservation of diagnostic fidelity. Traditional de-identification techniques often degrade essential pathological markers, while state-of-the-art generative approaches typically require computationally intensive inversion processes or extensive task-specific fine-tuning, limiting their feasibility for real-time deployment. This study introduces a zero-shot generative de-identification framework that utilizes an inversion-free pipeline for
privacy-preserving medical image analysis. By leveraging Rectified Flow
Transformers (FlowEdit), the proposed method achieves high-fidelity identity transformation in less than 20 seconds without requiring pathology-specific training or labeled
datasets. We introduce a novel "segment-by-synthesis" mechanism that generates counterfactual "healthy" and "pathological" digital twin pairs to isolate clinical signals from biometric identifiers in a zero-shot manner. Our approach specifically utilizes the CIELAB color space to decouple erythema-related pathological signals from semantic noise and individual skin characteristics. Pilot validation on high-resolution clinical samples demonstrates robust stability in preserving pathological features, achieving an Intersection over Union (IoU) stability exceeding 0.67, while ensuring rigorous de-identification. These results suggest that the proposed zero-shot, inversion-free approach provides a scalable and efficient solution for secure data sharing and collaborative biomedical research, bypassing the need for large-scale annotated medical
datasets while aligning with data protection standards.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
SyNeT: Synthetic Negatives for Traversability Learning
Bomena Kim, Hojun Lee, Younsoo Park et al. (+3 more)
Reliable traversability estimation is crucial for autonomous
robots to navigate complex outdoor environments safely. Existing self-supervised learning frameworks primarily rely on positive and unlabeled data; however, the lack of explicit negative data remains a critical limitation, hindering the model's ability to accurately identify diverse non-traversable regions. To address this issue, we introduce a method to explicitly construct synthetic negatives, representing plausible but non-traversable, and integrate them into
vision-based traversability learning. Our approach is formulated as a training strategy that can be seamlessly integrated into both Positive-Unlabeled (PU) and Positive-Negative (PN) frameworks without modifying inference architectures. Complementing standard pixel-wise metrics, we introduce an object-centric FPR evaluation approach that analyzes predictions in regions where synthetic negatives are inserted. This evaluation provides an indirect measure of the model's ability to consistently identify non-traversable regions without additional manual labeling. Extensive experiments on both public and self-collected
datasets demonstrate that our approach significantly enhances robustness and generalization across diverse environments. The source code and demonstration
videos will be publicly available.
Read the full abstract ▾
All authors Bomena Kim, Hojun Lee, Younsoo Park, Yaoyu Hu, Sebastian Scherer, Inwook Shim
Computer Vision
Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval
Tong Wang, Yunhan Zhao and Shu Kong
Composed Image
Retrieval (CIR) is the task of retrieving a target image from a database using a
multimodal query, which consists of a reference image and a modification text. The text specifies how to alter the reference image to form a ”mental image”, based on which CIR should find the target image in the database. The fundamental challenge of CIR is that this ”mental image” is not physically available and is only implicitly defined by the query. The contemporary literature pursues zero-shot methods and uses a Large
Multimodal Model (LMM) to generate a textual description for a given
multimodal query, and then employs a
Vision-
Language Model (VLM) for textual-visual matching to search for the target image. In contrast, we address CIR from first principles by directly generating the ”mental image” for more accurate matching. Particularly, we
prompt an LMM to generate a ”mental image” for a given
multimodal query and propose to use this ”mental image” to search for the target image. As the ”mental image” has a synthetic-to-real domain gap with real images, we also generate a synthetic counterpart for each real image in the database to facilitate matching. In this sense, our method uses LMM to construct a “paracosm”, where it matches the
multimodal query and database images. Hence, we call this method Paracosm. Notably, Paracosm is a training-free zero-shot CIR method. It significantly outperforms existing zero-shot methods on challenging
benchmarks, achieving state-of-the-art performance for zero-shot CIR.
Read the full abstract ▾
Computer Vision
VVLoc: Prior-free 3-DoF Vehicle Visual Localization
Ze Huang, Zhongyang Xiao, Mingliang Song et al. (+3 more)
Localization is a critical technology in autonomous driving, encompassing both topological localization, which identifies the most similar map keyframe to the current observation, and metric localization, which provides precise spatial coordinates. Conventional methods typically address these tasks independently, rely on single-camera setups, and often require additional 3D semantic or pose priors, while lacking mechanisms to quantify the confidence of localization results, making them less feasible for real industrial applications. In this paper, we propose VVLoc, a unified pipeline that employs a single neural network to concurrently achieve topological and metric vehicle localization using multi-camera system. VVLoc first evaluates the geo-proximity between visual observations, then estimates their relative metric poses using a matching strategy, while also providing a confidence measure. Additionally, the training process for VVLoc is highly efficient, requiring only pairs of visual data and corresponding ground-truth poses, eliminating the need for complex supplementary data. We evaluate VVLoc not only on the publicly available
datasets, but also on a more challenging self-collected
dataset, demonstrating its ability to deliver state-of-the-art localization accuracy across a wide range of localization tasks.
Read the full abstract ▾
All authors Ze Huang, Zhongyang Xiao, Mingliang Song, Longan Yang, Hongyuan Yuan, Li Sun
Computer Vision
Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds
Xianzhe Fan, Shengliang Deng, Xiaoyang Wu et al. (+7 more)
Existing
Vision-
Language-Action (VLA) models typically take 2D images as visual input, which limits their spatial understanding in complex scenes. How can we incorporate 3D information to enhance VLA capabilities? We conduct a pilot study across different observation spaces and visual representations. The results show that explicitly lifting visual input into point clouds yields representations that better complement their corresponding 2D representations. To address the challenges of (1) scarce 3D data and (2) the domain gap induced by cross-environment differences and depth-scale biases, we propose Any3D-VLA. It unifies the simulator, sensor, and model-estimated point clouds within a training pipeline, constructs diverse inputs, and learns domain-agnostic 3D representations that are fused with the corresponding 2D representations. Simulation and real-world experiments demonstrate Any3D-VLA's advantages in improving performance and mitigating the domain gap. Our project homepage is available at https://xianzhefan.github.io/Any3D-VLA.github.io.
Read the full abstract ▾
All authors Xianzhe Fan, Shengliang Deng, Xiaoyang Wu, Yuxiang Lu, Zhuoling Li, Mi Yan, Yujia Zhang, Zhizheng Zhang, He Wang, Hengshuang Zhao
Computer Vision
DVLA-RL: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learning
Wenhao Li, Xianjing Meng, Qiangchang Wang et al. (+3 more)
Few-shot learning (FSL) aims to generalize to novel categories with only a few samples. Recent approaches incorporate large
language models (LLMs) to enrich visual representations with semantic
embeddings derived from class names. However, they overlook progressive and adaptive
alignment between
vision and
language from low-level to high-level semantics, resulting in limited semantic gains. To address these challenges, we propose Dual-level
Vision-
Language Alignment with Reinforcement Learning gating (DVLA-RL), which consists of Dual-level Semantic Construction (DSC) and RL-gated
Attention (RLA). Specifically, DSC conditions LLMs on both class names and support samples to generate discriminative attributes, progressively selects the most relevant ones, and then synthesizes them into coherent class descriptions. This process provides complementary low-level attributes and high-level descriptions, enabling both fine-grained grounding and holistic class understanding. To dynamically integrate dual-level semantics along with the visual network layers, RLA formulates cross-modal fusion as a sequential decision process. A lightweight policy trained with episodic REINFORCE adaptively adjusts the contributions of self-
attention and cross-
attention to integrate textual and visual
tokens. As a result, shallow layers refine local attributes and deep layers emphasize global semantics, enabling more precise cross-modal
alignment. This achieves class-specific discrimination and generalized representations with merely a few support samples. DVLA-RL achieves new state-of-the-art performance across nine
benchmarks in three diverse FSL scenarios.
Read the full abstract ▾
All authors Wenhao Li, Xianjing Meng, Qiangchang Wang, Zhongyi Han, Zhibin Wu, Yilong Yin
Computer Vision
Evaluating Deep Learning-Based Nerve Segmentation in Brachial Plexus Ultrasound Under Realistic Data Constraints
Dylan Yves, Khush Agarwal, Jonathan Hoyin Chan et al. (+2 more)
Accurate nerve localization is critical for the success of ultrasound-guided regional anesthesia, yet manual identification remains challenging due to low image contrast, speckle noise, and inter-patient anatomical variability. This study evaluates deep learning-based nerve
segmentation in ultrasound images of the brachial plexus using a U-Net architecture, with a focus on how
dataset composition and annotation strategy influence
segmentation performance. We find that training on combined data from multiple ultrasound machines (SIEMENS ACUSON NX3 Elite and Philips EPIQ5) provides regularization benefits for lower-performing acquisition sources, though it does not surpass single-source training when matched to the target domain. Extending the task from binary nerve
segmentation to multi-class supervision (artery, vein, nerve, muscle) results in decreased nerve-specific Dice scores, with performance drops ranging from 9% to 61% depending on
dataset, likely due to class imbalance and boundary ambiguity. Additionally, we observe a moderate positive correlation between nerve size and
segmentation accuracy (Pearson r=0.587, p<0.001), indicating that smaller nerves remain a primary challenge. These findings provide methodological guidance for developing robust ultrasound nerve
segmentation systems under realistic clinical data constraints.
Read the full abstract ▾
All authors Dylan Yves, Khush Agarwal, Jonathan Hoyin Chan, Patcharapit Promoppatum, Aroonkamon Pattanasiricharoen
Computer Vision
HSI-VAR: Rethinking Hyperspectral Restoration through Spatial-Spectral Visual Autoregression
Xiangming Wang, Benteng Sun, Yungeng Liu et al. (+4 more)
Hyperspectral images (HSIs) capture richer spatial-spectral information beyond RGB, yet real-world HSIs often suffer from a composite mix of degradations, such as noise, blur, and missing bands. Existing generative approaches for HSI restoration like
diffusion models require hundreds of iterative steps, making them computationally impractical for high-dimensional HSIs. While regression models tend to produce oversmoothed results, failing to preserve critical structural details. We break this impasse by introducing HSI-VAR, rethinking HSI restoration as an autoregressive generation problem, where spectral and spatial dependencies can be progressively modeled rather than globally reconstructed. HSI-VAR incorporates three key innovations: (1) Latent-condition
alignment, which couples semantic consistency between latent priors and conditional
embeddings for precise reconstruction; (2) Degradation-aware guidance, which uniquely encodes mixed degradations as linear combinations in the
embedding space for automatic control, remarkably achieving a nearly $50\%$ reduction in computational cost at inference; (3) A spatial-spectral adaptation module that refines details across both domains in the decoding phase. Extensive experiments on nine all-in-one HSI restoration
benchmarks confirm HSI-VAR's state-of-the-art performance, achieving a 3.77 dB PSNR improvement on \textbf{ICVL} and offering superior structure preservation with an inference speed-up of up to $95.5 \times$ compared with
diffusion-based methods, making it a highly practical solution for real-world HSI restoration.
Read the full abstract ▾
All authors Xiangming Wang, Benteng Sun, Yungeng Liu, Haijin Zeng, Yongyong Chen, Jingyong Su, Jie Liu
Computer Vision · Cross-listed from cs.SE
Can Vision-Language Models Handle Long-Context Code? An Empirical Study on Visual Compression
Jianping Zhong, Guochang Li, Chen Zhi et al. (+6 more)
Large
Language Models (LLMs) struggle with long-context code due to window limitations. Existing textual code compression methods mitigate this via selective filtering but often disrupt dependency closure, causing semantic fragmentation. To address this, we introduce LongCodeOCR, a visual compression framework that renders code into compressed two-dimensional image sequences for
Vision-
Language Models (VLMs). By preserving a global view, this approach avoids the dependency breakage inherent in filtering. We systematically evaluate LongCodeOCR against the state-of-the-art LongCodeZip across four
benchmarks spanning code summarization, code question answering, and code completion. Our results demonstrate that visual code compression serves as a viable alternative for tasks requiring global understanding. At comparable compression ratios ($\sim$1.7$\times$), LongCodeOCR improves CompScore on Long Module Summarization by 36.85 points over LongCodeZip. At a 1M-
token context length with Glyph (a specialized 9B VLM), LongCodeOCR maintains higher accuracy than LongCodeZip while operating at about 4$\times$ higher compression. Moreover, compared with LongCodeZip, LongCodeOCR drastically reduces compression-stage overhead (reducing latency from $\sim$4.3 hours to $\sim$1 minute at 1M
tokens). Finally, our results characterize a fundamental coverage–fidelity trade-off: visual code compression retains broader context coverage to support global dependencies, yet faces fidelity bottlenecks on exactness-critical tasks; by contrast, textual code compression preserves symbol-level precision while sacrificing structural coverage.
Read the full abstract ▾
All authors Jianping Zhong, Guochang Li, Chen Zhi, Junxiao Han, Zhen Qin, Xinkui Zhao, Nan Wang, Shuiguang Deng, Jianwei Yin
Computer Vision
Diffusion-Driven Inter-Outer Surface Separation for Point Clouds with Open Boundaries
Zhengyan Qin and Liyuan Qiu
We propose a
diffusion-based
algorithm for separating the inter and outer layer surfaces from double-layered point clouds, particularly those exhibiting the "double surface artifact" caused by truncation in Truncated Signed Distance Function (TSDF) fusion during indoor or medical 3D reconstruction. This artifact arises from asymmetric truncation thresholds, leading to erroneous inter and outer shells in the fused volume, which our method addresses by extracting the true inter layer to mitigate challenges like overlapping surfaces and disordered normals. We focus on point clouds with open boundaries (i.e., sampled surfaces with topological openings/holes through which particles may escape), rather than point clouds with missing surface regions where no samples exist. Our approach enables robust processing of both watertight and open-boundary models, achieving extraction of the inter layer from 20,000 inter and 20,000 outer points in approximately 10 seconds. This solution is particularly effective for applications requiring accurate surface representations, such as indoor scene modeling and medical imaging, where double-layered point clouds are prevalent, and it accommodates both closed (watertight) and open-boundary surface geometries. Our goal is post-hoc inter/outer shell separation as a lightweight module after TSDF fusion; we do not aim to replace full variational or learning-based reconstruction pipelines.
Read the full abstract ▾
Computer Vision
Supervised makeup transfer with a curated dataset: Decoupling identity and makeup features for enhanced transformation
Qihe Pan, Yiming Wu, Xing Zhao et al. (+3 more)
Diffusion models have recently shown strong progress in generative tasks, offering a more stable alternative to GAN-based approaches for makeup transfer. Existing methods often suffer from limited
datasets, poor disentanglement between identity and makeup features, and weak controllability. To address these issues, we make three contributions. First, we construct a curated high-quality
dataset using a train-generate-filter-retrain strategy that combines synthetic, realistic, and filtered samples to improve diversity and fidelity. Second, we design a
diffusion-based framework that disentangles identity and makeup features, ensuring facial structure and skin tone are preserved while applying accurate and diverse cosmetic styles. Third, we propose a text-guided mechanism that allows fine-grained and region-specific control, enabling users to modify eyes, lips, or face makeup with natural
language prompts. Experiments on
benchmarks and real-world scenarios demonstrate improvements in fidelity, identity preservation, and flexibility. Examples of our
dataset can be found at: https://makeup-adapter.github.io.
Read the full abstract ▾
All authors Qihe Pan, Yiming Wu, Xing Zhao, Liang Xie, Guodao Sun, Ronghua Liang
Computer Vision
StomataSeg: Semi-Supervised Instance Segmentation for Sorghum Stomatal Components
Zhongtian Huang, Zhi Chen, Zi Huang et al. (+8 more)
Sorghum is a globally important cereal grown widely in water-limited and stress-prone regions. Its strong drought tolerance makes it a priority crop for
climate-resilient agriculture. Improving water-use efficiency in sorghum requires precise characterisation of stomatal traits, as stomata control of gas exchange, transpiration and photosynthesis have a major influence on crop performance. Automated analysis of sorghum stomata is difficult because the stomata are small (often less than 40 $\mu$m in length in grasses such as sorghum) and vary in shape across genotypes and leaf surfaces. Automated
segmentation contributes to high-throughput stomatal phenotyping, yet current methods still face challenges related to nested small structures and annotation bottlenecks. In this paper, we propose a semi-supervised instance
segmentation framework tailored for analysis of sorghum stomatal components. We collect and annotate a sorghum leaf imagery
dataset containing 11,060 human-annotated patches, covering the three stomatal components (pore, guard
cell and complex area) across multiple genotypes and leaf surfaces. To improve the detection of tiny structures, we split high-resolution microscopy images into overlapping small patches. We then apply a pseudo-labelling strategy to unannotated images, producing an additional 56,428 pseudo-labelled patches. Benchmarking across semantic and instance
segmentation models shows substantial performance gains: for semantic models the top mIoU increases from 65.93% to 70.35%, whereas for instance models the top AP rises from 28.30% to 46.10%. These results demonstrate that combining patch-based preprocessing with semi-supervised learning significantly improves the
segmentation of fine stomatal structures. The proposed framework supports scalable extraction of stomatal traits and facilitates broader adoption of AI-driven phenotyping in crop science.
Read the full abstract ▾
All authors Zhongtian Huang, Zhi Chen, Zi Huang, Xin Yu, Daniel Smith, Chaitanya Purushothama, Erik Van Oosterom, Alex Wu, William Salter, Yan Li, Scott Chapman
Computer Vision
JoyStreamer: Unlocking Highly Expressive Avatars via Harmonized Text-Audio Conditioning
Ruikui Wang, Jinheng Feng, Lang Tian et al. (+6 more)
Existing
video avatar models have demonstrated impressive capabilities in scenarios such as talking, public speaking, and singing. However, the majority of these methods exhibit limited
alignment with respect to text instructions, particularly when the
prompts involve complex elements including large full-body movement, dynamic camera trajectory, background transitions, or human-object interactions. To break out this limitation, we present JoyAvatar, a framework capable of generating long duration avatar
videos, featuring two key technical innovations. Firstly, we introduce a twin-teacher enhanced training
algorithm that enables the model to transfer inherent text-controllability from the foundation model while simultaneously learning audio-visual synchronization. Secondly, during training, we dynamically modulate the strength of multi-modal conditions (e.g., audio and text) based on the distinct denoising timestep, aiming to mitigate conflicts between the heterogeneous conditioning signals. These two key designs serve to substantially expand the avatar model's capacity to generate natural, temporally coherent full-body motions and dynamic camera movements as well as preserve the basic avatar capabilities, such as accurate lip-sync and identity consistency. GSB evaluation results demonstrate that our JoyStreamer model outperforms the state-of-the-art models such as Omnihuman-1.5 and KlingAvatar 2.0. Moreover, our approach enables complex applications including multi-person dialogues and non-human subjects role-playing. Some
video samples are provided on https://joystreamer.github.io/.
Read the full abstract ▾
All authors Ruikui Wang, Jinheng Feng, Lang Tian, Huaishao Luo, Chaochao Li, Liangbo Zhou, Huan Zhang, Youzheng Wu, Xiaodong He
Computer Vision · Cross-listed from cs.MM
Cross-Modal Binary Attention: An Energy-Efficient Fusion Framework for Audio-Visual Learning
Mohamed Saleh and Zahra Ahmadi
Effective
multimodal fusion requires mechanisms that can capture complex cross-modal dependencies while remaining computationally scalable for real-world deployment. Existing audio-visual fusion approaches face a fundamental trade-off:
attention-based methods effectively model cross-modal relationships but incur quadratic computational complexity that prevents hierarchical, multi-scale architectures, while efficient fusion strategies rely on simplistic concatenation that fails to extract complementary cross-modal information. We introduce CMQKA, a novel cross-modal fusion mechanism that achieves linear O(N) complexity through efficient binary operations, enabling scalable hierarchical fusion previously infeasible with conventional
attention. CMQKA employs bidirectional cross-modal Query-Key
attention to extract complementary spatiotemporal features and uses learnable residual fusion to preserve modality-specific characteristics while enriching representations with cross-modal information. Building upon CMQKA, we present SNNergy, an energy-efficient
multimodal fusion framework with a hierarchical architecture that processes inputs through progressively decreasing spatial resolutions and increasing semantic abstraction. This multi-scale fusion capability allows the framework to capture both local patterns and global context across modalities. Implemented with event-driven binary spike operations, SNNergy achieves remarkable energy efficiency while maintaining fusion effectiveness and establishing new state-of-the-art results on challenging audio-visual
benchmarks, including CREMA-D, AVE, and UrbanSound8K-AV, significantly outperforming existing
multimodal fusion baselines. Our framework advances
multimodal fusion by introducing a scalable fusion mechanism that enables hierarchical cross-modal integration with practical energy efficiency for real-world audio-visual intelligence systems.
Read the full abstract ▾
Computer Vision
V2X-DSC: Multi-Agent Collaborative Perception with Distributed Source Coding Guided Communication
Yuankun Zeng, Shaohui Li, Zhi Li et al. (+3 more)
Collaborative perception improves 3D understanding by fusing multi-
agent observations, yet intermediate-feature sharing faces strict bandwidth constraints as dense BEV features saturate V2X links. We observe that collaborators view the same physical world, making their features strongly correlated; thus receivers only need innovation beyond their local context. Revisiting this from a distributed source coding perspective, we propose V2X-DSC, a framework with a Conditional Codec (DCC) for bandwidth-constrained fusion. The sender compresses BEV features into compact codes, while the receiver performs conditional reconstruction using its local features as side information, allocating bits to complementary cues rather than redundant content. This conditional structure regularizes learning, encouraging incremental representation and yielding lower-noise features. Experiments on DAIR-V2X, OPV2V, and V2X-Real demonstrate state-of-the-art accuracy-bandwidth trade-offs under KB-level communication, and generalizes as a plug-and-play communication layer across multiple fusion backbones.
Read the full abstract ▾
All authors Yuankun Zeng, Shaohui Li, Zhi Li, Shulan Ruan, Yu Liu, You He
Computer Vision
Video Understanding: Through A Temporal Lens
Thong Thanh Nguyen
This thesis explores the central question of how to leverage temporal relations among
video elements to advance
video understanding. Addressing the limitations of existing methods, the work presents a five-fold contribution: (1) an automatic annotation framework that utilizes large
vision-
language models and a noise-robust contrastive learning objective with a subtractive angular margin; (2) a parameter-efficient fine-tuning strategy using "recurrent adapters" to capture temporal dynamics in low-data regimes; (3) the integration of State Space Layers (SSL) for efficient long-form
video modeling, supported by the introduction of two new long-term
benchmarks for
egocentric and feature-length content; (4) a novel contrastive learning framework designed to explicitly model fine-grained relations between motions and
video moments; and (5) a comprehensive empirical study on Large
Vision-
Language Models (LVLMs) that identifies the visual-
language interface as a bottleneck for temporal
reasoning, leading to a new "temporal-oriented recipe" for upscaled
video understanding. Collectively, these contributions demonstrate that explicit temporal modeling significantly enhances a model's ability to represent and reason about the fluid nature of
video content.
Read the full abstract ▾