Computer Vision
OMNI-Dent: Towards an Accessible and Explainable AI Framework for Automated Dental Diagnosis
Leeje Jang, Yao-Yi Chiang, Angela M. Hastings et al. (+8 more)
Accurate dental diagnosis is essential for oral healthcare, yet many individuals lack access to timely professional evaluation. Existing AI-based methods primarily treat diagnosis as a visual pattern recognition task and do not reflect the structured clinical
reasoning used by dental professionals. These approaches also require large amounts of expert-annotated data and often struggle to generalize across diverse real-world imaging conditions. To address these limitations, we present OMNI-Dent, a data-efficient and explainable diagnostic framework that incorporates clinical
reasoning principles into a
Vision-
Language Model (VLM)-based pipeline. The framework operates on multi-view smartphone photographs,embeds diagnostic heuristics from dental experts, and guides a general-purpose VLM to perform tooth-level evaluation without dental-specific fine-tuning of the VLM. By utilizing the VLM's existing visual-linguistic capabilities, OMNI-Dent aims to support diagnostic assessment in settings where curated clinical imaging is unavailable. Designed as an early-stage assistive tool, OMNI-Dent helps users identify potential abnormalities and determine when professional evaluation may be needed, offering a practical option for individuals with limited access to in-person care.
Read the full abstract ▾
All authors Leeje Jang, Yao-Yi Chiang, Angela M. Hastings, Patimaporn Pungchanchaikul, Martha B. Lucas, Emily C. Schultz, Jeffrey P. Louie, Mohamed Estai, Wen-Chen Wang, Ryan H.L. Ip, Boyen Huang
Computer Vision
UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents
Yifan Ji, Zhipeng Xu, Zhenghao Liu et al. (+7 more)
Key Information Extraction (KIE) from real-world documents remains challenging due to substantial variations in layout structures, visual quality, and task-specific information requirements. Recent Large
Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images. To enable a comprehensive and systematic evaluation across realistic and diverse application scenarios, we introduce UNIKIE-BENCH, a unified
benchmark designed to rigorously evaluate the KIE capabilities of LMMs. UNIKIE-BENCH consists of two complementary tracks: a constrained-category KIE track with scenario-predefined schemas that reflect practical application needs, and an open-category KIE track that extracts any key information that is explicitly present in the document. Experiments on 15 state-of-the-art LMMs reveal substantial performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios. These findings underscore persistent challenges in grounding accuracy and layout-aware
reasoning for LMM-based KIE. All codes and
datasets are available at https://github.com/NEUIR/UNIKIE-BENCH.
Read the full abstract ▾
All authors Yifan Ji, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Qian Zhang, Zhibo Yang, Junyang Lin, Yu Gu, Ge Yu, Maosong Sun
Computer Vision · Cross-listed from cs.NE
Huannan Zheng, Jingli Liu and Kezhou Yang
Uncertainty in biological neural systems appears to be computationally beneficial rather than detrimental. However, in neuromorphic computing systems, device variability often limits performance, including accuracy and efficiency. In this work, we propose a spiking
Bayesian neural network (SBNN) framework that unifies the dynamic models of intrinsic device stochasticity (based on
Magnetic Tunnel Junctions) and
stochastic threshold
neurons to leverage noise as a functional
Bayesian resource. Experiments demonstrate that SBNN achieves high accuracy (99.16% on MNIST, 94.84% on CIFAR10) with 8-bit precision. Meanwhile rate estimation method provides a ~20-fold training speedup. Furthermore, SBNN exhibits superior robustness, showing a 67% accuracy improvement under synaptic weight noise and 12% under input noise compared to standard spiking neural networks. Crucially, hardware validation confirms that physical device implementation causes invisible accuracy and calibration loss compared to the algorithmic model. Converting device stochasticity into neuronal uncertainty offers a route to compact, energy-efficient neuromorphic computing under uncertainty.
Read the full abstract ▾
Computer Vision
iSight: Towards expert-AI co-assessment for improved immunohistochemistry staining interpretation
Jacob S. Leiby, Jialu Yao, Pan Lu et al. (+17 more)
Immunohistochemistry (IHC) provides information on
protein expression in tissue sections and is commonly used to support pathology diagnosis and disease triage. While AI models for H&E-stained slides show promise, their applicability to IHC is limited due to domain-specific variations. Here we introduce HPA10M, a
dataset that contains 10,495,672 IHC images from the Human
Protein Atlas with comprehensive metadata included, and encompasses 45 normal tissue types and 20 major cancer types. Based on HPA10M, we trained iSight, a multi-task learning framework for automated IHC staining assessment. iSight combines visual features from whole-slide images with tissue metadata through a
token-level
attention mechanism, simultaneously predicting staining intensity, location, quantity, tissue type, and malignancy status. On held-out data, iSight achieved 85.5% accuracy for location, 76.6% for intensity, and 75.7% for quantity, outperforming fine-tuned foundation models (PLIP, CONCH) by 2.5–10.2%. In addition, iSight demonstrates well-calibrated predictions with expected calibration errors of 0.0150-0.0408. Furthermore, in a user study with eight pathologists evaluating 200 images from two
datasets, iSight outperformed initial pathologist assessments on the held-out HPA
dataset (79% vs 68% for location, 70% vs 57% for intensity, 68% vs 52% for quantity). Inter-pathologist agreement also improved after AI assistance in both held-out HPA (Cohen's $\kappa$ increased from 0.63 to 0.70) and Stanford TMAD
datasets (from 0.74 to 0.76), suggesting expert–AI co-assessment can improve IHC interpretation. This work establishes a foundation for AI systems that can improve IHC diagnostic accuracy and highlights the potential for integrating iSight into clinical workflows to enhance the consistency and reliability of IHC assessment.
Read the full abstract ▾
All authors Jacob S. Leiby, Jialu Yao, Pan Lu, George Hu, Anna Davidian, Shunsuke Koga, Olivia Leung, Pravin Patel, Isabella Tondi Resta, Rebecca Rojansky, Derek Sung, Eric Yang, Paul J. Zhang, Emma Lundberg, Dokyoon Kim, Serena Yeung-Levy, James Zou, Thomas Montine, Jeffrey Nirschl, Zhi Huang
Computer Vision · Cross-listed from cs.LG
SEIS: Subspace-based Equivariance and Invariance Scores for Neural Representations
Huahua Lin, Katayoun Farrahi and Xiaohao Cai
Understanding how neural representations respond to
geometric transformations is essential for evaluating whether learned features preserve meaningful spatial structure. Existing approaches primarily assess robustness primarily by comparing model outputs under transformed inputs, offering limited insight into how
geometric information is organized within internal representations and failing to distinguish between information loss and re-encoding. In this work, we introduce SEIS (Subspace-based Equivariance and Invariance Scores), a subspace metric for analyzing layer-wise feature representations under
geometric transformations, disentangling equivariance from invariance without requiring labels or explicit knowledge of the transformation. Through controlled experiments across diverse architectures, we uncover several consistent patterns. First, convolutional encoders exhibit a depth-wise transition from strong equivariance to increasing invariance, with both properties stabilizing within the first few training epochs. In
segmentation decoders, however, equivariance tends to recover in later layers. Second, this trade-off is not intrinsic but is shaped by training decisions: data augmentation actively strengthens both equivariance and invariance simultaneously, and multi-task learning induces synergistic gains in both properties beyond what either task achieves alone. Extending our analysis beyond convolutional networks, we find that
transformer-based models exhibit distinct
geometric behaviors, while MLP-Mixers display intermediate characteristics.
Read the full abstract ▾
Computer Vision
Seeing Through Clutter: Structured 3D Scene Reconstruction via Iterative Object Removal
Rio Aguina-Kang, Kevin James Blackburn-Matzen, Thibault Groueix et al. (+2 more)
We present SeeingThroughClutter, a method for reconstructing structured 3D representations from single images by segmenting and modeling objects individually. Prior approaches rely on intermediate tasks such as semantic
segmentation and depth estimation, which often underperform in complex scenes, particularly in the presence of occlusion and clutter. We address this by introducing an iterative object removal and reconstruction pipeline that decomposes complex scenes into a sequence of simpler subtasks. Using VLMs as orchestrators, foreground objects are removed one at a time via detection,
segmentation, object removal, and 3D fitting. We show that removing objects allows for cleaner
segmentations of subsequent objects, even in highly occluded scenes. Our method requires no task-specific training and benefits directly from ongoing advances in foundation models. We demonstrate stateof-the-art robustness on 3D-Front and ADE20K
datasets. Project Page: https://rioak.github.io/seeingthroughclutter/
Read the full abstract ▾
All authors Rio Aguina-Kang, Kevin James Blackburn-Matzen, Thibault Groueix, Vladimir Kim, Matheus Gadelha
Computer Vision
Artifact Removal and Image Restoration in AFM:A Structured Mask-Guided Directional Inpainting Approach
Juntao Zhang, Angona Biswas, Jaydeep Rade et al. (+5 more)
Atomic Force Microscopy (AFM) enables high-resolution surface imaging at the nanoscale, yet the output is often degraded by artifacts introduced by environmental noise, scanning imperfections, and tip-sample interactions. To address this challenge, a lightweight and fully automated framework for artifact detection and restoration in AFM image analysis is presented. The pipeline begins with a classification model that determines whether an AFM image contains artifacts. If necessary, a lightweight semantic
segmentation network, custom-designed and trained on AFM data, is applied to generate precise artifact masks. These masks are adaptively expanded based on their structural orientation and then inpainted using a directional neighbor-based interpolation strategy to preserve 3D surface continuity. A localized Gaussian smoothing operation is then applied for seamless restoration. The system is integrated into a user-friendly GUI that supports real-time parameter adjustments and batch processing. Experimental results demonstrate the effective artifact removal while preserving nanoscale structural details, providing a robust, geometry-aware solution for high-fidelity AFM data interpretation.
Read the full abstract ▾
All authors Juntao Zhang, Angona Biswas, Jaydeep Rade, Charchit Shukla, Juan Ren, Anwesha Sarkar, Adarsh Krishnamurthy, Aditya Balu
Computer Vision
Fast, Unsupervised Framework for Registration Quality Assessment of Multi-stain Histological Whole Slide Pairs
Shikha Dubey, Patricia Raciti, Kristopher Standish et al. (+2 more)
High-fidelity registration of histopathological whole slide images (WSIs), such as hematoxylin & eosin (H&E) and immunohistochemistry (IHC), is vital for integrated molecular analysis but challenging to evaluate without ground-truth (GT) annotations. Existing WSI-level assessments – using annotated landmarks or intensity-based similarity metrics – are often time-consuming, unreliable, and computationally intensive, limiting large-scale applicability. This study proposes a fast, unsupervised framework that jointly employs down-sampled tissue masks- and deformations-based metrics for registration quality assessment (RQA) of registered H&E and IHC WSI pairs. The masks-based metrics measure global structural correspondence, while the deformations-based metrics evaluate local smoothness, continuity, and transformation realism. Validation across multiple IHC markers and multi-expert assessments demonstrate a strong correlation between automated metrics and human evaluations. In the absence of GT, this framework offers reliable, real-time RQA with high fidelity and minimal computational resources, making it suitable for large-scale quality control in digital pathology.
Read the full abstract ▾
All authors Shikha Dubey, Patricia Raciti, Kristopher Standish, Albert Juan Ramon, Erik Ames Burlingame
Computer Vision
A Parameterizable Convolution Accelerator for Embedded Deep Learning Applications
Panagiotis Mousouliotis and Georgios Keramidas
Convolutional neural network (CNN) accelerators implemented on Field-Programmable Gate Arrays (FPGAs) are typically designed with a primary focus on maximizing performance, often measured in giga-operations per second (GOPS). However, real-life embedded deep learning (DL) applications impose multiple constraints related to latency, power consumption, area, and cost. This work presents a hardware-software (HW/SW) co-design methodology in which a CNN accelerator is described using high-level synthesis (HLS) tools that ease the parameterization of the design, facilitating more effective optimizations across multiple design constraints. Our experimental results demonstrate that the proposed design methodology is able to outperform non-parameterized design approaches, and it can be easily extended to other types of DL applications.
Read the full abstract ▾
Computer Vision
AnyStyle: Single-Pass Multimodal Stylization for 3D Gaussian Splatting
Joanna Kaleta, Bartosz Świrta, Kacper Kania et al. (+3 more)
The growing demand for rapid and scalable 3D asset creation has driven interest in feed-forward 3D reconstruction methods, with 3D Gaussian Splatting (3DGS) emerging as an effective scene representation. While recent approaches have demonstrated pose-free reconstruction from unposed image collections, integrating stylization or appearance control into such pipelines remains underexplored. Existing attempts largely rely on image-based conditioning, which limits both controllability and flexibility. In this work, we introduce AnyStyle, a feed-forward 3D reconstruction and stylization framework that enables pose-free, zero-shot stylization through
multimodal conditioning. Our method supports both textual and visual style inputs, allowing users to control the scene appearance using natural
language descriptions or reference images. We propose a modular stylization architecture that requires only minimal architectural modifications and can be integrated into existing feed-forward 3D reconstruction backbones. Experiments demonstrate that AnyStyle improves style controllability over prior feed-forward stylization methods while preserving high-quality
geometric reconstruction. A user study further confirms that AnyStyle achieves superior stylization quality compared to an existing state-of-the-art approach. Repository: https://github.com/joaxkal/AnyStyle.
Read the full abstract ▾
All authors Joanna Kaleta, Bartosz Świrta, Kacper Kania, Tomasz Trzciński, Przemysław Spurek, Marek Kowalski
Computer Vision · Cross-listed from eess.IV
MS-SCANet: A Multiscale Transformer-Based Architecture with Dual Attention for No-Reference Image Quality Assessment
Mayesha Maliha R. Mithila and Mylene C.Q. Farias
We present the Multi-Scale Spatial Channel
Attention Network (MS-SCANet), a
transformer-based architecture designed for no-reference image quality assessment (IQA). MS-SCANet features a dual-branch structure that processes images at multiple scales, effectively capturing both fine and coarse details, an improvement over traditional single-scale methods. By integrating tailored spatial and channel
attention mechanisms, our model emphasizes essential features while minimizing computational complexity. A key component of MS-SCANet is its cross-branch
attention mechanism, which enhances the integration of features across different scales, addressing limitations in previous approaches. We also introduce two new consistency loss functions, Cross-Branch Consistency Loss and Adaptive Pooling Consistency Loss, which maintain spatial integrity during feature scaling, outperforming conventional linear and bilinear techniques. Extensive evaluations on
datasets like KonIQ-10k, LIVE, LIVE Challenge, and CSIQ show that MS-SCANet consistently surpasses state-of-the-art methods, offering a robust framework with stronger correlations with subjective human scores.
Read the full abstract ▾
Computer Vision
TiCLS : Tightly Coupled Language Text Spotter
Leeje Jang, Yijun Lin, Yao-Yi Chiang et al. (+1 more)
Scene text spotting aims to detect and recognize text in real-world images, where instances are often short, fragmented, or visually ambiguous. Existing methods primarily rely on visual cues and implicitly capture local character dependencies, but they overlook the benefits of external linguistic knowledge. Prior attempts to integrate
language models either adapt
language modeling objectives without external knowledge or apply pretrained models that are misaligned with the word-level granularity of scene text. We propose TiCLS, an end-to-end text spotter that explicitly incorporates external linguistic knowledge from a character-level pretrained
language model. TiCLS introduces a linguistic decoder that fuses visual and linguistic features, yet can be initialized by a pretrained
language model, enabling robust recognition of ambiguous or fragmented text. Experiments on ICDAR 2015 and Total-Text demonstrate that TiCLS achieves state-of-the-art performance, validating the effectiveness of PLM-guided linguistic integration for scene text spotting.
Read the full abstract ▾
All authors Leeje Jang, Yijun Lin, Yao-Yi Chiang, Jerod Weinman
Computer Vision · Cross-listed from cs.LG
PromptSplit: Revealing Prompt-Level Disagreement in Generative Models
Mehdi Lotfian, Mohammad Jalali and Farzan Farnia
Prompt-guided generative AI models have rapidly expanded across
vision and
language domains, producing realistic and diverse outputs from textual inputs. The growing variety of such models, trained with different data and architectures, calls for principled methods to identify which types of
prompts lead to distinct model behaviors. In this work, we propose PromptSplit, a kernel-based framework for detecting and analyzing
prompt-dependent disagreement between generative models. For each compared model pair, PromptSplit constructs a joint
prompt–output representation by forming tensor-product
embeddings of the
prompt and image (or text) features, and then computes the corresponding kernel covariance
matrix. We utilize the eigenspace of the weighted difference between these
matrices to identify the main directions of behavioral difference across
prompts. To ensure scalability, we employ a random-projection approximation that reduces computational complexity to $O(nr^2 + r^3)$ for projection dimension $r$. We further provide a theoretical analysis showing that this approximation yields an eigenstructure estimate whose expected deviation from the full-dimensional result is bounded by $O(1/r^2)$. Experiments across text-to-image, text-to-text, and image-captioning settings demonstrate that PromptSplit accurately detects ground-truth behavioral differences and isolates the
prompts responsible, offering an interpretable tool for detecting where generative models disagree.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
AtlasPatch: Scalable Foundation Model-based Tissue Detection and Patch Extraction for Computational Pathology
Ahmed Alagha, Christopher Leclerc, Yousef Kotp et al. (+12 more)
Whole-slide image (WSI) preprocessing, including tissue detection and patch extraction, is critical computational pathology, yet remains a major bottleneck for large-scale workflows. Existing methods often rely either on threshold-based heuristics that are sensitive to staining variations, tissue fragmentation, and artifacts, or on patch-wise deep learning pipelines with substantially higher computational cost. We present AtlasPatch, a scalable high-throughput WSI preprocessing method built around a foundation-model-based tissue detector that operates at thumbnail resolution: a single thumbnail-level forward pass yields a tissue mask that directly guides patch coordinate generation at the target desired magnification, avoiding repeated patch-level inference. The proposed detector's robustness and efficiency is driven by two coupled contributions: (i) a parameter-efficient adaptation of the SAM2 foundation model that updates only its layer-normalization parameters (0.076% of model weights), and (ii) a curated and semi-manually annotated multi-cohort
dataset of 30,000 WSI thumbnail-mask pairs deliberately spanning multiple organs, scanners, tissue appearances, and artifacts. The detector is coupled with pyramid-aware contour mapping from thumbnail to full-resolution slide coordinates, enabling direct patch coordinate generation at the target magnification and parallelized high-throughput patch extraction. AtlasPatch's tissue detection achieves a precision of 0.986 and remains robust across slide variations. Compared with widely used deep-learning preprocessing methods, AtlasPatch is up to 16x faster while preserving downstream multiple-instance learning performance across six slide-level classification tasks. These results position AtlasPatch as a frontier of efficient preprocessing in large-scale computational pathology and pathology foundation models.
Read the full abstract ▾
All authors Ahmed Alagha, Christopher Leclerc, Yousef Kotp, Omar Metwally, Calvin Moras, Peter Rentopoulos, Ghodsiyeh Rostami, Bich Ngoc Nguyen, Jumanah Baig, Abdelhakim Khellaf, Vincent Quoc-Huy Trinh, Rabeb Mizouni, Hadi Otrok, Jamal Bentahar, Mahdi S. Hosseini
Computer Vision · Cross-listed from cs.RO
Efficient Long-Horizon Vision-Language-Action Models via Static-Dynamic Disentanglement
Weikang Qiu, Huashuo Lei, Tinglin Huang et al. (+1 more)
Vision-
Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon
vision-
language model (VLM) architectures, VLAs predict actions conditioned on visual observations and
language instructions, achieving strong performance and generalization across tasks. However, VLAs face two major challenges: a limited context window for input frames and inefficient inference due to the quadratic
attention complexity and large parameter counts. To this end, we propose DySta, a framework that disentangles visual inputs into multi-level static and dynamic
tokens, which enables (1) retaining a single copy of static
tokens across frames to significantly reduce context length, and (2) reusing the key-value (KV) cache of static
tokens through a lightweight recache gate that updates only when necessary. This design enables efficient multi-frame integration and efficient inference. In addition, we introduce a new
benchmark that more effectively evaluates the multi-frame integration ability of VLAs. Experiments show that Dysta improves multi-frame integration by 24.5% across metrics on our
benchmark and 23.3% in absolute success rate on real-world
memory-dependent tasks, while accelerating inference by 2.0x (with +2.3% success rate) on simulation
benchmarks and 2.2x (with +10.6% success rate) on real-world general tasks.
Read the full abstract ▾
All authors Weikang Qiu, Huashuo Lei, Tinglin Huang, Rex Ying
Computer Vision · Cross-listed from cs.RO
VLS: Steering Pretrained Robot Policies via Vision-Language Models
Shuo Liu, Ishneet Sukhvinder Singh, Yiqing Xu et al. (+2 more)
Why do pretrained
diffusion or flow-matching policies fail when the same task is performed near an obstacle, on a shifted support surface, or amid mild clutter? Such failures rarely reflect missing motor skills; instead, they expose a limitation of imitation learning under train-test shifts, where action generation is tightly coupled to training-specific spatial configurations and task specifications. Retraining or fine-tuning to address these failures is costly and conceptually misaligned, as the required behaviors already exist but cannot be selectively adapted at test time. We propose
Vision-
Language Steering (VLS), a training-free framework for inference-time adaptation of frozen generative
robot policies. VLS treats adaptation as an inference-time control problem, steering the sampling process of a pretrained
diffusion or flow-matching policy in response to out-of-distribution observation-
language inputs without modifying policy parameters. By leveraging
vision-
language models to synthesize trajectory-differentiable reward functions, VLS guides denoising toward action trajectories that satisfy test-time spatial and task requirements. Across simulation and real-world evaluations, VLS consistently outperforms prior steering methods, achieving a 31% improvement on CALVIN and a 13% gain on LIBERO-PRO. Real-world deployment on a Franka
robot further demonstrates robust inference-time adaptation under test-time spatial and semantic shifts. Project page: https://
vision-
language-steering.github.io/webpage/
Read the full abstract ▾
All authors Shuo Liu, Ishneet Sukhvinder Singh, Yiqing Xu, Jiafei Duan, Ranjay Krishna
Computer Vision · Cross-listed from cs.LG
Representation Geometry as a Diagnostic for Out-of-Distribution Robustness
Ali Zia and Farid Hazratian
Robust generalization under distribution shift remains difficult to monitor and optimize in the absence of target-domain labels, as models with similar in-distribution accuracy can exhibit markedly different out-of-distribution (OOD) performance. While prior work has focused on training-time regularization and low-order representation statistics, little is known about whether the
geometric structure of learned
embeddings provides reliable post-hoc signals of robustness. We propose a geometry-based diagnostic framework that constructs class-conditional mutual k-nearest-neighbor graphs from in-distribution
embeddings and extracts two complementary invariants: a global spectral complexity proxy based on the reduced log-determinant of the normalized
Laplacian, and a local smoothness measure based on Ollivier–
Ricci curvature. Across multiple architectures, training regimes, and corruption
benchmarks, we find that lower spectral complexity and higher mean
curvature consistently predict stronger OOD accuracy across checkpoints. Controlled perturbations and topological analyses further show that these signals reflect meaningful representation structure rather than superficial
embedding statistics. Our results demonstrate that representation geometry enables interpretable, label-free robustness diagnosis and supports reliable unsupervised checkpoint selection under distribution shift.
Read the full abstract ▾
Computer Vision
Entropy Reveals Block Importance in Masked Self-Supervised Vision Transformers
Peihao Xiang, Kaida Wu and Ou Bai
Masked self-supervised
vision transformers have become a dominant pretraining paradigm, yet their substantial model size poses significant challenges for resource-constrained deployment and efficient transfer learning. A fundamental question remains: are all
transformer blocks equally important for downstream performance? In this paper, we show that block importance in masked self-supervised
vision transformers can be accurately estimated without access to any data. Our key finding is that the information
entropy of pretrained block weights strongly correlates with oracle sensitivity obtained via iterative block removal and finetuning. This observation enables Gardener, a data-free, one-shot, block-level
pruning principle that identifies redundant blocks through simple information-theoretic measurements. We evaluate Gardener on VideoMAE-B across multiple
pruning ratios and downstream
video recognition
benchmarks. Despite its negligible computational overhead, Gardener consistently matches or outperforms existing data-free
pruning baselines and closely approaches sensitivity-based
pruning. Remarkably, even after
pruning up to 91.7% of blocks, the pruned model retains competitive transfer performance. Our results reveal substantial block-level redundancy in masked self-supervised
vision transformers and demonstrate that information-theoretic analysis offers a principled and efficient pathway for model compression and resource-efficient transfer learning.
Read the full abstract ▾
Computer Vision
SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
Azmine Toushik Wasi, Wahid Faisal, Abdur Rahman et al. (+12 more)
Spatial
reasoning is a fundamental aspect of human cognition, yet it remains a major challenge for contemporary
vision-
language models (VLMs). Prior work largely relied on synthetic or LLM-generated environments with limited task designs and puzzle-like setups, failing to capture the real-world complexity, visual noise, and diverse spatial relationships that VLMs encounter. To address this, we introduce SpatiaLab, a comprehensive
benchmark for evaluating VLMs' spatial
reasoning in realistic, unconstrained contexts. SpatiaLab comprises 1,400 visual question-answer pairs across six major categories: Relative Positioning, Depth & Occlusion, Orientation, Size & Scale, Spatial
Navigation, and 3D Geometry, each with five subcategories, yielding 30 distinct task types. Each subcategory contains at least 25 questions, and each main category includes at least 200 questions, supporting both multiple-choice and open-ended evaluation. Experiments across diverse state-of-the-art VLMs, including open- and closed-source models,
reasoning-focused, and specialized spatial
reasoning models, reveal a substantial gap in spatial
reasoning capabilities compared with humans. In the multiple-choice setup, InternVL3.5-72B achieves 54.93% accuracy versus 87.57% for humans. In the open-ended setting, all models show a performance drop of around 10-25%, with GPT-5-mini scoring highest at 40.93% versus 64.93% for humans. These results highlight key limitations in handling complex spatial relationships, depth perception,
navigation, and 3D geometry. By providing a diverse, real-world evaluation framework, SpatiaLab exposes critical challenges and opportunities for advancing VLMs' spatial
reasoning, offering a
benchmark to guide future research toward robust, human-aligned spatial understanding. SpatiaLab is available at: https://spatialab-
reasoning.github.io/.
Read the full abstract ▾
All authors Azmine Toushik Wasi, Wahid Faisal, Abdur Rahman, Mahfuz Ahmed Anik, Munem Shahriar, Mohsin Mahmud Topu, Sadia Tasnim Meem, Rahatun Nesa Priti, Sabrina Afroz Mitu, Md. Iqramul Hoque, Shahriyar Zaman Ridoy, Mohammed Eunus Ali, Majd Hawasly, Mohammad Raza, Md Rizwan Parvez
Computer Vision
Phaedra: Learning High-Fidelity Discrete Tokenization for the Physical Science
Levi Lingsch, Georgios Kissas, Johannes Jakubik et al. (+1 more)
Tokens are discrete representations that allow modern deep learning to scale by transforming high-dimensional data into sequences that can be efficiently learned, generated, and generalized to new tasks. While foundational for image and
video generation, the application of
tokens to physical simulation remains nascent. Because existing tokenizers are designed for the perceptual requirements of natural images, they struggle with scientific data, which exhibits large dynamic ranges and requires exact preservation of physical and spectral properties. In this work, we investigate the performance of a suite of image tokenizers across metrics designed to measure PDE fidelity. Observing that these baselines struggle to simultaneously capture fine
geometric details and precise physical magnitudes, we propose Phaedra, a novel tokenizer inspired by classical shape-gain quantization and the paradigm of basis functions coupled with continuous coefficients. Phaedra acts as a highly effective nonlinear compression
algorithm, massively reducing
dataset footprints while maintaining physical fidelity. We demonstrate that Phaedra consistently improves reconstruction across diverse 2D gridded PDE solutions, generalizes robustly to unseen PDE types and real-world Earth observation data, and is competitive with continuous models in downstream proof-of-concept operator learning and masked autoencoding tasks.
Read the full abstract ▾
All authors Levi Lingsch, Georgios Kissas, Johannes Jakubik, Siddhartha Mishra
Computer Vision
Entropy-Aware Structural Alignment for Zero-Shot Handwritten Chinese Character Recognition
Qiuming Luo, Tao Zeng, Feng Li et al. (+3 more)
Zero-shot Handwritten Chinese Character Recognition (HCCR) aims to recognize unseen characters by leveraging radical-based semantic compositions. However, existing approaches often treat characters as flat radical sequences, neglecting the hierarchical
topology and the uneven information density of different components. To address these limitations, we propose an
Entropy-Aware Structural
Alignment Network that bridges the visual-semantic gap through information-theoretic modeling. First, we introduce an Information
Entropy Prior to dynamically modulate positional
embeddings via multiplicative interaction, acting as a saliency detector that prioritizes discriminative roots over ubiquitous components. Second, we construct a Dual-View Radical Tree to extract multi-granularity structural features, which are integrated via an adaptive Sigmoid-based gating network to encode both global layout and local spatial roles. Finally, a Top-K Semantic Feature Fusion mechanism is devised to augment the decoding process by utilizing the centroid of semantic neighbors, effectively rectifying visual ambiguities through feature-level consensus. Extensive experiments demonstrate that our method establishes new state-of-the-art performance, achieving an accuracy of 55.04% on the ICDAR 2013
dataset ($m=1500$), significantly outperforming existing CLIP-based baselines in the challenging zero-shot setting. Furthermore, the framework exhibits exceptional data efficiency, demonstrating rapid adaptability with minimal support samples, achieving 92.41% accuracy with only one support sample per class.
Read the full abstract ▾
All authors Qiuming Luo, Tao Zeng, Feng Li, Heming Liu, Rui Mao, Chang Kong
Computer Vision · Cross-listed from cs.RO
Beyond the Vehicle: Cooperative Localization by Fusing Point Clouds for GPS-Challenged Urban Scenarios
Kuo-Yi Chao, Ralph Rasshofer and Alois Christian Knoll
Accurate vehicle localization is a critical challenge in urban environments where GPS signals are often unreliable. This paper presents a cooperative multi-sensor and multi-modal localization approach to address this issue by fusing data from vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) systems. Our approach integrates cooperative data with a point cloud registration-based simultaneous localization and mapping (SLAM)
algorithm. The system processes point clouds generated from diverse sensor modalities, including vehicle-mounted
LiDAR and stereo cameras, as well as sensors deployed at intersections. By leveraging shared data from infrastructure, our method significantly improves localization accuracy and robustness in complex, GPS-noisy urban scenarios.
Read the full abstract ▾
Computer Vision
HY3D-Bench: Generation of 3D Assets
Team Hunyuan3D: Bowen Zhang, Chunchao Guo, Dongyuan Guo et al. (+20 more)
While recent advances in neural representations and generative models have revolutionized 3D content creation, the field remains constrained by significant data processing bottlenecks. To address this, we introduce HY3D-Bench, an open-source ecosystem designed to establish a unified, high-quality foundation for 3D generation. Our contributions are threefold: (1) We curate a library of 250k high-fidelity 3D objects distilled from large-scale repositories, employing a rigorous pipeline to deliver training-ready artifacts, including watertight meshes and multi-view renderings; (2) We introduce structured part-level decomposition, providing the granularity essential for fine-grained perception and controllable editing; and (3) We bridge real-world distribution gaps via a scalable AIGC synthesis pipeline, contributing 125k synthetic assets to enhance diversity in long-tail categories. Validated empirically through the training of Hunyuan3D-2.1-Small, HY3D-Bench democratizes access to robust data resources, aiming to catalyze innovation across 3D perception, robotics, and digital content creation.
Read the full abstract ▾
All authors Team Hunyuan3D: Bowen Zhang, Chunchao Guo, Dongyuan Guo, Haolin Liu, Hongyu Yan, Huiwen Shi, Jiaao Yu, Jiachen Xu, Jingwei Huang, Kunhong Li, Lifu Wang, Linus, Penghao Wang, Qingxiang Lin, Ruining Tang, Xianghui Yang, Yang Li, Yirui Guan, Yunfei Zhao, Yunhan Yang, Zeqiang Lai, Zhihao Liang, Zibo Zhao
Computer Vision
Benchmarking Bias Mitigation Toward Fairness Without Harm from Vision to LVLMs
Xuwei Tan, Ziyu Hu and Xueru Zhang
Machine learning models trained on real-world data often inherit and amplify biases against certain social groups, raising urgent concerns about their deployment at scale. While numerous bias mitigation methods have been proposed, comparing the effectiveness of bias mitigation methods remains difficult due to heterogeneous
datasets, inconsistent
fairness metrics, isolated evaluation of
vision versus multi-modal models, and insufficient hyperparameter tuning that undermines fair comparisons. We introduce NH-Fair, a unified
benchmark for
fairness without harm that spans both
vision models and large
vision-
language models (LVLMs) under standardized data, metrics, and training protocols, covering supervised and zero-shot regimes. Our key contributions are: (1) a systematic ERM tuning study that identifies training choices with large influence on both utility and disparities, yielding empirically grounded guidelines to help practitioners reduce expensive hyperparameter tuning space in achieving strong
fairness and accuracy; (2) evidence that many debiasing methods do not reliably outperform a well-tuned ERM baseline, whereas a composite data-augmentation method consistently delivers parity gains without sacrificing utility, emerging as a promising practical strategy. (3) an analysis showing that while LVLMs achieve higher average accuracy, they still exhibit subgroup disparities, and gains from scaling are typically smaller than those from architectural or training-protocol choices. NH-Fair provides a reproducible, tuning-aware pipeline for rigorous, harm-aware
fairness evaluation.
Read the full abstract ▾
Computer Vision
Vision Transformers for Zero-Shot Clustering of Animal Images: A Comparative Benchmarking Study
Hugo Markoff, Stefan Hein Bengtson and Michael Ørsted
Manual labelling of animal images remains a significant bottleneck in ecological research, limiting the scale and efficiency of biodiversity monitoring efforts. This study investigates whether state-of-the-art
Vision Transformer (ViT) foundation models can reduce thousands of unlabelled animal images directly to species-level clusters. We present a comprehensive benchmarking framework evaluating five ViT models combined with five dimensionality reduction techniques and four clustering
algorithms, two supervised and two unsupervised, across 60 species (30 mammals and 30 birds), with each test using a random subset of 200 validated images per species. We investigate when clustering succeeds at species-level, where it fails, and whether clustering within the species-level reveals ecologically meaningful patterns such as sex, age, or phenotypic variation. Our primary, fully unsupervised finding is that DINOv3
embeddings with t-SNE and HDBSCAN reach a species-level V-measure of 0.943 with no prior knowledge of the number of species, rejecting only 1.14% of images as outliers requiring expert review. Assigning one label per cluster reduces manual labelling actions by 98.3%, with 94.6% of clustered images labelled correctly, close to the accuracy of manual annotation. When the true number of species is supplied, supervised hierarchical clustering raises this to 0.958, which we report as an upper bound as the species count is typically unknown in real deployments. We further demonstrate robustness to realistic long-tailed distributions of species and show that intentional over-clustering can reveal intra-specific variation including age classes, sexual dimorphism, and pelage differences. We introduce an open-source benchmarking toolkit and provide recommendations for ecologists to select appropriate methods for sorting their specific taxonomic groups and data.
Read the full abstract ▾