Computer Vision · Cross-listed from eess.IV
Guidestar-Free Adaptive Optics with Asymmetric Apertures
Weiyun Jiang, Haiyun Guo, Christopher A. Metzler et al. (+1 more)
This work introduces the first closed-loop adaptive optics (AO) system capable of optically correcting aberrations in real-time without a guidestar or a wavefront sensor. Nearly 40 years ago, Cederquist et al. demonstrated that asymmetric apertures enable phase
retrieval (PR)
algorithms to perform fully computational wavefront sensing, albeit at a high computational cost. More recently, Chimitt et al. extended this approach with machine learning and demonstrated real-time wavefront sensing using only a single (guidestar-based) point-spread-function (PSF) measurement. Inspired by these works, we introduce a guidestar-free AO framework built around asymmetric apertures and machine learning. Our approach combines three key elements: (1) an asymmetric aperture placed at the system's pupil plane that enables PR-based wavefront sensing, (2) a pair of machine learning
algorithms that estimate the PSF from natural scene measurements and reconstruct phase aberrations, and (3) a spatial light modulator that performs optical correction. We experimentally validate this framework on dense natural scenes imaged through unknown obscurants. Our method outperforms state-of-the-art guidestar-free wavefront shaping methods, using an order of magnitude fewer measurements and three orders of magnitude less computation.
Read the full abstract ▾
All authors Weiyun Jiang, Haiyun Guo, Christopher A. Metzler, Ashok Veeraraghavan
Computer Vision
A Comparative Study of Adversarial Robustness in CNN and CNN-ANFIS Architectures
Kaaustaaub Shankar, Bharadwaj Dogga and Kelly Cohen
Convolutional Neural Networks (CNNs) achieve strong image classification performance but lack interpretability and are vulnerable to
adversarial attacks. Neuro-fuzzy hybrids such as DCNFIS replace fully connected CNN classifiers with Adaptive Neuro-Fuzzy Inference Systems (ANFIS) to improve interpretability, yet their robustness remains underexplored. This work compares standard CNNs (ConvNet, VGG, ResNet18) with their ANFIS-augmented counterparts on MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 under gradient-based (PGD) and gradient-free (Square) attacks. Results show that ANFIS integration does not consistently improve clean accuracy and has architecture-dependent effects on robustness: ResNet18-ANFIS exhibits improved
adversarial robustness, while VGG-ANFIS often underperforms its baseline. These findings suggest that neuro-fuzzy augmentation can enhance robustness in specific architectures but is not universally beneficial.
Read the full abstract ▾
Computer Vision
Fair Context Learning for Evidence-Balanced Test-Time Adaptation in Vision-Language Models
Sanggeon Yun, Ryozo Masukawa, SungHeon Jeong et al. (+3 more)
Vision-
Language Models (VLMs) such as CLIP enable strong zero-shot recognition but suffer substantial degradation under distribution shifts. Test-Time Adaptation (TTA) aims to improve robustness using only unlabeled test samples, yet most
prompt-based TTA methods rely on
entropy minimization – an approach that can amplify spurious correlations and induce overconfident errors when classes share visual features. We propose Fair Context Learning (FCL), an episodic TTA framework that avoids
entropy minimization by explicitly addressing shared-evidence bias. Motivated by our additive evidence decomposition assumption, FCL decouples adaptation into (i) augmentation-based exploration to identify plausible class candidates, and (ii)
fairness-driven calibration that adapts text contexts to equalize sensitivity to common visual evidence. This
fairness constraint mitigates partial feature obsession and enables effective calibration of text
embeddings without relying on
entropy reduction. Through extensive evaluation, we empirically validate our theoretical motivation and show that FCL achieves competitive adaptation performance relative to state-of-the-art TTA methods across diverse domain-shift and fine-grained
benchmarks.
Read the full abstract ▾
All authors Sanggeon Yun, Ryozo Masukawa, SungHeon Jeong, Wenjun Huang, Hanning Chen, Mohsen Imani
Computer Vision
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
Xiaomin Yu, Yi Xin, Yuhui Zhang et al. (+12 more)
Despite the success of
multimodal contrastive learning in aligning visual and linguistic representations, a persistent
geometric anomaly, the Modality Gap, remains:
embeddings of distinct modalities expressing identical semantics occupy systematically offset regions. Prior approaches to bridge this gap are largely limited by oversimplified isotropic assumptions, hindering their application in large-scale scenarios. In this paper, we address these limitations by precisely characterizing the
geometric shape of the modality gap and leveraging it for efficient model scaling. First, we propose the Fixed-frame Modality Gap Theory, which decomposes the modality gap within a frozen reference frame into stable biases and anisotropic residuals. Guided by this precise modeling, we introduce ReAlign, a training-free modality
alignment strategy. Utilizing statistics from massive unpaired data, ReAlign aligns text representation into the image representation distribution via a three-step process comprising Anchor, Trace, and Centroid
Alignment, thereby explicitly rectifying
geometric misalignment. Building on ReAlign, we propose ReVision, a scalable training paradigm for
Multimodal Large
Language Models~(MLLMs). ReVision integrates ReAlign into the pretraining stage, enabling the model to learn the distribution of visual representations from unpaired text before visual instruction tuning, without the need for large-scale, high-quality image-text pairs. Our framework demonstrates that statistically aligned unpaired data can effectively substitute for expensive image-text pairs, offering a robust path for the efficient scaling of MLLMs.
Read the full abstract ▾
All authors Xiaomin Yu, Yi Xin, Yuhui Zhang, Wenjie Zhang, Chonghan Liu, Hanzhen Zhao, Chen Liu, Xiaoxing Hu, Ziyue Qiao, Hao Tang, Xiaobin Hu, Chengwei Qin, Hui Xiong, Yu Qiao, Shuicheng Yan
Computer Vision
The Geometry of Representational Failures in Vision Language Models
Daniele Savietto, Declan Campbell, André Panisson et al. (+4 more)
Vision-
Language Models (VLMs) exhibit puzzling failures in multi-object visual tasks, such as hallucinating non-existent elements or failing to identify the most similar objects among distractions. While these errors mirror human cognitive constraints, such as the 'Binding Problem', the internal mechanisms driving them in artificial systems remain poorly understood. Here, we propose a mechanistic insight by analyzing the representational geometry of open-weight VLMs (Qwen, InternVL, Gemma), comparing methodologies to distill "concept vectors” - latent directions encoding visual concepts. We validate our concept vectors via steering interventions that reliably manipulate model behavior in both simplified and naturalistic
vision tasks (e.g., forcing the model to perceive a red flower as blue). We observe that the
geometric overlap between these vectors strongly correlates with specific error patterns, offering a grounded quantitative framework to understand how internal representations shape model behavior and drive visual failures.
Read the full abstract ▾
All authors Daniele Savietto, Declan Campbell, André Panisson, Marco Nurisso, Giovanni Petri, Jonathan D. Cohen, Alan Perotti
Computer Vision · Cross-listed from cs.RO
A Distributed Multi-Modal Sensing Approach for Human Activity Recognition in Real-Time Human-Robot Collaboration
Valerio Belcamino, Nhat Minh Dinh Le, Quan Khanh Luu et al. (+3 more)
Human activity recognition (HAR) is fundamental in human-
robot collaboration (HRC), enabling
robots to respond to and dynamically adapt to human intentions. This paper introduces a HAR system combining a modular data glove equipped with Inertial Measurement Units and a
vision-based tactile sensor to capture hand activities in contact with a
robot. We tested our activity recognition approach under different conditions, including offline classification of segmented sequences, real-time classification under static conditions, and a realistic HRC scenario. The experimental results show a high accuracy for all the tasks, suggesting that multiple collaborative settings could benefit from this multi-modal approach.
Read the full abstract ▾
All authors Valerio Belcamino, Nhat Minh Dinh Le, Quan Khanh Luu, Alessandro Carfì, Van Anh Ho, Fulvio Mastrogiovanni
Computer Vision · Cross-listed from eess.IV
Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss
Yucheng Zhou, Hao Li and Jianbing Shen
Recent studies have explored autoregressive models for image generation, with promising results, and have combined
diffusion models with autoregressive frameworks to optimize image generation via
diffusion losses. In this study, we present a theoretical analysis of
diffusion and autoregressive models with
diffusion loss, highlighting the latter's advantages. We present a theoretical comparison of conditional
diffusion and autoregressive
diffusion with
diffusion loss, demonstrating that patch denoising optimization in autoregressive models effectively mitigates condition errors and leads to a stable condition distribution. Our analysis also reveals that autoregressive condition generation refines the condition, causing the condition error influence to decay exponentially. In addition, we introduce a novel condition refinement approach based on Optimal Transport (OT) theory to address “condition inconsistency”. We theoretically demonstrate that formulating condition refinement as a Wasserstein Gradient Flow ensures convergence toward the ideal condition distribution, effectively mitigating condition inconsistency. Experiments demonstrate the superiority of our method over
diffusion and autoregressive models with
diffusion loss methods.
Read the full abstract ▾
Computer Vision
DiGAN: Diffusion-Guided Attention Network for Early Alzheimer's Disease Detection
Maxx Richard Rahman, Mostafa Hammouda and Wolfgang Maass
Early diagnosis of Alzheimer's disease (AD) remains a major challenge due to the subtle and temporally irregular progression of structural
brain changes in the prodromal stages. Existing deep learning approaches require large longitudinal
datasets and often fail to model the temporal continuity and modality irregularities inherent in real-world clinical data. To address these limitations, we propose the
Diffusion-Guided
Attention Network (DiGAN), which integrates latent
diffusion modelling with an
attention-guided convolutional network. The
diffusion model synthesizes realistic longitudinal neuroimaging trajectories from limited training data, enriching temporal context and improving robustness to unevenly spaced visits. The
attention-convolutional layer then captures discriminative structural-temporal patterns that distinguish cognitively normal subjects from those with mild cognitive impairment and subjective cognitive decline. Experiments on the ADNI
dataset demonstrate that DiGAN outperforms existing state-of-the-art baselines, showing its potential for early-stage AD detection.
Read the full abstract ▾
Computer Vision
TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions
Ali Bayeh, Samira Sadaoui and Malek Mouhoub
To address the trade-off between computational efficiency and adherence to Kolmogorov-Arnold Network (KAN) principles, we propose TruKAN, a new architecture based on the KAN structure and learnable activation functions. TruKAN replaces the B-spline basis in KAN with a family of truncated power functions derived from k-order spline theory. This change maintains the KAN's expressiveness while enhancing accuracy and training time. Each TruKAN layer combines a truncated power term with a
polynomial term and employs either shared or individual
knots. TruKAN exhibits greater interpretability than other KAN variants due to its simplified basis functions and
knot configurations. By prioritizing interpretable basis functions, TruKAN aims to balance approximation efficacy with transparency. We develop the TruKAN model and integrate it into an advanced EfficientNet-V2-based framework, which is then evaluated on computer
vision benchmark datasets. To ensure a fair comparison, we develop various models: MLP-, KAN-, SineKAN and TruKAN-based EfficientNet frameworks and assess their training time and accuracy across small and deep architectures. The training phase uses hybrid optimization to improve convergence stability. Additionally, we investigate layer normalization techniques for all the models and assess the impact of shared versus individual
knots in TruKAN. Overall, TruKAN outperforms other KAN models in terms of accuracy, computational efficiency and
memory usage on the complex
vision task, demonstrating advantages beyond the limited settings explored in prior KAN studies.
Read the full abstract ▾
Computer Vision
Intellectual Property Protection for 3D Gaussian Splatting Assets: A Survey
Longjie Zhao, Ziming Hong, Jiaxin Huang et al. (+3 more)
3D Gaussian Splatting (3DGS) has become a mainstream representation for real-time 3D scene synthesis, enabling applications in virtual and augmented reality, robotics, and 3D content creation. Its rising commercial value and explicit parametric structure raise emerging intellectual property (IP) protection concerns, prompting a surge of research on 3DGS IP protection. However, current progress remains fragmented, lacking a unified view of the underlying mechanisms, protection paradigms, and robustness challenges. To address this gap, we present the first systematic survey on 3DGS IP protection and introduce a bottom-up framework that examines (i) underlying Gaussian-based perturbation mechanisms, (ii) passive and active protection paradigms, and (iii) robustness threats under emerging generative AI era, revealing gaps in technical foundations and robustness characterization and indicating opportunities for deeper investigation. Finally, we outline six research directions across robustness, efficiency, and protection paradigms, offering a roadmap toward reliable and trustworthy IP protection for 3DGS assets.
Read the full abstract ▾
All authors Longjie Zhao, Ziming Hong, Jiaxin Huang, Runnan Chen, Mingming Gong, Tongliang Liu
Computer Vision · Cross-listed from cs.LG
A Reproducible Framework for Bias-Resistant Machine Learning on Small-Sample Neuroimaging Data
Jagan Mohan Reddy Dwarampudi, Jennifer L Purks, Joshua Wong et al. (+2 more)
We introduce a reproducible, bias-resistant machine learning framework that integrates domain-informed feature engineering, nested cross-validation, and calibrated decision-threshold optimization for small-sample neuroimaging data. Conventional cross-validation frameworks that reuse the same folds for both model selection and performance estimation yield optimistically biased results, limiting reproducibility and generalization. Demonstrated on a high-dimensional structural MRI
dataset of deep
brain stimulation cognitive outcomes, the framework achieved a nested-CV balanced accuracy of 0.660\,$\pm$\,0.068 using a compact, interpretable subset selected via importance-guided ranking. By combining interpretability and unbiased evaluation, this work provides a generalizable computational blueprint for reliable machine learning in data-limited biomedical domains.
Read the full abstract ▾
All authors Jagan Mohan Reddy Dwarampudi, Jennifer L Purks, Joshua Wong, Renjie Hu, Tania Banerjee
Computer Vision
A Multi-scale Linear-time Encoder for Whole-Slide Image Analysis
Jagan Mohan Reddy Dwarampudi, Joshua Wong, Hien Van Nguyen et al. (+1 more)
We introduce Multi-scale Adaptive Recurrent Biomedical Linear-time Encoder (MARBLE), the first purely Mamba-based multi-state multiple instance learning (MIL) framework for whole-slide image (WSI) analysis. MARBLE processes multiple magnification levels in parallel and integrates coarse-to-fine
reasoning within a linear-time state-space model, efficiently capturing cross-scale dependencies with minimal parameter overhead. WSI analysis remains challenging due to gigapixel resolutions and hierarchical magnifications, while existing MIL methods typically operate at a single scale and
transformer-based approaches suffer from quadratic
attention costs. By coupling parallel multi-scale processing with linear-time sequence modeling, MARBLE provides a scalable and modular alternative to
attention-based architectures. Experiments on five public
datasets show improvements of up to 6.9% in AUC, 20.3% in accuracy, and 2.3% in C-index, establishing MARBLE as an efficient and generalizable framework for multi-scale WSI analysis.
Read the full abstract ▾
All authors Jagan Mohan Reddy Dwarampudi, Joshua Wong, Hien Van Nguyen, Tania Banerjee
Computer Vision
FaceLinkGen: A Re-evaluation of Identity Leakage in Privacy-Preserving Face Recognition and Face Anonymization Systems Using Simple Distillation
Wenqi Guo, Qingyun Qian, Mohamed Shehata et al. (+1 more)
Privacy-preserving face recognition (PPFR) and face anonymization have different goals, but both must retain some identity-related information for their intended use. We show that an adaptive attacker can learn this information. We propose FaceLinkGen, a simple
distillation-based attack that trains a face recognition model to map protected inputs back to standard face
embeddings. FaceLinkGen applies to keyless PPFR systems and perception-preserving face de-identification (De-ID) systems. For PPFR, the recovered
embeddings can be used to regenerate faces that match the original person. Across MinusFace, PartialFace, and DecoyFace, the regenerated faces achieve acceptance rates of 81.0–99.4% on Face++ and 74.9–99.6% on Amazon. For De-ID, FaceLinkGen links protected faces to unprotected images of the same person, reaching Recall@1 values of 48.4–89.6% in the one-side-protected setting across the evaluated methods. FaceLinkGen exposes identity leakage across all evaluated methods, including DecoyFace and WDP, whose protection resists the tested U-Net attacks on face recovery and protected-to-unprotected linkage, respectively. The attack also remains effective when trained with limited paired data. Code is available at https://github.com/weathon/FaceLinkGenRelease.
Read the full abstract ▾
All authors Wenqi Guo, Qingyun Qian, Mohamed Shehata, Shan Du
Computer Vision · Cross-listed from cs.LG
A Random Matrix Theory Perspective on the Consistency of Diffusion Models
Binxu Wang, Jacob Zavatone-Veth and Cengiz Pehlevan
Diffusion models trained on different, non-overlapping subsets of a
dataset often produce strikingly similar outputs when given the same noise seed. We trace this consistency to a simple linear effect: the shared Gaussian statistics across splits already predict much of the generated images. To formalize this, we develop a random
matrix theory (RMT) framework that quantifies how finite
datasets shape the expectation and variance of the learned denoiser and sampling map in the linear setting. For expectations, sampling variability acts as a renormalization of the noise level through a self-consistent relation $\sigma^2 \mapsto \kappa(\sigma^2)$, explaining why limited data overshrink low-variance directions and pull samples toward the
dataset mean. For fluctuations, our variance formulas reveal three key factors behind cross-split disagreement: anisotropy across eigenmodes, inhomogeneity across inputs, and overall scaling with
dataset size. Extending deterministic-equivalence tools to fractional
matrix powers further allows us to analyze entire sampling trajectories. The theory sharply predicts the behavior of linear
diffusion models, and we validate its predictions on UNet and DiT architectures in their non-memorization regime, identifying where and how samples deviates across training data split. This provides a principled baseline for reproducibility in
diffusion training, linking spectral properties of data to the stability of generative outputs.
Read the full abstract ▾
Computer Vision
DoubleTake: Contrastive Reasoning for Faithful Decision-Making in Medical Imaging
Daivik Patel and Shrenik Patel
Accurate decision making in medical imaging requires
reasoning over subtle visual differences between confusable conditions, yet most existing approaches rely on nearest neighbor
retrieval that returns redundant evidence and reinforces a single hypothesis. We introduce a contrastive, document-aware reference selection framework that constructs compact evidence sets optimized for discrimination rather than similarity by explicitly balancing visual relevance,
embedding diversity, and source-level provenance using ROCO
embeddings and metadata. While ROCO provides large-scale image-caption pairs, it does not specify how references should be selected for contrastive
reasoning, and naive
retrieval frequently yields near-duplicate figures from the same document. To address this gap, we release a reproducible reference selection protocol and curated reference bank that enable a systematic study of contrastive
retrieval in medical image
reasoning. Building on these contrastive evidence sets, we propose Counterfactual-Contrastive Inference, a confidence-aware
reasoning framework that performs structured pairwise visual comparisons and aggregates evidence using margin-based decision rules with faithful abstention. On the MediConfusion
benchmark, our approach achieves state-of-the-art performance, improving set-level accuracy by nearly 15% relative to prior methods while reducing confusion and improving individual accuracy.
Read the full abstract ▾
Computer Vision
ViThinker: Active Vision-Language Reasoning via Dynamic Perceptual Querying
Weihang You, Qingchan Zhu, David Liu et al. (+3 more)
Chain-of-Thought (CoT)
reasoning excels in
language models but struggles in
vision-
language models due to premature visual-to-text conversion that discards continuous information such as geometry and spatial layout. While recent methods enhance CoT through static enumeration or
attention-based selection, they remain passive, i.e., processing pre-computed inputs rather than actively seeking task-relevant details. Inspired by human active perception, we introduce ViThinker, a framework that enables
vision-
language models to autonomously generate decision (query)
tokens triggering the synthesis of expert-aligned visual features on demand. ViThinker internalizes
vision-expert capabilities during training, performing generative mental simulation during inference without external tool calls. Through a two-stage curriculum: first distilling frozen experts into model parameters, then learning task-driven querying via sparsity penalties, i.e., ViThinker discovers minimal sufficient perception for each
reasoning step. Evaluations across
vision-centric
benchmarks demonstrate consistent improvements, validating that active query generation outperforms passive approaches in both perceptual grounding and
reasoning accuracy.
Read the full abstract ▾
All authors Weihang You, Qingchan Zhu, David Liu, Yi Pan, Geng Yuan, Hanqi Jiang
Computer Vision
Self-Supervised Uncalibrated Multi-View Video Anonymization in the Operating Room
Keqi Chen, Vinkle Srivastav, Armine Vardazaryan et al. (+3 more)
Privacy preservation is a prerequisite for using
video data in Operating Room (OR) research. Effective anonymization relies on the exhaustive localization of every individual; even a single missed detection necessitates extensive manual correction. However, existing approaches face two critical scalability bottlenecks: (1) they usually require manual annotations of each new clinical site for high accuracy; (2) while multi-camera setups have been widely adopted to address single-view ambiguity, camera calibration is typically required whenever cameras are repositioned. To address these problems, we propose a self-supervised multi-view
video anonymization framework consisting of whole-body person detection and whole-body pose estimation, without annotation or camera calibration. Our core strategy is to enhance the single-view detector by "retrieving" false negatives using temporal and multi-view context, and conducting self-supervised domain adaptation. We first run an off-the-shelf whole-body person detector in each view with a low-score threshold to gather candidate detections. Then, we retrieve the low-score false negatives that exhibit consistency with the high-score detections via tracking and self-supervised uncalibrated multi-view association. These recovered detections serve as pseudo labels to iteratively fine-tune the whole-body detector. Finally, we apply whole-body pose estimation on each detected person, and fine-tune the pose model using its own high-score predictions. Experiments on the 4D-OR
dataset of simulated surgeries and our
dataset of real surgeries show the effectiveness of our approach achieving 99% and 97% recall, respectively. Moreover, we train a real-time whole-body detector using our pseudo labels, achieving comparable performance and highlighting our method's practical applicability. Code will be available at https://github.com/CAMMA-public/OR_anonymization.
Read the full abstract ▾
All authors Keqi Chen, Vinkle Srivastav, Armine Vardazaryan, Cindy Rolland, Didier Mutter, Nicolas Padoy
Computer Vision · Cross-listed from cs.LG
From Tokens to Numbers: Continuous Number Modeling for SVG Generation
Michael Ogezi, Martin Bell, Freda Shi et al. (+1 more)
For certain image generation tasks, vector graphics such as Scalable Vector Graphics (SVGs) offer clear benefits such as increased flexibility, size efficiency, and editing ease, but remain less explored than raster-based approaches. A core challenge is that the numerical,
geometric parameters, which make up a large proportion of SVGs, are inefficiently encoded as long sequences of
tokens. This slows training, reduces accuracy, and hurts generalization. To address these problems, we propose Continuous Number Modeling (CNM), an approach that directly models numbers as first-class, continuous values rather than discrete
tokens. This formulation restores the mathematical elegance of the representation by aligning the model's inputs with the data's continuous nature, removing discretization artifacts introduced by
token-based encoding. We then train a
multimodal transformer on 2 million raster-to-SVG samples, followed by fine-tuning via reinforcement learning using perceptual feedback to further improve visual quality. Our approach improves training speed by over 30% while maintaining higher perceptual fidelity compared to alternative approaches. This work establishes CNM as a practical and efficient approach for high-quality vector generation, with potential for broader applications. We make our code available http://github.com/mikeogezi/CNM.
Read the full abstract ▾
All authors Michael Ogezi, Martin Bell, Freda Shi, Ethan Smith
Computer Vision
LmPT: Conditional Point Transformer for Anatomical Landmark Detection on 3D Point Clouds
Matteo Bastico, Pierre Onghena, David Ryckelynck et al. (+5 more)
Accurate identification of anatomical landmarks is crucial for various medical applications. Traditional manual landmarking is time-consuming and prone to inter-observer variability, while rule-based methods are often tailored to specific geometries or limited sets of landmarks. In recent years, anatomical surfaces have been effectively represented as point clouds, which are lightweight structures composed of spatial coordinates. Following this strategy and to overcome the limitations of existing landmarking techniques, we propose Landmark Point
Transformer (LmPT), a method for automatic anatomical landmark detection on point clouds that can leverage homologous bones from different species for translational research. The LmPT model incorporates a conditioning mechanism that enables adaptability to different input types to conduct cross-species learning. We focus the evaluation of our approach on femoral landmarking using both human and newly annotated dog femurs, demonstrating its generalization and effectiveness across species. The code and dog femur
dataset will be publicly available at: https://github.com/Pierreoo/LandmarkPointTransformer.
Read the full abstract ▾
All authors Matteo Bastico, Pierre Onghena, David Ryckelynck, Beatriz Marcotegui, Santiago Velasco-Forero, Laurent Corté, Caroline Robine–Decourcelle, Etienne Decencière
Computer Vision · Cross-listed from eess.IV
Real-time topology-aware M-mode OCT segmentation for robotic deep anterior lamellar keratoplasty (DALK) guidance
Rosalinda Xiong, Jinglun Yu, Yaning Wang et al. (+2 more)
Robotic deep anterior lamellar keratoplasty (DALK) requires accurate real time depth feedback to approach Descemet's membrane (DM) without perforation. M-mode intraoperative optical coherence
tomography (OCT) provides high temporal resolution depth traces, but speckle noise, attenuation, and instrument induced shadowing often result in discontinuous or ambiguous layer interfaces that challenge anatomically consistent
segmentation at deployment frame rates. We present a lightweight,
topology aware M-mode
segmentation pipeline based on UNeXt that incorporates anatomical
topology regularization to stabilize boundary continuity and layer ordering under low signal to noise ratio conditions. The proposed system achieves end to end throughput exceeding 80 Hz measured over the complete preprocessing inference overlay pipeline on a single GPU, demonstrating practical real time guidance beyond model only timing. This operating margin provides temporal headroom to reject low quality or dropout frames while maintaining a stable effective depth update rate. Evaluation on a standard rabbit eye M-mode
dataset using an established baseline protocol shows improved qualitative boundary stability compared with
topology agnostic controls, while preserving deployable real time performance.
Read the full abstract ▾
All authors Rosalinda Xiong, Jinglun Yu, Yaning Wang, Ziyi Huang, Jin U. Kang
Computer Vision
SVD-ViT: Does SVD Make Vision Transformers Attend More to the Foreground?
Haruhiko Murata and Kazuhiro Hotta
Vision Transformers (ViT) have been established as large-scale foundation models. However, because self-
attention operates globally, they lack an explicit mechanism to distinguish foreground from background. As a result, ViT may learn unnecessary background features and artifacts, leading to degraded classification performance. To address this issue, we propose SVD-ViT, which leverages singular value decomposition (SVD) to prioritize the learning of foreground features. SVD-ViT consists of three components-SPC module, SSVA, and ID-RSVD-and suppresses task-irrelevant factors such as background noise and artifacts by extracting and aggregating singular vectors that capture object foreground information. Experimental results demonstrate that our method improves classification accuracy and effectively learns informative foreground representations while reducing the impact of background noise.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Physics-based generation of multilayer corneal OCT data via Gaussian modeling and MCML for AI-driven diagnostic and surgical guidance applications
Jinglun Yu, Yaning Wang, Rosalinda Xiong et al. (+3 more)
Training deep learning models for corneal optical coherence
tomography (OCT) imaging is limited by the availability of large, well-annotated
datasets. We present a configurable Monte Carlo simulation framework that generates synthetic corneal B-scan optical OCT images with pixel-level five-layer
segmentation labels derived directly from the simulation geometry. A five-layer corneal model with Gaussian surfaces captures
curvature and thickness variability in healthy and keratoconic eyes. Each layer is assigned optical properties from the literature and light transport is simulated using Monte Carlo modeling of light transport in multi-layered tissues (MCML), while incorporating system features such as the confocal PSF and sensitivity roll-off. This approach produces over 10,000 high-resolution (1024x1024) image-label pairs and supports customization of geometry,
photon count, noise, and system parameters. The resulting
dataset enables systematic training, validation, and benchmarking of AI models under controlled, ground-truth conditions, providing a reproducible and scalable resource to support the development of diagnostic and
surgical guidance applications in image-guided ophthalmology.
Read the full abstract ▾
All authors Jinglun Yu, Yaning Wang, Rosalinda Xiong, Ziyi Huang, Kristina Irsch, Jin U. Kang
Computer Vision · Cross-listed from cs.LG
Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion
Dan Haramati, Carl Qi, Tal Daniel et al. (+3 more)
We propose a hierarchical entity-centric framework for offline Goal-Conditioned Reinforcement Learning (GCRL) that combines subgoal decomposition with factored structure to solve long-horizon tasks in domains with multiple entities. Achieving long-horizon goals in complex environments remains a core challenge in Reinforcement Learning (RL). Domains with multiple entities are particularly difficult due to their combinatorial complexity. GCRL facilitates generalization across goals and the use of subgoal structure, but struggles with high-dimensional observations and combinatorial state-spaces, especially under sparse reward. We employ a two-level hierarchy composed of a value-based GCRL
agent and a factored subgoal-generating conditional
diffusion model. The RL
agent and subgoal generator are trained independently and composed post hoc through selective subgoal generation based on the value function, making the approach modular and compatible with existing GCRL
algorithms. We introduce new variations to
benchmark tasks that highlight the challenges of multi-entity domains, and show that our method consistently boosts performance of the underlying RL
agent on image-based long-horizon tasks with sparse rewards, achieving over 150% higher success rates on the hardest task in our suite and generalizing to increasing horizons and numbers of entities. Rollout
videos are provided at: https://sites.google.com/view/hecrl
Read the full abstract ▾
All authors Dan Haramati, Carl Qi, Tal Daniel, Amy Zhang, Aviv Tamar, George Konidaris