Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Computer Vision

cs.CV Watch Computer Vision

Friday 7 August

Computer Vision · Cross-listed from cs.LG

Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory

Kaustubh Kapil and Kishor P. Upla

Transformers have had a profound impact on the world of language processing and computer vision. As efforts to answer the million-dollar question of “How does a Transformer learn?" have been increasing, existing interpretability studies primarily analyze representations at isolated layers or the network as a whole, while the developmental evolution of individual representations and its manifolds across transformer layers remains underexplored. With this work, we aim at providing a comprehensive analysis of the evolution of representations as the representation point cloud transforms across the layers; thereby attempting to isolate layers or establish a trend which comes closer to justifying how and when raw input representations evolve into task-relevant feature representations. Thus, Transformer Geometry Observatory-TGO-IV introduces a topological framework for analysing the evolution of Transformer representations through the lens of Persistent Homology. Rather than studying local geometric properties alone, TGO-IV constructs Vietoris–Rips simplicial complexes from token-level representation point clouds and investigates the evolution of their persistent topological signatures across Transformer layers. The proposed framework comprises complementary topological observatories including Persistence Diagrams, Barcode Diagrams, Betti Curves, Persistence Landscapes, Bottleneck Distance, and Wasserstein Distance, enabling a comprehensive analysis of how the global topology of representation point clouds develops throughout the forward pass.
Read the full abstract

Computer Vision · Cross-listed from eess.IV

Energy and Performance Benchmarking of Deep Learning Models for Breast Cancer Detection

Samar Garrab and Ghada Achour

Recent advances in machine learning have greatly improved breast cancer detection, enabling more accurate and timely diagnosis. Deep learning (DL) models show strong potential for medical image analysis; however, as their architectural complexity increases, their environmental impacts are becoming a growing concern. In this paper, we present a comparative analysis of seven DL models for breast cancer detection on two medical datasets: Breast Ultrasound and BreakHis 400X. The evaluated architectures range from Convolutional Neural Networks (CNNs) and transformers to hybrid models. In addition to performance metrics, we assess CO2 emissions during both training and inference. Our results show that EfficientNet and ResNet consistently deliver strong performance, although with higher CO2 emissions. The selected transformers, such as DeiT-Tiny, perform competitively on both datasets, whereas DenseNet121 achieves lower accuracy. On the Breast Ultrasound Dataset, DeiT provides the most favourable balance between accuracy and energy consumption, whereas on the BreakHis dataset, the ViT and Swin models achieve the best results. Overall, our findings indicate that no single architecture category from the evaluated ones consistently dominates across the two selected datasets. Our results highlight the importance of jointly considering performance, emissions, and dataset characteristics when selecting models for medical applications.
Read the full abstract

Computer Vision · Cross-listed from eess.IV

Structural Guidance for Unified Joint Demosaicing and Denoising

Qixin Zheng, Ping Chen, Qiangqiang Shen et al. (+1 more)

Joint demosaicing and denoising is a fundamental step in camera image signal processing, yet remains challenging because different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content. Existing unified restoration networks explicitly model CFA geometry but are still driven primarily by pixel-level supervision, making them prone to structural degradation around edges, repetitive textures, and moiré patterns where local evidence is unreliable. We attribute this limitation partly to the absence of explicit structural guidance beyond pixel-level reconstruction supervision. Motivated by this observation, we propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration. Our model receives a unified five-channel observation consisting of the raw mosaic, CFA masks, and a noise-level map. A SwinIR restoration branch reconstructs pixel details under CFA-conditioned modulation, while a parallel structural reasoning branch extracts complementary structural cues from a sparse pseudo-RGB observation. To bridge the substantial domain gap between sparse noisy sensor data and the natural-image pretraining domain of the structural encoder, we introduce a lightweight trainable adapter before residually fusing structural and restoration features. A shared decoder jointly predicts the restored RGB image and an auxiliary clean mosaic, providing supervision in both image and sensor domains. Extensive experiments across multiple CFA patterns and noise levels demonstrate consistent improvements over state-of-the-art unified and CFA-specific methods, indicating that adapted structural priors can enhance robust camera image restoration. The source codes and dataset are provided in the supplementary material.
Read the full abstract

All authors Qixin Zheng, Ping Chen, Qiangqiang Shen, Haijin Zeng

Computer Vision · Cross-listed from eess.IV

SpecF2M: A Spectral-Aware Multi-task Network Estimating Axial Length and Refractive Error from Pediatric Fundus Photographs

Mengxian He, Xinyue Liu, Yunyun Sun et al. (+5 more)

Spherical Equivalent Refraction (SER) and Axial Length (AL) are core indicators for pediatric myopia screening, yet their measurements require dedicated biometry and cycloplegic refraction. Fundus photography offers an accessible imaging modality, as myopia-related posterior-pole changes are visible in 45$^\circ$ fundus images. However, these cues are often low-contrast, spatially diffuse, and multi-scale. Moreover, AL, Sphere (SPH), and Cylinder (CYL) share partially overlapping but non-identical anatomical correlates. We propose SpecF2M, a spectral-aware multi-task network for estimating AL and SER components from pediatric fundus photographs. SpecF2M integrates a deterministic anatomy-guided enhancement module, a hybrid spatial–spectral backbone combining MixCNN and Hybrid Spectral Learning (HSL) blocks, and an expert-routing head for component-level estimation of AL, SPH, and CYL. On a pediatric cohort of 4,359 eligible child visits and 6,966 fundus images, SpecF2M outperforms controlled CNN/ViT baselines for AL and SPH estimation, achieving MAEs of 0.5347 mm and 0.7062 D, respectively. Component-level analysis further reveals asymmetric task coupling, where CYL exhibits weaker association with fundus-derived myopic patterns than AL/SPH. These results support fundus-based, screening-oriented estimation of pediatric myopia indicators, while external validation remains necessary before deployment.
Read the full abstract

All authors Mengxian He, Xinyue Liu, Yunyun Sun, Wei Hao, Minqing Zhang, Lichun Wang, Shunyi Zhang, Wu Yuan

Computer Vision · Cross-listed from eess.IV

APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT–IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction

Xincan Zheng, Yaqi Wang, Zhi Li et al. (+4 more)

Registration between cone-beam computed tomography (CBCT) and intraoral scans (IOS) is essential for patient-specific surgical planning. However, disparate imaging modalities, limited overlap, and large pose offsets make automated registration unreliable. Consequently, clinical registration remains dependent on conventional geometry pipelines and manual clinician adjustment. To address these challenges, we propose APCReg, an anatomical-prior-guided coarse-to-fine framework for global registration and reliability-controlled residual correction. Specifically, multi-view anatomical coarse registration (MACR) performs ordered orthogonal projection alignment (buccal, proximal, and occlusal) to decompose the six-degree-of-freedom search before three-dimensional refinement. Overlap-aware residual registration (OARR) combines shared KPConv features, a folded arch-length cue, overlap-gated cross-attention, and Sinkhorn matching. Finally, dental-arch-structured hypothesis selection evaluates diverse poses on held-out reliable correspondences, while a ground-truth-free coarse-retention guard conditionally retains a geometrically reliable coarse pose. On 60 held-out jaw pairs, APCReg achieves a submillimeter mean Chamfer distance of 0.87 mm and a Hausdorff distance of 2.92 mm under this evaluation protocol, and ranks first across the six reported metrics among the evaluated open-source baselines.
Read the full abstract

All authors Xincan Zheng, Yaqi Wang, Zhi Li, Jiahao Bao, Lan Feng, Yiru Xia, Shuai Wang

Computer Vision · Cross-listed from eess.IV

Knowledge-Guided 3D CT Generation: A Conditioning-Centric Taxonomy

Francesca Pia Panaccione, Eugenio Lomurno and Matteo Matteucci

Controllable generation guided by external knowledge is a key requirement in modern generative deep learning applications, enabling the synthesis of samples with explicit constraints on semantic content, structural properties, and variability. In 3D Computed Tomography (CT), such control is essential for clinical applications, including data augmentation, privacy-preserving data sharing, and the simulation of specific anatomical or pathological scenarios. While research on conditional 3D CT generation has expanded rapidly, the diversity of existing approaches makes systematic comparison difficult and obscures fundamental design choices. In this survey, we propose a conditioning-centric taxonomy that organizes the literature along three orthogonal dimensions: the type of external knowledge (K), the knowledge integration paradigm (I), and the generative architecture (A). This factorization defines an explicit design space (K x I x A) that provides a unified perspective on prior work. Using this framework, we systematize existing methods, identify dominant trends and recurring design patterns, and highlight underexplored regions of the design space that point toward promising directions for future research.
Read the full abstract

Computer Vision · Cross-listed from eess.IV

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang et al. (+3 more)

Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging during treatment could support treatment response assessment. Many existing imaging-based approaches rely on a single static timepoint, which fails to capture changes that occur during treatment. In this work, we present a longitudinal framework that combines a frozen 3D foundation encoder (Pillar-0) with our Temporal Dynamics Network (TDN) to predict treatment response from serial Dynamic Contrast-Enhanced (DCE) MRI acquired across four clinical timepoints from pre-treatment to pre-surgery. The TDN combines time-aware volumetric embeddings with clinical and treatment data to predict pCR. Evaluated on 982 patients from the combined I-SPY2 and ACRIN-6698 cohort, the proposed model achieves strong performance across all reported metrics when longitudinal 3D imaging is fused with clinical data (test AUROC: 73.6%, balanced accuracy: 69.1%). While clinical variables provide the strongest individual predictive signal, longitudinal 3D imaging contributes complementary information when fused with clinical data, improving pCR prediction. Our source code is available at: https://github.com/omarftt/longitudinal_temporal_pillar.
Read the full abstract

All authors Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang, Pegah Khosravi, Yu Tian, Chen Chen

Computer Vision · Cross-listed from cs.RO

Drone-Assisted UAV-UGV Collaboration for Autonomous Navigation in Snow-Covered Terrain

Shreyam Gupta, P. Agrawal, Priyam Gupta et al. (+1 more)

This paper presents a collaborative UAV-UGV navigation framework for high-altitude, snow-covered terrain, where reduced visibility and unstable ground render conventional methods ineffective. We introduce a custom efficient U-Net architecture that falls under the computational constraints for real-time road segmentation, utilizing a novel synthetic snow data augmentation technique to achieve 96.5% segmentation accuracy. For UAV localization, we implement an Extended Kalman Filter (EKF) fusing onboard GPS and IMU data, achieving a maximum observed positional error of +-0.5 meters. The UGV position is determined via a visual tracking pipeline using YOLOv5 and depth data from the UAV's RGB-D camera. A dynamic path planning algorithm utilizes this segmentation to adjust for snow drifts, enabling successful navigation in obscured test environment with minimal deviation.
Read the full abstract

All authors Shreyam Gupta, P. Agrawal, Priyam Gupta, R. Gautam

Computer Vision · Cross-listed from cs.LG

From Benchmark Performance to Tool Deployment: Human-in-the-Loop Anomaly Detection

Mike Szklarzewski, CJ George, Gavin Smithson et al. (+10 more)

Automated anomaly detection methods often report strong performance on curated academic benchmarks, but their behavior under real-world industrial conditions is less clear. In this work, we evaluate 19 unsupervised anomaly detection models on the BowTie dataset, a challenging manufacturing dataset with reflective surfaces, subtle defects, and profile-specific variation. In contrast to benchmark results, we observe that model performance is less stable than typically reported on standard benchmarks such as MVTec AD, highly sensitive to preprocessing, and inconsistent across conditions, with no single approach emerging as uniformly robust; a consensus audit further indicates that nominal-data quality affects deployment. Motivated by these findings, we developed and initially deployed a unified human-in-the-loop framework for manufactured-part inspection that combines image annotation, AI-assisted defect detection, and an integrated validation engine, replacing a prior manual visual inspection and documentation workflow. The system supports heatmap-guided defect review, SAM-refined candidate regions for inspector acceptance, rejection, or boundary adjustment, mask evaluation where annotations exist, and review history for inspector consistency and onboarding. Together, the results highlight the gap between benchmark performance and deployment reality, and provide a practical framework for addressing it.
Read the full abstract

All authors Mike Szklarzewski, CJ George, Gavin Smithson, Christopher Stokes, Dakota Fulp, William M. Jones, Benjamin Wynn, Alexander Ur, Agit Yesiloz, Clint Kallenbach, Mark Swartz, Nathan DeBardeleben, Sharmistha Chakrabarti

Computer Vision

DINO-3DRA: Leveraging 2D Foundation Model Semantics for 3D Cerebral Aneurysm Segmentation

Jiayang Lu, Fengming Lin, Alejandro F. Frangi et al. (+1 more)

Accurate aneurysm segmentation in 3D rotational angiography (3DRA) is hindered by extreme class imbalance, morphological similarity to vessels, and absent large-scale 3D pretraining. 2D vision foundation models encode dense structural priors from 1.7 billion images, yet naïve slice-wise transfer fragments anatomical continuity and destabilises optimisation. We propose DINO-3DRA, a dual-path framework achieving effective cross-dimensional semantic transfer by injecting frozen DINOv3 features into a 3D U-Net backbone via Room-Lite spatial mixing and calibrated residual fusion. On multi-centre 3DRA data, DINO-3DRA achieves state-of-the-art aneurysm segmentation (Dice: 0.758; HD95: 2.75 mm; +13% over nnU-Net) with only 5.72M trainable parameters. Ablation studies confirm that gains arise from structured cross-dimensional transfer rather than loss design alone, with bridged foundation features improving anatomical continuity between aneurysms and parent vessels. Without fine-tuning on CADA and SHINY-ICARUS, DINO-3DRA eliminates all catastrophic failure cases observed in baseline architectures, demonstrating robust generalisation across heterogeneous imaging protocols.
Read the full abstract

All authors Jiayang Lu, Fengming Lin, Alejandro F. Frangi, Ali Sarrami-Foroushani

Computer Vision · Cross-listed from cs.CL

Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation

Anna Kołos, Grzegorz Statkiewicz, Karolina Seweryn et al. (+3 more)

Vision-language models (VLMs) have achieved strong performance on tasks such as image captioning, visual question answering, and image-to-text generation. However, they are predominantly trained on English-centric data, which limits their ability to handle culturally grounded visual understanding and leads to failures in interpreting region-specific meanings, symbolic content, and context-dependent visual cues. Existing benchmarks for cultural competence are often template-driven and focused on surface-level recognition, making them insufficient for evaluating deeper linguistic and pragmatic understanding in culturally situated settings. We introduce PoVisLE, a monocultural vision-language benchmark for Polish designed to evaluate culturally grounded multimodal understanding under a grounded evaluation paradigm, where language is interpreted in interaction with visual context. The dataset contains 1,117 images and 2,366 manually annotated VQA pairs. Overall, our dataset provides a controlled and challenging resource for assessing culturally grounded vision-language understanding beyond surface-level recognition.
Read the full abstract

All authors Anna Kołos, Grzegorz Statkiewicz, Karolina Seweryn, Katarzyna Kowol, Karolina Piosek, Wojciech Kusa

Computer Vision

XClipGS: Exact Half-Space Clipping for Medical Volume Gaussian Splatting

Zhongpai Gao, Benjamin Planche, Meng Zheng et al. (+4 more)

Gaussian-splatting proxies enable interactive rendering of volumetric medical scans, but a clipping plane exposes anatomy not constrained by external-view training and intersects primitives that conventional splatting can only keep or drop whole. We present XClipGS (eXact Clipping), which treats these as two separate problems: the render-time clip operator and supervision of the hidden interior. Under the local affine model used by EWA splatting, the ray integral of a half-space-restricted Gaussian factorizes exactly into its ordinary 2D footprint and a conditional Gaussian CDF whose argument is affine in pixel coordinates. The resulting closed-form per-pixel operator introduces no learned clipping parameters or auxiliary network and remains differentiable with respect to the primitive and plane. We use multi-distance reference views with varied clipping-plane axes and offsets to supervise the interior through the same operator. We also introduce a paired clipped/unclipped cut-face protocol with difference-referenced cut error (CDE) and culled-side leakage (Leak), because global image metrics dilute errors near the plane. On eight CT and MRI volumes with plane offsets not used for training, XClipGS attains the highest PSNR on every volume (33.56 versus 32.34 dB for ClipGS) while rendering at over 650 FPS, far above real time, versus 278 FPS. On voxel-axis cut-face views, it raises average band SSIM from 0.809 to 0.860 and leaks roughly 40 times less. Without retraining, it also achieves the best average across all four metrics on arbitrary-normal planes; on a fixed interior, it matches RaRa's face fidelity with about 16 times less leakage. Project page: https://gaozhongpai.github.io/XClipGS/
Read the full abstract

All authors Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Chaoyi Zhou, Terrence Chen, Ziyan Wu

Computer Vision

Rethinking 3D Segmentation from Individual LiDAR Scans: Incidence-Aware Sampling on the SIP Benchmark

Seongyong Kim, Jingdao Chen and Yong Kwon Cho

3D scene understanding is increasingly important in construction, yet most methods are developed on curated datasets that do not fully reflect real site sensing conditions. In many workflows, individual LiDAR scans provide rapid local updates rather than complete scene representations, producing limited surface coverage, acquisition-driven density variation, and severe imbalance between dominant planar surfaces and sparse construction elements. Because large point clouds must be downsampled, sampling resolution and point allocation directly affect the balance between geometric detail and spatial context. This study evaluates these effects under a fixed per-fragment point budget and introduces an incidence-aware sampling strategy for individual LiDAR scans. The method maps points to a geometry-normalized manifold space for voxel-based selection while preserving original Euclidean coordinates for downstream learning. It requires only point coordinates and normals and no backbone modification. Using the Site in Pieces (SIP) benchmark, experiments with Point Transformer and PointNeXt show improved resolution-averaged segmentation performance, especially for non-planar elements and ladders, while reducing sensitivity to sampling resolution. The results show that acquisition-aware sampling can provide a more stable geometric representation and should be treated as an active component of individual-scan 3D segmentation rather than generic preprocessing.
Read the full abstract

Computer Vision

Multi-Task Consistency-based Detection of Adversarial Attacks

Cong Chen, Jean-Philippe Monteuuis and Jonathan Petit

Deep Neural Networks (DNNs) have found successful deployment in numerous vision perception systems. However, their susceptibility to adversarial attacks has prompted concerns regarding their practical applications, specifically in the context of autonomous driving. Existing defenses often suffer from cost inefficiency, rendering their deployment impractical for resource-constrained applications. In this work, we propose an efficient and effective adversarial attack detection scheme leveraging the multi-task perception within a complex vision system. Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation. To this end, we developed a consistency score metric to measure the inconsistency between vision tasks. Next, we designed an approach to select the best model pairs for detecting inconsistencies effectively. Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset. The experimental results demonstrated that our defense achieved a ROC-AUC performance of 99.9% detection within the considered attacker model.
Read the full abstract

Computer Vision

LoRSA: Toward Generalizable Parameter-Efficient Fine-Tuning for Biomedical Downstream Tasks

Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj et al. (+2 more)

Parameter-efficient fine-tuning enables the adaptation of vision foundation models to biomedical tasks under limited computational resources, but a single low-rank update can constrain all task-specific changes to one narrow parameter subspace. This restriction may prevent the model from simultaneously representing globally shared task structure and localized residual directions required for generalization to unseen imaging domains. We introduce LoRSA, a global–residual adaptation framework that jointly learns a dense low-rank component and a dynamically structured-sparse low-rank component. The dense component captures globally coordinated task adaptation, while the structured component provides complementary residual corrections whose support evolves during training. We characterize the representational capacity, approximation properties, rank structure, and singular-subspace complementarity of this decomposition. We evaluate LoRSA for four-class breast-density classification using DINOv3-Base, with VinDr-Mammo as the source domain and MammosighTR and RSNA as unseen external domains. LoRSA remains competitive on the internal validation set and achieves the best external macro-F1 on both target datasets, improving upon the strongest competing method by 2.15 percentage points on MammosighTR and 3.09 percentage points on RSNA. Weight-matrix analysis further shows that approximately $92\%$ of the energy of each adaptation component lies outside the bilateral singular subspace of the other, indicating that the two components learn largely complementary update directions. These results suggest that organizing adaptation capacity into distinct global and residual paths can improve the external-domain generalization of parameter-efficiently adapted biomedical vision models.
Read the full abstract

All authors Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj, Yimin Yang, Ashirbani Saha

Computer Vision

BRUCE: Benchmarking Robustness Under Corruption Escalation for Scientific Vision-Language Reasoning

Saim Rehman and Muhammad Shafique

Visual-language models (VLMs) frequently struggle with robustness issues in real-world situations due to low- or varying-quality input images. In this paper, we aim at analyzing VLMs' robustness by applying perturbations and distortions to the input images, such as blur or low contrast. Toward this goal, we propose BRUCE (Benchmarking Robustness Under Corruption Escalation), a multimodal reasoning fragility framework for scientific vision-language reasoning. State-of-the-art evaluation frameworks/studies primarily focus on clean-task accuracy and rarely analyze how reasoning stability degrades across robustness dimensions. Besides varying over a wide-range of input perturbations, BRUCE employs two novel metrics – Robustness Corruption Index (RCI) and Traversal-RCI (T-RCI) – to quantify how rapidly multimodal reasoning performance deteriorates in VLMs as visual corruption severity increases under progressive perturbation scaling. We evaluate BRUCE across chemistry and mathematical reasoning tasks for multiple datasets, while analyzing corruption-induced prediction failures in terms of four high-level reasoning domains: OCR-dependent reasoning, spatial reasoning, symbolic reasoning, and semantic failures, with each containing fine-grained corruption specific failure subtypes, thereby enabling an interpretable failure analysis.
Read the full abstract

Computer Vision

Ghost Features and Spooky Transfer Learning for Hypercomplex-Valued Neural Networks

Guilherme Vieira Neto and Marcos Eduardo Valle

Hypercomplex numbers extend the concept of complex numbers by introducing additional imaginary components. Besides increasing dimensionality, operations on the imaginary parts provide algebraic and geometrical properties that can be beneficial for solving machine learning problems. In this paper, we show how to create hypercomplex-valued neural network layers where the real part corresponds to the output of a traditional real-valued layer. The additional imaginary parts of these hypercomplex-valued layers produce what we call “ghost features,” which contain enhanced information that is not present in the output of the real-valued layer. Moreover, ghost features can be effectively integrated into a trained neural network through a process we refer to as “spooky transfer learning.” This approach allows us to harness the richness of ghost features, leading to more efficient neural networks. The source code and Jupyter Notebook are available at https://github.com/mevalle/v-nets/.
Read the full abstract

Computer Vision

Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Properties

Ali Bahri, Hongliang Li, Soufiane Lamghari et al. (+2 more)

Estimating volumetric mechanical properties, including Young's modulus, Poisson's ratio, and density at each voxel, is intrinsically ambiguous from vision alone, as visually similar objects may have substantially different material compositions and physical behavior. Existing approaches predict these properties independently across voxels, overlooking the piecewise-constant material structure of real objects and producing noisy or inconsistent estimates for voxels that share the same material, while lacking an explicit mechanism to resolve visual ambiguity. We introduce ViWi (Vision Meets WiFi), an object-centric framework for volumetric mechanical-property estimation. ViWi represents each object using a compact set of material slots that aggregate evidence from voxels with a shared material identity and produce coherent slot-level property predictions. To complement visual appearance, ViWi incorporates a compact RF descriptor generated through WiFi-band electromagnetic simulation using permittivity and conductivity. The RF descriptor conditions the material slots with global composition cues that may be unavailable from images, while visual features preserve voxel-level spatial localization. On GVM, ViWi improves over the prior state of the art on four of six per-voxel metrics, while its vision-only variant improves all reported mass-estimation metrics on ABO-500. These results demonstrate that combining object-centric material structure with complementary RF evidence enables more accurate and physically coherent volumetric property estimation beyond what is possible from visual appearance alone.
Read the full abstract

All authors Ali Bahri, Hongliang Li, Soufiane Lamghari, Jie Chuai, Zhitang Chen

Computer Vision

Tokenizer-Generator Coupling in Medical Image Generation

Liam Chalcroft

Latent medical image generators usually treat the tokenizer as fixed preprocessing. We test whether this separation holds in a controlled ChestMNIST study at $64\times64$ that crosses discrete tokenizers, generator families, and sampler settings under a shared latent grid, with continuous-latent reference cells. The rankings depend jointly on the tokenizer, generator, and sampler. The best quantizer changes with the generator, and validation-based sampler selection changes the apparent generator ranking. We interpret this through a rate-distortion-modelability framing in which modelability is conditional on the generator, sampler, and inference budget. The interaction persists when the vocabulary-1024 block is retrained at three seeds, with 4 of 9 pairwise quantizer comparisons exceeding three seed standard deviations, including a reversal between LFQ and FSQ from MaskGIT to D3PM; the rest of the grid was trained at a single seed and is correspondingly less certain. Reconstruction PSNR alone is not a reliable selection criterion. On LFQ-1024, retuning the D3PM and SEDD samplers on a held-out validation split reduces FID-192 from 0.44/0.41 at the default budget to 0.09/0.10 at lower NFE, replicated across seeds, although the continuous references were not given an equivalent sampler sweep. FID-192, our internal ranking metric, ranks consistently with standard FID-2048 (Spearman 0.89) and with a label-free classifier two-sample test (0.86). All experiments are unconditional, use low-resolution $64\times64$ medical-style images and are evaluated with non-clinical FID-based metrics, and our claims are limited to that setting. Implementations are released at https://github.com/liamchalcroft/medtokenizers and https://github.com/liamchalcroft/medlatents.
Read the full abstract

Computer Vision

SpikeWorld: Fast-State Adaptation for Frozen Spiking World Models

Ziqiao Yu

A predictive model receives a self-supervised signal whenever the consequence of an action is observed. Using that signal after deployment is difficult when dynamics and semantics share parameters: freezing prevents adaptation, whereas weight updates require optimizer state and may alter the learned representation. Here we introduce SpikeWorld, a 1.45M-parameter sparse spiking model jointly trained for heterogeneous sensory prediction, semantics, image-text binding and action-conditioned dynamics. At deployment, all trained parameters are frozen. Delayed next-state residuals update two external paths: cumulative fixed-bank losses select the bounded action correction, while route-specific residual matrices refine next-state prediction. Neither path uses labels, teacher outputs, rewards, success signals or the true shift value. Joint optimization improves action next-state MSE by 17.10% while also improving multimodal prediction, semantic accuracy and image-text retrieval. On held-out shear and attenuation streams, the combined external state improves aggregate prediction by 5.48% and 30.01%; its fixed-bank action path improves tracking by 24.20% and 3.94%, respectively. In a six-arm study comprising 450 new Meta-World trajectories (75 per arm), SpikeWorld raises frozen-policy reward by 7.90 (95% CI [2.48, 14.06]); the 13.33-point success difference is descriptive (CI [0, 40]). For identical sensory inputs, model parameters and inherited semantic outputs remain bitwise unchanged. A 16-byte RLS estimator obtains the highest non-oracle reward on linear attenuation, showing that the contribution is not superior linear identification, but its integration with a frozen multimodal spiking checkpoint. Reference code is publicly available at https://github.com/Oooorca/SpikeWorld.
Read the full abstract

Computer Vision

CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecasting

Quang Minh Dinh and Tuan Kiet Doan

Generative traffic video forecasting aims to synthesize long-horizon, temporally coherent future videos of traffic scenes from a short observation history and textual descriptions. In this paper, we present CosmosAlign, a generative traffic video forecasting framework built upon the pretrained Cosmos3-Nano world foundation model. Our approach is motivated by the observation that successfully adapting large pretrained world models to downstream forecasting tasks depends primarily on distribution alignment rather than increased model capacity. To this end, we propose a two-stage LoRA adaptation strategy that first aligns the conditioning-mode distribution with the target forecasting task, and then aligns the training captions with the model's native structured prompting interface through an LLM-based re-captioning pipeline. During inference, we further improve prediction quality using a fully training-free procedure consisting of consensus-based medoid sample selection and motion-adaptive blending of static scene regions. CosmosAlign achieves a final score of 76.49 on the AI City Challenge 2026 Track 5 benchmark, ranking first on the final leaderboard. Our code is publicly available at https://quangminhdinh.github.io/CosmosAlign/.
Read the full abstract

Computer Vision

Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding

Yeeun Choi, Youngbeom Yoo, Joon-Young Lee et al. (+2 more)

When videos extend from hours to days, directly processing them end-to-end becomes impractical for current Multi-modal Large Language Models (MLLMs). This ultra-long setting necessitates a two-stage paradigm: query-agnostic memory construction followed by retrieval-based inference. Prior work invests in complex memory construction to pre-model high-level relations in videos, despite not knowing the downstream query at build time. We instead prioritize high-recall retrievability during memory building, and defer query-specific, high-level relation composition to inference time. To this end, we propose MERIT(Multi-key Episodic Retrieval with Inference-time Temporal expansion), a simple yet effective agentic framework for ultra-long video understanding. First, we formulate an episodic multi-key representation that enables precise retrieval of fine-grained memories through a simple key-matching mechanism. Second, we introduce a neighbor filtering mechanism to capture broader semantic context without the massive computational overhead of global memory construction. This is achieved by expanding the temporal scope exclusively around the retrieved segments at inference time. By leveraging simple key-matching with this on-demand temporal expansion, MERIT achieves state-of-the-art performance across three long-video benchmarks: EgoLifeQA, LVBench, and Video-MME (Long).
Read the full abstract

All authors Yeeun Choi, Youngbeom Yoo, Joon-Young Lee, Hyolim Kang, Seon Joo Kim

Computer Vision · Cross-listed from cs.AI

An Agentic AI Framework Overcomes Fundamental Limitations of Large Language Models for Glaucoma Detection from Fundus Photography

Jalil Jalili, Hossein Taghizad, Anuwat Jiravarnsirikul et al. (+8 more)

Large language models (LLMs) show promise in medical image interpretation but suffer from hallucination, limited accuracy, and run-to-run inconsistency. We developed and validated an agentic AI framework integrating LLMs with specialized deep learning tools for glaucoma detection from fundus photography. The workflow had three steps: (1) LLM initial assessment; (2) function calling to invoke specialized tools for image quality (QAModel, FundaQ-8), glaucoma classification (SwinV2-Tiny), and optic disc/cup segmentation (SegFormer-B0); and (3) LLM reflection integrating the initial impression with tool outputs. Two LLMs (Gemini 2.5 Flash, GPT-5.4 mini) were evaluated on two public datasets (ORIGA, n=100; RIM-ONE-v3, n=100) under uncropped and cropped fields of view; all images were independently graded by a masked fellowship-trained glaucoma specialist. The agentic workflow improved classification accuracy by 16 to 47 percentage points across all conditions, reaching within 6 points of the specialist; on RIM-ONE-v3 the best configurations matched the specialist accuracy of 88%. LLM-alone approaches failed in two ways: GPT-5.4 mini showed positive bias (sensitivity 95-100%, specificity 0-5%), while Gemini 2.5 Flash varied stochastically between runs; the agentic workflow corrected both. Cup-to-disc ratio error fell 15-50% (MAE 0.156-0.228 to 0.104-0.132), and correlation with specialist grading rose from weak (r=0.12-0.39) to moderate-strong (r=0.59-0.84). Run-to-run consistency rose from near-random (kappa as low as -0.01) to near-perfect (kappa up to 0.96). Integrating LLMs with specialized tools addressed key limitations of LLM-alone approaches, including over-diagnosis and run-to-run variability. Gains held for both LLMs, suggesting generalizability across backbones, and may signal a shift from monolithic models toward orchestrated multi-agent systems in medical AI.
Read the full abstract

All authors Jalil Jalili, Hossein Taghizad, Anuwat Jiravarnsirikul, Christopher Bowd, Akram Belghith, Raheleh Kafieh, Christopher A. Girkin, Sally L. Baxter, Robert N. Weinreb, Linda M. Zangwill, Mark Christopher

Computer Vision

Data collection from highways: a geometric, class-agnostic approach to embedded vehicle counting

Lucas Gouveia Omena Lopes, William W. M. Lira, Alexandre M. Lima et al. (+1 more)

Traffic data collection is dominated today by deep object detectors followed by tracking-by-detection, a pipeline that presupposes what is often missing in practice: a detector already trained on the class one wants to count. We revisit a purely geometric traffic-sensing pipeline for Single Board Computers in which detection is class-agnostic: moving objects come from background subtraction and thresholding, and counting is decided by a geometric rule on an imaginary line across the road, a software inductive loop detector. With no object model, training set or per-object trajectory, it runs faster than real time on Raspberry Pi class hardware. Two counting rules are described: a constant average speed rule, whose expected accuracy is derived analytically as about 86% under a Gaussian speed distribution, and a self-calibrating pre-calibration rule that recovers the lane geometry from blob statistics and counts edges of lane occupancy, additionally yielding per-vehicle average speed at no extra cost. Over four videos the latter counts with 83.3%-100% accuracy; in a field deployment it reaches 91% against 37.5% for a blob-tracking baseline under the same compute budget. We report the observations of that period in detail: the resolution floor below which accuracy collapses, the frame rate floor at which vehicles alias past the counting line, the gap between short curated clips and long uncontrolled footage, and the trade-off between Python (easier to tune, 100% CPU) and C++ (40% CPU, thermally viable). These are properties of the sampling geometry, not of the hardware of the time, and still constrain edge deployments. We close by arguing where motion-based, class-agnostic detection remains the right tool: open-set classes with no annotated data, tight power budgets, privacy-constrained installations, and the cold start of mining training crops to bootstrap a learned detector.
Read the full abstract

All authors Lucas Gouveia Omena Lopes, William W. M. Lira, Alexandre M. Lima, Thales M. A. Vieira

Computer Vision

HeatCast: A Benchmark for Neighborhood-Scale LST Forecasting across 124 U.S. Cities

Jesus Guerrero, Isaac Corley, Leon Najafirad et al. (+2 more)

Land Surface Temperature (LST) is a widely used satellite-derived measure of urban surface heat, but there is no shared benchmark for forecasting it at 30 m. Prior studies usually cover one to three cities, use kilometer-scale products, or do not release data and code. We introduce HeatCast, a Landsat-based benchmark for monthly LST forecasting across 124 U.S. cities from 2013 through June 2025. HeatCast contains 30 m monthly tiles with LST, elevation, surfacereflectance RGB, three spectral indices, broadband albedo, quality masks, and Local Climate Zone (LCZ) labels, together with a fixed temporal split, LCZ-stratified metrics, and a reference evaluation harness. We evaluate a CNN+LSTM and Earthformer on next-month forecasting, where Earthformer reaches 7.74 K RMSE against 10.42 K for the CNN+LSTM. Forecasting from the eight nonLST channels alone reaches 7.72 K, against 8.15 K from LST history and 8.68 K from RGB. The data, code, and weights are released under MIT at https://doi.org/10.57967/hf/9889.
Read the full abstract

All authors Jesus Guerrero, Isaac Corley, Leon Najafirad, Maryam Tabar, Paul Rad

Computer Vision · Cross-listed from cs.CR

Adversarial Attacks on Deep OCR Systems

Wenbo Sun, Hongzong LI, Yanyun Wang et al. (+5 more)

Deep-OCR (DeepSeek-OCR) advances document recognition by treating the visual modality as an optical compression medium, enabling long-context OCR at low token cost. However, its increased complexity may introduce new security vulnerabilities. In this paper, we present, to the best of our knowledge, the first pure black-box adversarial attack against a generative OCR vision-language model, where only the decoded string can be queried and no gradients, logits, or model internals are available. We recast the attack as a zeroth-order optimization problem driven by a bounded scalar loss defined directly on the string output via sequence similarity, and estimate the gradient with a random-direction finite-difference scheme whose query cost is independent of the image dimension. An Adam update with ell_infinity projection yields imperceptible perturbations for both untargeted and targeted objectives. Pilot experiments on Deep-OCR validate the string-only attack and evaluation pipeline and expose severe qualitative decoder failures, including repetition, truncation, and prompt leakage. They also show that controlled targeted rewriting remains substantially harder than untargeted degradation; we avoid claiming targeted success until the pre-registered evaluation is complete.
Read the full abstract

All authors Wenbo Sun, Hongzong LI, Yanyun Wang, Jiahao MA, Shuxin Zhuang, Rong Feng, Shiqin Tang, Zi Liang

Computer Vision · Cross-listed from q-bio.QM

JUMP-lite: Compact, reproducible benchmarking of cell representations

Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen et al. (+2 more)

Image-based profiling captures rich phenotypic signatures for drug discovery and functional genomics. Large public datasets like JUMP Cell Painting now provide millions of images for systematic study. However, JUMP alone occupies 115 TB, and fragmented evaluation practices make systematic comparisons of representation methods impractical for many researchers. Here we present Nahual, an open-source framework for reproducible model deployment, and JUMP-lite, a 92.0 GB subset of JUMP that is approximately 1,250-fold smaller, selected to cover genetic modalities and compound annotations and reduced via lossy JPEG XL compression. Using these resources, we benchmark five representation methods, including classical features (CellProfiler) and deep learning models (MorphEM, OpenPhenom, SubCell, DINOv2). Moderate compression broadly retains signal relative to uncompressed images. Standardized phenotypic activity and consistency metrics reveal meaningful performance differences across methods. Together, JUMP-lite and Nahual provide a foundation for accessible, reproducible benchmarking of image-based cell representations.
Read the full abstract

All authors Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen, Anne E. Carpenter, Shantanu Singh

Computer Vision · Cross-listed from cs.AI

CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Models

Hsu-kuang Chiu and Stephen F. Smith

Vision-Language-Action (VLA) models have recently achieved impressive performance for end-to-end autonomous driving, yet existing approaches are primarily designed for an individual single autonomous driving agent with limited support for cooperative perception, reasoning, and planning. We present Cooperative Multi-agent Unified Driving with Reasoning (CMU-Drive), a closed-loop end-to-end benchmark for evaluating cooperative autonomous driving with multiple connected autonomous vehicles (CAVs) operating in safety-critical driving scenarios with background traffic participants. We further propose Vehicle-to-Vehicle Vision-Language-Action (V2V-VLA), a cooperative VLA model that integrates cooperative driving into a single forward pass by jointly generating driving actions, future waypoints, language reasoning, and communication policies. Experiments on CMU-Drive establish the first benchmark and baseline for cooperative VLA driving and provide a foundation for future research on multi-agent, closed-loop, end-to-end cooperative autonomous driving. Our code, benchmark, and model checkpoint will be publicly released to facilitate open-source research.
Read the full abstract

Computer Vision

FlowErase-OPD: Multi-Concept Erasure via Anchored On-Policy Distillation in Flow Matching Models

Yi Sun, Yimin Zhou, Xinhao Zhong et al. (+3 more)

Recent advances in flow matching models have substantially improved the quality of text-to-image generation, but have also raised increasing safety concerns due to their potential to generate harmful or undesirable content. Existing concept erasure methods for flow matching models predominantly focus on removing individual concepts, while effectively erasing multiple concepts simultaneously remains challenging. We propose FlowErase-OPD, a framework for multi-concept erasure based on on-policy distillation (OPD). Our approach first distills multiple single-concept erased models into a unified LoRA module and introduces Anchored Multi-Teacher Distillation (AMTD), which incorporates a retention teacher to mitigate the trade-off between concept erasure and preservation of generative capabilities. To further improve the coordination of multiple erasure objectives, we develop Adaptive Retention Control (ARC), which dynamically adjusts the sampling frequency and loss weight of each erasure teacher, together with the relative contribution of erasure and retention teachers throughout training. Extensive experiments on nudity, object, and artistic-style erasure demonstrate that FlowErase-OPD consistently improves the trade-off between erasure effectiveness, image quality, and semantic alignment, achieving state-of-the-art performance across diverse multi-concept erasure settings. Furthermore, the resulting models exhibit strong robustness against adversarial attacks. These results highlight the potential of on-policy distillation as a principled framework for safe and controllable generation in flow matching models.
Read the full abstract

All authors Yi Sun, Yimin Zhou, Xinhao Zhong, Zhiqi Zhang, Junhao Li, Bin Chen

Computer Vision

HSMLA: Hierarchical Softmax Multi-scale Linear Attention for Efficient Vision Transformers

Dong Liu, Yanxuan Yu, Renata Borovica-Gajic et al. (+2 more)

Vision transformers face significant computational overheads in high-resolution dense prediction due to the quadratic complexity of self-attention. Linear attention offers efficiency but sacrifices local context modeling. We propose HSMLA (Hierarchical Softmax Multi-scale Linear Attention), which combines ReLU-based linear attention for global context, selective softmax refinement for critical local features, and multi-scale token representations via depthwise convolutions. HSMLA achieves superior accuracy-efficiency trade-offs: up to $4.2\times$ inference-time speedup across dense prediction tasks, $87.3%$ Dice with $3.2\times$ speedup on CT organ segmentation, and $94.2%$ AUC with $4.1\times$ speedup on pathology WSI.
Read the full abstract

All authors Dong Liu, Yanxuan Yu, Renata Borovica-Gajic, Tong Geng, Ying Nian Wu