Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Computer Vision

cs.CV Watch Computer Vision

Monday 21 September

Computer Vision

WebMRIQC: A Web-Based Implementation of MRIQC for Accessible MRI Image Quality Assessment in Resource-Constrained Settings

Philip Nkwam, Ifeoluwa Oladeji, Sekinat Zurakat-Aderibigbe et al. (+9 more)

Reliable quality control (QC) of magnetic resonance imaging (MRI) is essential for reliable diagnostic neuroimaging, yet standard manual assessment is subjective and time-consuming. MRIQC has established standardized automated extraction of image-quality metrics (IQMs), but its reliance on local computational imaging skills and capacity including high-performance computing, limits its adoption in resource-constrained settings (RCS). We present WebMRIQC (webmriqc.mailab.io), an open-source browser-based platform that wraps the validated MRIQC engine behind a zero-installation web interface. WebMRIQC automates the DICOM-to-BIDS conversion of de-identified MRI scans, executes the unmodified containerized MRIQC pipeline on a shared compute node governed by a fair-share job queue, and returns an interactive in-browser dashboard. The dashboard grounds every IQM in published quality thresholds, benchmarks each scan against the normative distribution of high-resource open datasets, and supports cross-site multicentre implementation of optimized scan protocols in RCS.We describe the system architecture and a validation framework establishing measurement equivalence between WebMRIQC and native MRIQC across thirteen IQMs on the BraTS-Africa and BraTS 2021 datasets. Preliminary results indicate strong agreement for contrast-, signal and noise-based metrics, demonstrating that web-based implementation lowers the barrier to standardized MRI QC and provides a foundation for harmonized, regionally adapted quality benchmarks across RCS imaging sites. The code is publicly available here https://github.com/CAMERA-MRI/WebMRIqc.
Read the full abstract

All authors Philip Nkwam, Ifeoluwa Oladeji, Sekinat Zurakat-Aderibigbe, Jasmine Cakmak, Harrison Aduluwa, Confidence Raymond, Cliff Mokua, Abdulrazaq Zubair, Daniel Champanda, Tolulope Olusuyi, Maruf Adewole, Udunna Anazodo

Computer Vision

Evaluating the Generalization of Neuroimaging Foundation Models on African Brain MRI

Oluwatobi Iyanuoluwa Akinmuleya, Olatokun Shamsudeen Akano, Samuel Danquah Ankapong et al. (+2 more)

Neuroimaging foundation models pretrained on large, predominantly western cohorts are increasingly proposed as general-purpose backbones for brain MRI analysis. Yet, their ability to generalize to underrepresented clinical populations remains largely untested. We evaluate four recent foundation models (BrainIAC, Neuro-JEPA, NeuroVFM, and Primus) on a three-way diagnostic classification task (Control, Dementia, Parkinson's disease) using a cohort of 88 subjects from a Nigerian clinical brain MRI dataset, across four modality configurations (T1w, T2w, T1w+T2w, FLAIR), and compare against an end-to-end trained ViT3D baseline. The frozen backbones collapse to majority-class predictions, while Neuro-JEPA on FLAIR shows modest but still limited discrimination. In contrast, the end-to-end trained ViT3D achieves higher accuracy and MCC on every task (up to 53.4% accuracy, MCC=0.27) and is the only model with non-trivial recall. Our findings suggest that these frozen neuroimaging foundation models are insufficient for fine-grained diagnostic classification in small, non-western clinical cohorts, motivating parameter-efficient adaptation and broader multi-site external validation for equitable deployment in global health settings.
Read the full abstract

All authors Oluwatobi Iyanuoluwa Akinmuleya, Olatokun Shamsudeen Akano, Samuel Danquah Ankapong, Olamide Lawal, Toufiq Musah

Computer Vision · Cross-listed from cs.AI

UniK: Universal Knowledge Perception for Digital and Physical AI

Nirmit Desai, Kunal Sawarkar, Aditya Mahakali et al. (+3 more)

Two transformative classes of AI systems are reshaping how organizations operate: digital AI, which reasons over enterprise knowledge to power chatbots and agent workflows; and physical AI, which learns to control robots and autonomous systems from video, gameplay, and sensor telemetry. Both face the same foundational bottleneck: raw knowledge at scale, spanning heterogeneous modalities, locked in private corpora that existing AI infrastructure cannot access reliably or efficiently. We propose Universal Knowledge Perception (UniK) as a common platform for both classes, covering the full knowledge lifecycle (ingestion, enrichment, indexing, retrieval, and continuous evaluation) across modalities from rich text and video to molecular data and sensor telemetry. We present UniK, built on Polymath Retrieval (multi-index fusion over automatically enriched indices) with no task-specific fine-tuning. Across five digital AI domains (medical literature, open-domain QA, chemistry, legal video proceedings, and government open data) UniK combined with an open-source 70-billion-parameter model consistently matches or outperforms frontier proprietary LLMs that are orders of magnitude larger: 76% RAG accuracy on government data versus 47% for GPT-5; 77.9% on medical QA without fine-tuning; topping all open-source chemistry pipelines. We show that the same infrastructure directly addresses the data curation, indexing, and retrieval challenges facing physical AI world model training, where the knowledge problem is harder but structurally identical.
Read the full abstract

All authors Nirmit Desai, Kunal Sawarkar, Aditya Mahakali, Dongkon Lee, Kevin Park, Eric Song

Computer Vision

Rethinking Diffusion Segmentation: When Does It Rely on Its Noisy State, and Does Diffusion Matter?

Hengzhuo Yang, Yuming Zeng and Yuling Yang

Diffusion models are increasingly adapted from generation to conditional prediction, where a conditioning signal is combined with an evolving noisy representation of the target. In fully supervised segmentation, however, the conditioning image can already support direct target prediction, so endpoint performance alone establishes neither reliance on the added diffusion state nor a deterministic advantage over image-only prediction. For state reliance, we disrupt target-derived state content or correct image-state pairing during retraining of twelve published methods across three datasets, with ten matched seeds per setting. All 40 original-method comparisons whose evaluated-mask routes remained downstream of noised-quantity reconstruction exhibited state reliance, whereas all 30 comparisons with a segmentation-supervised bypass preserved reference performance. Rerouting five originally bypass-capable methods by forcing segmentation supervision through noise-to-mask reconstruction converted all 30 corresponding comparisons from preserved performance to state reliance. For deterministic utility, matched image-only counterparts achieved similar or better performance in 28 of 35 settings overall, including 16 of 20 whose native methods relied on both audited state properties. These results identify supervision path as a determinant of state reliance in the audited methods. Separately, matched image-only counterparts show that diffusion-specific computation often provides no deterministic endpoint advantage, including in methods that rely on the audited state properties. More generally, when conditioning already supports strong target prediction, diffusion-specific claims require additional evidence that the added state is used and that diffusion-specific computation improves the claimed capability beyond a matched condition-only counterpart.
Read the full abstract

Computer Vision

Colon3R: Cross-Domain 3D Reconstruction from Monocular Colonoscopic Video

Zhihao Xing, Yingyu Wang, Liang Zhao et al. (+1 more)

Monocular colonoscopic 3D reconstruction is important for surgical robotic colonoscopy, but remains challenging due to weak texture, specular reflections, limited view overlap, and non-rigid tissue motion. Conventional multi-view 3D reconstruction methods rely on stable correspondences and approximate rigidity, which are often violated in colonoscopy. Existing endoscopic methods often rely on domain-specific supervision, whereas there are not enough in-vivo labeled data available to adapt geometry foundation models to clinical colonoscopy. We present Colon3R, a cross-domain semi-supervised framework built on pretrained VGGT that transfers coupled camera, depth, and pointmap geometry from labeled phantom and simulated data to unlabeled in-vivo colonoscopy without requiring target-domain geometric annotations. Unlike source-only fine-tuning, which learns only from phantom and simulated data, Colon3R directly exploits unlabeled in-vivo video through teacher-derived cross-view supervision. Our proposed hierarchical quasi-rigid reliability selects reliable supervision at the sequence, directed-pair, and pixel levels, while source-preserving adaptation retains the learned coupled geometry during target-domain adaptation. Extensive experiments demonstrate that our method achieves superior overall performance over state-of-the-art approaches in depth, pointmap, and camera pose estimation. Qualitative comparisons on real in-vivo colonoscopy further show substantially more complete and geometrically consistent reconstructions than competing methods under clinical domain shift. The code is publicly available at https://github.com/vglx/Colon3R.
Read the full abstract

All authors Zhihao Xing, Yingyu Wang, Liang Zhao, Shoudong Huang

Sunday 20 September

Computer Vision

Query Independent Variable Rate Visual Token Coding

Hongbo Zhang, Zihao Yang, Liuyang Song et al. (+4 more)

Visual-token compression for vision–language models is posed almost entirely as a selection problem: decide which tokens to keep and discard the rest. The criteria that work best rank tokens by the attention the language model pays them, which makes the ranking a function of the question being asked. That is invisible in a single-turn benchmark and decisive whenever a compressed representation is written once and read many times, as when it is cached across the turns of a conversation or transmitted between a device and a server. We take the other half of the classical transform-coding toolkit instead: keep every token and vary its rate. A transform code exposes each token's measured distortion–rate curve, and a fixed bit budget is distributed across tokens by exact integer rate–distortion optimisation on those curves. No text enters the pipeline, so one compressed representation serves any query. At equal bit budgets, on two datasets and two capacities, it preserves the model's output distribution and its answers better than uniform-rate coding, the closed-form water-fill and distortion-ranked pruning. It matches attention-ranked pruning on the question pruning was tuned for, and overtakes it once the compressed image must answer a different question about the same image.
Read the full abstract

All authors Hongbo Zhang, Zihao Yang, Liuyang Song, Daqian Yang, Haoyang Yao, Yan Wen, Zhengtao Yao

Computer Vision

Cross-Dataset Generalization of Bangladeshi Rice Leaf Disease Classifiers: Benchmark, Diagnosis, and Mitigation

Anindya Paul

Cross-dataset transfer in rice leaf disease classification remains a significant challenge, with models trained on one image collection performing substantially worse when deployed on another. We conduct a systematic benchmark across three Bangladeshi rice leaf disease datasets (5,419 images, 6 transfer pairs, 3 CNN backbones, 3 random seeds) to characterize and diagnose this failure. Strong augmentation recovers a mean cross-dataset macro-F1 improvement of +0.070 (Wilcoxon p < 0.001, 15 of 18 transfer pairs positive). Removing non-leaf image content via segmentation shows directional benefit (mean +0.066, p = 0.062, n = 36 paired observations) that is consistent across two independent segmentation methods but does not reach conventional significance. A self-supervised ViT control (DINOv2 linear probe) exhibits equivalent cross-dataset collapse to CNNs, ruling out architecture inductive bias as the primary driver and pointing to acquisition-condition shift. Adaptive batch normalization uniformly harms transfer performance, with harm magnitude correlating with source-target label-prior divergence and model depth (Spearman rho = 0.621, p = 0.009). Grad-CAM attribution analysis on 12 sampled predictions does not distinguish correct from incorrect cross-domain predictions (p = 0.462), indicating that common attribution proxies are insufficient for diagnosing shift at practical sample sizes. We document all frozen results, prespecified analysis criteria, and reproducibility artifacts in a public repository with SHA-256 integrity verification. This work establishes a rigorous empirical baseline for understanding cross-dataset generalization in agricultural computer vision and identifies both effective (augmentation) and ineffective (AdaBN) adaptation strategies.
Read the full abstract

Computer Vision

Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Level

Alexandra Schild and Gerard de Melo

Visual Language Models (VLMs) have achieved remarkable success across diverse tasks, yet they struggle with high-resolution inputs where critical information resides in small regions or detailed, cluttered scenes. While several approaches address this limitation, a systematic understanding of why models fail at high resolutions is lacking. We introduce a controlled evaluation framework that disentangles resolution-related performance degradation from task difficulty through semantics-preserving transformations. We propose two simple metrics: Area Under the Scaling Curve (AUSC), which quantifies scaling robustness independent of baseline accuracy, and Prediction Variance Score (PVS), which measures resolution-induced prediction instability. Through comprehensive experiments across 5 model families and 5 benchmarks, we identify three primary failure modes: (1) information loss from downsampling at vision token limits, (2) tokenization artifacts from patch boundary shifts and positional encoding fragility under non-standard aspect ratios, and (3) attention dilution as token counts increase. Our analysis reveals that even state-of-the-art models suffer from performance drops when processing high-resolution images, with degradation patterns varying systematically by architectural family. We provide actionable insights for model architecture design and data augmentation strategies to mitigate these limitations.
Read the full abstract

Computer Vision

MDL-Calibrated Significance-Gain Pair Encoding: Replication-Aware Automatic Stopping for Subword Tokenization

Azam Nouri

Byte-Pair Encoding (BPE) constructs subword vocabularies through greedy pair merging, but conventional BPE requires the number of merges or target vocabulary size to be specified externally. Significance-Gain Pair Encoding (SG-BPE) replaces frequency-only selection with a statistical criterion based on how strongly an observed pair exceeds its expected co-occurrence under an independence model. This paper introduces MDL-Calibrated Significance-Gain Pair Encoding (MDL-SG), a three-stage procedure separating discovery, replication, and utility. Candidate pairs are ranked by Significance-Gain on a discovery partition, tested for replication on a separate partition using an exact one-sided hypergeometric test with per-iteration Benjamini-Hochberg correction, and then evaluated on a utility partition using a Minimum Description Length (MDL) criterion. Merging stops automatically when no replicated candidate yields positive held-out MDL gain. On WikiText-103, MDL-SG stops at 209, 433, and 847 merges for 120K, 250K, and 500K-character tokenizer-training samples, respectively. At 500K characters, it selects a stored vocabulary of 1,017 tokens without prescribing the vocabulary size in advance. In a compute-matched TinyGPT experiment with identical 2,024,448-parameter models and 500 optimizer updates per language model, MDL-SG achieves validation/test BPC of 3.2612/3.2436, compared with test BPC of 3.2894 for SG-BPE and 3.3493 for frequency BPE. Frequency BPE achieves stronger raw compression, while MDL-SG achieves lower BPC, showing that compression-oriented merge selection and language-model utility need not coincide.
Read the full abstract

Computer Vision · Cross-listed from eess.IV

Does DCGAN-Based Synthetic Augmentation Improve Brain Tumor MRI Classification? An Empirical Study

Irhum Jawad Khan and Talha bin Aslam

Generative adversarial networks (GANs) are increasingly used to augment medical imaging datasets, but synthetic images do not necessarily provide downstream classification benefits. This study investigates whether class-specific Deep Convolutional Generative Adversarial Network (DCGAN) augmentation improves brain tumor classification when the classifier and evaluation set are held constant. Experiments were conducted on 7,200 brain magnetic resonance imaging (MRI) scans across four classes: glioma, meningioma, pituitary tumor, and no tumor. For each class, 1,400 real images were used for training and 400 were reserved for testing. A baseline Swin Transformer classifier was trained using only the real training images and compared with a second model trained using the same real images augmented with 500 DCGAN-generated images per class. Both conditions were evaluated on the identical held-out test set. The two models achieved the same overall accuracy of 96%, while macro F1 remained effectively unchanged and ROC-AUC decreased slightly from 0.987 to 0.982 after augmentation. Class-level analysis showed small redistributions in errors rather than a consistent performance gain. FID values between real and synthetic images ranged from 209.15 to 314.27, indicating substantial distributional differences under the adopted evaluation setup. These results suggest that synthetic augmentation should not be assumed to improve medical image classification and should instead be evaluated for both distributional fidelity and downstream task utility.
Read the full abstract

Computer Vision · Cross-listed from q-bio.QM

WILSON - Multi-magnification composites as a foundation for patient-level pathology analysis and diagnostic text generation

Saghir Alfasly, Wataru Uegami, Sobhan Hemati et al. (+17 more)

Pathologists examine tissue sections across multiple magnifications and slides, integrating morphological information to establish a diagnosis. In contrast, most pathology foundation models encode thousands of image patches from a single representative ("best") slide and aggregate their features, limiting multiscale and patient-level contextual integration. Here we introduce WILSON, a vision-language foundation model that represents whole-slide images and multi-slide cases through multi-magnification composites. Trained on approximately 189,000 slides spanning 42 organs and 838 diagnostic entities, WILSON enables patient-level analysis, multimodal retrieval and diagnostic text generation. Without task-specific training, WILSON outperformed a dedicated case-level model (macro-F1, 0.52 versus 0.38) and matched substantially larger slide-level models while requiring 400-1,350 times fewer computational operations. End-to-end fine-tuning improved breast cancer subtyping and stromal lymphocyte grading. WILSON also surpassed PRISM in image-text retrieval (recall@1, 75.6% versus 58.1%) and generated diagnostic descriptions that received higher ratings from five board-certified pathologists than competing models. These findings demonstrate that compact, multiscale representations can support computationally efficient, clinically grounded pathology foundation models.
Read the full abstract

All authors Saghir Alfasly, Wataru Uegami, Sobhan Hemati, Wenchao Han, Xiaojia Tang, Kevin Thompson, Daniel Stone, Ghazal Alabtah, Chris Hartley, Chady Meroueh, John C. Cheville, Thomas J. Flotte, Saba Yasir, Michael R. Lucas, Eric W. Klee, Cheryl L. Willman, Judy C. Boughey, Matthew P. Goetz, Krishna R. Kalari, H.R. Tizhoosh

Computer Vision

You've Seen Enough: Quality-Constrained Image Coding for Machines

Khoa Pham-Dinh, Sanaz Nami, Hamed Rezazadegan Tavakoli et al. (+2 more)

Visual data is increasingly consumed by machine-vision systems rather than by human observers. Image Coding for Machines (ICM) compresses images assuming the main observer is a computer vision application and that the human observer needs to inspect or validate the decisions. Inspired by just-noticeable distortion, we cap human-observed quality at a desired level and devote the remaining bits to machine performance. Specifically, joint compression-segmentation training is recast as a constrained optimization problem in which the codec must meet a predefined acceptable target visual quality while a task term consumes the remaining coding capacity. This paper proposes two variants of a penalty function that guides the quality toward the target: an absolute function and a bilinear function, the latter applying a steeper slope once the target visual quality is exceeded. Experimental results show that, under the quality constraint, the proposed method achieves BD-rates of $-22.82\%$ and $-29.81\%$ relative to an unconstrained joint rate–distortion–task optimization and a simple rate–distortion baseline, respectively, showcasing bitrate reduction with the same task performance. This is achieved while the codec also meets the target visual quality with a reasonable error and without adding any complexity overhead.
Read the full abstract

All authors Khoa Pham-Dinh, Sanaz Nami, Hamed Rezazadegan Tavakoli, Moncef Gabbouj, Farhad Pakdaman

Computer Vision · Cross-listed from eess.IV

ORION-CMR: On-scanner Reporting with Integrated Foundation Model for End-to-End Cardiac MRI Analysis and Interpretation

Omer Burak Demirel, Kelly K. Horst, Alessio Perazzolo et al. (+15 more)

Cardiovascular magnetic resonance (CMR) provides comprehensive cardiac assessment but remains underutilized because of the complexity of acquisition, post-processing, and interpretation. Existing artificial intelligence (AI) methods address isolated tasks, limiting clinical integration. We present ORION-CMR (On-scanner Reporting with Integrated fOunda-tioN Model), the first clinically evaluated scanner-native end-to-end CMR foundation model. Pretrained on 12,896,733 CMR images from 9,258 studies, ORION-CMR performs sequence classification, ventricular function assessment, late gadolinium enhancement (LGE) detection, binary and multiclass disease classification, and local large language model-based report generation in approximately 90 seconds. The framework. was evaluated on public benchmarks and clinically validated in a multi-vendor cohort of 68 subjects with normal examinations, congenital heart disease, dilated cardiomyopathy, and myocardial infarction. ORION-CMR outperformed supervised baselines and the previously published CMR foundation model (CMR-FM), achieving state-of-the-art performance for LGE classification and scar segmentation. Clinical evaluation achieved an AUC of 0.96 for normal-versus abnormal classification and 0.88 for multiclass disease classification, while generated reports demonstrated 81.4% agreement with expert interpretation. These results demonstrate the feasibility of real-time scanner-native AI-assisted CMR analysis and automated report generation.
Read the full abstract

All authors Omer Burak Demirel, Kelly K. Horst, Alessio Perazzolo, Elisa Bruno, Kenan Kaya, Rongzhen Ouyang, Enas Ahmed, Jouke Smink, Spencer L. Waddle, Zainudeen Kallumpurath, Tzu Cheng Chao, Dinghui Wang, Steve G. Langer, Timothy L. Kline, Panagiotis Korfiatis, Jacinta Browne, Ivana Isgum, Tim Leiner

Computer Vision · Cross-listed from eess.IV

VGG16-MCA UNet: Whole-Tumor Segmentation in 2D FLAIR MRI with Decoder-Side Channel Attention

Shubham Gajjar, Deep Joshi, Avi Poptani et al. (+1 more)

Automated brain tumor segmentation supports diagnosis, treatment planning, and monitoring of disease progression, but building models that generalize across heterogeneous tumors and limited annotated data remains difficult. We present VGG16-MCA UNet, a hybrid architecture pairing an ImageNet-pretrained VGG16 encoder with a decoder in which a Multi-Channel Attention (MCA) module recalibrates features after each skip-connection fusion, trained with the Focal Tversky loss to counter severe foreground-background imbalance. We evaluate the model as a 2D, FLAIR-only, whole-tumor segmenter on tumor-positive slices from two public datasets: the BraTS 2020 benchmark and the LGG MRI Segmentation dataset. Using 5-fold cross-validation and a single network formed by averaging the weights of the five fold models, the method attains an aggregate pixel-level Dice (F1) of 95.10% on our held-out BraTS 2020 split and 88.32% on LGG. These scores are computed over all test pixels pooled into a single confusion matrix rather than averaged per case, and are therefore not directly comparable to the per-case mean Dice used in the BraTS challenge protocol. All partitions were drawn over individual slices rather than over patients, so every patient contributes slices to both training and test; the figures above therefore measure interpolation within known patients and should be read as an upper bound rather than as generalization to new ones. The model segments a 256x256 slice in 66.32 ms on a single 6 GB NVIDIA RTX 2060, approximately 8 ms more than an equivalent VGG16-UNet without MCA. We release the split records and report the protocol in full, with the aim of providing a precisely specified and reproducible 2D FLAIR baseline.
Read the full abstract

All authors Shubham Gajjar, Deep Joshi, Avi Poptani, Vishal Barot

Computer Vision · Cross-listed from eess.IV

Learning-Based 3D Reconstruction of Power Networks from Aerial Point Clouds

Rishabh Jain, Anuja Saini and Vishal Jain

This paper presents an end-to-end framework for reconstructing overhead power utility network topology and extracting span-level physical metadata from large-scale aerial LiDAR. The pipeline begins with semantic segmentation of the input point cloud using an improved KPConv-based model, in which data sampling and loss functions are adapted to emphasize pole and conductor (wire) classes. Network topology inference then proceeds in two stages: (i) pole instances are obtained by clustering pole-class points and validating candidates using geometric criteria, including height and verticality estimated via PCA, and (ii) candidate pole pairs are evaluated using a heuristic method and a lightweight ResNet-based classifier on 2D top-view projections of pole and wire point distributions to determine whether a physical conductor span exists. By explicitly classifying candidate spans, the approach mitigates common failure modes of heuristic connectivity rules in dense or cluttered scenes and under partial wire observation. For each validated wire, attributes regarding utility infrastructure geometry are computed, including endpoint conductor heights, ground elevation, sag-related lowest-point features, conductor arrangement, and wire width. Evaluation on multiple real-world aerial LiDAR datasets demonstrates decimeter-level endpoint height accuracy and approximately 9% relative improvement in recall for topology reconstruction compared to heuristic nearest-neighbor baselines, with larger gains in complex layouts.
Read the full abstract

Computer Vision

MotionJEPA: Preventing Temporal Feature Collapse by Capturing Visual Changes in Latent Space

Markus Karmann, Shile Li, Christian Internò et al. (+10 more)

Joint Embedding Predictive Architectures (JEPAs) are a promising paradigm for learning task-agnostic latent world models without visual reconstruction. However, standard JEPA training exhibits a strong inductive bias towards slow features, causing feature suppression and the collapse of latent representation. While inverse dynamics provides temporal anti-collapse, it relies on action labels and offers little incentive to embed general, unlabeled dynamics. We introduce Difference Image and Single image embedding Regularization (DISReg), a novel regularizer that builds on an inverse-dynamics-style module that predicts temporal difference image embeddings without any pixel reconstruction loss, encouraging balanced static and dynamic feature learning. DISReg consists of a static term that shapes the distribution of the image embedding and encourages slow features, and a dynamic term, which, unlike direct regularization on the embedding, imposes no constraint on the image embedding's shape or distribution and instead only incentivizes that dynamic features be present. By integrating this regularizer into a standard JEPA, we establish our new architecture, MotionJEPA. Latent probing demonstrates that MotionJEPA produces more complete representations than other methods, and our trajectory analysis shows it maintains geometrically simple latent embeddings with low curvature. We further show that MotionJEPA improves downstream planning success under static-background distractors across four environments.
Read the full abstract

All authors Markus Karmann, Shile Li, Christian Internò, Bruno Andreis, David Klindt, Randall Balestriero, Jindong Gu, Philip Torr, Qi Zhang, Peng-Tao Jiang, Hao Zhang, Bo Li, Onay Urfalioglu

Computer Vision

Comparative Performance and Parameter-Efficient Adaptation of DINOv2 for Active Trachoma Classification

Kibrom Gebremedhin, Hadush Hailu, Bruk Gebregziabher et al. (+1 more)

Automated grading of conjunctival photographs could reduce the cost and variability of trachoma prevalence surveys, but the relative value of modern pretrained visual representations, lightweight feature adaptation, and training-objective design has not been established under a common protocol. This study presents a controlled evaluation for binary classification of Trachomatous Inflammation-Follicular (TF) versus Normal using 1,546 images from the public UCSF/Lietman collection. Images are processed using the OPTED pipeline for zero-shot tarsal-conjunctiva segmentation, alignment, cropping, and standardization. We first compare six pretrained backbones using a common classification pipeline and then evaluate four lightweight adaptation mechanisms on DINOv2 ViT-B/14. Under stratified five-fold cross-validation, DINOv2 with Efficient Channel Attention (ECA) and focal-plus-center loss achieved 91.66 +/- 0.97% accuracy, 90.69 +/- 1.10% macro-F1, and 96.06 +/- 0.71% AUC. ECA introduces only five learnable parameters while matching the performance of substantially larger alternatives. Objective ablation further showed that ECA did not consistently improve plain DINOv2 across loss functions; the lowest-variance 91.66% accuracy was obtained with cross-entropy plus center loss. Overall, the fine-tuned DINOv2 representation provided most of the predictive performance, while ECA offered a highly parameter-efficient refinement whose effect depended on the training objective. The resulting workflow provides a reproducible benchmark for active trachoma image classification.
Read the full abstract

All authors Kibrom Gebremedhin, Hadush Hailu, Bruk Gebregziabher, Yordanos Hailu

Computer Vision

DFD-Lab: A Modular Audio-Visual Deepfake Detection Pipeline

Jan Rybarczyk, Mateusz Roszkowski and Jacek Komorowski

Comparing audio-visual deepfake detectors requires coordinating dataset adaptation, temporal input representation, model interfaces and experimental conditions. We present DFD-Lab, a modular pipeline that separates these responsibilities while supporting shared training and evaluation workflows. We integrate three implementations: Xception-based maximum-logit fusion, ResNet with temporal LSTM fusion, and our AVFF reimplementation. Experiments cover external testing, degradation-based training augmentation and evaluation-time corruption. On a filtered subset of Deepfake-Eval-2024, models trained on FakeAVCeleb attain baseline AUROC values of 0.504, 0.538 and 0.458. JPEG50 training augmentation raises these to 0.691, 0.605 and 0.570, respectively, while all three accuracies decrease. These results illustrate why training interventions, evaluation corruptions and metric-dependent outcomes should remain distinct within a common pipeline. The contribution is the integration of audio-visual processing, interchangeable detectors and configurable experimental workflows, supported by empirical case studies. The findings highlight the challenge of cross-dataset detection and the complementary information provided by ranking and classification metrics.
Read the full abstract

Computer Vision

VISTA: Video-Injected Stylized Text-to-Animation

Monseej Purkayastha, Anindita Ghosh and Philipp Slusallek

We present VISTA, a two-stage framework for generating stylized 3D human motion by fusing structural content from text prompts with expressive style from reference videos, without requiring jointly paired (text, video, stylized motion) triplets. A Dual-channel Autoencoder first maps motion sequences and video clips into a shared latent manifold. A masked autoregressive diffusion backbone then operates within this manifold, injecting video-derived style through a dedicated late-fusion Dual-AdaLN pathway while preserving text-conditioned content structure. A cross-batch unpaired training protocol with latent cycle consistency enables joint learning across separate semantically rich and stylistically diverse datasets. As a proof-of-concept for controllable animation synthesis, we validate VISTA on rendered motion-capture references: it achieves the highest style recognition accuracy among video-conditioned methods while preserving competitive content alignment, and its decomposed 3-way classifier-free guidance provides independent, user-controllable calibration of the content-style balance at inference time.
Read the full abstract

Computer Vision

Confidence-Aware Teacher-Student Distillation for 3D Medical Segmentation

Georgios Triantafyllou and Dimitris K. Iakovidis

Medical image segmentation models typically rely on large amounts of densely annotated volumetric data, limiting their scalability across tasks and imaging modalities. This work addresses the challenge of predicting entire 3D anatomical structures from extreme annotation sparsity. An annotation-efficient student-teacher framework is proposed for automatic 3D medical segmentation that requires only a set of point prompts on a single 2D slice per volume, as input. A foundation model serves as an offline teacher, utilizing the provided point prompts from the selected slice to full-volume pseudo-annotations alongside their corresponding spatial confidence scores prior to student training. To mitigate the error propagation of noisy pseudo-annotations, a task-specific 3D student network is trained using a confidence-aware optimization strategy. By leveraging the teacher's pre-computed confidence scores, this strategy explicitly excludes statically uncertain regions of the pseudo-annotations from the loss calculation, while simultaneously emphasizing regions with higher confidence. Evaluated on 3D cardiac MRI datasets, our framework outperforms state-of-the-art semi-supervised methods, improving segmentation performance by up to 43.6%. Furthermore, it drastically reduces the manual annotation burden to just a few positive point prompts per volume, while improving surface boundary precision by up to 14.7% over the teacher and successfully recovering up to 34.1% of the performance gap toward the fully supervised upper bound.
Read the full abstract

Computer Vision · Cross-listed from cs.GR

MoSAT: Human Motion Generation from Spatial Audio and Textual Description

Shuyang Xu, Zhiyang Dou, Yiduo Hao et al. (+8 more)

Human motion is shaped by both external acoustic events and behavioral intent: spatial audio conveys environmental cues that elicit or guide a response, while text specifies the desired action and how it should be performed. In this paper, we study the novel task of human motion synthesis jointly conditioned on spatial audio and natural language, a problem that has been largely overlooked in previous research. To support this task, We introduce STAM, a dataset of motion sequences paired with spatial audio and detailed textual annotations whose rich vocabulary affords precise and nuanced specification of human motions. We further introduce MoSAT, a latent flow-matching framework for full-body motion generation jointly conditioned on natural-language intent and directional spatial-audio cues through hierarchical cross-attention before generating motion. Such a hierarchical design enhances temporally coherent and semantically aligned motion sequences. We also develop tri-modal evaluators for comprehensive evaluation on this novel task. Extensive experiments show that MoSAT achieves the SOTA performance by leveraging spatial audio's intrinsic motion-shaping properties alongside textual semantics, enabling precise and diverse motion in various scenarios.
Read the full abstract

All authors Shuyang Xu, Zhiyang Dou, Yiduo Hao, Zekun Li, Liang Pan, Jingbo Wang, Cheng Lin, Yuan Liu, Wenping Wang, Mingmin Zhao, Taku Komura

Computer Vision

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction

Yang-Tian Sun, Tianjia Liu, Zehuan Huang et al. (+7 more)

Single-image 3D object generation can now produce high-fidelity assets, yet accurately placing them into a coherent scene layout remains an open challenge. A central difficulty lies in how object layout is represented. Holistic methods absorb placement into a scene-level generation process, sacrificing object-level detail. Compositional methods preserve object fidelity by decoupling geometry from layout, but typically parameterize layout as sparse, unbounded pose variables that are difficult to learn and generalize poorly under scarce scene-level supervision. We present Mira-Scene, a compositional 3D scene reconstruction framework that replaces sparse pose regression with dense, bounded correspondence recovery. At its core is the Canonical Coordinate Map (CCM), a pixel-aligned field that maps each visible object pixel to a surface coordinate in the object's bounded canonical space. When paired with a scene-space Point Cloud Map (PCM) from monocular geometry estimation, CCM induces dense canonical-to-scene correspondences from which object transformations are recovered through robust geometric alignment. Because CCM operates in bounded canonical space, it provides a stable prediction target that can be trained from scalable object-level 3D data without requiring scene-level layout annotations. Mira-Scene further introduces a multimodal diffusion transformer that jointly generates object geometry and CCMs, using modality-specific expert streams with shared attention and positional encoding to promote geometry-layout consistency. Experiments on indoor, outdoor, synthetic, and in-the-wild scenes show that Mira-Scene substantially outperforms strong baselines in layout accuracy, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D, using limited open-source training data.
Read the full abstract

All authors Yang-Tian Sun, Tianjia Liu, Zehuan Huang, Yi-Hua Huang, Xiaoyang Lyu, Ziyi Yang, Zi-Xin Zou, Yuan-Chen Guo, Yan-Pei Cao, Xiaojuan Qi

Computer Vision

PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning

Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs et al. (+1 more)

The disambiguation of semantically similar statutory text across jurisdictions is a retrieval problem that existing methods do not solve. This inter-context conflict can steer generative models toward confidently produced answers grounded in topically relevant but jurisdictionally incorrect sources. Tobacco and nicotine regulations vary by US jurisdiction, often sharing similar language, thus, robust reasoning requires identifying which jurisdiction's law governs a given product, not merely retrieving relevant text. Emerging products (e.g., pouches) exploit ambiguous definitions to evade regulation. State-of-the-art (SOTA) document retrieval-augmented generation (RAG) methods struggle to address this inter-context conflict, and thus struggle to connect image attributes (e.g., rich attribute captions) to the set of similar legislation texts. We introduce NicoPRISM (Nicotine Product and Regulation Image-and-Text Surveillance Multimodal), comprising 161,563 images, attribute captions, a knowledge base of product, health, and legislative documents spanning 13 US jurisdictions, and 1,495 validated question-answer pairs across two tasks: policy compliance QA and product knowledge QA. We also propose PRISM-RAG, a multimodal hypergraph RAG framework built over images, captions, and entities without any LLM calls at index time, grounding every query in a product image and routes retrieval through a jurisdiction-aware context assembly mechanism guaranteeing that statutory text from the queried jurisdiction reaches the language model by construction. PRISM-RAG retrieves passages from the correct jurisdiction in 93.9% of policy compliance queries, a 48.6 percentage point advantage over standard RAG (p<0.001), using zero LLM calls at index time and one at query time, and is competitive with or outperforms SOTA RAG frameworks across keyword, semantic, jurisdiction-, and compliance-accuracy metrics.
Read the full abstract

All authors Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs, Khoa Luu

Computer Vision

Training-Free Spectral Transductive Refinement for Cross-Domain Few-Shot Classification

Fahim Rahman, S.M. Tanjeeb Meheran Rohan, Md. Taimum Ibne Sayed et al. (+2 more)

Few-shot recognition with frozen visual features is especially fragile under domain shift and one-shot supervision, where a single labelled image is an unreliable estimate of its class. We ask how far this fragility can be reduced purely at test time, without retraining the encoder or augmenting the source domain. We present Spectral Transductive Refinement (STR), a training-free transductive inference rule that exploits the geometry of the complete support-query episode. Given frozen embeddings, STR builds a joint k-nearest-neighbour graph, maps the episode into a normalized-Laplacian spectral coordinate system, initializes class representatives from the labelled support, and iteratively refines them using pseudo-labelled queries. We evaluate STR under two protocols. A controlled component study with frozen ResNet-18 features shows that spectral refinement consistently improves over single-prototype spectral initialization across five shifted domains, with the largest gains in the one-shot regime where support estimates are weakest. We then benchmark STR against recent Cross-Domain Few-Shot Learning (CD-FSL) methods using the standard miniImageNet-pretrained ResNet-10 backbone over eight established target domains. Operating entirely at inference time, STR attains the highest 1-shot average among compared methods and remains competitive at 5-shot, rivalling approaches relying on heavy source-domain meta-training augmentations. Because STR is transductive, we report its setting explicitly. Diagnostics attribute its gains to iterative refinement in spectral coordinates rather than added prototype capacity, which remains inactive in our configuration.
Read the full abstract

All authors Fahim Rahman, S.M. Tanjeeb Meheran Rohan, Md. Taimum Ibne Sayed, Asaduzzaman Herok, Md. Bakhtiar Hasan

Computer Vision

OnlineWM: Causality-Aware Active Online Learning for Effective World Modeling

Yikun Miao, Fangqi Zhu, Quanxin Shou et al. (+6 more)

Generative world models aim to predict future states conditioned on actions, where action controllability is fundamental for reliable dynamics modeling. While recent efforts leverage simulator-generated data to enhance this capability, existing training pipelines face two fundamental limitations. First, static offline data collection leads to a distribution misalignment between training sets and the model's evolving error patterns, failing to resolve critical long-tail scenarios where dynamics predictions remain unreliable. Second, the standard objective of minimizing observational discrepancy often encourages the model to exploit spurious correlations instead of capturing the underlying action-effect causality. To address these limitations, we propose OnlineWM, an online training framework that continuously improves world modeling through active simulator interaction and causality-aware optimization. OnlineWM introduces two key innovations: (1) Active Online Learning: Instead of using fixed datasets, OnlineWM adaptively queries the simulator for new interaction sequences that target the model's current predictive weaknesses, ensuring high-utility data acquisition. (2) Causality-Aware Fine-Tuning: We propose a counterfactual learning strategy that contrasts the outcomes of different actions from identical states, forcing the model to attribute state transitions to specific actions rather than ambient environmental evolution, thereby grounding its predictions in reliable causal mechanisms. By integrating active data acquisition with causal optimization, OnlineWM establishes a closed-loop refinement process that ensures the model is both robust to diverse scenarios and precise in its causal attribution. Extensive experiments demonstrate that OnlineWM significantly enhances action controllability and generalizes effectively to unseen domains.
Read the full abstract

All authors Yikun Miao, Fangqi Zhu, Quanxin Shou, Xiaoyi Pang, Zhengyang Yan, Junhao Li, Haodong Wang, Zicong Hong, Song Guo

Computer Vision

VGGT-Prime: Compute-Adaptive Mixture-of-Heads for Efficient Visual Geometry Transformers

Abteen Arab, Guile Wu, Chengjie Huang et al. (+1 more)

Feed-forward visual geometry models such as the Visual Geometry Grounded Transformer (VGGT) have recently enabled direct 3D reconstruction from multi-view images. Despite their promising performance, these models scale quadratically with the number of input views due to their global attention mechanism, resulting in substantial latency for long sequence inputs. There have been some recent efforts to accelerate VGGT, but they primarily focus on reducing token redundancy through token merging or key/value sparsification. Our work resolves this bottleneck from a different perspective by investigating architectural redundancy in visual geometry transformers. We show that the multi-head attention modules in VGGT's global-attention layers contain substantial architectural redundancy, with only a subset of heads carrying critical geometric information. In light of this observation, we propose VGGT-Prime, a compute-adaptive mixture-of-heads model that resolves this redundancy to accelerate visual geometry transformers while maintaining competitive reconstruction quality. The key idea of VGGT-Prime is to estimate the appropriate computation level for each global-attention head using a lightweight router and then dynamically assign each head to different computation modes. Extensive experiments on multiple datasets demonstrate that VGGT-Prime can achieve an {$8\times$} inference speedup over VGGT while maintaining competitive performance on camera pose, depth, and point-cloud predictions. We further show that VGGT-Prime is complementary to existing acceleration methods, such as token merging, further improving inference speed by up to $14{\times}$ over VGGT. An overview of our work is available on our \href{https://vggt-prime.github.io}{project page}.
Read the full abstract

All authors Abteen Arab, Guile Wu, Chengjie Huang, Dongfeng Bai

Computer Vision

BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring

Liuyang Song, Yi Zhang, Zhongyi Deng et al. (+2 more)

Global vision–language similarities compress an image and a caption into one vector, preserving semantics but not which word corresponds to which region or how those regions are arranged; a model can recognize every word and object yet prefer a compositionally incorrect caption. We argue that a frozen encoder retains this association structure, so the problem is to read it rather than to rebuild it beside the pretrained similarity. We introduce BindCLIP, a pairwise scorer built on one latent object: a balanced token–patch–depth optimal-transport coupling that places both candidate captions and several visual depths in a single plan. Semantic, entity, order, and spatial evidence are read as energies of this state, and exchanging the candidates permutes the plan, making the score exactly antisymmetric. A geometric refinement inside the coupling contracts moves that the candidates and the visual depths do not support. No task label, parser, relation inventory, or detector is used. One checkpoint and one inference path improve the official What'sUp, ARO, and SugarCrepe benchmarks over frozen global CLIP, with the strongest transfer on the relation splits. Controls rule out patch access and caption-length shortcuts, and an inference-time lesion localizes spatial arrangement to the coupling.
Read the full abstract

All authors Liuyang Song, Yi Zhang, Zhongyi Deng, Daqian Yang, Hongbo Zhang

Computer Vision

Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models

Yahong Wang, Zhangkai Ni, Juncheng Wu et al. (+3 more)

Multimodal large language models (MLLMs) incur substantial computational overhead due to the reliance on hundreds of visual tokens to represent images. While token pruning has emerged as a promising approach to reduce the inference cost of MLLMs, existing methods typically reassign position embeddings to the retained tokens using either sparse or continuous position embeddings, each introducing distinct limitations. Sparse position embeddings tend to decrease the attention value allocated to visual tokens, thereby degrading the perception capability of MLLMs, whereas continuous position embeddings disrupt the original spatial correspondence of visual tokens, leading to weakened grounding capability. To mitigate this issue, we perform layer-wise analysis of the language decoder and observe that intermediate layers play a critical role for maintaining the grounding capability of MLLMs under token pruning. Based on this observation, we propose a layer-aware position embedding strategy, which switches to sparse position embeddings at grounding-sensitive layers while maintaining continuous position embeddings elsewhere. Extensive experiments across representative pruning methods and diverse benchmarks demonstrate that our approach improves the comprehensive multimodal performance of pruned MLLMs compared with standard sparse and continuous position embeddings.
Read the full abstract

All authors Yahong Wang, Zhangkai Ni, Juncheng Wu, Yuyin Zhou, Ying Wen, Lianghua He

Computer Vision

Infectious Bovine Pinkeye Detection Using Computer Vision and Imbalance-Aware Learning

Michael Abalo, Jameson Brennan and Hossein Moradi Rekabdarkolaee

Infectious bovine pinkeye is a contagious ocular disease that adversely affects cattle health, welfare, and agricultural productivity. Conventional diagnosis relies primarily on clinical observation, which can be subjective, time-consuming, and difficult to implement efficiently in large herds or remote settings. This study evaluated and compared You Only Look Once (YOLO) v11 and YOLOv26 for automated bovine pinkeye classification and investigated the effects of class-balancing strategies on model performance. Five variants (n, s, m, l, and x) of each architecture were trained and evaluated using the original imbalanced dataset, Random Minority Oversampling (RMO), and an adapted Synthetic Minority Oversampling Technique (SMOTE). Both YOLOv11 and YOLOv26 demonstrated strong classification performance, although the effects of class balancing varied across model variants. For YOLOv11, RMO-s achieved an accuracy of 0.99, a macro F1-score of 0.98, and a true positive rate (TPR) of 1.00, with no false-negative classifications. RMO-m also achieved a TPR of 1.00 with no false negatives. For YOLOv26, the original l, RMO-m, and RMO-l variants each achieved an accuracy of 0.99 and a macro F1-score of 0.98, with RMO-l attaining a TPR of 1.00 and no false negatives. Overall, RMO generally provided greater improvements in minority-class detection than adapted SMOTE, whereas the strong performance of the original YOLOv26-l demonstrates that oversampling was not necessary for all model variants. These findings demonstrate the potential of YOLOv11 and YOLOv26 for automated detection of bovine pinkeye and support further evaluation for livestock health monitoring.
Read the full abstract

Computer Vision · Cross-listed from cs.LG

A multi-temporal dataset for mapping burned areas in the Brazilian Cerrado using time series of remote sensing imagery

Alisson Cleiton de Oliveira and Thales Sehn Körting

This paper introduces a multi-temporal tabular dataset derived from satellite images to map burned areas in the Chapada dos Veadeiros National Park, in Goiás, Brazil, covering the years 2020 to 2022. The dataset contains blue, green, red, and near-infrared bands, as well as the BAI, EVI, GEMI, NDVI, and NDWI spectral indices from the WFI sensor on the CBERS-4A, CBERS-4, and AMAZONIA-1 satellites, organized into a regular grid. We applied the Random Forest classifier to develop and validate models based on samples labeled as totally burned, partially burned, and non-burned. Two classification approaches were tested: one combining burned and non-burned areas into binary classes and another distinguishing between totally burned (TB), partially burned (PB), and non-burned (NB) classes. Seven validation approaches assessed different post-classification combinations, focusing on accuracy, precision, recall, and intersection over union (IoU) metrics. Results showed higher IoU when TB, PB, and NB were used as individual classes and TB was reclassified as burned area (BA) while PB and NB were grouped as non-burned. Comparing the annual results of this approach to the MCD64A1 product, the errors of omission for the BA class were 22% in 2020, 28% in 2021 and 59% in 2022, while the errors of commission were 46%, 43% and 46%, respectively. The study highlights the utility of the WFI sensor for burned area mapping without inter-satellite spectral calibration and suggests further exploration with other machine learning algorithms to evaluate the dataset potential and limitations.
Read the full abstract