Computer Vision
SPECTRA-Net: Scalable Pipeline for Explainable Cross-domain Tensor Representations for AI-generated Images Detection
Sarra Arab, Anfal Achouri and Seif Eddine Bouziane
The rapid proliferation of AI-generated images (AIGI) presents a significant challenge to digital information integrity. While human observers and existing detection models struggle to keep pace with the increasing sophistication of generative models, the need for robust, real-time detection systems has become critical. This paper introduces SPECTRA-Net, a scalable pipeline for explainable, cross-domain tensor representations for AIGI detection. Our approach leverages a multi-view representation of images, combining global semantic features from a
Vision Foundation Model (VFM), spectral analysis, local patch-based anomaly detection, and statistical descriptors. By fusing these complementary data streams, SPECTRA-Net achieves state-of-the-art performance in both in-domain and cross-domain settings, demonstrating high accuracy and generalization capabilities across a wide range of challenging
datasets, including WildFake, Chameleon, and RRDataset. The proposed pipeline not only provides a robust solution for AIGI detection but also offers explainability through artifact localization, paving the way for more trustworthy and reliable content verification in real-world applications.
Read the full abstract ▾
Computer Vision
From Historical Tabular Image to Knowledge Graphs: A Provenance-Aware Modular Pipeline
Sarah Binta Alam Shoilee, Victor de Boer, Jacco van Ossenbruggen et al. (+1 more)
Handwritten archival tables contain rich historical information, yet transforming them into structured representations, such as Knowledge Graphs, requires integrating table structure recognition, handwriting recognition, and semantic interpretation - a complex
multimodal process. End-to-end AI implementations can obscure these steps, resulting in opaque algorithmic operations that hinder human oversight, critical assessment, and trust. To address this, we present a modular, provenance-aware pipeline to convert handwritten tabular images into KGs supporting human-AI collaboration. The pipeline decomposes the workflow into three stages - table reconstruction, information extraction, and KG construction - while exposing intermediate representations for inspection, evaluation, and correction. A key contribution of our approach is the systematic integration of data provenance at every stage, ensuring that all extracted entities and literals remain traceable to their visual and textual origins. The proposed pipeline is demonstrated through a number of experiments on real-world archival material concerning military careers. The results across three different table reconstruction variants highlight the importance of modularisation. By coupling modularity with data provenance, our work advances transparent and collaboratively controllable image-to-KG pipelines for complex historical data.
Read the full abstract ▾
All authors Sarah Binta Alam Shoilee, Victor de Boer, Jacco van Ossenbruggen, Susan Legêne
Computer Vision · Cross-listed from cs.AI
Spatial Priming Outperforms Semantic Prompting: A Grid-Based Approach to Improving LLM Accuracy on Chart Data Extraction
Andrei Lazarev, Dmitrii Sedov and Alexander Galkin
The automated extraction of data from scientific charts is a critical task for large-scale literature analysis. While
multimodal Large
Language Models (LLMs) show promise, their accuracy on non-standardized charts remains a challenge. This raises a key research question: what is the most effective strategy to improve model performance (high-level semantic priming) or low-level spatial priming? This paper presents a comparative investigation into these two distinct strategies. We describe our exploratory experiments with semantic methods, such as a two-stage metadata-first framework and Chain-of-Thought, which failed to produce a statistically significant improvement. In contrast, we present a simple but highly effective spatial priming method: overlaying a coordinate grid onto the chart image before analysis. Our quantitative experiment on a synthetic
dataset demonstrates that this grid-based approach provides a statistically significant reduction in data extraction error (SMAPE reduced from 25.5% to 19.5%, p < 0.05) compared to a baseline. We conclude that for the current generation of
multimodal models, providing explicit spatial context is a more effective and reliable strategy than high-level semantic guidance for this class of tasks.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models
Adam Szokalski and Mateusz Modrzejewski
This paper proposes latent visualization by optimization (LVO), a mechanistic interpretability technique that extends feature visualization by optimization - originally developed for convolutional neural networks - to latent
diffusion models. LVO employs sparse autoencoders (SAEs) to disentangle polysemantic layer representations into monosemantic features. Key contributions include latent-space optimization, time-step activity analysis, schedule-matched noise injection, prior initialization through feature steering, and suitable regularization strategies. We demonstrate the method on Stable
Diffusion 1.5 fine-tuned on the Style50
dataset, showing that SAE features produce clear visualizations of recognizable concepts - including diagonal compositions, human figures, roses, cables, and waterfall foam - that correlate with
dataset examples, while the baseline without disentanglement produces less coherent results. We further show that regularization techniques from pixel-space feature visualization transfer to the latent domain, though they require different configurations for the raw-layer and SAE variants. Compared to
dataset examples and steering, LVO provides complementary insights by directly revealing what activates a feature rather than its downstream effects.
Read the full abstract ▾
Computer Vision
Test-Time Training for Visual Foresight Vision-Language-Action Models
Sangwu Park, Wonjoong Kim, Yeonjun In et al. (+3 more)
Visual Foresight VLA (VF-VLA) has become a prominent architectural choice in the recent VLA due to its impressive performance. Nevertheless, the inherent design of VF-VLA makes it particularly vulnerable to out-of-distribution (OOD) shifts. Because the quality of action directly depends on the accuracy of the predicted future visual information, OOD conditions affect both stages at once. To address this vulnerability, we propose Test-Time Training Visual Foresight VLA ($T^3$VF), a test-time training approach motivated by the observation that the predicted future image and its subsequent observation form a natural supervision pair. To further address the practical challenges that arise from indiscriminate test-time updates, we introduce an adaptive update filtering mechanism. Empirically, $T^3$VF mitigates the OOD vulnerability of VF-VLA at a modest additional inference cost, without requiring any architectural modification or auxiliary modules.
Read the full abstract ▾
All authors Sangwu Park, Wonjoong Kim, Yeonjun In, Sein Kim, Hongseok Kang, Chanyoung Park
Computer Vision
Low-Cost Stereo Vision for Robust 3D Positioning of Thin Radiata Pine Branches in Autonomous Drone Pruning
Yida Lin, Bing Xue, Mengjie Zhang et al. (+2 more)
Manual
pruning of radiata pine, a species of major economic importance to New Zealand forestry, is hazardous, labour-intensive, and increasingly constrained by workforce shortages. Existing autonomous
pruning platforms typically rely on expensive sensors such as
LiDAR and are limited to thick branches, which restricts their wider adoption. This paper investigates whether a single low-cost stereo camera mounted on a drone can provide sufficiently accurate branch detection and three-dimensional positioning to support autonomous
pruning of branches as thin as 10 mm, thereby removing the need for auxiliary depth sensors. The proposed pipeline comprises two stages: branch
segmentation and depth estimation. For
segmentation, Mask R-CNN variants and the YOLOv8 and YOLOv9 families are compared on a custom
dataset of 71 stereo image pairs captured with a ZED Mini camera; YOLOv8 and YOLOv9 are selected as representative state-of-the-art real-time segmentors at the time of data collection, and the framework is designed to remain compatible with newer YOLO releases. For depth estimation, a traditional method (SGBM with WLS filtering) and deep-learning-based methods (PSMNet, ACVNet, GWCNet, MobileStereoNet, RAFT-Stereo, and NeRF-Supervised Deep Stereo) are evaluated, including cross-
dataset fine-tuning experiments that expose the domain gap between urban driving
benchmarks and natural forestry scenes. The main novelty of this work lies in coupling stereo
segmentation with a centroid-based triangulation
algorithm and Median-Absolute-Deviation outlier rejection that converts a
segmentation mask and disparity map into a single robust branch-to-camera distance, addressing the challenges of sparse texture, thin structures, and noisy disparity values typical of forest scenes. Qualitative evaluations at distances of 1-2 m show that the learning-based stereo methods produce more coherent depth es...
Read the full abstract ▾
All authors Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green
Computer Vision
Harmonized Feature Conditioning and Frequency-Prompt Personalization for Multi-Rater Medical Segmentation
Sanaz Karimijafarbigloo, Armin Khosravi, Alireza Kheyrkhah et al. (+3 more)
Multi-rater medical image
segmentation captures the inherent ambiguity of clinical interpretation, where diagnostic boundaries vary across experts and imaging devices. Existing approaches often reduce this diversity to consensus labels or treat rater differences as noise, resulting in overconfident and poorly calibrated models. We propose a harmonized probabilistic framework that disentangles acquisition artifacts from genuine annotator variability through adaptive feature conditioning and frequency-domain personalization. A lightweight Harmonizer Network implicitly models scanner-specific artifacts and performs dynamic feature modulation to standardize latent representations, ensuring that uncertainty reflects anatomy rather than noise. To represent rater-specific styles, we introduce a novel High-Frequency
Prompt Modules that operate in the spectral domain to encode annotator-dependent boundary precision and textural sensitivity. These
prompts adaptively modulate harmonized features to produce personalized yet anatomically consistent
segmentations. Furthermore, a Generalized Energy Distance based regularization aligns the generative distribution with empirical annotation variability, promoting diversity where experts disagree and consensus where they converge. Experiments on LIDC-IDRI and NPC-170 show SOTA aggregated and individualized
segmentation, with notable GED reductions and improved Dice scores, especially on noisy cases. Beyond accuracy, the model exhibits clinically meaningful uncertainty. Confidence rises in agreement regions and declines in ambiguous areas, supporting its use as a reliable and interpretable tool for multi-expert clinical workflows.
Read the full abstract ▾
All authors Sanaz Karimijafarbigloo, Armin Khosravi, Alireza Kheyrkhah, Reza Azad, Mauricio Reyes, Dorit Merhof
Computer Vision
A Breast Vision Pathology Foundation Model for Real-world Clinical Utility
Yingxue Xu, Zhengyu Zhang, Xiuming Zhang et al. (+32 more)
Pathology foundation models have shown strong retrospective performance, but whether such systems can support clinically relevant use remains unclear. This challenge is particularly important in breast cancer, where pathological assessment serves as the gold standard for diagnosis and guides treatment planning,
surgical decision-making and risk stratification across pre-, intra- and post-operative stages. Here we present BRAVE, a breast-adaptive pathology foundation model developed and evaluated using a total resource of 101,638 breast whole-slide images from 32 sources across Asia, Europe and North America. We assessed BRAVE across 34 tasks in 82 cohorts spanning pre-operative biopsy, intra-operative frozen section and post-operative resection, using an evidence chain comprising retrospective benchmarking, clinically challenging scenarios, workflow-oriented clinical impact simulations, prospective observational validation with the thresholds locked in the retrospective cohorts and crossover pathologist-AI interaction studies. Across these settings, BRAVE supported practical roles in the clinical workflow, including safe exclusion of low-risk cases from routine review, AI-assisted second-review rescue of initially missed positives and prioritization of cases for further assessment. In prospective validation across three centres, BRAVE excluded 76.9% of negative biopsy cases (NPV 0.953) and 70.1% of negative frozen-section cases (NPV 0.973), and triaged 78.8% of post-operative subtyping cases as high-confidence clear-cut cases (NPV 1.000). In reader studies, AI assistance improved balanced accuracy from 88.5% to 95.1% (OR 3.14, P<0.001), with better efficiency, confidence and inter-rater agreement. BRAVE-derived scores also independently predicted disease-free survival (adjusted HR 4.79, P<0.001) and overall survival (adjusted HR 8.14, P<0.001).
Read the full abstract ▾
All authors Yingxue Xu, Zhengyu Zhang, Xiuming Zhang, Mengwei Xu, Fengtao Zhou, Yihui Wang, Jiabo Ma, Yi Xin, Danyi Li, Chengyu Lu, Zhijian Cen, Ying Tan, Qingbing Yao, Qi Wang, Zizhao Gao, Yong Zhang, Jingjing Chen, Feifei Liu, Qian Xu, Yi Dai, Hongxuan Tan, Cheng Jin, Huajun Zhou, Zhengrui Guo, Ling Liang, Hongyi Wang, Yingcong Chen, Xi Wang, Zhenhui Li, Ronald Cheong Kin Chan, Ning Mao, Muyan Cai, Zhe Wang, Li Liang, Hao Chen
Computer Vision
Visual Text Compression as Measure Transport
Lv Tang, Tianyi Zheng, Yang Liu et al. (+2 more)
Visual text compression (VTC) promises efficient long-context processing by rendering text into an image and re-encoding it with a
vision-
language model, often producing $3$–$20\times$ fewer decoder
tokens than subword tokenization. Yet
token savings do not translate predictably into downstream utility: on some tasks the visual path matches or exceeds the text path, on others it collapses, and the compression ratio itself does not predict which regime will occur. The missing quantity is therefore not another summary of efficiency, but a principled measure of task-relevant information loss induced by visual encoding. We address this problem by formulating VTC in the
language of measure transport. Treating text and visual
tokens as empirical probability measures, we show that the ViT patch encoder induces a push-forward map whose transport cost decomposes into a precision cost from within-patch aggregation and a coverage cost from cross-patch fragmentation. Both terms are estimable from downstream-label-free probes. This formulation yields two operational consequences: a downstream-label-free routing criterion that selects whether to use the visual path for a given input or
benchmark instance, and a transport-informed foveation mechanism that re-encodes high-cost regions at higher resolution. Across $24$ NLP
datasets at Qwen3-4B, our label-free rule matches the per-
dataset oracle on $17/24$
datasets ($70.8\%$), and improves the average task score by $+3.3\%$ with $-10.3\%$ average
tokens relative to a pure-LLM.
Read the full abstract ▾
All authors Lv Tang, Tianyi Zheng, Yang Liu, Bo Li, Xingyu Li
Computer Vision · Cross-listed from cs.CL
HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities
Esra Dönmez, Pascal Tilli, Hsiu-Yu Yang et al. (+2 more)
Image-Text-Matching (ITM) is one of the defacto methods of learning generalized representations from a large corpus in
Vision and
Language (VL). However, due to the weak association between the web-collected image-text pairs, models fail to show a fine-grained understanding of the combined semantics of these modalities. To address this issue we propose Hard Negative Captions (HNC): an automatically created
dataset containing foiled hard negative captions for ITM training towards achieving fine-grained cross-modal comprehension in VL. Additionally, we provide a challenging manually-created test set for benchmarking models on a fine-grained cross-modal mismatch task with varying levels of compositional complexity. Our results show the effectiveness of training on HNC by improving the models' zero-shot capabilities in detecting mismatches on diagnostic tasks and performing robustly under noisy visual input scenarios. Also, we demonstrate that HNC models yield a comparable or better initialization for fine-tuning
Read the full abstract ▾
All authors Esra Dönmez, Pascal Tilli, Hsiu-Yu Yang, Thang Vu, Carina Silberer
Computer Vision · Cross-listed from eess.IV
Tumor-aware augmentation with task-guided attention analysis improves rectal cancer segmentation from magnetic resonance images
Aneesh Rangnekar, Joao Miranda, Natally Horvat et al. (+13 more)
Although self-supervised pretraining is expected to learn broadly transferable representations, its effectiveness across imaging modalities substantially different from the pretraining domain, and on complex tumor-
segmentation tasks, remains understudied. Evaluating CT-pretrained
transformers on MRI rectal cancer
segmentation, we identified two interacting failure modes in CT-to-MRI transfer: (a) inefficient
token usage caused by zero-padding to match pretrained input dimensions, and (b) ineffective feature adaptation. We investigated these vulnerabilities using two primary CT-pretrained hierarchical shifted-window
transformer backbones, SMIT and Swin UNETR, together with VoCo as a large-scale-pretrained supporting
benchmark; these models differ in pretraining objectives and
datasets. Mechanistic analysis leveraged an
attention dilution index (ADI), an
entropy-based metric quantifying
attention diverted toward uninformative padding
tokens, and centered kernel
alignment (CKA) to measure feature reuse during MRI adaptation. ADI increased with zero-padding, while high feature reuse did not necessarily translate to improved downstream accuracy. To mitigate these issues, we introduced two interventions: a tumor-aware augmentation strategy to expand tumor appearance heterogeneity coverage, and an anisotropic cropping strategy to restore
token efficiency. Fine-tuning with these strategies on identical rectal MRI
datasets yielded detection rates of 91.1% (225/247) and 88.7% (219/247) for the primary SMIT and Swin UNETR backbones, with the supporting VoCo
benchmark reaching 90.3% (223/247), demonstrating significantly improved robustness under CT-to-MRI transfer. This study is among the first to examine when pretrained
transformers fail to transfer across imaging modalities and demonstrates how targeted mitigation strategies can systematically overcome cross-modality transfer limitations.
Read the full abstract ▾
All authors Aneesh Rangnekar, Joao Miranda, Natally Horvat, Stephanie Chahwan, Samir Alrayess, Aditya Apte, Aditi Iyer, Eve LoCastro, Revathi Ravella, Marc J Gollub, Iva Petkovska, Jesse Joshua Smith, Paul Romesser, Julio Garcia-Aguilar, Harini Veeraraghavan, Joseph O Deasy
Computer Vision
The First Controllable Bokeh Rendering Challenge at NTIRE 2026
Tim Seizinger, Florin-Alexandru Vasluianu, Jeffrey Chen et al. (+23 more)
This study presents the outcomes of the first Controllable Bokeh Rendering Challenge at NTIRE and highlights the most effective submitted methodologies. In total, 44 participants registered for the competition, of which 8 teams submitted valid solutions after the conclusion of the final test phase. All submissions were evaluated on unseen images, focusing on portraits and intricate subjects with complex and visually appealing bokeh phenomena. In addition to the first track focusing on established quantitative fidelity metrics, we conducted a qualitative user study with a panel of experts for a second track focusing on perceptual assessment. As this was the inaugural challenge on this topic, most of the participants focused on refining and extending the Bokehlicious baseline method.
Read the full abstract ▾
All authors Tim Seizinger, Florin-Alexandru Vasluianu, Jeffrey Chen, Zhuyun Zhou, Zongwei Wu, Radu Timofte, Dafeng Zhang, Yipeng Lin, Qi Yan, Junhao Chen, Yang Yang, Divyavardhan Singh, Hariom Thacker, Hammad Mohammad, Aanchal Maurya, Kishor Upla, Kiran Raja, Wei Zhou, Hongyu Huang, Yujin Cho, Grigory Malivenko, Jiachen Tu, Yaokun Shi, Guoyi Xu, Yaoxin Jiang, Jiajia Liu
Computer Vision
EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function
Elias Stenhede, Joanna Sulkowska, Eivind Bjørkan Orstad et al. (+2 more)
We introduce EchoXFlow, a clinical echocardiography
dataset with ultrasound in its native acquisition geometry. Existing public
datasets offer limited opportunities to study how cardiac anatomy, myocardial motion, and blood flow relate: Doppler velocities are typically absent or fused as RGB overlays on grayscale anatomical images, and acquisitions are released only after scan conversion and lossy vendor display processing. EchoXFlow comprises 35,377 recordings from 666 routine-care examinations of 622 patients. It preserves the timing, geometry, and modality relationships needed for physically grounded echo learning. Each recording is retained as separable modality-specific streams: temporally resolved 1D, 2D, and 3D data alongside multiple Doppler modalities, paired with a synchronized ECG. Clinical annotations range from guideline-defined measurements to dense 2D myocardial contours and 3D left-ventricular endocardial meshes. With its associated open-source tooling, EchoXFlow enables cross-modal, acquisition-aware learning tasks that cannot be formulated from conventional scan-converted
videos alone, and serves as a testbed for 4D
vision and physically grounded multi-modal learning more broadly. The
dataset is available at https://huggingface.co/
datasets/Ahus-AIM/EchoXFlow and the tooling at https://github.com/Ahus-AIM/EchoXFlow
Read the full abstract ▾
All authors Elias Stenhede, Joanna Sulkowska, Eivind Bjørkan Orstad, Henrik Schirmer, Arian Ranjbar
Computer Vision
Safety-Critical Camera Reliability Monitoring for ADAS via Degradation-Aware Uncertainty Pattern Analysis
Shiva Aher
Reliable camera input is essential for safety-critical ADAS perception, but most monitoring approaches detect sensor failures only after downstream performance has degraded. We propose a proactive camera reliability monitoring framework that estimates perception risk from degradation-induced uncertainty patterns before downstream failure becomes observable. The method introduces a Global Sensor Health Index (GSHI), a continuous reliability score that aggregates per-degradation severities using a risk-aware multiplicative formulation, allowing severe single-mode failures such as lens occlusion or motion blur to dominate the health estimate. A lightweight multi-task network predicts degradation type, severity, GSHI, and spatial uncertainty maps from a single RGB image without downstream task feedback. Training uses physics- and geometry-aware synthetic supervision over twelve camera degradation modes. Experiments on KITTI-derived degradations show that GSHI decreases monotonically with severity, achieves a health-estimation MAE of 0.064, and provides positive early-warning lead time of 0.47 $\pm$ 0.25 severity units before YOLOv8 detection failure. GSHI also outperforms IQA, detector-confidence, and clean-feature OOD baselines, and transfers zero-shot to real adverse-weather driving data. These results support degradation-aware uncertainty analysis as a practical direction for proactive camera reliability monitoring in intelligent vehicles.
Read the full abstract ▾
Computer Vision
Zero-Shot Satellite Image Retrieval through Joint Embeddings: Application to Crisis Response
James Walsh, William Fawcett, Grace Colverd et al. (+1 more)
Semantic search of Earth observation archives remains challenging. Visual foundation models such as CLAY produce rich
embeddings of satellite imagery but lack the natural-
language grounding needed for intuitive query, and full contrastive training of a remote-sensing CLIP-style model requires paired data and compute that are unavailable at global scale. To allow natural
language querying at global scales, we present GeoQuery, a zero-shot
retrieval system that sidesteps data and compute constraints through a two-stage semantic and visual search, leveraging a natural
language embedding of a subset (proxy) of global data. Rather than training a joint encoder, we generate
language descriptions for a 100k proxy subset of global Sentinel-2 tiles and optimise the description-generation
prompt so that distances in the resulting text-
embedding space correlate with distances in the frozen CLAY visual-
embedding space. Queries are resolved in two stages, with a text-similarity search over the proxy subset followed by a visual nearest-neighbour search over worldwide CLAY
embeddings On 76 disaster-location queries covering UK floods, US wildfires, and US droughts, GeoQuery achieves 31.6% accuracy within 50\,km, with the strongest performance on floods (50% within 50\,km) where terrain features are well captured by RGB
embeddings. Deployed within a crisis response system called \ECHO{}, GeoQuery identified vulnerable areas during Brisbane's 2025 Cyclone Alfred, with downstream flood simulations reproducing historical patterns.
Prompt-aligned proxies offer a practical bridge between EO foundation models and operational
retrieval when full contrastive training is out of reach.
Read the full abstract ▾
All authors James Walsh, William Fawcett, Grace Colverd, Raúl Ramos-Pollán
Computer Vision · Cross-listed from cs.AI
Intelligent CCTV for Urban Design: AI-Based Analysis of Soft Infrastructure at Intersections
Vinit Katariya, Seungjin Kim, Curtis Craig et al. (+2 more)
Artificial intelligence (AI) and computer
vision are transforming transportation data collection. This study introduces an AI-enabled analytics framework leveraging existing CCTV infrastructure to evaluate the impact of soft interventions, such as temporary pedestrian refuges and curb extensions, on vehicle speed and safety. Using deep learning and perspective-based speed estimation, we evaluated driver behavior before and after interventions, with repeated post-installation monitoring in Week 1 and Week 2, in Minneapolis. Findings reveal that at unsignalized intersections, mean and 85th-percentile speeds fell by up to 18.75% and 16.56%, respectively, while pass-through traffic decreased by as much as 12.2%. Signalized intersections showed comparable reductions except one location, with mean and 85th-percentile speeds dropping by up to 20.0% and 17.19%. These results demonstrate the traffic-calming effectiveness of soft infrastructure and underscore the utility of AI-powered methods for rapid, low-cost, and evidence-based transport policy evaluation.
Read the full abstract ▾
All authors Vinit Katariya, Seungjin Kim, Curtis Craig, Nichole Morris, Hamed Tabkhi
Computer Vision
LAMP: Localization Aware Multi-camera People Tracking in Metric 3D World
Nan Yang, Julian Straub, Fan Zhang et al. (+3 more)
Tracking 3D human motion from
egocentric multi-camera headset is challenged by severe egomotion, partial visibility or occlusions and lack of training data. Existing methods designed for
monocular video often require static or slowly-moving cameras and cannot efficiently leverage multi-view, calibrated and localized input. This makes them brittle and prone to fail on dynamic
egocentric captures. We propose LAMP (Localization Aware Multi-camera People Tracking): a novel, simple framework to solve this via early disentanglement of observer and target motion. LAMP introduces a two-step process. First, we leverage the known device 6 DoF motion and calibration to convert detected 2D body keypoints from all cameras over a temporal window into a unified 3D world reference frame. Second, an end-to-end-trained spatio-temporal
transformer fits 3D human motion directly to this 3D ray cloud. This "lift-then-fit" approach allows LAMP to learn and leverage a natural human motion prior in the world-space, as well as providing an elegant framework to flexibly incorporate information from multiple temporally asynchronous, partially observing and moving cameras. LAMP achieves state-of-the-art results on
monocular benchmarks, while significantly outperforming baselines for our targeted
egocentric setting.
Read the full abstract ▾
All authors Nan Yang, Julian Straub, Fan Zhang, Richard Newcombe, Jakob Engel, Lingni Ma
Computer Vision
Two Steps Are All You Need: Efficient 3D Point Cloud Anomaly Detection with Consistency Models
Pranav A, Shashank B, Pranav Siddappa et al. (+3 more)
Diffusion models are rapidly redefining 3D anomaly detection in point cloud data. As 3D sensing becomes integral to modern manufacturing, reliable anomaly detection is essential for high-throughput quality assurance and process control. Yet practical deployment on resource-constrained, latency-critical systems remains limited. Existing methods are often computationally prohibitive or unreliable in complex, unmasked regions, and
diffusion pipelines are inherently bottlenecked by iterative denoising. In this work, we address this bottleneck by reformulating reconstructionbased anomaly detection through consistency learning, enabling direct prediction of anomaly-free geometry in one or two network evaluations. We further introduce a novel hybrid loss formulation that explicitly enforces reconstruction toward clean data. This design substantially reduces inference cost, achieving up to 80x faster runtime than the current state-of-the-art method, without GPU acceleration, while preserving strong detection performance. It outperforms R3D-AD on Anomaly-ShapeNet with 76.20% I-AUROC and remains competitive on Real3DAD with 72.80% I-AUROC, enabling efficient, low-latency anomaly detection on resource-constrained platforms, including drones, smart industrial cameras, and other edge devices.
Read the full abstract ▾
All authors Pranav A, Shashank B, Pranav Siddappa, Dominik Seuss, Minal Moharir, Subramanya KN
Computer Vision
Tamaththul3D: High-Fidelity 3D Saudi Sign Language Avatars from Monocular Video
Eyad Alghamdi, Sattam Altuuaim, Obay Ghulam et al. (+2 more)
Existing 3D sign
language avatar reconstruction methods are developed and evaluated exclusively on Western sign
languages, and no 3D parametric annotations exist for any Arabic Sign
Language dataset, a gap that blocks the development of avatar-based accessibility applications for the Arab Deaf community. We release the first SMPL-X parametric annotations for the Ishara-500 Saudi Sign
Language dataset, enabling quantitative evaluation and downstream sign
language generation for Arabic Sign
Language. We introduce Tamaththul3D, a reconstruction pipeline that aligns hand and body estimates through
geometric inverse kinematics on the forearm chain followed by 2D-supervised shoulder refinement. The closed-form integration is decoupled from the specific choice of body and hand
estimators: any SMPL-X-compatible body
estimator and any MANO-compatible hand
estimator can be substituted, as we demonstrate by swapping each module independently. Tamaththul3D achieves up to 32% lower hand error than prior methods, runs 32x faster than the strongest baseline, and generalizes across five typologically distinct sign
languages without
dataset-specific adaptation.
Read the full abstract ▾
All authors Eyad Alghamdi, Sattam Altuuaim, Obay Ghulam, Abdulrahman Qutah, Yousef Basoodan
Computer Vision · Cross-listed from cs.LG
Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
Alaa Zniber, Ouassim Karrakchou and Mounir Ghogho
Early-exiting neural networks enable adaptive inference by allowing inputs to exit at intermediate classifiers, reducing computation for easy samples while maintaining high accuracy. In practice, exits can be trained sequentially by incrementally adding them to a shared backbone; however, this sequential training can cause newly introduced exits to interfere with previously learned ones, degrading the performance of earlier classifiers. We address this problem by retaining the knowledge embedded in existing exits while allowing new ones to specialize. We propose two alternative approaches that operate at different levels of the model. The first constrains learning by protecting parameters that are important for previously trained exits, while the second preserves the output distributions of earlier exits as the network adapts. These alternatives directly reflect the stability-plasticity trade-off studied in continual learning. Accordingly, we leverage Elastic Weight Consolidation to constrain critical weights and Learning without Forgetting to preserve output distributions. Experiments on standard
benchmarks show that our approaches consistently improve early-exit performance, achieving higher accuracy over existing sequential training methods and significant performance speedups at low computational budgets.
Read the full abstract ▾
Computer Vision
egenioussBench: A New Dataset for Geospatial Visual Localisation
Phillipp Fanta-Jende, Francesco Vultaggio, Alexander Kern et al. (+2 more)
We present egenioussBench, a visual localisation
benchmark built on geospatial reference data: a city-scale airborne 3D mesh and a CityGML LoD2 model. This pairing reflects deployable mapping assets and supports true scalability beyond traditional SfM-based approaches. The query data comprise smartphone images with centimetre-accurate, map-independent ground truth obtained via PPK and GCP/CP-aided adjustment. From 2,709 images, we derive a non-co-visible subset by estimating the full co-visibility
matrix from rendered depth and selecting a maximum independent set; the released data include a test split of 42 non-co-visible images with withheld ground truth and a validation split of 412 sequential images with poses, e.g. for training of pose regressors and self-validation. The
benchmark features a public leaderboard evaluated with binning metrics at multiple pose-error thresholds alongside global statistics (median, RMSE, outlier ratio), ensuring fair, like-for-like comparison across mesh- and LoD2-based methods. Together, these design choices expose realistic cross-view and cross-domain challenges while providing a rigorous, scalable path for advancing large-scale visual localisation. We make the evaluation code and data availeable at https://github.com/fratopa/egenioussBench and https://www.egeniouss.eu/
Read the full abstract ▾
All authors Phillipp Fanta-Jende, Francesco Vultaggio, Alexander Kern, Yasmin Loeper, Markus Gerke
Computer Vision
Open-SAT: LLM-Guided Query Embedding Refinement for Open-Vocabulary Object Retrieval in Satellite Imagery
Md Adnan Arefeen, Biplob Debnath, Ravi K. Rajendran et al. (+2 more)
In satellite applications, user queries often take the form of open-ended natural
language, extending beyond a fixed set of predefined categories. This open-vocabulary nature poses significant challenges for retrieving relevant image tiles, as the
retrieval system must generalize to a wide range of unseen objects and concepts. While
vision-
language models (VLMs) such as CLIP are widely used for text-image
retrieval, even fine-tuned variants often struggle to accurately align such queries with satellite imagery. To address this, we propose Open-SAT, a training-free query
embedding refinement
algorithm that operates at inference time to improve
alignment between user queries and satellite image content. Open-SAT uses VLMs to compute
embeddings for image tiles, which are stored in a vector database for efficient
retrieval. At query time, it leverages Large
Language Models (LLMs) to refine the text
embeddings by incorporating contextual information about objects of interest and their surroundings. A threshold-free
retrieval mechanism further enhances accuracy and efficiency. Experimental results in three public
benchmarks demonstrate that Open-SAT improves the F1 score by up to 16.04%, while retrieving a comparable number of image tiles. These results demonstrate the effectiveness of Open-SAT in open-vocabulary satellite image
retrieval, leveraging LLM guidance without the need for additional training or supervision.
Read the full abstract ▾
All authors Md Adnan Arefeen, Biplob Debnath, Ravi K. Rajendran, Murugan Sankaradas, Srimat T. Chakradhar
Computer Vision
ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parameters
Philippe Hansen-Estruch, Jiahui Chen, Vivek Ramanujan et al. (+9 more)
Vision Transformer (ViT) autoencoders have emerged as compelling tokenizers for images, offering improved reconstruction over convolutional tokenizers. However, existing ViT tokenizers cannot explore this landscape as performance degrades outside training resolutions, and reliance on
adversarial losses prevents stable scaling. ViTok (Hansen-Estruch et al., 2025) found that the compression ratio r mediates a reconstruction-generation trade-off where lower r means better reconstructions but harder generations, so improving tokenizer reconstruction is key to more Pareto-optimal tokenizers. We introduce ViTok-v2, which addresses these limitations with native resolution support via NaFlex for generalization across resolutions and aspect ratios, and a novel DINOv3 perceptual loss that replaces both LPIPS and GAN objectives for stable training at any scale. ViTok-v2 is trained on about 2B images and scaled to 5B parameters, the largest image autoencoder to date. ViTok-v2 matches or exceeds state-of-the-art reconstruction at 256p and outperforms all baselines at 512p and above. In joint scaling experiments with flow matching generators, we show that scaling both the autoencoder and the generator advances the Pareto frontier of this trade-off.
Read the full abstract ▾
All authors Philippe Hansen-Estruch, Jiahui Chen, Vivek Ramanujan, Orr Zohar, Yan Ping, Animesh Sinha, Markos Georgopoulos, Edgar Schoenfeld, Ji Hou, Felix Juefei-Xu, Sriram Vishwanath, Ali Thabet
Computer Vision
Query2Uncertainty: Robust Uncertainty Quantification and Calibration for 3D Object Detection under Distribution Shift
Till Beemelmanns, Alexey Nekrasov, Stefan Vilceanu et al. (+4 more)
Reliable uncertainty estimation for 3D object detection is critical for deploying safe autonomous systems, yet modern detectors remain poorly calibrated, especially under distribution shifts. Although post-hoc calibration methods address this issue and provide improved calibration for in-distribution tests, they fail to adapt in distribution-shifted scenarios. In this work, we address this issue and introduce a density-aware calibration method that couples post-hoc calibrators with the feature density of latent object queries from DETR-style 3D object detectors. These queries form a compact, location and class-aware feature, ideal for density estimation, allowing our approach to adjust model confidences in distribution-shift scenarios. By fitting a density
estimator on these query features, our approach jointly recalibrates both classification and bounding box regression uncertainties. On both a multi-view camera and
LiDAR-based detector, our approach consistently outperforms standard post-hoc methods in both in-distribution and distribution-shifted scenarios. Code available https://tillbeemelmanns.github.io/query2uncertainty/ .
Read the full abstract ▾
All authors Till Beemelmanns, Alexey Nekrasov, Stefan Vilceanu, Jonas Steinhaus, Timo Woopen, Bastian Leibe, Lutz Eckstein
Computer Vision
Seeing What Shouldn't Be There: Counterfactual GANs for Medical Image Attribution
Shakeeb Murtaza
Ascription of an image gives insights into the objects that influence the classification of the whole image or its pixels towards a specific category. These insights help radiologists to visualize deformities in medical imaging. Most of the existing visualization techniques are based on discriminative models and highlight regions of the input image participating in the decision-making of a classifier. However, these approaches do not take all noticeable objects into account as their objective is to classify the input by using a minimal set of discriminative features. To overcome the issue, a counterfactual explanation (CX) based class-oriented feature attribution method is proposed. A counterfactual explanation (CX) explicates a causal
reasoning process of the form: "if X had not happened, then Y would not have happened". The method is built on generative
adversarial networks (GANs) with a cyclical-consistent loss function. We evaluate our method on three
datasets: synthetic, tuberculosis and BraTS. All experiments confirm the efficacy of the proposed method. This study also highlighted the limitations of existing counterfactual explanation techniques in producing plausible counterfactual instances (CIs). Accompanying CXs with believable CIs thus provides self-explanatory analogy-based explanations. To this end, a CI generation method is proposed. Also, a novel technique is used to evaluate the quality of CI. The baseline results are produced on the BraTS
dataset.
Read the full abstract ▾
Computer Vision
Syn4D: A Multiview Synthetic 4D Dataset
Zeren Jiang, Yushi Lan, Yihang Luo et al. (+8 more)
Dense 3D reconstruction and tracking of dynamic scenes from
monocular video remains an important open challenge in computer
vision. Progress in this area has been constrained by the scarcity of high-quality
datasets with dense, complete, and accurate
geometric annotations. To address this limitation, we introduce Syn4D, a multiview synthetic
dataset of dynamic scenes that includes ground-truth camera motion, depth maps, dense tracking, and parametric human pose annotations. A key feature of Syn4D is the ability to unproject any pixel into 3D to any time and to any camera. We conduct extensive evaluations across multiple downstream tasks to demonstrate the utility and effectiveness of the proposed
dataset, including 4D scene reconstruction, 3D point tracking, geometry-aware camera retargeting, and human pose estimation. The experimental results highlight Syn4D's potential to facilitate research in dynamic scene understanding and spatiotemporal modeling.
Read the full abstract ▾
All authors Zeren Jiang, Yushi Lan, Yihang Luo, Yufan Deng, Zihang Lai, Edgar Sucar, Christian Rupprecht, Iro Laina, Diane Larlus, Chuanxia Zheng, Andrea Vedaldi
Computer Vision
Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu et al. (+3 more)
We study outlier
tokens in
Diffusion Transformers (DiTs) for image generation. Prior work has shown that
Vision Transformers (ViTs) can produce a small number of high-norm
tokens that attract disproportionate
attention while carrying limited local information, but their role in generative models remains underexplored. We show that this phenomenon appears in both the encoder and denoiser of modern Representation Autoencoder (RAE)-DiT pipelines: pretrained ViT encoders can produce outlier representations, and DiTs themselves can develop internal outlier
tokens, especially in intermediate layers. Moreover, simply masking high-norm
tokens does not improve performance, indicating that the problem is not only caused by a few extreme values, but is more closely related to corrupted local patch semantics. To address this issue, we introduce Dual-Stage Registers (DSR), a register-based intervention for both components: trained registers when available, recursive test-time registers otherwise, and
diffusion registers for the denoiser. Across ImageNet and large-scale text-to-image generation, these interventions consistently reduce outlier artifacts and improve generation quality. Our results highlight outlier-
token control as an important ingredient in building stronger DiTs.
Read the full abstract ▾
All authors Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu, Liang-Chieh Chen, Zhe Gan, Chen Wei
Computer Vision
D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models
Dengyang Jiang, Xin Jin, Dongyang Liu et al. (+9 more)
The landscape of high-performance image generation models is currently shifting from the inefficient multi-step ones to the efficient few-step counterparts (e.g, Z-Image-Turbo and FLUX.2-klein). However, these models present significant challenges for direct continuous supervised fine-tuning. For example, applying the commonly used fine-tuning technique would compromise their inherent few-step inference capability. To address this, we propose D-OPSD, a novel training paradigm for step-distilled
diffusion models that enables on-policy learning during supervised fine-tuning. We first find that the modern
diffusion models, where the LLM/VLM serves as the encoder, can inherit its encoder's in-context capabilities. This enables us to formulate the training as an on-policy self-
distillation process. Specifically, during training, we make the model act as both the teacher and the student with different contexts, where the student is conditioned only on the text feature, while the teacher is conditioned on the
multimodal feature of both the text
prompt and the target image. Training minimizes the two predicted distributions over the student's own roll-outs. By optimizing on the model's own trajectory and under its own supervision, D-OPSD enables the model to learn new concepts, styles, etc., without sacrificing the original few-step capacity.
Read the full abstract ▾
All authors Dengyang Jiang, Xin Jin, Dongyang Liu, Zanyi Wang, Mingzhe Zheng, Ruoyi Du, Xiangpeng Yang, Qilong Wu, Zhen Li, Peng Gao, Harry Yang, Steven Hoi
Computer Vision
LoViF 2026 The First Challenge on Holistic Quality Assessment for 4D World Model (PhyScore)
Wei Luo, Yiting Lu, Xin Li et al. (+32 more)
This paper reports on the LoViF 2026 PhyScore challenge, a competition on holistic quality assessment of world-model-generated
videos across both 2D and 4D generation settings. The challenge is motivated by a central gap in current evaluation practice: perceptual quality alone is insufficient to judge whether generated dynamics are physically plausible, temporally coherent, and consistent with input conditions. Participants are required to build a metric that jointly predicts four dimensions, i.e.,
Video Quality, Physical Realism, Condition-
Video Alignment, and Temporal Consistency. Depart from that, participants also need to localize physical anomaly timestamps for fine-grained diagnosis. The
benchmark dataset contains 1,554
videos generated by seven representative world generative models, organized into three tracks (text-2D, image-to-4D, and
video-to-4D) and spanning 26 categories. These categories explicitly cover physics-relevant scenarios, including dynamics, optics, and thermodynamics, together with diverse real-world and creative content. To ensure label reliability, scores and anomaly timestamps are produced through trained human annotation with an additional automated quality-control pass. Evaluation is based on both score prediction and anomaly localization, with a composite protocol that combines TimeStamp_IOU and SRCC/PLCC. This report summarizes the challenge design and provides method-level insights from submitted solutions.
Read the full abstract ▾
All authors Wei Luo, Yiting Lu, Xin Li, Haoran Li, Fengbin Guan, Chen Gao, Xin Jin, Yong Li, Zhibo Chen, Sijing Wu, Kang Fu, Yunhao Li, Ziang Xiao, Huiyu Duan, Jing Liu, Qiang Hu, Xiongkuo Min, Guangtao Zhai, Manxi Sun, Zixuan Guo, Yun Li, Ziyang Chen, Manabu Tsukada, Zhengyang Li, Zhenglin Du, Yi Wen, Licheng Jiao, Fang Liu, Lingling Li, Yiwen Ren, Zhilong Song, Dubing Chen, Yucheng Zhou, Tianyi Yan, Huan Zheng