Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Wednesday 23 September

Image and Video Processing · Cross-listed from cs.LG

Geospatial embeddings detect old-growth forests but buffered spatial validation narrows their advantage over Sentinel features

Thomas Ratsakatika, Mihai Zotta, Srinivasan Keshav et al. (+1 more)

Old-growth forests develop over centuries under minimal anthropogenic disturbance, producing structurally complex and biodiverse stands. In Europe, protecting them requires mapping that is accurate for individual forest parcels yet deployable continent-wide. Geospatial foundation model (GFM) embeddings enable label-scarce land classification, but their value for old-growth detection remains unknown. Here, we map old-growth forests across 211,893 ha of Romania's Southern Carpathians, a beech-spruce landscape typical of the Alpine Biogeographic Region. We construct high-confidence, expert-informed reference labels for old-growth and non-old-growth parcels. We add AlphaEarth, TESSERA v2 and Sentinel-1/2 features to a common baseline of topographic and human-access predictors, then compare them under spatially blocked validation with and without 10 km train-test buffers to limit residual autocorrelation. With buffering, GFM and Sentinel-1/2 predictors increase precision-recall AUC by 0.21-0.25 [95% CIs: 0.15-0.34] relative to baseline, indicating spectral data contain a spatially robust old-growth signal. With a PR-AUC of 0.84 [0.79-0.88], TESSERA outperforms Sentinel-1/2 (+0.08 [+0.05 to +0.11]) and AlphaEarth (+0.08 [+0.04 to +0.12]) under unbuffered spatial validation. At a 10 km buffer, however, this advantage narrows to +0.04 [-0.01 to +0.11] and +0.03 [-0.04 to +0.10], intervals consistent with no difference. At 10 m resolution, convolutional neural networks add no benefit over pixel-based XGBoost. Comparisons with four national- and continental-scale products show the importance of non-old-growth labels, and reveal 81% agreement between our predictions and a field-calibrated map. We conclude that buffered spatial validation is vital when transferring old-growth detection models to unseen landscapes, and provide our labels and predictions for future work.
Read the full abstract

All authors Thomas Ratsakatika, Mihai Zotta, Srinivasan Keshav, Emily R. Lines

2609.28194 · 23 Sep · 34 pages, including supplementary material (19-page main article with 7 figures and 3 tables; 15-page supplement with 9 figures and 21 tables). Submitted for publication. Data: https://doi.org/10.5281/zenodo.22693148 (embargoed until publication); code: https://github.com/ratsakatika/detecting-old-growth-forests

Abstract PDF $EMBD ×2

Image and Video Processing · Cross-listed from cs.CV

A comparative assessment of global building and settlement datasets across geographic and settlement contexts

Rufai Omowunmi Balogun, Caroline Margaux Gevaert, Capucine Riom et al. (+5 more)

Global building and settlement products increasingly support population mapping, exposure assessment, and urban monitoring, yet comparative evidence remains fragmented across representations, geographic contexts, and evaluation methods. We benchmark seven global or near-global products against harmonized reference building footprints across 135 study areas. Our representation-aware framework integrates complementary measures of detection, geometric agreement, and aggregate quantity accuracy with stratified analyses of settlement characteristics, evaluation resolution, reference provenance, and temporal alignment. Overture Maps achieved the highest median city-level vector F1 (0.786), while 3D-GloBFP showed the closest aggregate building-area agreement despite weaker detection performance. Raster rankings were scale-dependent: Google Open Buildings Temporal (OBT) performed best at 10\,m (F1 = 0.680), whereas WSF Tracker led at 100\,m (F1 = 0.904) but substantially overestimated reference building area (relative bias = 1.845). Raster accuracy increased with building density ($\rho=0.58$–$0.75$), while smaller candidate footprints were disproportionately associated with vector false positives. Temporal alignment improved WSF Tracker mean city-level F1 by 0.060. These findings demonstrate that product reliability depends on representation, spatial scale, and settlement context, providing a reproducible framework for assessing the suitability of EO-derived urban information for different applications.
Read the full abstract

All authors Rufai Omowunmi Balogun, Caroline Margaux Gevaert, Capucine Riom, Derrick Mirindi, Aaron Opdyke, Hamed Alemohammad, Pierre Chrzanowski, Edward Charles Anderson

Image and Video Processing

Recursive Uncertainty-Gated Image Registration for Learning-based Algorithms

Clara Rodrigo González, Oscar Bates, Fu Siong Ng et al. (+1 more)

Conventional image registration algorithms are robust to domain shifts and achieve low errors, but they are slow and computationally expensive. Deep-learning methods are efficient at inference-time, but face challenges in out-of-domain samples. We propose Recursive Uncertainty-Gated Image Registration (RUGI), an algorithm for iteratively refining deformation fields predicted by learning-based registration models. At each iteration, the registration model predicts an incremental deformation, and a gating map modulates the update. Refinements are hence concentrated in regions that remain difficult to register. We explore two gating strategies: a learned uncertainty-based approach and an image residual error approach. We evaluate RUGI on cardiac MRI and echocardiography datasets and show consistent improvements over single-step inference. Ablation experiments demonstrate that iterative refinement alone improves registration, but informative spatial gating provides a significant additional benefit. The error-gated variant of RUGI can also be applied directly to existing pretrained models; applied to VoxelMorph, TransMorph, and CycleMorph, it yields MSE reductions of 27-37% with no modification to the original training procedure. The improvements in registration performance are reflected in decreased errors in ejection fraction estimation relative to ground truths. These results demonstrate that spatially selective iterative refinement provides an effective strategy to improve registration accuracy at inference-time.
Read the full abstract

All authors Clara Rodrigo González, Oscar Bates, Fu Siong Ng, Meng-Xing Tang

Image and Video Processing · Cross-listed from cs.CV

SatUnreal: A High-Precision Synthetic Dataset for Satellite Stereo Matching via Unreal Engine

Han-Gyeol Kim, JaeWan Park, Junmin Park et al. (+1 more)

3D reconstruction from satellite imagery is essential for large-scale topographic analysis, yet the lack of high-fidelity training datasets with accurate occlusion labels remains a primary bottleneck. Existing benchmarks, such as US3D and WHU-Stereo, face inherent challenges in spatio-temporal mismatch – environmental changes and shadow displacements between multi-view acquisitions – and provide ambiguous ground truth in occluded regions due to LiDAR sparsity. In this paper, we propose SatUnreal, a high-precision synthetic dataset designed to fundamentally overcome these limitations through an Unreal Engine-based simulation pipeline. SatUnreal provides 10,000 stereo pairs with high resolution (0.3m GSD) and is characterized by: (1) Physical Geometry Simulation, replicating realistic satellite orbits by systematically varying baselines and azimuths; (2) Spatio-temporal Consistency, eliminating temporal noise through fixed virtual environments; (3) Topographic Diversity, spanning dense urban canyons to low-texture natural terrains; and (4) Mathematical Label Integrity, utilizing a novel two-step linetrace algorithm to generate flawless occlusion masks. Experimental results using SOTA iterative models demonstrate that models trained exclusively on SatUnreal achieve superior zero-shot transfer performance on real-world benchmarks (US3D, WHU-Stereo) compared to those trained on real datasets. Our findings prove that physically accurate synthetic data provides a more effective supervisory signal for learning geometric features than complex real-world observations, establishing a new paradigm for Sim-to-Real transfer in Earth Observation. Code and dataset are available at https://github.com/jmp-Telepix/SatUnreal_A_High-Precision_Synthetic_Dataset_for_Satellite_Stereo_Matching_via_UnrealEngine
Read the full abstract

All authors Han-Gyeol Kim, JaeWan Park, Junmin Park, Darongsae Kwon

2609.27442 · 23 Sep · Accepted at CVPR 2026 Workshop on EarthVision (CVPRW 2026), pp. 7990-7999. Code and dataset: https://github.com/jmp-Telepix/SatUnreal_A_High-Precision_Synthetic_Dataset_for_Satellite_Stereo_Matching_via_UnrealEngine Supplementary material: https://openaccess.thecvf.com/content/CVPR2026W/EarthVision/supplemental/Kim_SatUnreal_A_High-Precision_CVPRW_2026_supplemental.pdf

Abstract PDF $DATA ×6$BNCH ×2$ALGO ×1$GEOM ×1 +1

Tuesday 22 September

Image and Video Processing · Cross-listed from physics.ao-ph

Deep Learning Denoising of Real SWOT Sea Surface Height Observations

Gaétan Meis, Anaëlle Tréboutte, Maxime Ballarotta et al. (+2 more)

The SWOT (Surface Water Ocean Topography) mission is currently providing unpreceded high-resolution measurements of Sea Surface Height (SSH), revealing ocean features at finer scales. Nevertheless, the two-dimensional observations of KaRIn altimeter of SWOT suffer from instrumental errors. This noise degradation is altering the high frequencies of SWOT signal, and the small to sub-mesoscale dynamics of interest for some oceanographers. For this reason, Tréboutte et al. (2023) have developed a convolutional neural network (CNN) based on U-Net architecture to separate the noise from the physical signals contained in the SSH. Their approach has demonstrated great potential on simulated SWOT measurements, and Dibarboure et al. (2024) report a positive influence on actual flight data from SWOT. However, degraded denoising performance, and occasional negative side-effects have been observed in atypical conditions (e.g. very high surface waves, internal tides solitons). In this study, we illustrate some of these limitations and we present an improved approach of the CNN-based denoising: we modified the training procedure to obtain a more robust version of the algorithm, to avoid biases and artifacts in the denoised SSH. This paper also presents a more complete validation process with a robust and standardized evaluation benchmark: these metrics could be of interest to assess other SWOT filtering and denoising algorithms.
Read the full abstract

All authors Gaétan Meis, Anaëlle Tréboutte, Maxime Ballarotta, Marie-Isabelle Pujol, Gérald Dibarboure

Image and Video Processing

Adaptive Tiling for Least-Squares Phase Unwrapping: Runtime and Accuracy

Antoine Moevus and Max Mignotte

Phase unwrapping estimates the missing multiples of $2\pi$ in measured phase images. For large images, tiling limits the size of local reconstruction problems and enables parallel processing. Adaptive tiling could further reduce the number of local problems and boundaries by retaining large tiles where little refinement is needed. We investigate whether this reduction makes reconstruction faster. We compare complete reconstruction time and accuracy for a regular grid, quadtree, and kd-tree partitions. We also evaluate nine criteria for deciding where quadtree tiles should be subdivided, including residue count, fringe density, and measures of phase variation, at different tile sizes and budgets. In single-threaded experiments on a heterogeneous image dataset, optimized adaptive partitions use fewer tiles but remain slower than the optimized grid, and some reconstructions lose substantial accuracy. Stage measurements explain why: constructing the partition and solving larger retained tiles outweigh the savings at tile boundaries. The criterion comparison also shows that more refinement does not consistently improve accuracy. These results motivate evaluating adaptive partitions by the complete time needed to reach a chosen reconstruction accuracy, including whether limited refinement can provide a faster approximate result.
Read the full abstract

Image and Video Processing

BRiDCT: Fast Two-Dimensional DCTs Using SIMD: SIMD Organization, Register Blocking, and Numerical Verification

Antoine Moevus and Max Mignotte

Repeated two-dimensional discrete cosine transforms (DCTs) require implementations that remain fast across several array sizes while preserving the mathematical transform. Fewer arithmetic operations do not guarantee a faster DCT: execution time also depends on how the computation uses the CPU's single-instruction, multiple-data (SIMD) units, registers and memory hierarchy. We implement established DCT algorithms in SIMD and compare execution strategies for forward, inverse, and normalized round-trip transforms on float32 arrays from $8\times8$ to $256\times256$. The resulting library, BRiDCT, combines the Shao–Johnson factorization with SIMD lane organization, banded intermediate storage, padding, and register blocking in C11/ARM NEON. On one Apple M3 Max, the final native library has lower execution times than the tested general-purpose routes—Apple's Accelerate framework (vDSP), FFTW and Ooura—across their available cases from $8\times8$ to $256\times256$. It also outperforms the tested libjxl and floating-point libjpeg-turbo routes. The compiled NumPy interface outperforms OpenCV in all 130 tested contracts per session, including squares up to $1024\times1024$ and two rectangles, with call and allocation costs included. Complementary comparisons evaluate SciPy, DUCC and cached/planned pyFFTW. The measured gains come from organizing SIMD execution and intermediate storage, rather than introducing a new arithmetic factorization. A reproducible 474-input verification corpus probes cancellation, neighboring floats, dynamic range, and extreme amplitudes: all 1 260 core checks of the final configuration pass, while boundary diagnostics reveal overflow and weak-component limitations. Larger native transforms show size-dependent limits. Code, versioned releases, dataset and recorded measurements: https://github.com/antmoev/bridct.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

nnFoundation: 3D Foundation Models for Radiology

Constantin Ulrich Harsy, Tassilo Wald, Karol Gotkowski et al. (+80 more)

Radiological artificial intelligence has advanced rapidly, yet most systems remain narrowly task-specific, data-intensive, and fragile under domain shift. Foundation models promise more transferable and data-efficient solutions, but existing approaches are limited in scale, evaluated narrowly, and often assume that a single pretrained model can support diverse downstream tasks. Here we present nnFoundation, complementary convolutional and transformer-based 3D radiological foundation models. Developed within the Human Radiome Project (THRP), nnFoundation is trained on 2.1 million CT, MRI, and PET image volumes from 125 institutional and public datasets. We evaluate them across 108 tasks spanning segmentation, detection, classification, report generation, and image retrieval, including evaluations under domain shift, by external partners and in low-data and low-compute regimes. Across all task types, our convolution- and transformer-based nnFoundation models consistently outperform both prior 3D foundation models and training from scratch, establishing state-of-the-art performance for radiological imaging. However, performance follows a consistent task-dependent structure: the convolutional nnFoundation model dominates spatially localized tasks, whereas the transformer-based nnFoundation model excels in tasks requiring global semantic reasoning and in frozen-feature settings. Dynamically aligning the foundation model topology with the dataset characteristics post-hoc further improves transfer across heterogeneous 3D settings. These results show that transferable 3D radiological performance is governed not by a single universal model, but by the interplay of scalable pretraining, complementary architectures, and dataset-aware adaptation. We release nnFoundation models integrated into nnU-Net and nnDetection, enabling immediate application across established radiology workflows.
Read the full abstract

All authors Constantin Ulrich Harsy, Tassilo Wald, Karol Gotkowski, Yannick Kirchhoff, Marcel Knopp, Maximilian Rokuss, Elisa Stegmeier, Philipp Schader, Dasha Trofimova, Raphael Stock, Kim-Celine Kahl, Stephen Schaumann, Selen Erkan, David Zimmerer, Stefan Denner, Moritz Langenberg, Sebastian Ziegler, Katharina Eckstein, Maximilian Fischer, Jonathan Suprijadi, Bálint Kovács, Benjamin Hamm, Anand Deshpande, Dimitrios Bounias, Nico Disch, Shuhan Xiao, Jessica Kächele, Jan Sellner, Rajesh Baidya, Jeremias Traub, Lars Krämer, Maximilian Zenk, Tim Rädsch, Stefan Dvoretskii, Robin Peretzke, Jonathan Deissler, Alexandra Ertl, Partha Ghosh, Kris Dreher, Stefan Dinkelacker, Annika Reinke, Evangelia Christodoulou, Numan Saeed, Yoland Savriama, Santiago Estrada, David Kügler, Laura Alexandra Daza Barragan, Cristina Isabel Gonzalez Osorio, Jan Peeken, Michael Baumgartner, Marvin Teichmann, Guillaume Chabin, Matthias Kirchler, Valentin Koch, for the ALFA study, Markus Hohenhaus, Dimitri Koslov, Nina Decker, Mohammad Yaqub, Arnd Heuser, Martin Reuter, Julia A. Schnabel, Tobias Heimann, Florin Ghesu, Paul Brachmann, Claus P. Heußel, Alexander Radbruch, Gianluca Brugnara, Aditya Rastogi, Martha Foltyn-Dumitru, Heinz-Peter Schlemmer, Ignaz Reicht, Julius C. Holzschuh, Michael Bach, Bram Stieltjes, Kai Schlamp, Lena Maier-Hein, Marco Nolden, Ralf Floca, Paul F. Jäger, Philipp Vollmuth, Fabian Isensee, Klaus H. Maier-Hein

Image and Video Processing · Cross-listed from cs.MM

ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion

Pu Wang and Hugo Van hamme

Active speaker detection (ASD) requires reliable association between visible faces and acoustic speech, yet existing systems often degrade under challenging domains or incomplete observations. We introduce ROAM-ASD, a robust audiovisual framework that jointly models audio, full-face, and fine-grained mouth representations. A unified joint self-attention mechanism processes all input streams together with modality-agnostic query tokens, enabling direct interaction among available modality inputs. Modality dropout further improves robustness when input streams are unavailable. ROAM-ASD achieves state-of-the-art performance across five ASD benchmarks: 98.8% mAP on WASD, 87.9% on UniTalk, 96.5% on AVA, 99.3% on ASW, and 98.2% on Talkies, improving over previous best systems by 5.1, 4.7, 0.9, 1.0, and 2.1 mAP points, respectively. ROAM-ASD also substantially improves zero-shot cross-dataset generalization and remains robust to missing observations.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

Laryngeal Structure Segmentation in High-Speed Videoendoscopy Using Deep Learning

Sardar Nafis Bin Ali, Mohsen Zayernouri, Dimitar D. Deliyski et al. (+1 more)

Laryngeal high-speed videoendoscopy (HSV) offers an effective means of observing the motion of different laryngeal structures along with vibratory behaviors of the vocal folds under various voicing conditions. Segmentation of laryngeal tissues enables analysis of different tissue structures and their dynamics, helping characterize the involvement of laryngeal muscles in voice production. Given the large number of HSV frames, automating this task is imperative. While deep learning-based methods have been implemented in previous studies to segment laryngeal structures, they have not been applied to HSV data during connected speech, which poses significant challenges due to excessive tissue movements and image quality limitations associated with fiberoptic image acquisition. The application of deep learning to connected speech data is critical for capturing nonstationary laryngeal behaviors and identifying anomalous patterns associated with voice disorders. The present study aims to address these gaps by training U-Net models to detect the aryepiglottic folds and arytenoid cartilages, vocal folds, epiglottis, and glottal area, using HSV data from both sustained vowel phonation and connected speech obtained from normophonic and disordered voices. Image pre-processing techniques, including noise removal and histogram equalization, were applied to improve the quality of the training HSV images and enhance network performance. Finally, to evaluate the accuracy and reliability of the networks, quantitative performance metrics were used alongside qualitative visual inspection of the test images. The high performance of the developed networks, with overall accuracies exceeding 95%, establishes their potential as reliable tools for automated laryngeal image analysis, quantitative characterization of laryngeal dynamics, and future detection of anomalous laryngeal behaviors in clinical settings.
Read the full abstract

All authors Sardar Nafis Bin Ali, Mohsen Zayernouri, Dimitar D. Deliyski, Maryam Naghibolhosseini

Image and Video Processing · Cross-listed from cs.CV

PP-Net: A Hybrid Physical-Prior Neural Network for Scattered Light Removal in Biomedical Images on Embedded Devices

Yongfei Guo, Tingjin Chu, Mengzhuo Liu et al. (+2 more)

Scattered light is common in biomedical images, yet its removal remains challenging. The difficulty arises from three aspects: first, aligned scattered-light-free biomedical ground truth is often unavailable; second, scattering is coupled with weak illumination and sensor-induced noise; and third, many learning-based restoration models are computationally expensive for embedded devices in Internet of Medical Things (IoMT) scenarios. To address these issues, this paper proposes PP-Net, a hybrid physical-prior neural network for biomedical scattered light removal. The proposed method consists of three components: DFN-Net suppresses sensor-induced noise, ASAP estimates the scattering map and recovers a physics-based prior map, and GF-Net refines the prior map by fusing it with the denoised observation. To reduce the dependence on paired biomedical ground truth, a progressive synthetic training and cross-domain transfer strategy is developed. Experiments show that the physical-prior branch improves the peak signal-to-noise ratio (PSNR) by up to 1.26 dB on paired synthetic benchmarks. Under joint noise-and-scattering degradation, PP-Net improves PSNR by more than 10.8 dB and the structural similarity index measure (SSIM) by more than 0.62 compared with representative baseline methods. On real W2S biomedical images, the proposed method reduces the average Natural Image Quality Evaluator (NIQE) score by 43.3%. Edge deployment with RKNN conversion and INT8 quantization achieves an average inference latency of approximately 200 ms per $512\times512$ image over 360 test images. These results demonstrate that PP-Net provides an effective and deployable solution for microscopic imaging, endoscopic inspection, and edge-assisted biomedical analysis in IoMT scenarios.
Read the full abstract

All authors Yongfei Guo, Tingjin Chu, Mengzhuo Liu, Hongwei Lou, Yuanhao Gong

Image and Video Processing · Cross-listed from cs.CV

Mammo-LIFE: Longitudinal Mammographic Imaging and Clinical Feature Enrichment for Post-Radiotherapy Outcome Prediction

Farnoush Bayatmakou, Maryam Hosseini, Reza Taleei et al. (+1 more)

Recent advances in Artificial Intelligence (AI)-powered Computer-Aided Diagnosis (CAD) systems have substantially improved breast cancer screening, diagnosis, and prognosis. Comparatively, postradiotherapy outcome prediction using paired longitudinal mammograms has received considerably less attention. This is largely due to the limited availability of well-annotated longitudinal datasets. Longitudinal mammograms, coupled with paired pre- and post-treatment information, provide a unique opportunity to characterize treatment-induced breast tissue changes following radiotherapy. The resulting learned representations can serve as a valuable asset for advancing personalized radiotherapy planning and post-treatment management. In this context, we propose Mammo-LIFE, a patient-level multimodal framework for post-radiotherapy outcome prediction that combines longitudinal mammographic features with patient-level clinical variables. The imaging branch processes paired pre- and post-treatment mammograms acquired from the four standard views using a mammography-specific encoder adapted via Low-Rank Adaptation (LoRA). Within each view, preand post-treatment representations are explicitly compared through a longitudinal comparison module to capture treatment-related changes. The resulting view-level embeddings are then aggregated using learned view-attention pooling to form a unified patient-level mammographic representation. Selected clinical variables are subsequently combined with the image-derived prediction probability through a late-fusion strategy. To evaluate the effectiveness of combining paired longitudinal mammograms with clinical information, experiments were conducted on an in-house clinical cohort using patient-level stratified five-fold cross-validation.
Read the full abstract

All authors Farnoush Bayatmakou, Maryam Hosseini, Reza Taleei, Arash Mohammadi

Image and Video Processing · Cross-listed from physics.app-ph

Evaluation of effective wave velocities in polycrystalline materials using the ultrasonic reflection matrix

Gatien Clément, Alexandre Aubry, Cécile Brütt et al. (+2 more)

In-depth characterization of heterogeneous materials has long been a challenge in non-destructive testing. Here, a method is proposed to determine the elastic constants of metallic polycrystalline materials using back-scattered ultrasound. The waves scattered by the microstructure are analyzed to image the effective bulk velocities. To this end, a reflection matrix is acquired with an array of transducers. The projection of this matrix onto a focused basis is used to estimate an average point spread function. Optimizing this function with respect to the propagation model leads to an estimation of the longitudinal velocity. Additional treatments are developed to adapt the method to map the shear wave velocity. The local Poisson's ratio is then deduced from the ratio between those two velocities. Young's modulus and shear modulus can also be obtained assuming known densities. This matrix approach is experimentally validated on different polycrystalline materials. A sample displaying heterogeneous mechanical properties is then simulated to assess the accuracy and the resolution of the method. Its strengths and limitations are discussed, demonstrating its potential for quantitative non-destructive material characterization.
Read the full abstract

All authors Gatien Clément, Alexandre Aubry, Cécile Brütt, Benoît Gérardin, Claire Prada

Image and Video Processing

A Reference-Based Protocol for Assessing Image Displacement and Scale Stability

Fatih Ozturk, Emir Mercanoglu, Irem Sayin et al. (+1 more)

We present a reference-based protocol for assessing image stability, illustrated with a passive lens support. Pre/post-event wall references separate image-centre displacement from scale change. All 70 trials were retained: 30 baseline and 40 supported, acquired sequentially. Displacement (p = 0.597) and scale spread (p = 0.356) showed no significant difference. Lower median absolute scale deviation in the supported run persisted with scene-only processing, but its cause remains unresolved. Marker-estimation eligibility differed completely between runs for an unknown reason. The contribution is a workflow that reports completerun variability, typical deviations and estimator coverage together; the case study does not establish support efficacy, equivalence or transient damping.
Read the full abstract

All authors Fatih Ozturk, Emir Mercanoglu, Irem Sayin, Huseyin Uvet

Image and Video Processing · Cross-listed from cs.CV

Faithful Faithfulness Evaluations: Challenges & Pitfalls Learned from a Breast MRI Case Study

Peachapong Poolpol, Henrik H. J. Detjen and Eike Petersen

Saliency maps are widely used to explain deep learning predictions in medical imaging, yet visually plausible explanations do not necessarily reflect a model's true decision process and may therefore mislead clinicians. We investigate this problem using a Vision Transformer-based breast MRI classifier trained on the ODELIA Breast MRI Challenge dataset and evaluate multiple saliency methods, including Last-layer Attention, Attention Rollout, Grad-SAM, Gradient Attention Rollout, GMAR, Grad-CAM, and HiResCAM. Our study highlights two often-overlooked challenges in perturbation-based faithfulness evaluation. First, method rankings depend strongly on the perturbation strategy, varying across intensity-based perturbations and transformer-based attention masking. Second, benchmarking saliency methods requires distinguishing between class-specific and class-agnostic explanations. To enable fair comparisons, we introduce non-class-specific variants of gradient-based methods and evaluate both settings separately. Across protocols, Grad-CAM and Gradient Attention Rollout consistently emerged as the strongest class-specific methods, although their relative ranking depended on the evaluation design. These findings expose important limitations of current saliency-based explainability approaches and highlight the need for more robust and standardized evaluation frameworks for trustworthy clinical AI systems.
Read the full abstract

Image and Video Processing · Cross-listed from eess.AS

Interactive TTS: Dynamic Speaking Style Adaptation for Expressive Speech Synthesis

Wenjie Tian, Kangxiang Xia, Jingbin Hu et al. (+7 more)

Dynamic speaking style adaptation in multi-turn multimodal interaction remains a major challenge for text-to-speech (TTS) systems. Existing context-aware TTS (CTTS) methods typically map dialogue context to speech in an end-to-end manner. Such implicit modeling makes contextual style decisions difficult to supervise, while the entanglement of style, timbre, and content often leads to weak instruction-following and severe timbre drift across turns. To overcome these limitations, we propose Interactive TTS, a dynamic, style-adaptive framework for contextually appropriate and speaker-consistent speech generation. Interactive TTS decouples the process by explicitly modeling contextual style decisions as executable instructions. To bridge the gap between style decisions and speech generation, we introduce Iterative Rejection Sampling Fine-Tuning (Iterative RSFT) and Context-Aware Direct Preference Optimization (CADPO), which significantly enhance instruction-following and align the generated speech with conversational contexts. Extensive experiments demonstrate that Interactive TTS outperforms state-of-the-art models on VStyle and SpeechParaling-Bench. Demo is available at https://wjtian-wonderful.github.io/InteractiveTTS/
Read the full abstract

All authors Wenjie Tian, Kangxiang Xia, Jingbin Hu, Xinfa Zhu, HangRui Hu, Ziyue Jiang, Kexin Huang, Ting He, Lei Xie, Jin Xu

Image and Video Processing · Cross-listed from cs.CV

Initialization and Stopping Tolerance in CPU Dermoscopic Segmentation

Wenhao Xu, Yixian Kong, Ting Pan et al. (+2 more)

Contour initialization and numerical stopping can jointly affect the evaluation of active-contour segmentation. We examine their interaction using the open-source scikit-image Chan-Vese implementation on a resized ISIC 2017 mirror. A fixed development set of 100 images selects a common input channel; all 600 images in the repository's held-out partition are then evaluated. Otsu thresholding is compared with checkerboard-, disk-, and Otsu-initialized contours under default and tighter level-set tolerances. At the default tolerance, Otsu initialization increases mean image Dice from 0.6011 to 0.6660 relative to checkerboard initialization, a paired difference of 0.0649 (95% image-bootstrap interval [0.0452, 0.0860]). Otsu thresholding alone achieves 0.6897. The default disk initializer stops after one iteration on 471 images. Tightening the tolerance reduces the Otsu-seed advantage over checkerboard initialization to 0.0197, with most runs reaching the 500-iteration limit. The default-tolerance advantage also reverses between small- and large-lesion strata. These findings show that an improvement over a generic initializer can coexist with deterioration relative to the threshold baseline. Evaluations should retain the unrefined mask as a comparator and report the initial-field definition, stopping tolerance, and observed iteration counts together.
Read the full abstract

All authors Wenhao Xu, Yixian Kong, Ting Pan, Feilong Wang, Changwei Wang

Image and Video Processing

Robust, Estimator-Agnostic Dynamic 3DGS Compression

Chenjunjie Wang, Zixi Huang, Yao Wang et al. (+1 more)

Dynamic 3D Gaussian splats (3DGS) model time-varying scenes using a separate Gaussian set per frame. While neighboring video frames are highly correlated due to smooth motion, Gaussian representations retain this correlation to varying degrees, depending on whether the estimator tracks them across time. Some 3DGS compression methods integrate the estimation to exploit temporal redundancy; here, we focus on robust compression regardless of the estimator. We concatenate groups of frames into one Gaussian set, augment each Gaussian with a frame index, and pass it to a static (i.e., non-temporal) 3DGS codec, converting temporal redundancy into spatial redundancy. Concatenated sets are spatially partitioned to limit memory. Our technique requires neither a motion model nor knowledge of the training method. Averaged over six N3DV sequences, all six static codecs achieve gains on tracked sets (-42.0% to -71.8% BD-rate) over per-frame coding. On untracked sets, all codecs except HGSC, which appears incompatible with our technique, remain competitive with per-frame coding (-3.5% to +5.0%). We further replace D-FCGS's I-frame coding with our technique while retaining its P-frame coding, yielding an overall BD-rate of -46.2%. We propose to visualize "trackedness" using an inter-frame similarity metric. The project is available at https://wcjj1236.github.io/d3dgs-benchmark.
Read the full abstract

All authors Chenjunjie Wang, Zixi Huang, Yao Wang, Jona Ballé

Image and Video Processing · Cross-listed from cs.CV

Observer Choice and Threshold Selection in Retinal Vessel Segmentation: A Subject-Separated Evaluation

Wenhao Xu, Yixian Kong, Ting Pan et al. (+2 more)

The annotation used to select a segmentation threshold is part of the evaluation protocol, yet its effect is easily conflated with model quality. We examine this choice for retinal vessel segmentation using all 28 CHASE DB1 images and both human annotations. A fixed seven-fold protocol keeps both eyes of each of the 14 subjects together. Random forests and Extra Trees are fitted against observer 1 with three random seeds, yielding 42 fits. Five threshold policies share identical score maps: fixed 0.50, observer-1 tuning, observer-2 tuning, mean-observer tuning, and maximin tuning of the per-image lower observer Dice. For random forests, maximin changes the threshold in 19 of 21 fits, but worst-observer Dice decreases from 70.53 percent to 70.45 percent. The paired difference is -0.073 percentage points, with a conditional subject-bootstrap 95 percent interval of [-0.384, 0.238]. Extra Trees shows the same direction. Identical observer-1-tuned random-forest masks score 73.66 percent against observer 1 and 71.06 percent against observer 2. The results support explicit reporting of both the threshold-selection reference and evaluation reference; they do not support an accuracy benefit from maximin tuning in this cohort. All splits, raw predictions, metrics and code are supplied. AI assistance is disclosed.
Read the full abstract

All authors Wenhao Xu, Yixian Kong, Ting Pan, Feilong Wang, Changwei Wang

Monday 21 September

Image and Video Processing · Cross-listed from cs.CV

PanoFuse: Panorama-Enhanced Vision-Language-Action Learning with Decoupled Semantic-Geometric Routing

Peng Xu, Haoran Lin, Wanjun Jia et al. (+4 more)

Vision-Language-Action (VLA) policies have shown promising performance in language-conditioned robotic manipulation. However, most existing VLA systems rely on conventional perspective cameras with limited fields of view, often missing global scene context and leading to unreliable manipulation under visual occlusions, distractors, and unseen environments. In this work, we propose PanoFuse, a panorama-enhanced VLA framework that complements local manipulation observations with global panoramic perception. PanoFuse introduces a dedicated panoramic branch that leverages a pretrained panoramic foundation model to extract complementary semantic and geometric representations from omnidirectional observations. Rather than directly mixing these heterogeneous features, we introduce Decoupled Semantic-Geometric Routing (DSGR), which maintains semantic and geometric representations as separate context streams and selectively routes both to downstream state and action representations through structured block-wise attention. This design provides the action expert with global spatial context while preserving task-relevant semantic information from the pretrained VLA backbone. We further develop a synchronized data collection pipeline and construct a new real-world manipulation dataset containing panoramic RGB observations, wrist-view images, language instructions, robot states, and actions. Across seven evaluation settings, PanoFuse achieves an average success rate of 52.9%, outperforming the evaluated baselines and achieving consistent gains under novel-object, unseen-background, and distractor-rich settings. Code and data will be released publicly at https://xux-hnu.github.io/PanoFuse.
Read the full abstract

All authors Peng Xu, Haoran Lin, Wanjun Jia, Kai Luo, Wenrui Chen, Zhiyong Li, Kailun Yang

Image and Video Processing · Cross-listed from cs.CV

PanOVOcc: Panoramic Embodied Open-Vocabulary Occupancy Mapping with Long-term Spatial Voxel Memory

Di Kuang, Mengfei Duan, Yuhang Wang et al. (+2 more)

Persistent semantic occupancy mapping is essential for embodied scene understanding. However, perspective-based systems provide limited spatial coverage, while existing panoramic methods primarily predict local volumes from single observations. We introduce PanOVOcc, a training-free framework for persistent open-vocabulary semantic occupancy mapping from panoramic sequences. PanOVOcc unifies panoramic SLAM, open-vocabulary perception, and long-term spatial voxel memory within an online architecture, continuously integrating geometric and semantic evidence into a global, language-queryable map. To facilitate systematic evaluation of this setting, we establish Pan-Replica and Pan-Holo360D, two benchmarks pairing continuous panoramic RGB-D sequences with scene-level semantic occupancy ground truth across synthetic and real-world scenes. Compared with the strongest evaluated baseline for each metric, PanOVOcc improves occupancy IoU and semantic mIoU by absolute +20.03 and +7.06 on Pan-Replica, and by +43.26 and +20.16 on Pan-Holo360D, respectively. The source code and the established benchmarks will be available at https://github.com/bakereet/PanOVOcc.
Read the full abstract

All authors Di Kuang, Mengfei Duan, Yuhang Wang, Weixing Peng, Kailun Yang

Image and Video Processing

Entropy-map SSIM analysis of Salt and Pepper Noise Removal via Recursive Median Filterring

Petr Boriskov and Andrei Velichko

This paper studies the removal of salt-and-pepper (SP) noise from grayscale images using a median filter (MF) within a recursive thresholding algorithm. Denoising performance is assessed using two complementary metrics: SSIM-Img and SSIM-Map.SSIM-Img is standard Image Quality Assessment (IQA), the conventional Structural Similarity Index (SSIM) computed between the restored and clean images. SSIM-Map is novel IQA, an SSIM-based evaluation computed between entropy maps of these images, where the maps are obtained using singular value-decomposition entropy in sliding windows. We show that SSIM-Map is more sensitive to residual impulse artifacts, blur, and local intensity transitions, and therefore complements the conventional SSIM-Img metric.
Read the full abstract

Image and Video Processing

When is a closed-form RGB->S/P ratio adequate? A hyperspectral characterization on natural scenes for mesopic display

Naoyuki Uchida

Mesopic and low-light display transforms require, as their driving signal, a per-pixel scotopic-to-photopic luminance ratio (S/P); the exact spectral S/P is unavailable for ordinary RGB content, so a low-cost closed form that estimates S/P from a linear-RGB triplet is used in its place. Such closed forms exist but have been characterized only on narrowband / LED sources, i.e. spectrally sparse spectra, where a relative error of ~41% has been reported for a three-channel projection. Display content, however, is natural and broadband. We ask whether the same closed form is adequate there, using per-pixel spectral S/P from hyperspectral imagery as ground truth. On a daylight radiance time-series, a six-scalar closed form (three photopic and three scotopic channel weights) reproduces spectral S/P with a median error of ~0.07 that is time-invariant once the RGB input is chromatically adapted to D65; evaluated in un-adapted sRGB the error instead carries a color-temperature tilt across illuminants (~0.19), so adaptation is the enabling step for this use case. The result generalizes to an independent fifty-scene set (pooled median 0.024; 45/50 scenes within a pre-registered 0.10 band), with the few exceedances concentrated in saturated, spectrally-peaky surfaces that approach the narrowband regime (floral close-ups in this set). The scotopic weight vector is shown to be primary-model dependent, but the value used here is corroborated by a primary-free XYZ projection, and the median error stays within the band across all principled coefficient choices. We do not claim observer-validated appearance fidelity or adequacy on narrowband sources; both are out of scope. Both outcomes follow from the same three-channel projection: it is overwhelmed by spectrally sparse inputs and adequate on spectrally smooth ones.
Read the full abstract

Image and Video Processing

Disparity Estimation of Planar Reflective Surfaces Using Specular Reflections From a Single Light Source

Katja Kossira, Frank Sippel, Jürgen Seiler et al. (+1 more)

Multi-camera imaging and camera arrays have become ubiquitous in many applications, such as autonomous driving, robot control, or virtual reality, and accurate disparity maps of objects and their environment are essential for reliable operation. Despite recent advances in neural networks, correctly estimating the disparity of flat and textureless objects remains challenging. In particular, we consider a scenario defined by flat, textureless surfaces illuminated by a single fixed light source, resulting in one dominant specular reflection visible on the object surface. Under these conditions, reliable geometric and photometric cues are missing, and the specular reflection often causes mispredictions, especially when using conventional methods that rely on texture information to match corresponding pixels. To address this issue, the novel Specular Reflection Disparity Estimation SRDE algorithm is introduced, which is specifically designed for the constrained scenario of planar, textureless objects and single-source illumination. Unlike conventional stereo matching methods, SRDE ignores texture and instead leverages the geometric properties of specular reflections by incorporating the position information of the reflective region, the light source, and the camera setup. We show that SRDE outperforms existing methods by a notable margin, achieving more than a 52% improvement in End Point Error on synthetic images. Further tests demonstrate superior performance on real-world data. Furthermore, we integrate SRDE into existing neural disparity estimation pipelines by selectively replacing predictions in specular regions without modifying the backbone model. This hybrid strategy enables additional performance gains without requiring network retraining.
Read the full abstract

All authors Katja Kossira, Frank Sippel, Jürgen Seiler, André Kaup

Image and Video Processing · Cross-listed from cs.CV

GraphSVR: q-Space–Aware Graph-Based Slice-to-Volume Registration for Diffusion MRI

Noga Kertes, Daphna Link Sourani, Alex M. Bronstein et al. (+1 more)

Diffusion-weighted imaging (DWI) remains highly vulnerable to subject motion, particularly in time-efficient protocols and in motion-prone populations. While slice-to-volume registration (SVR) can mitigate inter-slice and inter-stack misalignment, diffusion MRI introduces additional complexity due to diffusion-direction-dependent contrast and the requirement to align dozens of measurements within a common reference frame, effectively yielding a 4D registration problem. Existing approaches rely primarily on sequential modeling or pairwise similarity and often degrade under sparse gradient sampling or severe motion. We introduce GraphSVR, a q-space-aware graph-based framework for 4D SVR registration in DWI. GraphSVR represents slice groups as nodes in an acquisition-structured graph, with edges encoding temporal proximity, spatial slice geometry and diffusion encoding relationships. A graph neural network predicts globally consistent stack-wise rigid motion, optimized in a self-supervised, zero-shot manner using only an anatomical reference image, without requiring paired ground-truth motion. We evaluate GraphSVR using both fully synthetic diffusion simulations and realistic recombination-based simulations from real acquisitions with controllable motion severity and gradient sparsity. Performance is quantified using grid error (mm) and rotation error relative to known ground-truth transforms. Under severe motion, GraphSVR reduces grid error and rotation error by 73% compared to FSL eddy, the standard DWI motion-correction method, with the largest gains observed in sparse-direction regimes. These results demonstrate that explicitly modeling acquisition structure through graph-based reasoning improves robustness and global consistency in 4D DWI motion estimation. Code is available at https://github.com/nogakertes/GraphSVR.git.
Read the full abstract

All authors Noga Kertes, Daphna Link Sourani, Alex M. Bronstein, Moti Freiman

Image and Video Processing

Recurrent Convolutional Neural Networks for LiDAR-Based Attitude Initialization of Rotating Spacecraft

Luca Bechis, Jean-Luc Sarvadon, Petre Ricioppo et al. (+1 more)

Accurate attitude estimation is essential for autonomous in-orbit servicing and proximity operations. This work proposes a Recurrent Convolutional Neural Network (RCNN) used in coarse attitude initialization of known, possibly tumbling spacecraft using LiDAR-derived depth images. By processing temporal sequences of 2D point-cloud projections, the RCNN effectively handles symmetries, occlusions, and degraded sensing. Simulations across various spacecraft geometries, angular velocities, and ranges show that the RCNN yields lower initialization errors and higher convergence rates than conventional CNN baseline within the adopted experimental framework, with performance varying across angular velocity conditions.
Read the full abstract

All authors Luca Bechis, Jean-Luc Sarvadon, Petre Ricioppo, Mauro Mancini

Image and Video Processing

JEVQA - Video Quality from Metadata, Bitstream, and Pixel Features with a General-Purpose Decision Model

Werner Robitza

Instrumental quality models for video quality prediction are usually trained for a fixed set of codecs or other input features, and every new input variable requires retraining. Novel, general-purpose decision models can answer questions without task-specific training, but it is unclear whether they can judge video quality. We evaluate Jev, a commercial “System One” model that returns probability distributions over a provided answer scale, as a zero-shot video quality model. We call the resulting method JEVQA. In a first study on 1,936 AV1, H.264, HEVC, and VP9 encodes of 22 sources, scored against VMAF as ground truth, using encoding metadata only, JEVQA reached a Pearson correlation of 0.737, on par with the standardized ITU-T P.1204.1 model (0.733). Giving the model bitstream data raised the accuracy to 0.797, and combined pixel-based and bitstream features raised it to 0.824. A pixel-only variant failed in our tests. In a second study, using H.264, HEVC, and VP9 encodes in the AVT-VQDB-UHD-1 database, the metadata-only model reached a correlation of 0.879 with MOS, close to P.1204.1 (0.898). Bitstream statistics did not help there. Our results show that trained models on the same features remain clearly ahead in both studies, but that zero-shot classifiers are promising.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning

Santi Ram Tiwari, Nihal Naik, Devbrat Pandey et al. (+1 more)

Multimodal large language models (MLLMs) fail at fine-grained visual questions less because they cannot reason than because they never see the evidence: high-resolution images are downsampled before encoding, so the model answers from linguistic priors. The standard remedies are expensive: annotated answers (SFT), hand-engineered verifiers (RLVR), or a large external teacher (on-policy distillation). We ask whether the visual evidence itself can supply the signal for free. We formalize the contrastive evidence gap, the per-token log-likelihood ratio that a model assigns to its own output when conditioned on a question-relevant region versus an irrelevant one, and study it across Qwen2.5-VL-7B, Qwen3-VL-8B, and Qwen3-VL-30B-A3B on V*Bench. Our main positive result is training-free: selecting the candidate crop under which the model's answer distribution is most peaked, using a single-view, label-free criterion, discovers the answer-bearing region with no bounding boxes, training, or labels. It localizes the target 4.4 to 5.1 times better than chance and raises fine-grained accuracy from 70 percent to 85 percent at inference. We further show that the gap is complementary to the model's own confidence. Combining them predicts correctness better than either alone, with AUC up to 0.99, and flags confidently wrong answers, with AUC ranging from 0.97 to 1.00 within the high-confidence subset. All effects concentrate on perception-bottleneck questions and vanish on a global-context control. Finally, we report an honest negative result: converting the same signal into a training method, gated self-distillation (SEG-Distill), does not outperform the base model at pilot scale across three gate designs, while more aggressive gating degrades accuracy. The signal is real, but converting it into training gains remains an open problem.
Read the full abstract

All authors Santi Ram Tiwari, Nihal Naik, Devbrat Pandey, Nishant Sinha

Sunday 20 September

Image and Video Processing · Cross-listed from cs.AR

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator

Mikolaj Walczak, Edward Humes, Chao Fang et al. (+2 more)

Due to limited support for intra-tensor heterogeneous precision in conventional accelerators, neural network quantization remains largely restricted to per-tensor precision assignment. We present ShatterQuant, a hardware-software co-designed framework enabling mixed-precision quantization within each tensor by assigning independent bit-widths to blocks of a weight projection. ShatterQuant couples precision granularity with PE configuration, such that each precision determines an effective block height. We introduce (1) a hardware-aware post-training method that assigns intra-tensor precision based on block-level standard deviation and weight sensitivity; (2) the ShatterQuant Transformer Accelerator supporting 1/2/4/8-bit weight precision, precision-dependent PE configuration, block rescaling, and integrated softmax and piecewise-linear nonlinearities; and (3) an evaluation of model-hardware tradeoffs using an implementation in the TSMC 16nm PDK operating at 1 GHz, achieving 1.5 TOPS, 760 GOPS/$mm^2$ area efficiency, and 2.8 TOPS/W energy efficiency. On DeiT and ImageNet-1K, ShatterQuant achieves accuracy within $3.3\%$ of state-of-the-art mixed-precision techniques while using a 2 bit lower effective bitwidth, while for PixelDiT demonstrates comparable generation quality. ShatterQuant demonstrates how fine-grained intra-tensor mixed-precision can be realized through hardware-software co-design.
Read the full abstract

All authors Mikolaj Walczak, Edward Humes, Chao Fang, Marian Verhelst, Tinoosh Mohsenin

Image and Video Processing

Does DCGAN-Based Synthetic Augmentation Improve Brain Tumor MRI Classification? An Empirical Study

Irhum Jawad Khan and Talha bin Aslam

Generative adversarial networks (GANs) are increasingly used to augment medical imaging datasets, but synthetic images do not necessarily provide downstream classification benefits. This study investigates whether class-specific Deep Convolutional Generative Adversarial Network (DCGAN) augmentation improves brain tumor classification when the classifier and evaluation set are held constant. Experiments were conducted on 7,200 brain magnetic resonance imaging (MRI) scans across four classes: glioma, meningioma, pituitary tumor, and no tumor. For each class, 1,400 real images were used for training and 400 were reserved for testing. A baseline Swin Transformer classifier was trained using only the real training images and compared with a second model trained using the same real images augmented with 500 DCGAN-generated images per class. Both conditions were evaluated on the identical held-out test set. The two models achieved the same overall accuracy of 96%, while macro F1 remained effectively unchanged and ROC-AUC decreased slightly from 0.987 to 0.982 after augmentation. Class-level analysis showed small redistributions in errors rather than a consistent performance gain. FID values between real and synthetic images ranged from 209.15 to 314.27, indicating substantial distributional differences under the adopted evaluation setup. These results suggest that synthetic augmentation should not be assumed to improve medical image classification and should instead be evaluated for both distributional fidelity and downstream task utility.
Read the full abstract