Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Wednesday 14 October · still arriving

Image and Video Processing · Cross-listed from cs.CV

A$^3$PreCore: Robust Rigid Registration for Native FLAIR-Space Brain-Mask Transfer

Elmira Mirzabeigi, Zahra Khalilzad, Taylor Snowden-Richardson et al. (+4 more)

Reliable brain-mask transfer into native fluid-attenuated inversion recovery (FLAIR) space can fail when T1-to-FLAIR rigid registration is unstable. We developed A$^3$PreCore, a six-degree-of-freedom rigid-registration strategy that aligns the full bias-corrected T1-weighted image to native FLAIR using normalized mutual information while preserving the established T1-space anatomical mask. We also introduced extended quantitative quality control (E-QC), which characterizes mask extent, field-of-view-relative position, FLAIR intensity support, boundary support, inferior morphology, and pairwise geometric disagreement. In 78 participants with paired T1-weighted and FLAIR imaging, A$^3$PreCore was compared with fsl_anat and the Brain Extraction Tool (BET). Gross rigid-body misregistration occurred in 21/78 fsl_anat outputs (26.9%) and 0/78 A$^3$PreCore outputs (exact two-sided McNemar p=9.54 x 10^-7). Among 57 participants without gross fsl_anat misregistration, A$^3$PreCore preserved close agreement (median Dice=0.993; median paired Delta NMI=-0.00007, p=0.429). A$^3$PreCore remained broadly concordant with BET (median Dice=0.955), with less inferior and extracranial extension in representative comparisons. Method-blinded neuroscientists ranked A$^3$PreCore first in 35/40 participants (87.5%). E-QC additionally exposed severe orientation-related disagreement not evident from global descriptors or centroid displacement alone. These findings support A$^3$PreCore as a robust native-FLAIR brain-mask transfer strategy and E-QC as a complementary framework for multidimensional mask characterization and participant-specific geometric quality control.
Read the full abstract

All authors Elmira Mirzabeigi, Zahra Khalilzad, Taylor Snowden-Richardson, Jesus Eduardo Lugo Arce, Jodie R.Gawryluk, Brian R. Christie, Jocelyn Faubert

Tuesday 13 October · still arriving

Image and Video Processing

Deep Learning for Atmospheric Turbulence Mitigation: A Comparison Between GANs and Transformers

Juan S. Carrillo and Jérôme Gilles

Atmospheric turbulence creates complex spatial and temporal distortions that significantly reduce image quality. This work compares two deep learning methods for turbulence reduction: Generative Adversarial Networks (GANs) and Transformer-based models. Both methods are trained and tested on the SOTIS dataset, which includes multi-frame sequences under different turbulence conditions with ground truth references. Our results show an important finding: GAN-based models perform better in severely degraded, low-quality situations, while Transformer-based models achieve better results in moderate to high-quality settings, especially in correcting geometric distortions. We also enhance the Turbulence Mitigation Transformer (TMT) by introducing a refined training strategy, an augmented input design that includes a temporally averaged frame as prior structural information, and a modified loss function that incorporates the Structural Similarity Index (SSIM). These improvements lead to consistent performance gains across various turbulence conditions. Overall, this study provides an in-depth evaluation of deep learning methods for turbulence reduction and offers practical guidance for selecting models in different imaging environments.
Read the full abstract

Image and Video Processing · Cross-listed from physics.optics

High-resolution retinal matrix imaging via computational motion and aberration correction

Aymeric Melt, Léo Puyo, Victor Barolle et al. (+3 more)

High-resolution optical imaging of the human retina is essential for diagnosing ophthalmic and neurodegenerative diseases. While adaptive optics combined with coherence gating can resolve retinal structures at the cellular scale, it requires complex, costly, and cumbersome hardware to physically correct for ocular aberrations and motion. Matrix imaging offers a powerful digital alternative by leveraging wave linearity to undo aberrations and multiple scattering; however, its in vivo deployment has been hindered by continuous, involuntary eye movements that destroy phase coherence during acquisition. Here, we overcome this fundamental barrier and demonstrate the first in vivo reflection matrix imaging of the human retina using an ultra-simplified optical setup without physical correctors. We introduce a fully computational framework that tracks and compensates for multi-axial ocular motion directly from raw interferometric data, thereby restoring the phase coherence of a multi-spectral distortion matrix. This enables local, high-order aberration correction to achieve diffraction-limited, volumetric imaging of retinal tissues. By shifting the hardware burden to physics-guided numerical computation, this paradigm opens the way for accessible, non-invasive, and high-resolution digital ophthalmology.
Read the full abstract

All authors Aymeric Melt, Léo Puyo, Victor Barolle, Claude Boccara, Gereon Hüttmann, Alexandre Aubry

Image and Video Processing

Per-Bundle, Pre-Reconstruction Inversion for Temporal CT: Flux-Compensation Design and Multi-Patient Generalization

Guy M Besson

Paper 1 established the per-bundle statistical limits of Temporal CT (TCT) and benchmarked a learned prior on a single patient at a uniform incident count $N_0$. This paper lifts both restrictions. SNN2, a Gauss-Newton estimator, converges to the per-bundle Poisson maximum-likelihood solution and is the prior-free baseline. N$_0$FC, a bow-tie and tube-current-modulation co-design, compresses within-bundle photon-flux dynamic range. V8c is a neural-network residual corrector trained on a 19-patient corpus stratified by water-equivalent diameter (WED). T2bp, a per-patient empirical-Bayes post-processor, combines per-bundle Poisson likelihoods with a prior built from the patient's own bundles; it requires no training, so it cannot inherit a mismatched training prior. Results use Paper 1's bias-charged Dose Inflation Factor (DIF), extended to per-ray budgets. V8c beats SNN2 on in-distribution patients under baseline acquisition but fails out of distribution. Composed with SNN2, T2bp matches or beats T2bp$\circ$V8c on three of four N$_0$FC test cells (WED 207-342 mm), with the largest gains on the high-WED and out-of-distribution patients, and trails by 3% on the fourth. On 23 non-test corpus volumes, N$_0$FC with closed-loop tube-current modulation lowers the per-volume median middle-path Cramer-Rao floor, as a DIF, from 23-48 under baseline acquisition to 3.3-6.3, near the equal-attenuation value 13/3; T2bp$\circ$SNN2 reaches this floor without training, its mean squared error 0.74-1.26 times the floor (median 0.89) over the bulk of each volume. Acquisition co-design removes most of the attenuation-contrast penalty, and a training-free estimator attains the resulting limit. The remaining three- to six-fold gap to dose parity with single-source CT, and the high-attenuation tail, are the target of inter-bundle priors in later work.
Read the full abstract

Image and Video Processing

Perceptual Characterization of Scan-induced Artifacts in Dynamic 3D Meshes

Huong Hoang, Truong Nguyen and Pamela Cosman

Dynamic meshes reconstructed from real-world scenes often exhibit scan-induced artifacts such as surface jitter and missing geometry, which strongly impact perceived visual quality. Objective assessment remains challenging due to the absence of artifact-free ground truth and limited perceptual understanding of artifact interactions. This paper presents a systematic and controlled perceptual study of scan-induced artifacts in dynamic meshes, focusing on surface jitter and holes. To enable systematic analysis, we introduce a physically grounded artifact synthesis framework in which holes are synthesized using a reconstruction uncertainty model derived from multi-view geometry. The study reveals asymmetric cross-artifact interactions, non-additive integration, and motion-dependent temporal masking. Based on these findings, we propose a compact, perceptually grounded no-reference quality assessment framework that encodes the observed perceptual mechanisms.
Read the full abstract

Image and Video Processing

DoReMe: Dual optimization of Reconstruction and Motion estimation for self-navigated free-breathing abdominal MRI using Gaussian Splatting

Majd Helo, Dominik Nickel and Thomas Kuestner

Purpose: Free-breathing abdominal MRI is limited by respiratory motion, which introduces image artifacts and k-space inconsistencies. Existing reconstruction approaches often depend on pre-estimated motion fields or external respiratory surrogate signals. We propose a coupled reconstruction-motion estimation algorithm for self-navigated free-breathing acquisition, in which motion is represented by compact Gaussian primitives and jointly refined with the image estimates during iterative reconstruction. Methods: An iterative reconstruction that alternates between data consistency updates and motion modelling was proposed. Respiratory motion was represented using Gaussian primitives. A Gaussian splatting was integrated into the reconstruction algorithm. The method was evaluated on self-navigated free-breathing Cartesian gradient-echo (GRE) acquisition in four healthy volunteers, including retrospective scan shortening experiments, and was compared with regularized compressed sensing (CS) reconstruction. Results: Respiratory motion states were reconstructed in approximately 1.5 min. Compared with CS, the proposed method improved visual image quality, particularly in coronal liver views, and better preserved anatomical structures in retrospectively shortened acquisitions. These findings provide proof of concept for the proposed method. Conclusion: The proposed coupled reconstruction-motion estimation algorithm improved visual image quality compared with CS alone. Gaussian primitives provided a compact and sufficiently flexible representation of nonrigid respiratory motion with practical runtime, supporting further scanner integration and validation in larger cohorts.
Read the full abstract

Image and Video Processing

Focus, Align, and Diffuse: Time-Series-Aware Keyframe Diffusion for Cardiac Dynamic Synthesis from Sparse Observations

Junkai Liu, Haofan Wu, Nay Aung et al. (+3 more)

Dense cardiac dynamics are essential for assessing cardiac function, yet in practice clinicians often observe only extremely sparse measurements. However, recovering full dynamics from such sparsity is a highly ill-posed problem due to the lack of temporal cues. While auxiliary time-series signals like ECG provide physiological guidance, effectively exploiting them is challenging due to the cross-modal representation gap and phase misalignment. In this work, we present CardioFAD, a Focus–Align–Diffuse paradigm that reformulates sparse cardiac dynamic synthesis as a recoverability–alignability anchor discovery problem. Our core insight is that resolving extreme temporal ambiguity requires a compact set of anchor states that are both sufficient to recover the visual trajectory and reliable for ECG–visual alignment. We therefore learn motion-salient keyframes under two complementary principles: trajectory sufficiency, which preserves trajectory-critical intra-modal motion content, and anchor alignment, which promotes consistent inter-modal correspondence at these anchors. We implement the Focus and Align modules optimized by the two aspects. Diffuse then performs two-stage generation by first synthesizing keyframes and subsequently interpolating the remaining frames. Extensive experiments on cardiac MRI and echocardiography videos demonstrate superior visual quality, temporal coherence, and improved functional consistency under the extreme one-shot regime.
Read the full abstract

All authors Junkai Liu, Haofan Wu, Nay Aung, Joao A. C. Lima, Steffen E. Petersen, Le Zhang

Image and Video Processing

Personalising Deformable Image Registration with Clinical Context and Deformation History for Adaptive Radiotherapy

Caiwen Jiang, Songlin Zhao, Yuzhen Ding et al. (+23 more)

Adaptive radiotherapy must account for anatomical changes during treatment, but doing so depends on reliably mapping targets, organs at risk and dose across serial images. Existing registration methods treat each image pair independently, without using radiotherapy-specific context or the patient's preceding anatomical trajectory. We developed Clinical-context-Aware and Patient-adaptive Deformable Image Registration (CAP-DIR), integrating source contours, dose, clinical text and prior-registration features. We evaluated 16,361 CT examinations from 3,473 patients across four institutional cohorts and five public benchmarks. CAP-DIR increased internal-test organ-at-risk-region normalized cross-correlation from 84.76% to 96.92% relative to the strongest learning-based comparator, with low deformation folding, and remained superior across seven external cohorts. In a blinded review of 20 longitudinal cases, clinically acceptable contours increased from 45.0% to 82.5% for clinical target volumes and from 22.5% to 62.5% for organs at risk, while estimated editing time decreased. In a controlled benchmark, CAP-DIR reduced global dose-mapping error by 34.5% relative to the strongest comparator. These findings show that clinically informed, longitudinal registration can improve contour usability and dose mapping for adaptive radiotherapy.
Read the full abstract

All authors Caiwen Jiang, Songlin Zhao, Yuzhen Ding, Jia Mi, Samir H. Patel, Alexandra Bennett, Keldon K. Lin, Terence T. Sio, Jonathan B. Ashman, Lisa A. McGee, Jean-Claude M. Rwigema, William G. Rule, Sameer R. Keole, Sujay A. Vora, William W. Wong, Nathan Y. Yu, Michele Y. Halyard, Steven E. Schild, Safia K. Ahmed, Daniel J. Ma, Robert L. Foote, Michelle A. Neben Wittich, Thomas J. Whitaker, Haibo Lin, Dinggang Shen, Wei Liu

Image and Video Processing · Cross-listed from cs.CV

Simplifying and Improving On-Policy Distillation for Few-Step Diffusion

Jiawei Zhang, Ziyu Wan, Nanye Ma et al. (+5 more)

Few-step distillation reduces the sampling cost of diffusion and flow models, but existing approaches often require auxiliary score models or specialized architectures. We show that simple on-policy velocity regression already provides a strong foundation for 4-NFE generation. To represent teacher velocity variation within each coarse transition, we introduce Pivot, a training-only residual proxy that separates local corrections from the student's interval-mean prediction. We further repel the student from weaker teacher predictions, providing a lightweight mechanism for avoiding undesirable modes. The resulting framework, Simplified Matching Distillation (SMD), remains entirely regression-based, requires neither online score estimation nor adversarial training, and leaves the deployed generator unchanged. We evaluate SMD on image generation and editing, 3D generation, video synthesis, and joint audio–video generation, scaling to 33 billion parameters. At 4-NFE, SMD outperforms strong few-step baselines and surpasses multi-step teachers on several benchmarks. It also supports cross-model, multi-teacher, and causal-video distillation.
Read the full abstract

All authors Jiawei Zhang, Ziyu Wan, Nanye Ma, Long Zhao, Dong Chen, Hongyu Liu, Qifeng Chen, Dongdong Chen

Image and Video Processing

Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling

Anuska Sarkar, Partha Acharya, Vishnu Nandury et al. (+2 more)

We present a real-time, hybrid framework for occlusion handling and depth-aware tracking in minimally invasive surgeries (MIS). Occlusions caused by surgical instruments often obscure the region of interest (ROI), affecting surgical precision. To address this, our approach integrates enhanced You Only Look Once (YOLO) based detection, Particle Filtering (PF), and monocular depth estimation using Multiple Depth Estimation Accuracy with Single Network (MiDaS). Occlusion is detected using a distance-based threshold between instruments and ROI. Experimental results demonstrate robust instrument detection with a confidence score of 0.85, effective occlusion event handling, and accurate depth prediction with an MAE of 0.34 cm and RMSE of 0.53 cm. Independent phantom-based validation against stereo depth measurements yielded an MAE of 0.65 cm and RMSE of 0.66 cm. The framework further achieved an occlusion detection accuracy of 81.95% while maintaining real-time performance at 38.7 FPS. Preliminary surgeon feedback indicates that the proposed visualization framework is clinically relevant and holds promise for future intraoperative image-guided navigation and decision support in MIS.
Read the full abstract

All authors Anuska Sarkar, Partha Acharya, Vishnu Nandury, Kaustav Basu, Subhamoy Mandal

Image and Video Processing · Cross-listed from physics.optics

Recovery-Oriented Optical System Design Based on Fisher Information

Zhuang Qian, Yan Mo, Hao Tan et al. (+5 more)

Traditional optical system design for imaging applications has long been guided by image quality metrics such as geometrical RMS spot size, wavefront error, and modulation transfer function (MTF). Although these metrics effectively characterize direct imaging performance, they do not necessarily indicate optimal behavior when image restoration is subsequently applied. In modern imaging pipelines, however, optical systems are increasingly combined with physics-based or learning-based restoration methods, making the information transfer properties of the optical system critically important. In this work, we propose a recovery-oriented optical design approach that explicitly links optical system optimization with image restoration performance through an estimation-theoretic framework based on Fisher information. A new design criterion is adopted to ensure that the information encoded by the optical system can be reliably recovered during image restoration. To validate the proposed approach, two dual separated aspheric lenses are designed under identical configurations using conventional RMS-based optimization and the proposed recovery-oriented optimization, respectively. Although the RMS-optimized lens exhibits superior direct imaging quality, it underperforms after image restoration compared to the recovery-oriented design. These results demonstrate that recovery-oriented optical design provides a physically interpretable and effective extension of traditional optical design criteria for imaging systems operating within modern computational imaging pipelines.
Read the full abstract

All authors Zhuang Qian, Yan Mo, Hao Tan, Yu Chen, Haibo Wang, Weining Guo, Yijie Ni, Donglin Ma

Image and Video Processing

Deficit-Driven Stratified Acquisition for Building STRATA-S12: An Open SAR-Optical Corpus for Self-Supervised Learning

Christian Sanchez-Leon, Dongming Chen and Dongqi Wang

Building a co-registered SAR-optical corpus for self-supervised Earth Observation is above all an acquisition problem, yet the main multi-modal datasets are collected opportunistically, with no per-stratum quota control and no mechanism against the geographic dropout that non-uniform cloud cover leaves behind. We present Deficit-Driven Stratified Acquisition (DDSA), an algorithm that treats building a co-registered Sentinel-1/Sentinel-2 corpus as a constrained maximum-coverage problem over a hemisphere-corrected (class x season x region) stratification, enforces strict temporal pairing and a multi-stage quality gate, and closes the loop during acquisition with a human-in-the-loop deficit-repair stage. Measured on its own product, the algorithm yields three results a single-pass acquisition cannot report. Deficit-driven ordering makes a partial run a covering one: it holds every one of the 192 strata populated from the first few percent of the budget, where an abundance-first order leaves most strata empty until the end. Whether a failed attempt can be recovered depends on the temporal persistence of its cause, not the kind of defect: geometric failures such as nodata are among the most repairable (eta = 41.4%) while cloud is among the least (eta = 18.8%), because a cloudy location is cloudy on most dates of its season. The repair loop supplies 39.5% of the released pairs at no measurable quality cost, with 89.8% surviving manual review. The quality guarantee has a price: 2,928,370 acquisition attempts produced 740,746 released pairs. We release the corpus, STRATA-S12, with STAC/PDS4-aligned FAIR sidecars, and a pre-registered linear-probe check confirms it is fit for self-supervised pre-training. A metadata-level coverage comparison against SSL4EO-S12 and SEN12MS situates the corpus over a shared frame; systematic cross-corpus and cross-method comparison is left to follow-up work.
Read the full abstract

Image and Video Processing

Low-Altitude Agentic Networks: From UAV Connectivity to Workflow-Driven Capability Orchestration

Baolin Chong, Fuhui Zhou, Qihui Wu et al. (+1 more)

Low-altitude platforms are evolving from connected sensing nodes into autonomous agents capable of bounded task interpretation, skill invocation, and local decision making. Increasing the intelligence of individual unmanned aerial vehicles (UAVs), however, does not resolve the organization problem that arises when mission workflows depend on capabilities distributed across multiple platforms and provider feasibility changes with network, resource, and mobility conditions. We propose low-altitude agentic networking as a task-level networking paradigm and develop a reference architecture that separates workflow logic from executor identity. Within a validated domain-specific workflow space, each ready operation is translated into a capability requirement and dynamically bound to an agent that remains feasible under current conditions. Task contracts and structured evidence govern workflow execution, while routine provider and link changes are handled through workflow-preserving local adaptation before high-level repair is invoked. A trace-calibrated discrete-event case study identifies a bounded operating region: conventional organization remains effective for simple workflows under stable conditions, whereas agentic networking becomes beneficial when workflow dependencies are tightly coupled, capability feasibility changes during execution, and viable local recovery paths remain available.
Read the full abstract

All authors Baolin Chong, Fuhui Zhou, Qihui Wu, Ran Tu

Image and Video Processing

Background2Background: a self-supervised background removal framework for high dimensional microscopy

Guangrui Ding, Haonan Lin, Rong Tang et al. (+8 more)

High-dimensional microscopy captures features across space, time, and spectrum, but complex backgrounds obscure signals. We present Background2Background (B2B), a self-supervised background removal framework leveraging the backgrounds larger correlation relative to the signal. Local pixel shuffling disrupts signal correlations while preserving background structure, transforming signals into effective noise for Noise2Noise-based background removal. Across fluorescence, stimulated Raman scattering and mid-infrared photothermal imaging, B2B achieves over 40-fold background suppression and reveals hidden signatures.
Read the full abstract

All authors Guangrui Ding, Haonan Lin, Rong Tang, Bethany Weinberg, Jianpeng Ao, Qing Xia, Guorong Hu, Dingcheng Sun, Jiaze Yin, Lei Tian, Ji-Xin Cheng

Monday 12 October

Image and Video Processing

Calibrating Generative Flows through Regional Probability Matching

Yubo Huang, Xiongyu Hu, Xin Lai et al. (+4 more)

Generative models offer a powerful route to scientific simulation and discovery by enabling the study of complex populations and rare phenomena. Flow matching can generate realistic samples, yet inaccurate occurrence frequencies can distort population statistics and compromise inferences about rare events. Here we developed generalized mass-preserving coupled flow matching (G-MPCF), which matches the probability masses of Gaussian source regions to those of weighted data regions. This construction encodes the desired regional frequencies in the correspondence between noise and observations, shaping the learned dynamics while preserving the prescribed target distribution. On a rare-mode benchmark, G-MPCF reduces relative score error by 95.0% and numerical integration error by 89.9% compared with independent flow matching. On CIFAR-10 with a 100:1 class-probability ratio, it reduces corrected class-mass error by 44.6% compared with minibatch optimal transport, while improving mean image fidelity with matched computational budgets. We envision that incorporating regional probability calibration into generative modelling will support more reliable scientific simulation and inference, enabling the study of systems in which rare events and their occurrence frequencies are central to understanding the underlying mechanisms.
Read the full abstract

All authors Yubo Huang, Xiongyu Hu, Xin Lai, Zixi Wang, Liao Jian, Jingzehua Xu, Chuan He

Image and Video Processing

Uncertainty-Aware Cortical Surface Reconstruction and Spherical Registration for Heterogeneous Brain MRI

Jesper Duemose Nielsen, Karthik Gopinath, Andrew Hoopes et al. (+4 more)

Surface-based cortical analysis is useful in many applications as it is a natural way to model the highly folded structure of the cerebral cortex, allowing quantities such as curvature and thickness to be easily calculated. Existing deep learning (DL) methods have focused on 1x1x1 mm3 T1-weighted (T1w) scans, thus replicating classical pipelines. Here we present a neural network that is trained exclusively on synthetic scans with random contrast and resolution. Consequently, it can be applied to almost any kind of magnetic resonance imaging (MRI) scan. In addition, our model provides vertex-wise uncertainty estimates of the predicted surface as well as a spherical registration to fsaverage space. Our model performs on par with other state-of-the-art DL methods on 1x1x1 mm3 T1w scans and outperforms the only other method capable of estimating cortical surfaces from low resolution, non-T1w scans. The predicted uncertainty, moreover, correlates with overall scan quality on low resolution scans and highlights areas of poor surface reconstruction on 1x1x1 mm3 T1w scans, suggesting that it can be used for quality control. Finally, the predicted spherical registrations prove comparable to other DL and classical registration methods. These qualities, when combined with the fast inference time, will provide the ability to analyze large, existing databases of clinical MRI scans, with the potential for finding disease-specific patterns that would otherwise be infeasible in a research setting.
Read the full abstract

All authors Jesper Duemose Nielsen, Karthik Gopinath, Andrew Hoopes, Adrian Dalca, Axel Thielscher, Juan Eugenio Iglesias, Oula Puonti

Image and Video Processing

Hybrid Context Forests and Bilateral Viterbi Matching for 3D Rib Centerline Tracing in CT

Quan Wang

Automatic extraction and anatomical numbering (R01-R12, L01-L12) of all 24 rib centerlines in thoracic CT is essential for unfolded rib visualization, fracture detection, and surgical navigation, but remains challenging due to adjacent-bone confluence, calcified costal cartilage bridges, kyphotic pose variation, and partial visibility of terminal ribs. We present ribline, an open-source three-stage framework combining: (1) an automatically detected lung-apex and vertebral-midline coordinate frame with a kyphosis-aware azimuthal angle for point ordering along each rib; (2) a hybrid voxel detector fusing an anatomically anchored Semantic Context Forest (SCF) with a lightweight 3D U-Net; and (3) a global stage that decomposes the probability skeleton into candidate curves, assigns rib indices via a bilateral Viterbi dynamic program coupled across both hemithoraces, and refines curves via template recovery and normal-plane centering (RibSnake). On the Siemens Corporate Research 2 mm thoracic CT benchmark (47 training and 12 evaluation scans), our pipeline achieves a labeled Chamfer distance of 2.05 mm, labeled clDice of 0.886 (3 mm) and 0.945 (4 mm), 97.2% labeling accuracy (278/286 ribs), and zero false-positive curves on the evaluation split, and 3.64 mm (median 2.39 mm) with 91.8% labeling accuracy across all 59 annotated scans (1,402 ribs) in 5-fold cross-validation. Code: https://github.com/wq2012/ribline Pretrained Models: https://huggingface.co/collections/wq2012/ribline
Read the full abstract

Sunday 11 October

Image and Video Processing · Cross-listed from stat.CO

Jensen-Bregman split Gibbs sampling

Nicolas Dobigeon, Mame Diarra Fall and Eric Barat

Split Gibbs sampling (SGS) is a versatile tool to sample from complex posterior distributions, such as those arising in imaging inverse problems. However, existing SGS instances are limited to quadratic variable couplings, restricting their applicability to Gaussian likelihoods. This letter genearalizes SGS to non-Euclidean geometry by introducing a coupling kernel defined through a Jensen-Bregman divergence. This framework not only enables posterior sampling in these settings but also unifies several ad hoc sampling schemes from the literature under a principled theoretical foundation.
Read the full abstract

Saturday 10 October

Image and Video Processing

Generative Quantum Machine Learning Methods for Cancer Detection using Augmented Datasets

Sophia Knutson, Nandika Goyal, Gennaro De Luca et al. (+3 more)

Cancer detection using AI methods and image databases faces several challenges, including limited and imbalanced image datasets, long algorithm training times, and high-complexity feature generation. In particular, many cancer image datasets contain very few malignant examples compared to benign ones, limiting the amount of minority-class data available for training. Although classical AI and especially deep learning methods have had some success in providing cancer diagnostics, Quantum Machine Learning (QML) methods may offer advantages through high-dimensional feature representations, generative data augmentation, and the exploitation of feature correlations through quantum properties such as entanglement. In this paper, we propose using a generative QML method on the compressed breast ultrasound image dataset (BUSI) to generate synthetic images and hence augmented datasets for classifier training. In particular, we train a convolutional neural network with real, synthetic, and a combination of real and synthetic images. We test the algorithm on real images and provide comparative results for all three training cases. Our results show that the classifier trained entirely on synthetic images achieved an AUC-ROC of 0.802 (80.2%), compared with 0.760 (76.0%) for the real-data baseline.
Read the full abstract

All authors Sophia Knutson, Nandika Goyal, Gennaro De Luca, Glen Uehara, Tanay Kamlesh Patel, Andreas Spanias

Image and Video Processing · Cross-listed from eess.SP

Efficient Multimodal Beam Prediction for Real-World 6G UAV Communications via Image-Position-Synthetic Fusion

Rifat Bin Reza, Md Ahadul Haque Suvo, Jia Uddin et al. (+1 more)

Camera-based beam selection can reduce the need for exhaustive channel measurement in highly mobile millimeter-wave UAV links, but the complementary value of visual and position-derived representations and their behavior under modality degradation require further investigation. This paper develops a staged multimodal framework for current-beam prediction using real-world UAV measurements. The proposed 32-beam classifier combines an ImageNet-pretrained EfficientNet-B3 visual encoder with two deterministic position-derived representations: an explicit Geometric-Kinematic vector and a nonlinear synthetic feature basis constructed from position. The image and synthetic representations are fused first, followed by late fusion with the geometry representation. Experiments on DeepSense 6G Scenario 23 show that Image-Synthetic fusion achieves 86.92% Top-1, 99.03% Top-3, and 99.56% Top-5 accuracy, improving Top-1 performance by 2.64 percentage points over image-only inference and outperforming the 60.71% Top-1 accuracy of synthetic-only inference. Adding the Geometric-Kinematic branch yields the highest measured performance, reaching 87.14% Top-1, 99.12% Top-3, and 99.60% Top-5 accuracy. Controlled input interventions further demonstrate complementary behavior under visual degradation and strong sensitivity to severe synthetic-input corruption. Position-derived nonlinear features provide complementary information to visual context and improve current-beam prediction in the evaluated real-world UAV setting.
Read the full abstract

All authors Rifat Bin Reza, Md Ahadul Haque Suvo, Jia Uddin, Saifur Rahman Sabuj

Friday 9 October

Image and Video Processing

Beyond Gate Weights: Decision-Level Interpretation of Radiomics-Image Feature Fusion for RCC Classification

Yuan Liang, Sourav Bhattacharjee and Abraham Campbell

Gate coefficients describe how a fusion layer weights intermediate features, but they do not necessarily reflect the magnitude of branch contributions at the classifier output. We investigate this distinction in a fixed 3D MedVAE–radiomics model for clear cell versus non-clear cell renal cell carcinoma classification from contrast-enhanced CT. Because the model uses a linear classification head, the binary decision logit can be decomposed exactly into a gated image term, a direct radiomics term, and a bias. We define effective radiomics dominance as the direct radiomics term's share of the summed absolute non-bias branch contributions and compare it with structural radiomics weighting. Internally, mean structural radiomics weighting was 0.823, whereas mean effective radiomics dominance was 0.148, with a modest case-wise association (h̊o} = 0.290). The gated image term retained substantially stronger discriminative ranking information than the direct radiomics term. The same structural-to-effective discrepancy was observed in exploratory external evaluation. Branch-targeted permutation, integrated-gradient, and Shapley-style analyses further identified radiomics descriptors influencing the direct radiomics term. Because radiomics also generates the gate, the gated image term remains radiomics-dependent, so additive branch contribution should not be interpreted as total input influence. These findings show that structural gate weighting and decision-level branch contribution represent distinct aspects of gated feature fusion and should be interpreted separately.
Read the full abstract

Image and Video Processing

Beyond Discrimination: Evaluating Probability Calibration in Radiomics-Image Fusion for Renal Cell Carcinoma Classification

Yuan Liang, Morgan May, Abraham Campbell et al. (+3 more)

Reliable probability estimates are important for the clinical interpretation of machine-learning classifiers, yet radiomics–image fusion models for renal cell carcinoma (RCC) are commonly evaluated primarily by discrimination. Using 396 patients from the KiTS23 dataset, we evaluated whether adding radiomics to four image-model families improved probabilistic performance in addition to discrimination. Seventeen individual models and four probability-average ensembles were assessed using discrimination, probability calibration, and proper scoring measures. Across 12 matched image-only versus fusion comparisons, AUC and AP improved in all 12, while Brier score and negative log likelihood (NLL) improved in 11 of 12 and expected calibration error (ECE) in eight of 12. The 3D MedVAE gated-fusion model achieved the strongest overall individual-model profile. Probability averaging across concatenation, cross-attention, and gated fusion produced favourable point estimates across all five primary metrics relative to image-only models in all four families, although paired-bootstrap support was stronger for Brier score and NLL than for ECE. These findings show that radiomics–image fusion can improve probabilistic prediction quality beyond discrimination, while calibration behaviour varies across image architectures and fusion strategies. We show that our proposed probability-average ensemble models improve discrimination and calibration simultaneously across all architectures in the study.
Read the full abstract

All authors Yuan Liang, Morgan May, Abraham Campbell, Sourav Bhattacharjee, Simon Caton, Pierpaolo Dondio

Image and Video Processing

C$^2$LUE: Clinically Aligned Patient-Level Ultrasound Diagnosis via Evidence-Aware Multi-Instance Learning

Qin-Yi Zhang, Linying Cao, De-Xing Huang et al. (+7 more)

Patient-level ultrasound diagnosis demands not only high predictive accuracy but also clinically aligned mitigation of asymmetric diagnostic risks. While Multi-Instance Learning (MIL) effectively models routine multi-view exams, standard aggregators often dilute heterogeneous diagnostic cues and optimize solely for global accuracy. This frequently yields undesirable sensitivity-specificity trade-offs. To bridge this gap, we propose C$^2$LUE, a novel MIL framework coupling evidence aware aggregation with risk-aligned regularization. To preserve evidence heterogeneity and prevent unimodal attention collapse, C$^2$LUE employs a multi-query cross-attention mechanism built upon a LoRA-adapted DINOv3 backbone. Multiple latent queries explicitly extract and disentangle diverse morphological patterns from the ultrasound sequence. Furthermore, to improve clinical risk alignment, we introduce a dynamic trade-off regularization objective that penalizes sensitivity-specificity imbalance during training. Rather than leaving the clinical trade-off entirely to post-hoc threshold adjustment, this objective encourages the learned score distribution toward a more balanced operating region. Extensive evaluations on patient-level, biopsy-proven thyroid and breast ultrasound datasets demonstrate that C$^2$LUE consistently outperforms prior alternatives. By uniting strong discrimination with a balanced error profile, C$^2$LUE provides a promising solution for real-world ultrasound screening and diagnosis.
Read the full abstract

All authors Qin-Yi Zhang, Linying Cao, De-Xing Huang, Haichuan Lin, Yi Yang, Manzhi Wang, Xiao-Hu Zhou, Chen Chen, Shuangyi Wang, Zeng-Guang Hou

Image and Video Processing

Frozen Site-Probe Accuracy Is Not a Standalone Harmonization Score: An Audit of Evaluation Practice for Multi-Site MRI

Muhammad Muqsit Islam, Gloria García Cuenco, José M. Martínez et al. (+1 more)

Image-level MRI harmonization is often evaluated with a frozen site probe, a site classifier trained on raw scans and applied unchanged to harmonized ones: a drop in its accuracy is read as removal of site effects. We audit this practice on ABIDE I (1,112 T1-weighted scans, 17 sites) with ten output sets from seven harmonization methods and controls for chance, head shape, intensity, preprocessing and target alignment. We find that this verdict depends on how the probe is trained, even among probes that classify raw scans well: across random seeds, its balanced accuracy on the same CycleGAN outputs ranges from 0.13 to 0.59 (chance: 0.06). Low frozen-probe accuracy can also arise without harmonization: a Gaussian blur, which increases the intensity distance to the target site, lowers this accuracy to chance. Moreover, where the frozen probe fails, probes trained on the harmonized scans still recover the site, even from intensity histograms alone: on HACA3 outputs, the frozen probe scores 0.25, a probe trained on those outputs 0.75, and one using only their intensity histograms 0.71. Low frozen-probe accuracy therefore does not show that site information has been removed. We recommend reporting several probes trained to convergence, probes trained on harmonized scans, and target alignment.
Read the full abstract

All authors Muhammad Muqsit Islam, Gloria García Cuenco, José M. Martínez, Pierrick Coupé

2610.14140 · 9 Oct · 4 pages, 2 figures, 1 table. Submitted to IEEE ISBI 2027. Code and results: https://github.com/muqsitamir/HarmonIt. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Abstract PDF $ALIGN ×2

Image and Video Processing

Exploring the Latent Structure of AMD Severity Using Retinal Foundation Models

Niccolò Fineschi, Jovana Panic, Atsushi Watanabe et al. (+7 more)

Retinal foundation models have demonstrated strong performance across a wide range of ophthalmic tasks, yet relatively little is known about how disease-related information is represented within their learned embeddings and whether this distribution influences downstream prediction performance. In this work, we use Age-related Macular Degeneration (AMD) as a case study to investigate how disease severity is represented within the embedding space of RET-CLIP, a retinal vision-language foundation model, using bilateral fundus images from the AREDS dataset, benchmarked against non-retinal encoders. We first characterize the distribution of AMD severity within the learned representation space and then evaluate how this structure influences classification performance across three clinically relevant task formulations of increasing granularity. Finally, we analyze the most influential latent dimensions combining feature attribution, statistical phenotype association, and qualitative image analyses. Our results show that RET-CLIP represents AMD severity as a continuous spectrum rather than a set of discrete categories, with healthy and advanced disease stages exhibiting clearer separation than intermediate ones. This organization was reflected in downstream performance, which decreased from a balanced accuracy of 0.844 and 0.814 for binary AMD and Late-AMD detection respectively to 0.446 for six-class grading. Furthermore, the most influential latent dimensions were statistically associated to distinct retinal phenotypes, including drusen accumulation, pigmentary alterations, and advanced retinal lesions. These findings suggest that the structure of retinal foundation-model representations influences the difficulty of clinically relevant prediction tasks and demonstrate the potential of representation-level analysis for understanding disease organization and improving interpretability.
Read the full abstract

All authors Niccolò Fineschi, Jovana Panic, Atsushi Watanabe, Hideaki Hayashi, Kazuya Nishimura, Yuki Maeno, Samanta Rosati, Gabriella Balestra, Kohji Nishida, Hajime Nagahara

Image and Video Processing

A Miniaturized Wi-Fi HaLow-Based Wireless Imaging Transfer System for Capsule Endoscopy

Michele Bertozzi, Lorenzo Amelio, Sebastiano Strangio et al. (+5 more)

Wireless capsule endoscopy is constrained by severe limitations in size, available power, and reliable data transmission through biological tissues. Although recent advances in miniature imaging sensors have improved image quality, the wireless transmission of color images at practical frame rates remains a challenging task under the tight capsule constraints. We present the design and the experimental validation of a miniaturized Wireless Imaging Transfer (WIT) system for capsule endoscopy based on a sub-GHz IEEE 802.11ah (Wi-Fi HaLow) wireless link. In addition, a preliminary in-vivo validation of Wi-Fi HaLow communication is reported, showing that a stable through-body wireless link can be established across biological tissues in a clinically relevant porcine model. These in-vivo results provide direct experimental support for the adoption of sub-GHz Wi-Fi HaLow for capsule endoscopy applications. Experimental results demonstrate stable wireless image transmission at a resolution of 320x320 pixels, with measured end-to-end data rates above 12 Mbps and a transmission energy efficiency of 20.7 nJ/bit. We also report a detailed power consumption analysis, with a breakdown of the contributions from the imaging sensor, illumination, digital processing, and RF transmission. When all active subsystem contributions are considered, including the camera, digital processing, RF transmission, and illumination at 60% LED brightness, the total power consumption is approximately 477 mW, corresponding to a complete-chain energy consumption of approximately 39.6 nJ/bit. Overall, the measured performance confirms that sub-GHz Wi-Fi HaLow provides a promising trade-off among throughput, robustness, and energy efficiency for wireless imaging in capsule endoscopy.
Read the full abstract

All authors Michele Bertozzi, Lorenzo Amelio, Sebastiano Strangio, Nikita J. Greenidge, Benjamin Calmé, Pietro Valdastri, Giuseppe Iannaccone Fellow, Alessandro Catania

Image and Video Processing

Can Multimodal Decision Models Classify Breast Ultrasound Images? Benchmarking CLEF and CLEF-Flash Across Prompts and Datasets

Tran Cao Minh, Hai Tran and Ha Minh Tan

Multimodal decision models return probabilities over predefined alternatives, offering a structured interface for image classification. This study benchmarks inference with CLEF and CLEF-Flash on 2818 breast ultrasound images from Dataset B, BUSI, and BUS-BRA. Dataset B and BUS-BRA use binary benign-malignant classification, whereas BUSI includes normal, benign, and malignant categories. Model parameters remain unchanged, and original images are submitted without user-defined preprocessing. Three prompts provide minimal labels, an explicit classification task, or sonographic criteria. We report accuracy, macro F1, balanced accuracy, macro precision, throughput, and latency. CLEF-Flash with the clinical prompt achieves the highest accuracy on all three datasets: 74.85% on Dataset B, 66.28% on BUSI, and 51.63% on BUS-BRA. On BUSI, however, CLEF with the task-guided prompt achieves the highest macro F1 (64.51%) and balanced accuracy (68.72%). CLEF-Flash has lower latency in every matched comparison. Clinical instructions improve CLEF-Flash accuracy relative to the minimal prompt but reduce CLEF accuracy on all datasets. Confusion matrices reveal frequent benign-to-malignant errors, particularly on BUS-BRA, where all configurations fall below the 67.63% accuracy of a majority-class predictor. These findings demonstrate dataset-and prompt-dependent performance and motivate class-specific evaluation before applying multimodal decision models to breast ultrasound analysis.
Read the full abstract

Image and Video Processing

Model-agnostic Knowledge Augmented Machine Learning for Weakly Annotated Bioimaging Data

Jayanie Bogahawatte, Sachith Seneviratne, Simon Maksour et al. (+6 more)

Machine learning models are increasingly used for automated feature extraction from weakly annotated bioimaging data, such as microscopy and cytomorphology images. Due to the sole dependency of automated feature extraction, existing methods yield inconsistent classification performance across weakly annotated datasets. Here, we propose a model-agnostic knowledge augmentation method that enables ML models to exploit domain knowledge beyond what automated feature extraction can capture. The proposed approach improves classification performance across seven ML architectures. When integrated with masked autoencoder-based self-supervised learning, our method achieves the strongest overall performance, improving precision by 2.9% over the next-best method on three in-house microscopy datasets and by an average of 6.8% across three public cytomorphology benchmarks. We perform nucleus-level classification based on mislocalization of TAR DNA-binding protein 43 (TDP-43) from the nucleus to the cytoplasm, a phenotype linked to Amyotrophic Lateral Sclerosis (ALS), using three independent microscopy datasets of motor neurons derived from human induced pluripotent stem cells. Accurate quantification of this phenotype at scale remains challenging and may contribute to inconsistencies across studies. Application of the best-performing model to these microscopy datasets reveals biological interpretations related to ALS, including dose-dependent effects of the proteasomal stressor, MG132, and the oxidative stressor, sodium arsenite (SA), on TDP-43 mislocalization, variable pathology in motor neurons from sporadic and familial ALS donors, and a lack of protection of the pharmacological compounds, edaravone and niclosamide, against SA-induced TDP-43 mislocalization. Together, our pipeline enables in-vitro cellular image analysis to advance understanding of ALS-associated pathology and potential treatments.
Read the full abstract

All authors Jayanie Bogahawatte, Sachith Seneviratne, Simon Maksour, Dzung Do-Ha, Elizabeth Qian, Samuel Brennan, Matthew Keon, Lezanne Ooi, Saman Halgamuge

Image and Video Processing

Desertification Forecasting in Semi-Arid Regions: Using Deep Learning to Probe the Observability Limits of Land Degradation Dynamics

Filipe Moura Da Silva and Cláubio Landney Lima Bandeira

Forecasting desertification has received far less attention than mapping it. We use a convolutional LSTM (ConvLSTM) trained on ten annual Earth Observation (EO) variables (2000-2023) over the semi-arid state of Ceará, Brazil, on a 256x206 grid at roughly 2.2 km resolution, to measure how much of the year-to-year evolution of a MapBiomas-derived degradation index (IDV) can be predicted from satellite data. The network predicts a bounded change on top of the last observed state, is trained in two stages (single-step, then recursive rollout) and is evaluated on a contiguous spatial and temporal holdout. State-wide mean IDV rose from 0.238 to 0.350 between 2000 and 2023, about 0.005 per year, but cell-level change is dominated by year-to-year jumps, and a gradient-boosting model given all channels explains only 4.7% of its variance. Against this ceiling, the ConvLSTM lowers RMSE relative to persistence by 1.6% one year ahead and 4.4% four years ahead in a held-out southern band (p < 0.001), beats persistence with a constant drift and reproduces 53-69% of the regional drift. Drift-aware baselines only help from the third year onward, by up to 4.4% in RMSE at six years. An earlier formulation that redrew the whole field each year predicted changes unrelated to the observed ones (correlation -0.06) and produced a spurious decline when run recursively. Channel knockouts suggest that elevation and EVI degrade the forecast, while slope and aspect have no measurable effect. Monte Carlo dropout and stochastic scenario maps convey the uncertainty that the expected-value forecast hides. Under climatological forcing, mean IDV rises from 0.350 in 2023 to 0.370 in 2030. EO-based desertification forecasting is therefore reliable for the regional trend, not for individual cells.
Read the full abstract

Image and Video Processing

Radiomics-Based Thyroid Nodule Classification with Label-Conditional Conformal Prediction

Laura Testa, Pierpaolo Brutti and Umberto Ferraro Petrillo

Thyroid nodule malignancy assessment relies on ultrasound imaging, yet current risk stratification systems suffer from substantial interobserver variability, contributing to a high rate of unnecessary biopsies. We propose a radiomics-based framework combining quantitative feature extraction via PyRadiomics with a Generalized Additive Model (GAM) classifier, selected among six candidates for its favorable trade-off between discriminative performance and interpretability. To complement point predictions with uncertainty estimates, Cross-Conformal Prediction with label-conditional guarantees is applied, flagging ambiguous cases for clinical review rather than forcing a binary decision. Experiments are conducted on TN5000, a publicly available ultrasound dataset of thyroid nodules. After Spearman correlation filtering, 36 features are retained and the GAM achieves an AUC of $0.89$, a sensitivity of $0.83$, and a specificity of $0.82$ on the validation set. At a significance level $\epsilon=0.09$, selected to ensure empirical coverage guarantees on both classes, 63% of nodules on the test set receive a certain (i.e., single-label) prediction, with conditional precision of $0.94$ for certain malignant and conditional Negative Predictive Value (NPV) of $0.73$ for certain benign cases, while the remaining 37% are referred to the clinician.
Read the full abstract