Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Thursday 17 September

Image and Video Processing

Uncertainty-driven training for three-dimensional calibrated lung nodule classification

Giuseppe Tripodi, Alessandro De Rosis and Saleh Rezaeiravesh

In this work, we present an uncertainty-driven training framework for three-dimensional computed tomography (CT) lung nodule classification, where validation-based uncertainty estimates guide loss reweighting to enhance predictive performance and probability calibration. Two Uncertainty Quantification (UQ) methods are considered: Monte Carlo Dropout (MCD) and Evidential Deep Learning (EDL). Both provide per-class uncertainty estimates that modulate the loss and encourage focus on hard or unreliable classes. The framework is evaluated with ResNet, DenseNet, EfficientNet, Vision Transformer (ViT), and Swin Transformer backbones on two datasets: the clinical LIDC-IDRI cohort and the NoduleMNIST3D benchmark. Uncertainty-driven training achieves classification performance similar to conventional training while substantially improving calibration, with an expected calibration error (ECE) reduced by up to 65% on LIDC-IDRI. EDL attains competitive performance on shallower architectures with single-pass inference, whereas MCD is more robust on deeper networks. Analysis across architectural families reveals that uncertainty-driven training benefits convolutional backbones more consistently than transformer-based architectures: EDL in particular degrades on ViT, suggesting that the Dirichlet evidence parameterisation may interact unfavourably with attention-based architectures at lower input resolutions. A posteriori temperature scaling proves highly effective across all configurations, indicating that a simple scalar calibration can be competitive even without explicit uncertainty-aware training. Our results indicate that integrating UQ into the training loop can significantly improve probabilistic calibration and support more trustworthy deployment of three-dimensional medical imaging models.
Read the full abstract

Image and Video Processing

Synthetic Fingerprints for Children Under Four: Generation and Biometric Evaluation

Faheem Ahmad, Ajan Ahmed, Mst Rumana Sumi et al. (+2 more)

Fingerprint recognition in children under four is of interest for longitudinal identity applications, but research in this age range is constrained by the limited availability and sensitivity of real fingerprint data. Synthetic data may provide a useful complementary resource if generated samples are carefully evaluated for biometric quality, similarity to the real training data, and identity diversity. This paper presents an evaluation and selection protocol for synthetic fingerprints generated from a small pediatric dataset. The protocol was applied to 32,000 candidates produced by an age-conditioned generator fine-tuned with 205 fingerprints from nine children. Candidates were evaluated using NFIQ 2, NBIS minutiae extrac-tion and matching, fingerprint-pattern classification, similarity to the complete real reference set, and pairwise similarity among retained synthetic samples. After candidate filtering and a final symmetric pairwise verification, 1,985 synthetic fingerprints remained. The youngest age group continued to produce retained samples within the sampling budget, whereas the oldest original age group produced 24 retained prints. A data-derived two-group age representation improved pattern-distribution agreement for the younger group. Repeated renderings of fixed synthetic iden-tities produced minutiae-based mated scores comparable to the real mated scores, although a DINOv2 texture representation showed substantially greater within-generator similarity. The results indicate that synthetic pediatric fingerprints can support controlled research use, but their evaluation should include both real-to-synthetic similarity and synthetic-to-synthetic diversity, and conclusions should remain specific to the matchers and measurements used.
Read the full abstract

All authors Faheem Ahmad, Ajan Ahmed, Mst Rumana Sumi, Stephanie Schuckers, Masudul Imtiaz

Image and Video Processing · Cross-listed from cs.RO

CoRef-GS: Cooperative Referring Gaussian Splatting for Multi-Agent Scene Understanding

Zhikun Zhou, Kunyu Peng, Runyi Yang et al. (+7 more)

Referring scene understanding for embodied robots requires grounding object- and relation-centric language queries from a designated viewpoint. While a local semantic Gaussian map can support such grounding within one agent's observations, cooperative settings require this ability to remain effective after independently reconstructed maps are aligned and fused. In this setting, the referred target or its contextual landmark may come from another agent's observations, while spatial relations must still be interpreted from the querying robot's viewpoint. We formulate this problem as cooperative referring Gaussian grounding over fused maps, which requires geometric alignability, instance-level semantic comparability, and view-conditioned relation reasoning. Existing language-aware Gaussian methods mainly focus on single-map querying, whereas Gaussian registration methods optimize geometric or photometric alignment without preserving language-grounding-oriented semantic compatibility. We propose CoRef-GS, a cooperative referring Gaussian splatting framework. CoRef-GS constructs local open-vocabulary instance-aware Gaussian maps, then aligns partially overlapping maps with a cross-agent alignment module by geometric and semantic consistency, and grounds queries using a view-conditioned mask relation graph. We further introduce CoQuad-Ref, a dual-quadruped benchmark spanning both real-world and simulated indoor scenes. Experiments show that, on simulated scenes, CoRef-GS reduces the rotation error from 2.58{\deg} after coarse initialization to 0.15{\deg} after refinement, and improves real-world referring mIoU over ReferSplat from 52.6% to 68.8%. The established benchmark and source code will be publicly released at https://github.com/ruojiruoli17/CoRef-GS.git.
Read the full abstract

All authors Zhikun Zhou, Kunyu Peng, Runyi Yang, Junhao Cai, Di Wen, Ruiping Liu, Danda Pani Paudel, Yi Zhou, Luc Van Gool, Kailun Yang

Image and Video Processing · Cross-listed from cs.RO

OmniMimic: Dynamics-completed Motion Augmentation for Multi-style Omnidirectional Quadruped Locomotion

Sheng Wu, Guoqiang Zhao, Zhe Yang et al. (+7 more)

Animal demonstrations provide quadruped robots with natural and distinctive gait styles that are difficult to specify through hand-crafted rewards. However, their narrow directional coverage leaves little style-consistent supervision for backward, lateral, and turning commands. We present OmniMimic, a training framework that turns directionally limited animal demonstrations into a single multi-gait policy over target per-axis velocity ranges. OmniMimic first combines temporal reversal, constrained dynamics completion, and sagittal reflection to construct robot-specific kinematic and physical supervision beyond the observed directions. It then expands commands progressively from the demonstrated velocity distribution toward the target per-axis bounds, and uses a shared actor with soft-gated, gait-specialized residual experts to balance reusable locomotion skills with gait-specific corrections. Across four gaits in simulation, OmniMimic reduces mean foot-position RMSE at forward and backward reference velocities by 12.9% and velocity-tracking RMSE on a uniform Cartesian command grid by 63.1%, compared with the matched APEX baseline. The project page is at https://OmniMimic.github.io.
Read the full abstract

All authors Sheng Wu, Guoqiang Zhao, Zhe Yang, Fei Teng, Zhikun Zhou, Yanlin Yang, Zheng Fang, Hong Zheng, Yaonan Wang, Kailun Yang

Image and Video Processing

Unifying Image Quality Assessment Datasets: MOSAIQ-500K and MOSAIQ-Bench

Wenbo Yang, Zhongling Wang, Jialu Xu et al. (+2 more)

Image quality assessment (IQA) datasets use different subjective protocols and rating scales, so their scores are not directly comparable. The lack of a common perceptual scale hinders multi-dataset training and precludes direct inter-dataset evaluation. We address this by conducting a new subjective experiment and using its ratings as perceptual anchors to fit monotonic mappings that place the existing scores of 23 IQA datasets on a common quality scale while preserving within-dataset rankings. The resulting dataset, MOSAIQ-500K, contains over 500,000 images and is, to our knowledge, the largest IQA dataset with perceptually aligned subjective scores. We also propose MOSAIQ-Bench, an inter-dataset benchmark, and use it to evaluate 31 IQA methods, revealing substantial gaps between intra- and inter-dataset performance, particularly on authentic distortions. The aligned scores offer a key advantage: replacing specialized multi-dataset training mechanisms with standard regression losses yields comparable intra-dataset accuracy and better inter-dataset performance. MOSAIQ thus provides a practical foundation for combining heterogeneous IQA datasets to train more generalizable IQA models. Code and dataset will be made public at ivc.uwaterloo.ca/projects/unifying_iqa_datasets/ upon acceptance.
Read the full abstract

All authors Wenbo Yang, Zhongling Wang, Jialu Xu, Jinghan Zhou, Zhou Wang

Image and Video Processing

Effects of Sequence Timing on the Spatio-Temporal Properties of 3D BOLD fMRI: A Formal Framework and Analysis

Samuel Bianchi and Klaas P. Pruessmann

3D sequences provide an alternative to 2D or multiband sequences for BOLD fMRI. The impact of acquisition time differences between slices is well understood for 2D or multiband sequences. For 3D sequences, k-space is partitioned into multiple segments and the final image depends on samples taken over an extended period of time. Any precise timing information is lost during reconstruction. A theoretical description of how sequence timing impacts 3D BOLD fMRI data properties is lacking in the literature. We present a formal framework that models sequence timing effects and draw connections to existing literature. The framework rests on the statistical description of a spin system that allows incorporation of BOLD signal changes and is completed by a general description of spatial encoding and image reconstruction. Using this formulation, we define three key images: the actual image reconstructed from a segmented 3D acquisition, an optimal reference image unaffected by sequence timing, and an error image representing their difference. Two sets of operators are defined that describe the generation of these images and allow us to analyse the effects of sequence timing independent of the object. All operators take the form of spatio-temporal filters. They mix signal content spatially and based on its temporal waveform. High-frequency BOLD signal content is affected more strongly compared to low-frequency content. Simulations demonstrate that while the total variance of 3D image time series can be reduced, spatio-temporal fidelity is increasingly lost. Furthermore, we hypothesize that the heightened sensitivity of 3D sequences to physiological noise (compared to 2D) is caused by sequence timing damping thermal noise more strongly than physiological fluctuations. The impact of motion and system instabilities on image time series needs to be studied further.
Read the full abstract

Image and Video Processing

Video Based Assessment of Surgical Skills Using Frozen Pretrained Video Foundation Models

Sangrock Lee, FNU Rahul and Suvranu De

Automated video-based surgical skill assessment has advanced rapidly, yet rigorous evaluation of continuous standardized score prediction under participant-level generalization to unseen trainees remains limited. We introduce VBA-Net+, a video-only framework for Fundamentals of Laparoscopic Surgery (FLS) score regression and pass-fail classification using pretrained video foundation models as frozen feature extractors. We evaluate two FLS datasets, suturing and pattern cutting, using VideoPrism, V-JEPA2, and VideoMAE v2, with a frame-level SimCLR baseline. A lightweight fully convolutional head is trained on embeddings offline and evaluated using participant-level leave-one-user-out (LOUO) cross-validation within the standardized assessment protocol. For continuous score prediction, the best representation achieves $R^2$=0.6367 for suturing and 0.9261 for pattern cutting. For pass-fail classification at official FLS thresholds, area under the receiver operating characteristic curve (AUC) reaches 0.9073 and 0.9906, respectively. Frozen video-encoder pipelines generally outperformed the frame-level SimCLR pipeline, particularly for suturing, providing a benchmark for video-only FLS assessment.
Read the full abstract

Image and Video Processing

HyperAMS-Net: Adaptive Multi-Scale Spatial Hypergraph Network for Brain Disorder Classification

Proloy Kumar Mondal, Md Kamran Hussin Chowdhury and Hoi Leong Lee

Accurate classification of brain disorders from neuroimaging data remains challenging because of substantial inter-subject heterogeneity and the complex multi-scale patterns present in functional connectivity and morphological representations. To address these challenges, we propose HyperAMS-Net, a deep learning framework for brain disorder classification using neuroimaging representations derived from resting-state functional MRI or structural MRI. HyperAMS-Net integrates adaptive multi-scale convolution, hypergraph attention, spatial-channel attention, and adaptive feature fusion. Specifically, adaptive multi-scale convolution learns data-driven weights over multiple receptive fields to capture complementary patterns at different scales. Hypergraph attention models higher-order dependencies among learned feature representations through node–hyperedge–node message passing, while spatial-channel attention enhances discriminative feature learning. Adaptive feature fusion further aggregates complementary information across parallel network branches. HyperAMS-Net is evaluated on three benchmark datasets spanning distinct brain disorders: ABIDE for autism spectrum disorder, REST-meta-MDD for major depressive disorder, and ADNI for Alzheimer's disease, using 5-fold stratified cross-validation. HyperAMS-Net achieves state-of-the-art performance across all evaluated datasets, attaining the highest accuracy and AUC among the compared methods. Ablation studies further demonstrate the contribution of each proposed component, with the largest performance degradation observed when hypergraph attention is removed.
Read the full abstract

Image and Video Processing

The segmentation ceiling: why explicit left-ventricular masks do not improve learned ejection-fraction regression

Farshid Farhadi Khouzani, Paul La Plante, Bryar Mustafa Shareef et al. (+1 more)

Accurate estimation of left ventricular ejection fraction (EF) from echocardiography is central to cardiovascular care, and deep learning enables automated EF prediction from echocardiographic video. Because EF is clinically derived from left-ventricular (LV) volumes, a widely held intuition is that explicit LV segmentation should improve prediction. We introduce a quantitative criterion, the segmentation ceiling, that makes this testable: from EF as a normalized difference of end-diastolic and end-systolic volumes, we derive in closed form how per-frame segmentation area error propagates into EF error, and thus the accuracy a mask must reach before it can improve on direct regression. Using EchoNet-Dynamic, a UniFormer-S backbone, and the empirically measured within-patient error correlation, the criterion places the break-even near 10% per-frame area error, whereas a representative segmenter operates at roughly 14%, above the ceiling. Consistent with this, four strategies for injecting segmentation or area information (a predicted-mask channel, end-diastolic/end-systolic clip sampling, and per-bin and amplitude area-consistency objectives) fail to beat a raw-video baseline; ground-truth masks help only through label leakage. Input representation thus not being the limit, we identify generalization as the practical lever: weight averaging with strong augmentation attains a test R^2 of 0.806 (MAE 4.08) under a matched dense-clip protocol, comparable to an R(2+1)D baseline (0.811) while tightening the validation-to-test gap. Finally, a heteroscedastic beta-NLL formulation yields informative, well-calibrated per-prediction uncertainty, larger for clinically harder low-EF cases, where Monte-Carlo dropout does not. The segmentation ceiling gives a concrete design criterion for when mask-guided EF estimation is worthwhile, plus a simple, uncertainty-aware recipe for EF regression.
Read the full abstract

All authors Farshid Farhadi Khouzani, Paul La Plante, Bryar Mustafa Shareef, Laxmi Gewali

Image and Video Processing

Compression Hurts, Pooling Helps: Information Loss in Rayleigh-Scale Estimation from B-Mode Ultrasound

D. Hudson Smith and Ahmer Raza

Clinical B-mode images are widely available as potential data sources for quantitative ultrasound (QUS) analysis for tissue characterization. However, standard clinical ultrasound devices apply unknown log-compression to RF envelope data before display and storage. Previous work has demonstrated estimation of the underlying RF envelope statistics in the presence of an unknown compression law. Using Fisher information analysis, we show that finite-offset log compression causes severe information loss when estimating the Rayleigh scale $\sigma$, which controls diffuse speckle. For a single image window, unknown compression raises the minimum achievable variance for unbiased estimation of $\sigma$ by a compression-independent factor of approximately 34.98. When $M$ equal-sized windows share the same unknown compression settings, the excess variance decays as $1/M$; even in the most favorable regime, reducing the variance inflation factor below 1.1 requires 340 windows. Our analysis treats the contrast parameter $a$ as unknown and the boundary offset $b$ as known; estimating $b$ experimentally shows even larger variance. We validate this theory using synthetic estimation experiments and demonstrate RF-scale recovery on real RF-envelope windows from the OASBUD dataset. Together, these results clarify the limitations of using routine B-mode images for QUS.
Read the full abstract

Wednesday 16 September

Image and Video Processing

Semantic-Guided Fusion Network for Multi-Source Remote Sensing Image Classification

Yuwei Zhao, Chuanzheng Gong, Baogui Huan et al. (+3 more)

Multi-source remote sensing image classification has attracted increasing attention due to the complementary spectral, structural, and geometric information. However, existing methods still suffer from two limitations: insufficient semantic contextual modeling and unreliable feature fusion caused by slight spatial misalignment. To address these issues, we propose a Semantic-Guided Fusion Network (SGFNet) for multi-source remote sensing image classification. Specifically, the Semantic Mixing Convolution Block (SMCB) is designed to dynamically generate semantic-aware convolution kernels according to contextual relationships among feature representations. In addition, the Frequency Modulated Fusion Block (FMFB) is introduced to perform cross-modal interaction in the frequency domain, which effectively alleviates the influence of slight spatial misalignment and improves complementary information fusion. Extensive experiments conducted on the Augsburg and Houston 2018 datasets demonstrate that the proposed SGFNet consistently outperforms several state-of-the-art methods. The codes are publicly available at https://github.com/oucailab/SGFNet .
Read the full abstract

All authors Yuwei Zhao, Chuanzheng Gong, Baogui Huan, Feng Gao, Junyu Dong, Qian Du

Image and Video Processing

Mammography Foundation Models for Opportunistic Prediction of Major Adverse Cardiovascular Events

Paula Feldman, Nusrat Binta Nizam, Sunwoo Kwak et al. (+3 more)

Cardiovascular disease (CVD) remains the leading cause of death among women, yet cardiovascular risk assessment often relies on clinical variables that may be missing, outdated, or unavailable in routine care. Screening mammography offers an opportunity for opportunistic cardiovascular risk stratification because it is routinely acquired and contains vascular features, including breast arterial calcifications (BAC), that are associated with cardiovascular risk and events. We evaluate whether mammography specific foundation models, originally pretrained for breast cancer-related tasks, can transfer to cardiovascular risk prediction without cardiovascular specific supervision or explicit BAC annotation. We constructed a 5-year major adverse cardiovascular event (MACE) cohort of 22,497 women linked to electronic health record outcomes, including 500 events (2.22% prevalence). The foundation models achieved AUROCs of 0.823 and 0.822 substantially exceeding an age-only model (AUROC 0.765), despite using only the screening mammogram as input, with no clinical variables. Both foundation models evaluated assigned substantially higher predicted risk to patients with radiologist-documented BAC, despite BAC never being used as a training label, and showed activation patterns consistent with vascular findings. Together, these findings suggest that mammography foundation models can recover clinically relevant cardiovascular risk information directly from mammographic pixels and suggest that screening mammography may provide an opportunistic source of cardiovascular risk information to complement conventional clinical assessment without additional imaging. Code is available in https://github.com/PauFeld/MammoCVD
Read the full abstract

All authors Paula Feldman, Nusrat Binta Nizam, Sunwoo Kwak, Batuhan Karaman, Katerina Dodelzon, Mert Sabuncu

Image and Video Processing

NIMARC-MRI: Abdominal HASTE Dataset and a Baseline U-Net Exposing the Synthetic-to-Real Gap in Low-Resource Motion Correction

Abdulrazaq A. Zubair, Nafiu Musa Muhammad, Simeon Krah et al. (+7 more)

Respiratory motion degrades abdominal T2 HASTE MRI in low- and middle-income countries where vendor motion-correction licenses are unavailable and failed scans are deleted during routine PACS cleanup, precluding supervised training. To address this, we introduce NIMARC-MRI, the first public abdominal MRI dataset from West Africa, comprising 139 clean HASTE, 25 native motion-degraded HASTE, and 79 paired HASTE-TSE TRIGGER acquisitions from a Nigerian centre operating 1.5 T Siemens scanner without integrated motion-correction licenses. A 2D U-Net with 7.7 million parameters was trained on 3D-consistent synthetic respiratory motion necessitated by local infrastructure reality and validated via a three-tier strategy: held-out synthetic data, blinded radiologist Likert scoring on native real motion, and cross-sequence TRIGGER generalisation. On synthetic test data the model achieved SSIM 0.863 +/- 0.042 and PSNR 30.06 +/- 1.80 dB. On native real motion, however, blinded radiologist and radiographer scores showed no significant improvement (mean Likert 4.06 +/- 0.55 original versus 4.04 +/- 0.61 corrected, P = 0.914), with 28% of cases rated worse after correction. Cross-sequence TRIGGER evaluation showed modest SSIM improvement (0.404 +/- 0.073 versus 0.334 +/- 0.055, P < 0.001) without radiologist-perceived gain. These findings demonstrate that conservative synthetic motion fails to capture clinical motion severity, exposing a reproducible synthetic-to-real gap. NIMARC-MRI is released on Zenodo (https://doi.org) under a controlled data-use agreement requiring citation. Sequence metadata and a sample subset are publicly accessible to facilitate discovery, while patient-level data remain restricted to approved collaborators. The aim is to establish a reproducible baseline for motion correction in resource-constrained settings.
Read the full abstract

All authors Abdulrazaq A. Zubair, Nafiu Musa Muhammad, Simeon Krah, Ummasalma Usman Ibrahim, Yusuf Tijjani Abdumumin, Ismail Ismail Tijjani, Ummulkhairi Ibrahim, Alyasaa Anas, Mubaraq Yakubu, Abbas Rabiu Muhammad

Image and Video Processing

Physics-Informed Hemodynamic Modeling for Data-Free Prediction and Sparse-Data Assimilation

Xi Chen, Jianchuan Yang, Hongde Li et al. (+5 more)

Clinical decision-making for coronary intervention relies mainly on angiography and fractional flow reserve (FFR). However, angiography is two-dimensional and lacks depth information for 3D lesion characterization, while FFR provides only a single functional index, offering limited hemodynamic insight. Among existing methods, numerical analysis is computationally expensive, whereas learning-based approaches require extensive supervision and often lack physical consistency. To address these limitations, we propose physics-informed hemodynamic modeling, an integrated deep learning framework for 3D coronary blood flow analysis from dual-view angiography. First, an attention-enhanced CNN reconstructs coronary geometry from angiography. The resulting point clouds are then mapped to a reference domain and Fourier-encoded for joint representation. A decoupled network separately predicts velocity and pressure fields, with embedded physical priors enabling efficient transfer across physiological conditions. Across 32 clinical patients evaluated under four flow conditions, the trans-stenotic pressure-drop mean absolute percentage error was 2.02%, while the velocity and pressure relative-L2 errors were 0.054 and 0.023, respectively. Validation against hospital-measured FFR further achieved 93.8% diagnostic accuracy (30/32; exact 95% CI, 79.2%-99.2%). The framework also supports illustrative revascularization comparisons and sparse-data assimilation, with the full angiography-to-hemodynamics pipeline completed within 20 minutes per patient.
Read the full abstract

All authors Xi Chen, Jianchuan Yang, Hongde Li, Guangxin He, Qiuyu Ye, Qiang Luo, Mao Chen, Wenqi Hu

Image and Video Processing

Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers

Emanuele Artioli, Farzad Tashtarian and Christian Timmerer

Traditional codecs treat every region of a frame alike; a generative layer can instead degrade the regions a viewer attends to least and reconstruct them at the client. We present PRESLEY, which extends the prior conference work ELVIS by replacing destructive block removal with adaptive in-place degradation under a removability mask, signaling per-block strength in a bit-packed side channel, and restoring via generative backbones conditioned on transmitted visual priors rather than unconditioned in-painting. We separate the problem into three goals: choosing which blocks to degrade, degrading them so the encoder spends fewer bits, and restoring them. Against its predecessor at matched rate, PRESLEY achieves a decisive mean -56.4% BD-rate reduction on delivered background quality across 13 rate ladders spanning multiple codecs and dataset families. Against pristine baselines, PRESLEY defines the operating regime of generative transport: delivering substantial bitrate savings (up to -29.4% BD-rate) and superior background quality (17/23 sequences) in the target bit-starved regime, while maintaining foreground fidelity bit-exact. We further map where the theoretical headroom in this class of architecture lies. Using an exact leave-one-superblock-out combinatorial oracle as an additive empirical bound, we show that existing complexity heuristics already capture 83.3% of bit-cost savings, bounding remaining cost-axis headroom at about 5% of total bitrate. We then identify and model the primary unaddressed axis – post-restoration damage – which disperses widely (4.9-8.4 dB). We prove that this damage is predictable before transmission (held-out rho = +0.400), establishing the feasibility of transmit-time restorability modeling and defining the roadmap for joint rate-distortion-restoration selection rules.
Read the full abstract

Image and Video Processing

Highly accelerated 3D Cartesian MPnRAGE with implicit neural representation reconstruction

Natascha Niessen, Ana Beatriz Solana, Carolin M. Pirkl et al. (+9 more)

MPnRAGE enables multiple inversion contrast images in a single scan, allowing quantitative T1 mapping, tissue nulled contrasts, and standard MPRAGE synthesis. However, current 3D scan times remain clinically impractical, motivating accelerated 3D MPnRAGE. This work provides a highly accelerated Cartesian 3D MPnRAGE sequence with joint implicit neural representation (INR) reconstruction. The sequence uses a tailored view-ordering strategy, flip angle schedule and complementary variable-density Poisson-disk undersampling. Calibration data acquired during otherwise unused delay time are used for sensitivity map estimation and complementary high-frequency sampling. Ten INR-reconstructed inversion images at 1.5 mm$^3$ are evaluated against fully sampled references via retrospective undersampling. Prospectively accelerated 1 mm$^3$ images at R = 20 (5.39 min) demonstrate clinical feasibility. INR reconstruction outperforms subspace and iterative local low rank reconstruction on highly accelerated data. The proposed highly undersampled 3D Cartesian MPnRAGE with INR reconstruction generates multiple high-quality inversion contrasts in substantially reduced scan time. Scan-specific INR reconstruction improves image quality while reducing reconstruction time versus state-of-the-art methods.
Read the full abstract

All authors Natascha Niessen, Ana Beatriz Solana, Carolin M. Pirkl, Tim Sprenger, Hannah Eichhorn, Veronika Spieker, Wenqi Huang, Rolf F. Schulte, Florian Wiesinger, Tobias C. Wood, Marion I. Menzel, Julia A. Schnabel on behalf of the PREDICTOM consortium

Image and Video Processing

Informed Sinogram Interpolation for Sparse View Reconstruction

Yuejie Liu, Alessandro Lupoli, David Uribe Gallo et al. (+2 more)

Computed tomography (CT) has been widely used in medical examinations and non-destructive testing. Micro-CT (microfocus X-ray CT system) is an advanced version that can observe the internal structures of small objects. However, secondary radiation can prevent micro-CT from imaging the full structure of the object. Given the same amount of secondary scans as primary scans, simple subtraction solves the problem at the price of doubled acquisition time. To reduce the acquisition time, one aims to limit the secondary scans to as few imaging angles as possible and interpolate to recover the missing data. Different interpolation methods have been explored for limited-angle tomography, such as polynomial or spline interpolation, compressive sensing, deep learning, etc. In contrast to many of these scenarios, the secondary imaging setup considered in this paper often exhibits simple sinusoidal structures that are not exploited directly by any of the aforementioned approaches. This paper aims to fill this gap with a continuous mathematical model describing the secondary sinogram only with information from the projection space. Building on this model, we propose an interpolation method that effectively interpolates secondary scans from limited observations.
Read the full abstract

All authors Yuejie Liu, Alessandro Lupoli, David Uribe Gallo, Andreas Fischer, Felix Krahmer

Image and Video Processing

Constrained Color Carrier: Characterization-Preserving Conditional Color Rendering in Multi-Illuminant Camera Profiles

Xilai Liang

In Digital Negative (DNG) multi-illuminant profiles, characterization matrices and nonlinear rendering payloads share condition-dependent interpolation slots, so adding a slot for rendering capacity also introduces an additional characterization state. We introduce Constrained Color Carrier (CCC), which constructs the three pre-serialization ColorMatrix and ForwardMatrix states from the original dual matrix segments while allowing the same shared slots to carry three HueSatMap rendering bases. Condition-specific HueSatMap payloads are solved as numerical preimages of the target through a fixed serialized downstream rendering operator, and carrier selection enforces a preservation bound on the host-effective interpolated characterization. We evaluate CCC on the Sony ILCE-7RM5 Adobe Standard dual-illuminant profile using a white-balance-dependent Standard ColorCorrect target recovered from Phocus 4.0.1 for the Hasselblad X2D 100C. Using Euclidean Oklab error, CCC yields a worst-temperature 95th-percentile preservation error of 0.003857, below the prescribed tolerance of 0.004, and reduces the worst-condition 95th-percentile target error from 0.08208 for the Dual representation to 0.03508. The synthetic Ordinary Triple yields a preservation error of 0.014802 and exceeds the same tolerance. The serialized DNG Camera Profile (DCP) and Extensible Metadata Platform (XMP) artifacts reproduce the final color-table payloads exactly and yield zero numerical difference from solver-side offline execution, showing that the CCC solution is representable within the evaluated serialized profile format.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

Visual Autoregressive Priors for RAW-to-sRGB Image Signal Processing

Tailai Chen, Xiaotong Luo, Yuan Gao et al. (+2 more)

RAW-to-sRGB image signal processing (ISP) must recover perceptually faithful colors and fine details from sensor measurements, often under imperfect spatial alignment and missing camera metadata. This paper presents, to the best of our knowledge, the first application of visual autoregressive (VAR) next-scale prediction over a discrete image codebook to the RAW-to-sRGB ISP task. We adapt a frozen 1.10\,B-parameter VAR backbone for RAW-conditioned ISP with only 32.93\,M trainable parameters (2.99%), and propose a frequency-decomposed color loss that separately supervises low-frequency tone via wavelet LL cosine similarity and chromatic edges via detail-band $\ell_1$. On the Zurich RAW-to-sRGB benchmark, the method improves PSNR-Y from 21.31 to 21.89\,dB and reduces LPIPS from 0.276 to 0.218 on the full 1,204-image test set. Diagnostic experiments show that the VAR prior preserves structure well, but continuous color transfer remains the dominant bottleneck: oracle affine correction recovers 3.8\,dB, while learned color heads yield marginal gains.
Read the full abstract

All authors Tailai Chen, Xiaotong Luo, Yuan Gao, Xin Jin, Wenjun Zeng

Tuesday 15 September

Image and Video Processing

TTTIR: Unlocking Instance-Specific State Evolution via Test-Time Training for Image Restoration

Kaihang Zheng, Jun Li, Hang Guo et al. (+5 more)

Image restoration is inherently challenging due to the diverse and highly input-dependent nature of real-world degradations. While recent architectures like Transformers and state-space models have advanced the field, they predominantly rely on static, globally shared parameters, which struggle to fully accommodate instance-specific degradation patterns. Test-Time Training (TTT) offers a promising paradigm for generating data-dependent operators, yet its standard self-supervised inner loop lacks the explicit guidance required to transition degraded features toward clean structures. To address this, we propose TTTIR, a novel framework that reformulates image restoration as an instance-specific state evolution process. Specifically, we design Progressive State Sequence Generation (PSSG) to construct complementary spatial-frequency target states (defining what to recover), and State Transition Evolution (STE) to adapt lightweight transition operators via a restoration-oriented TTT inner loop (determining how the features should evolve). Extensive experiments demonstrate that TTTIR consistently outperforms state-of-the-art models across multiple image restoration benchmarks, achieving dynamic instance-specific recovery with favorable computational scalability. The code is available at https://github.com/Elysiaaaaaaaa/TTTIR.git.
Read the full abstract

All authors Kaihang Zheng, Jun Li, Hang Guo, Hongyu Chi, Zimo Liu, Tao Dai, Jinpeng Wang, Yaowei Wang

Image and Video Processing · Cross-listed from cs.CV

Yarn tracking of large-scale 3D textile reinforcements using topological material features

Hafsa El Herichi, Arturo Mendoza, Yanneck Wielhorski et al. (+2 more)

Automated segmentation of CT images has become increasingly important to enhance the reliability of simulations through the generation of high fidelity numerical models. This study addresses the challenging task of semi-automatically tracking textile reinforcements in fan blade dry preforms using X-ray CT images captured at coarse resolutions (i.e., above 140 $\mu$m). Our approach offers a scalable, slice-based analysis conducted on planes orthogonal to the main yarn directions, applied to a large-scale real industrial component. This enables accurate identification and tracking of yarn paths while requiring minimal training. The method models three key yarn properties statistically: their typical cross-section shape, their continuity and movement in the 3D space, and their spatial relative arrangement with respect to neighboring yarns. These statistical properties are integrated into a tracking framework via a variational formulation that optimizes all yarn center positions in successive cross-section planes. The method tracks more than 3,000 warp yarns across 1,500 slices and achieves a tracking success rate above 90%. Overall, this work demonstrates a promising approach toward large-scale, automated textile reinforcement annotation, paving the way for more efficient material characterization in complex composite structures.
Read the full abstract

All authors Hafsa El Herichi, Arturo Mendoza, Yanneck Wielhorski, Hugues Talbot, Stéphane Roux

Image and Video Processing · Cross-listed from cs.CV

TAPe+ML: A Compact Structured Representation for Multi-Task Computer Vision

Sergey Kurinov and Alexey Upatov

We present TAPe+ML v3, a compact computer vision system based on TAPe (Theory of Active Perception), a structured representation that encodes relations among perceptual elements before recognition. Instead of operating directly on pixel tensors, the system uses a shared TAPe representation and a modular recognition architecture for image classification, object detection, and instance segmentation. TAPe+ML v3 combines background and contour processing, local object localization, prototype-based classification, and a coordinator for specialized submodels. Across the reported experiments, it uses fewer than 100,000 parameters. On COCO object detection, it obtains 84.7 mAP50 and 65.3 mAP50-95. On COCO instance segmentation, it obtains 80.7 mask mAP50 and 58.4 mask mAP50-95. In classification experiments, it reaches 92 percent validation accuracy on Imagenette under an identical-training comparison with a raw-pixel baseline, and 89.9 percent Top-1 accuracy on ImageNet-Real. We also evaluate compactness in video scene detection and adaptation under distribution shift in an industrial pilot. The results suggest that shifting part of the modeling burden from network parameters to a structured input representation can support compact multi-task vision systems with reduced data, memory, and compute requirements.
Read the full abstract

Image and Video Processing · Cross-listed from q-bio.OT

Flash-Radiomics: A Scalable Hybrid CPU-CUDA Engine for Standardized Scalar Radiomics and Accelerated Spatial Mapping

Shanli Ding, Yiyi Hu, Ziyu Fu et al. (+5 more)

Background and Objectives: Spatial mapping retains the spatial distribution of radiomic features, but computational cost and fragmented software limit its use. We developed Flash-Radiomics with scalar extraction and spatial mapping, a central processing unit (CPU) backend, a hybrid Compute Unified Device Architecture (CUDA) backend, consistent feature names, and Hierarchical Data Format version 5 (HDF5) storage. Methods: We evaluated Image Biomarker Standardisation Initiative (IBSI) compliance, CPU-CUDA concordance, and end-to-end processing time. Compliance testing included 825 chapter 1 (IBSI-1) tests covering 165 high-consensus features and 323 chapter 2 (IBSI-2) tests with numerical references. Concordance testing included 1,148 scalar pairs and 93 spatial-map pairs. End-to-end processing time was measured five times per input volume of interest (VOI) size. Comparisons included the Medical Image Radiomics Processor (MIRP) and PyRadiomics for 102 shared scalar features and PyRadiomics for 93 shared spatial maps. Results: Both backends passed all 1,148 IBSI tests, and all paired results were concordant. At the largest scalar input, CPU required 76.343 s and hybrid CUDA 81.915 s; CPU was 4.7 times faster than MIRP and 190.7 times faster than PyRadiomics. At the largest spatial input completed by both backends, hybrid CUDA reduced processing time by 68.6% relative to CPU (79.280 versus 252.791 s). At PyRadiomics' largest completed spatial input, hybrid CUDA was 84.9 times faster. Conclusions: Flash-Radiomics unified standardized scalar extraction, spatial mapping, concordant CPU-CUDA results, and HDF5 storage. CPU processing time was similar or shorter for scalar extraction, whereas hybrid CUDA was faster for spatial mapping under the tested conditions.
Read the full abstract

All authors Shanli Ding, Yiyi Hu, Ziyu Fu, Chia-Hsin Lin, Ruihan Luo, Jaehee Chun, Xinyue Zhang, Osama Mawlawi

Image and Video Processing

Coupling Hyperspectral and 3D Data for the preventive Conservation of Palace-museums

Julie Fromager, Vincent Gauthier, Loïc Martinez et al. (+2 more)

In the current context of energy and climate transition, the preventive conservation of historic buildings is particularly important due to their impact on architecture and works of art. Establishing the correlation between environmental variables and the condition of artworks in situ requires comprehensive and individualized monitoring, allowing for an understanding of cause-and-effect mechanisms. To address this challenge, the EPICO method provides a systematic framework for assessing deterioration risks in palace-museums through multi-scale monitoring and correlation between environmental parameters and object condition. The aim of the proposed topic is to enhance this decision-making tool with the creation of digital twins. These digital twins being fed with three-dimensional hyperspectral and LiDAR mapping of spaces and objects.
Read the full abstract

All authors Julie Fromager, Vincent Gauthier, Loïc Martinez, Danilo Forleo, Stéphane Serfaty

Image and Video Processing

Sustainable MRI: Interpretable Deep Learning for estimating energy and power consumption, revealing key acquisition parameters and their interactions

Florian Leonhard Raab, Fiona Mankertz, Nour Maalouf et al. (+16 more)

Magnetic resonance imaging (MRI) is among the most energy-intensive medical imaging modalities. To facilitate the development of more energy-efficient MRI protocols and sequences, we developed an interpretable data-driven deep learning (DL) framework to characterize the factors driving energy and power consumption. The aim was to identify the most influential acquisition parameters and their interactions on MRI energy and power demand, and to prospectively predict energy and power consumption on a per-sequence level. Two separate DL models were trained to predict energy and power consumption from MRI acquisition parameters. Attention weights were analyzed to identify the most influential parameters and their interactions. The proposed DL models successfully captured the variability in the data (energy model: $R^2 = 0.963$, power model: $R^2 = 0.843$). Energy was primarily driven by temporal parameters, whereas power depended on more complex sequence, gradient, and RF-related interactions. These results demonstrate that energy and power consumption can be accurately predicted from MRI acquisition parameters alone. The proposed framework reveals that a small subset of parameters governs energy and power demand, providing interpretable insights to support the development of more energy-efficient scanning protocols and sequences.
Read the full abstract

All authors Florian Leonhard Raab, Fiona Mankertz, Nour Maalouf, Josephine Berger, Andreas Lingg, Reza Dehdab, Sebastian Werner, Judith Herrmann, Andreas Brendlin, Sebastian Gassenmaier, Fabian Wagner, Julian Wohlers, Shreeja Varadarajan, Gurlal Singh, Jens Gühring, Rainer Schneider, Konstantin Nikolaou, Saif Afat, Thomas Küstner

Image and Video Processing

Optimal Transport Metric Learning for Feature Alignment in Partially Supervised Segmentation

Dakini Mallam Garba and Salim Abdou Daoura

Multi-organ segmentation is often challenged by partially annotated datasets and domain shifts across different imaging sources. To address these limitations, we propose a two-stage learning framework that efficiently leverages partial supervision. In the first stage, the model learns from available annotations to produce accurate segmentations of annotated organs, establishing robust feature representations. In the second stage, we introduce learnable organ prototypes and a Sinkhorn-triplet loss to enforce organ-wise feature consistency across datasets. This encourages latent embeddings of the same organ to remain close, while increasing separation between different organs, even when annotations are missing. Our approach achieves performance comparable to state-of-the-art methods on the BTCV dataset, while remaining computationally efficient. By explicitly aligning feature distributions rather than relying solely on pseudo-labels, the framework effectively mitigates domain shift, making it particularly suitable for medical image segmentation tasks with limited annotation resources.
Read the full abstract

Image and Video Processing

Quantum-Inspired Trainable and Parameter-Efficient Tensor Networks for Image Inpainting

Shiwen An and Konstantinos Slavakis

This work introduces quantum-inspired tensor-network circuits as trainable transforms for image inpainting. Among the proposed architectures, the diagonal quantum Fourier transform (QFT) relaxation is invertible with $O(N^2 \log N)$ computational cost for $N\times N$ images, inherently preserving minimum coherence throughout training via its circuit structure and eliminating the need for explicit coherence penalties. Unconstrained gradient-based phase optimization (Riemannian-optimization free) enables efficient learning from randomly sampled training data, allowing the learned transform to generalize to test images observed through fixed sampling masks. Numerical tests show that the learned models outperform fixed transforms and per-image optimization while matching the performance of much larger unitary architectures, yet with far fewer parameters.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

From Foundation Embeddings to Cropland Maps: Label Efficiency, Temporal Transferability and Independent Human Validation

Mohammad Ammar Mughees, Giovanni Montefoschi, Zhongxin Chen et al. (+1 more)

Geospatial foundation models provide reusable representations of satellite imagery that support downstream mapping with limited task-specific modelling. We evaluate whether annual AlphaEarth embeddings support binary cultivated-versus-non-cultivated mapping in Maine, USA, using 192 spatially separated patches and labels derived from the USDA Cropland Data Layer (CDL). Without fine-tuning the foundation model, a lightweight classifier reaches 93.7% overall accuracy and 90.8% balanced accuracy on held-out patches. Logistic regression is within 0.3 percentage points of a gradient-boosted ensemble, while a nearest-class-centroid rule, which uses class centroids but fits no parameters, reaches 90.2%. A balanced sample of 60,000 labelled pixels is within 1.3 percentage points of the full pool of 8.6 million pixels; because pixels are spatially autocorrelated, this result concerns pixel-sample efficiency rather than 60,000 independent annotation sites. In a same-region transfer experiment, classifiers trained in one year remain accurate across 2018 to 2023. Against a blind, two-interpreter consensus at 385 randomly sampled points in one contiguous 2023 block, the AlphaEarth-plus-random-forest map agrees at 95.3% ($\kappa=0.82$), compared with 91.7% for the CDL ($\kappa=0.72$; exact two-sided McNemar $p=0.0161$). This local result is consistent with partial smoothing of CDL label noise, but it does not establish statewide correction of the reference product. On the same points, the difference from a fine-tuned TerraMind segmentation model is not statistically significant (95.3% versus 93.5%; $p=0.14$), and the experiment is not a controlled comparison of computational cost. These results support frozen geospatial embeddings as a low-compute candidate for regional cropland mapping, subject to the limits of a single-state study, a 30 m-derived training reference, and a one-block human validation.
Read the full abstract

All authors Mohammad Ammar Mughees, Giovanni Montefoschi, Zhongxin Chen, Maria Antonia Brovelli

Image and Video Processing

StainBridge: Stain-Aware Pairwise Registration of Serial Renal Biopsy Whole-Slide Images Across Structural and Immunohistochemical Stains

Ellen Wei, Bohang Jiang, Yanfan Zhu et al. (+8 more)

Three-dimensional (3D) reconstruction of histopathology tissue requires accurate pairwise registration of serial whole-slide images (WSIs). Cross-stain benchmarks have advanced registration of differently stained histology, including structural-to-immunohistochemistry (IHC) pairs, but serial renal biopsy stacks remain difficult: they interleave several structural stains with diverse IHC markers whose expression can be sparse or absent, leaving few shared features to match. We present StainBridge, a stain-aware framework for registering serial renal biopsy WSIs across structural and IHC stains. StainBridge couples three preprocessing components, stain deconvolution, intensity normalization, and tissue-mask injection, with XFeat-based affine initialization and four nonrigid backends (VoxelMorph, ConvexAdam, FireANTs, and DeeperHistReg). We evaluate it on 23 cases comprising 338 WSIs, four structural stains, and ten IHC markers, with functional tissue units annotated on consecutive sections to give 1,468 landmark correspondences across 272 image pairs, and report tissue-mask Dice, functional-unit centroid error in micrometers, and tissue-restricted structural similarity. Nonrigid refinement improves on the affine initialization for three of four backends, VoxelMorph being the exception. DeeperHistReg, which computes its own initialization rather than relying on XFeat, gives the best pooled landmark accuracy and registers the most pairs, including every attempted structural-IHC pair. Preprocessing improves landmark accuracy for ConvexAdam and FireANTs in every stain-pairing category, and FireANTs shows both the largest single preprocessing gain on structural-IHC pairs and the best pooled tissue overlap. These results offer practical guidance for cross-stain registration and a foundation for integrated 3D analysis of renal tissue architecture and molecular expression.
Read the full abstract

All authors Ellen Wei, Bohang Jiang, Yanfan Zhu, Daniel Reisenbüchler, Kenji Ikemura, Steven Salvatore, Surya Seshan, Thangamani Muthukumar, Mert R. Sabuncu, Yihe Yang, Ruining Deng

Image and Video Processing · Cross-listed from cs.LG

Noise2Noise Revisited: Training Pair Distributions Dominate Loss Choice in Self-Supervised Denoising

Dingyan Shang, Zhenyu Xu, Youting Wang et al. (+2 more)

Noise2Noise (N2N) trains denoisers on pairs of independently corrupted observations, eliminating clean references. We stress-test two natural conjectures about why the L1 loss outperforms L2 here. First, the hypothesis that the L1 loss confers robustness via parameter sparsity confuses the loss with Lasso regularization: an explicit Lasso penalty produces the predicted sparsity yet fails to reproduce L1's cross-noise behavior, while L1- and L2-trained weight distributions are indistinguishable. Second, the population optima of the two losses coincide exactly for symmetric signal posteriors and nearly so for concentrated ones. Measured differences are therefore dominated by optimization dynamics (bounded-influence gradients), which we probe with gradient statistics and contaminated-target training. On Kodak24 with five synthetic noise families, the L1 loss holds a statistically significant edge over L2, below 1 dB PSNR, holding across three seeds on 13 of the 14 noise columns. On real camera noise the loss is not the decisive variable in distribution: on official SIDD validation blocks, synthetic-Gaussian-trained N2N models gain only 0.8 to 3.7 dB over the noisy input regardless of loss, while retraining on SIDD's own noisy pairs, never reading ground truth, gains 9.4 to 11.0 dB, far ahead of BM3D. All metrics are on raw network outputs, and the study makes no leaderboard claim. The training pair distribution, not the loss, carries the inductive bias. That design rule applies wherever clean references are unobtainable, from microscopy to industrial inspection sensors.
Read the full abstract

All authors Dingyan Shang, Zhenyu Xu, Youting Wang, Bonan Shen, Bowen Liu