Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Friday 21 August

Image and Video Processing

Multimodal pseudo-CT synthesis for PET attenuation correction using separate modality encoding and topogram conditioning

Rory Bell, Artemis Bouzaki, Jiaming Cao et al. (+2 more)

We participated in the BIC-MAC Challenge with a multimodal 3D patch-based U-Net for pseudo-CT generation from NAC-PET, MRI, and 2D topograms. By using separate PET and MR encoders, multi-scale feature fusion, and FiLM-based topogram conditioning at the bottleneck, we obtain a model that integrates complementary cross-modal information while reducing reliance on precise voxel-wise correspondence between modalities. Our final submission can be found: https://github.com/rrr-uom-projects/BIC-MAC-MICCAI2026
Read the full abstract

All authors Rory Bell, Artemis Bouzaki, Jiaming Cao, Jasmine Morrison, Chelsea Sargeant

Image and Video Processing

Toward Vision Language Model-based Assessment of Clinical Quality and Usability of LGE-MR Images for Cardiac Ablation Planning

Bipasha Kundu, Abhishek Chaturvedi, Axel W. E. Wismueller et al. (+2 more)

LGE cardiac MRI is widely used for left atrial fibrosis assessment and ablation planning in atrial fibrillation patients as knowledge of fibrotic tissue regions identified from LGE-MRI is critical for catheter ablation. Often, poor quality images used during ablation planning can cause mis-localization of ablation targets, directly impacting procedure safety and outcome. The decision of whether a scan meets the minimum quality threshold for ablation planning is currently made informally by the reviewing radiologist and is not captured by any automated system, yet it is arguably the most safety-critical output of the image quality assessment (IQA) process. However, variations in image quality caused by noise, motion artifacts, and poor boundary definition significantly compromise the reliability of downstream segmentation and clinical decision-making tasks. Manual quality assessment by expert radiologists is subjective and difficult to scale, while existing automated methods produce scalar scores without interpretable clinical reasoning. In this work, we propose a two-stage vision language model (VLM) framework for clinically grounded image quality assessment of left atrial LGE-MRI. In the first stage, a fine-tuned VLM generates structured radiology-style quality reports predicting five radiologist-defined criteria: Noise, Motion Artifact, LA Boundary Accuracy, PV Region Accuracy, and Under-segmentation Severity. In the second stage, a GPT-based reasoning module maps the predicted quality and reports to a structured quality scores and binary clinical usability decision for ablation planning. We curate a dataset of 60 annotated image slice-text pairs from 20 patients and benchmark four state-of-the-art VLM architectures. InternVL2 achieves the highest criterion-level accuracy (Avg ACC=0.65, PLCC=0.79), while DeepSeek achieves perfect clinical usability agreement (Acc=1.00, kappa=1.00).
Read the full abstract

All authors Bipasha Kundu, Abhishek Chaturvedi, Axel W. E. Wismueller, Richard Simon, Cristian A. Linte

Thursday 20 August

Image and Video Processing

Noise Stability and Rotational Equivariance of Image-Induced Hamiltonian Spectra for Robust Unsupervised Segmentation

Soumic Sarkar

Unsupervised segmentation methods that embed an image as the potential of a Schrödinger-type Hamiltonian and read out object structure from its low-lying eigenstates have demonstrated strong empirical performance across natural and volumetric imagery, yet the robustness of this spectral representation to sensor noise and geometric transformation has never been established theoretically. This paper closes that gap. A discrete graph-Hamiltonian formulation is adopted, consistent with existing quantum-inspired segmentation frameworks, and an explicit finite-sample perturbation bound is derived that links the drift of its eigenvalues and ground-state eigenvector under additive noise of known variance to the operator's spectral gap, extending a one-dimensional noise-error analysis originally developed for semiclassical signal analysis to the two-dimensional graph setting used in image segmentation. The eigenvector-level bound is further propagated to the induced binary segmentation mask, giving, for the first time, a quantitative guarantee on segmentation stability under noise rather than an empirical observation of it. A complementary equivariance analysis characterizes the conditions under which the operator's spectrum transforms exactly under the planar rigid-motion group, and quantifies the equivariance gap introduced by anisotropic potential terms commonly used in practice. Theoretical predictions are validated numerically on natural-image and noise-perturbed benchmarks, showing close agreement between the derived bounds and observed spectral and segmentation-mask deviation, and offering practitioners a principled criterion for selecting operator parameters under known noise conditions.
Read the full abstract

Image and Video Processing

Sparse Light Field Sampling Improves Casual 3D and 4D Reconstruction

Shamus Li, Ruiming Cao, Laura Waller et al. (+2 more)

Many consumer smartphones, stereo cameras, and light field cameras record multiple synchronized viewpoints in a single exposure event. However, novel view synthesis pipelines commonly use only a monocular stream and rely on camera motion or learned priors to obtain angular coverage. In this paper, we ask: why do we use only one viewpoint? We analyze sensor-limited multi-view, where one sensor trades off spatial and angular resolution, and exposure-limited multi-view, where multiple sensors on one commodity device observe each event simultaneously. We introduce a new dataset incorporating three types of commodity multi-view cameras, and evaluate sparse-view 3DGS and 4DGS baselines measuring reconstruction quality as a function of number of exposures and angle between extreme views. Our results demonstrate that using multiple cameras, even with a low baseline, significantly improves reconstruction quality in single-shot, few-shot, and casual video settings. In addition, under a fixed sensor budget, angular sampling improves reconstruction when exposures are scarce despite lower spatial resolution. The gains are most pronounced for single-shot and dynamic scenes, where a stationary monocular camera lacks the angular diversity to recover scene geometry and motion.
Read the full abstract

All authors Shamus Li, Ruiming Cao, Laura Waller, Kristina Monakhova, Sara Fridovich-Keil

Image and Video Processing

Consistency Models for Fast MRI Reconstruction Using Regularization by Denoising

Merve Gülle, Junno Yun, Yaşar Utku Alçalar et al. (+1 more)

Diffusion models (DMs) have emerged as powerful generative priors for MRI reconstruction with promising results. Yet DM-based methods require extensive iterative refinement, limiting their practical deployment. Consistency models (CMs) provide a compelling alternative, aiming to map out the diffusion trajectory in a single pass, enabling faster generation. In this work, we propose CM-RED, a novel MRI reconstruction method that integrates a pretrained CM into the regularization by denoising (RED) scheme. Our method builds on accelerated proximal gradient RED (RED-APG), and further incorporates controlled noise injection during the update steps to enhance generative diversity and accelerate convergence. Extensive experiments on the fastMRI knee and brain datasets demonstrate that CM-RED achieves high-quality reconstructions across multiple anatomies, contrast weights, acceleration factors, and undersampling patterns, using only 4 network function evaluations (NFEs). The proposed method consistently outperforms existing DM- and CM-based approaches in both quantitative metrics and visual fidelity, and exhibits strong robustness to hyperparameter variations, highlighting CM-RED as an efficient and effective generative framework for accelerated MRI reconstruction. The source code and pretrained models are publicly available at https://github.com/MerveGulle/CM-RED.
Read the full abstract

All authors Merve Gülle, Junno Yun, Yaşar Utku Alçalar, Mehmet Akçakaya

Image and Video Processing

Frozen CLIP Priors for Robust Self-Supervised Poisson Inverse Problems

Laura C. Diaz-Delgado, Emmanuel Martinez and Henry Arguello

Self-supervised learning for imaging inverse problems is increasingly important in photon-limited settings, where acquiring clean ground truth is impractical and reconstruction must remain stable under dataset and acquisition shifts. This challenge is amplified under Poisson noise, whose signal-dependent statistics interact with sampling operators (e.g., CFA mosaicing). Meanwhile, foundation vision encoders trained at web scale offer distortion-invariant, content-related representations that generalize well across domains, suggesting a promising route to build priors that transfer beyond the training distribution without expensive fine-tuning. This paper proposes an ADMM-inspired unrolled plug-and-play solver for Poisson inverse problems that decouples a closed-form data-consistency update from a parameter-efficient prior. The prior is implemented as a lightweight decoder operating on frozen CLIP RN50 dense multi-scale features, adapting foundation representations with less trainable parameters. For self-supervision, the method integrates GR2R measurement-domain re-corruption with an Equivariant Imaging regularizer via virtual acquisitions. Experiments on Poisson CFA demosaicing and deblurring show competitive quality, improved robustness under shifts, and self-supervised performance approaching supervised training.
Read the full abstract

Image and Video Processing

Flow Matching-Based PET Image Reconstruction

Fumio Hashimoto, Ziqian Huang, Tatsuya Yokota et al. (+1 more)

Generative models have shown strong potential for positron emission tomography (PET) image reconstruction. Although diffusion model-based reconstruction methods have demonstrated promising performance, they often require many reverse sampling steps with data-consistency updates incorporated into the sampling process. Flow matching offers an attractive alternative because it can directly estimate clean images from intermediate states, allowing data-consistency refinement to be separated from flow propagation. In this work, we proposed flow matching-based PET image reconstruction methods. We first established PET-FlowDPS by incorporating Poisson likelihood guidance with an expectation-maximization (EM)-based preconditioner into the FlowDPS framework. We then proposed a model-based PET reconstruction method that used a pretrained flow matching model as a prior, in which the flow-based prior, PET data refinement, and stochastic propagation were interpreted within an approximate Bayesian framework. Experimental results using [$^{\text{18}}\text{F}$]FDG brain PET datasets showed that the proposed method achieved better bias-variance trade-offs across different dose levels compared with other reference methods. These results demonstrated the potential of flow matching as a generative prior for quantitative PET image reconstruction.
Read the full abstract

All authors Fumio Hashimoto, Ziqian Huang, Tatsuya Yokota, Kuang Gong

Image and Video Processing

Simulation-to-Real First-Break Segmentation for Efficient Inversion in Musculoskeletal Ultrasound Tomography

Yifei Sun, Yubing Li, Yannick Benezeth et al. (+6 more)

Full-waveform inversion (FWI) is a promising strategy for quantitative musculoskeletal ultrasound computed tomography (USCT), but bone-related scattering, attenuation, and signal degradation make it highly sensitive to the accuracy of the initial acoustic-property distributions and prone to cycle skipping. First-arrival traveltimes provide important kinematic information for initial-model construction, yet conventional trace-wise picking is unreliable when arrivals are weak, spatially heterogeneous, or buried in system noise. We propose a learning-assisted reconstruction pipeline that combines segmentation-based first-arrival extraction with hybrid full-waveform inversion (HFWI), which incorporates Rytov-approximation-based traveltime information together with waveform fitting during the early inversion stage. A lightweight 2D U-Net treats the first-arrival trajectory across receiver channels as a first-break segmentation target and exploits its spatial continuity rather than processing each trace independently. To address both limited manual annotations and the simulation-to-real gap, the network is pretrained on task-specific simulations augmented with real system-noise recordings, followed by stage-wise training with progressively increased signal degradation and decoder-only fine-tuning using limited weakly labeled experimental data. The method is evaluated on in vitro phantom, ex vivo bovine-limb, and in vivo human-thigh datasets. Compared with conventional STA/LTA picking, the proposed network yields more spatially coherent first-arrival trajectories, lower mean extraction errors, and processes a full-matrix-capture dataset within seconds. When integrated into HFWI, the extracted arrivals improve initial-model construction and lead to stable subsequent FWI reconstructions, including challenging cases with estimated local first-arrival SNRs below 3 dB.
Read the full abstract

All authors Yifei Sun, Yubing Li, Yannick Benezeth, Stéphanie Bricq, Yunrong Zhang, Lekang Jiang, Chang Su, Ligang Cui, Weijun Lin

Image and Video Processing

Energy-Mamba: A Physics-Constrained State-Space Model for Medical Image Classification

Mohamed Mabrok, Yalda Zafari and Essam A. Rashed

State-Space Models (SSMs), particularly Mamba, offer linear-time complexity for long-range dependencies, making them attractive for medical imaging with limited annotated data. However, adapting these sequential models to 2D images through unconstrained state evolution causes representational drift, the dynamic hidden state progressively loses fidelity to local image features. We introduce Energy-Mamba, integrating SSM dynamics with physics-informed constraints via a learnable potential energy function that quantifies compatibility between evolving states and static local features. Our Energy-Mamba Block introduces a gradient-based forcing term, computed dynamically via automatic differentiation, that pulls states toward low-energy configurations maintaining local visual fidelity. This formulation mirrors Hamiltonian dynamics: kinetic energy (SSM scan) plus potential energy (our constraint function) govern state trajectories. This architectural prior enables learning implicit constraints for robust, faithful representations, crucial in medical imaging where fine-grained local detail drives accurate diagnosis. Evaluated on four datasets (retinal OCT, chest X-ray, microscopy, abdominal CT), Energy-Mamba achieves state-of-the-art classification performance with significantly fewer parameters, demonstrating that physics-informed grounding can enhance both efficiency and representational quality in medical vision tasks.
Read the full abstract

Image and Video Processing

MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalities

Tarun Kumar Garg and Vaanathi Sundaresan

Trustworthy multimodal fusion in clinical settings requires handling incomplete and heterogeneous modality subsets across institutions, where privacy constraints prohibit centralized data sharing. Federated learning (FL) mitigates data-sharing constraints but suffers from client-specific missing modalities, where institutions possess incomplete multimodal subsets, degrading fusion quality and segmentation performance. While FL and weak supervision have been studied separately, their joint use with image-level labels under heterogeneous missing modalities remains unaddressed. We propose MOSAIC, the first modality-agnostic federated framework for weakly supervised binary tumor segmentation under client-specific missing modalities. We introduce a client-specific modality-alignment module that fuses available channels into a shared latent space without prior knowledge of modality identity, a spectral prototype alignment loss that reconciles cross-client distribution shift using compact non-invertible frequency-domain statistics, and a dedicated federated refinement network that denoises the resulting CAM pseudo-labels into accurate masks, breaking the accuracy ceiling of weak supervision. Experiments on three multi-institutional brain tumor benchmarks (FeTS2022, BraTS-MEN, and BraTS-SSA) demonstrate significant improvements over all image, box, and point-supervised baselines, approaching fully supervised accuracy using only image-level labels and reaching 0.84 Dice on FeTS2022. Dynamic new client addition enables previously unseen institutions to join an already-trained federation within 0.01-0.04 Dice without retraining. Code is available at https://github.com/Tarun2201/MOSAIC.
Read the full abstract

Image and Video Processing

AsymFeX: A Symmetry-Driven Framework for Ischemic Stroke Segmentation Across Imaging Modalities and Stroke Stages

Maunil Shah and Vaanathi Sundaresan

Fast and accurate segmentation of Acute Ischemic Stroke (AIS) lesions is essential for stroke prognosis and treatment planning. Non-contrast CT (NCCT), the first-line imaging modality for diagnosing ischemic infarcts, exhibits subtle infarct contrast, making manual delineation slow and labor-intensive. Motivated by this, and by the clinical practice of comparing brain hemispheres to localize infarcts, we propose a two-stage, nnU-Net-compatible 3D segmentation method. The first stage corrects head tilt to align each scan to its true anatomical mid-sagittal plane; the second applies a novel Asymmetric Feature Extraction (AsymFeX) module, comparing each voxel to its true contralateral counterpart within a local 3 x 3 x 3 neighborhood via cross-hemispheric attention, feature disparity estimation, and dual-scale gating to capture both large and small infarcts. On AISD, our method achieves 0.6796 Dice, 23.53 mm HD95, and 7.69 mL AVD, significantly outperforming existing state-of-the-art methods, with clinically relevant volumetric analysis at the 70 mL thrombolysis-eligibility threshold. Proof-of-concept evaluation on ATLAS v2.1 and ISLES'24 demonstrates that the same symmetry-driven design generalizes across imaging modalities and stroke time points without architectural changes, further supported by an uncertainty analysis assessing reliability under clinical deployment. Code is publicly available at https://github.com/biomedia-lab/AIS-detection.
Read the full abstract

Image and Video Processing

Loss-Resilient Semantic Communication over Packet-Loss Networks at Extreme-Low Bandwidth

Shengshi Yao, Jincheng Dai, Sixian Wang et al. (+5 more)

In extreme-low bandwidth network scenarios, generative semantic codecs have emerged as promising solutions to reduce bandwidth cost for visual communications. However, these learned codecs are usually optimized solely for compression efficiency and thus not robust against transmission errors. Corruptions due to packet-loss among these highly compact generative latent representations often cause more critical degradation in fidelity and realism, intensified by the severe error propagation across the latent contexts and multi-step decoding process. In this paper, we propose ResiGLC, a novel loss-resilient generative latent coding framework designed for robust semantic communication over extreme-low bandwidth packet-loss networks. Motivated by the inherent goal-consistency between generation and compression, we sufficiently exploit the impressive in-context predictive capabilities of language models. Integrated with the masked learning strategy, our model supports arbitrary context modeling of latent codes, which could mitigate the error propagation and handle unpredictable packet loss patterns. At the receiver, a progressive resilient decoding pipeline is presented, which leverages both the contextual relationship of the latent codes and the multi-modal semantic prior in the generative latent space, separately. By jointly optimizing toward both compression efficiency and packet-loss resilience, our proposed progressive decoding mechanism offers graceful performance when dealing with dynamic packet losses. Through extensive experimental evaluations, we establish that under packet-loss network conditions, ResiGLC can effectively improve the loss-resilience in terms of perceptual fidelity and realism qualities with extreme-low bandwidth cost.
Read the full abstract

All authors Shengshi Yao, Jincheng Dai, Sixian Wang, Guo Lu, Kai Niu, Wenjun Xu, Wenjun Zhang, Ping Zhang

Wednesday 19 August

Image and Video Processing

SatoyamaCT: A Multi-Axis Night-IR Camera-Trap Benchmark for Monitoring Crop-Damaging Wildlife in Japanese Agroforestry

Keito Inoshita, Kohei Hisayama, Haruto Sugeno et al. (+1 more)

Crop and forest damage from sika deer, wild boar, and Japanese macaque is a serious economic problem in Japanese satoyama, where farmland and forest intermingle. Camera traps enable scalable monitoring, yet existing benchmarks evaluate recognition in-distribution, rarely prioritize night infrared imagery, and do not jointly address regional domain shift, novel-species detection, and uncertainty-based triage on a single dataset. We introduce the Satoyama Camera Trap Dataset (SatoyamaCT), 12,642 expert-verified crops from night-IR-dominant camera traps across three satoyama regions. An iterative annotation protocol combining BioCLIP embeddings with confidence-ordered confirmation reduced expert effort while all labels were verified by domain ecologists; species-level inter-annotator agreement reached Cohen's kappa = 0.900. A multi-axis protocol jointly evaluates domain generalization across region, camera placement, and illumination; open-set novel-species detection; and selective prediction, all under capture-event-based leakage control. Difficulty separates into distinct failure modes: a data-inherent regional gap of 16 to 27 percentage points persists in Wakayama across four backbones and domain-generalization methods including CORAL, DANN, and GroupDRO, and open-set detection reaches AUROC 0.93 to 0.96 overall yet degrades jointly with classification in Wakayama. Selective prediction recovers Wakayama accuracy from 0.593 to 0.815 at 50% coverage, supporting an uncertainty-aware triage workflow for practical pest monitoring in agroforestry.
Read the full abstract

All authors Keito Inoshita, Kohei Hisayama, Haruto Sugeno, Kota Nojiri

Image and Video Processing

HiFiC-G: Adapting HiFiC for Hi-C Contact Matrices

Andre Antonio Straton

We study whether the loss design of High-Fidelity Generative Image Compression (HiFiC), a GAN-based neural codec originally built for natural photographs, can be adapted to preserve biologically meaningful structure in Hi-C chromatin contact maps under lossy compression. Standard image compression, including HiFiC in its original form, optimizes for human visual perception; but a Hi-C contact map is normally distributed together with its numeric matrix file (.cool/.mcool), which downstream genomic analysis tools consume directly. Aggressive compression that looks acceptable to the eye can nonetheless blur or delete loops and topologically associating domain (TAD) boundaries that these tools depend on. We modify HiFiC's distortion term with a spatially-weighted MSE that up-weights biologically salient regions (loops, TAD boundaries, stripes, compartment structure) and add an insulation-score loss term that directly penalizes loss of TAD boundary sharpness. We describe a three-phase fine-tuning strategy that adapts a pretrained HiFiC checkpoint to the Hi-C domain without catastrophic forgetting. We evaluate the resulting system, HiFiC-G, using both conventional image-quality metrics (PSNR, SSIM) and genomics-domain preservation metrics (loop/TAD/compartment/stripe preservation percentage) across two cell lines. HiFiC-G preserves local structure, meaning stripes and TAD boundaries, substantially better than the metrics alone would suggest, while long-range A/B compartment structure remains poorly preserved; we show this gap tracks genomic scale and is consistent with a specific architectural cause, the fixed-size tiling that both HiFiC-G and the original HiFiC rely on for memory efficiency.
Read the full abstract

Image and Video Processing

MOSAIC: A Self-supervised Dynamic Multi-encoding Reconstruction Framework for 3D Late Gadolinium Enhancement MRI

Muhammad A. Sultan, Yingmin Liu, Katherine Binzel et al. (+3 more)

Purpose: To develop and evaluate a self-supervised dynamic reconstruction framework for highly undersampled dual-echo three-dimensional late gadolinium enhancement (3D LGE) MRI. Methods: MOSAIC jointly models multi-echo image content, coil sensitivity maps, and beat-specific nonrigid motion directly from acquired undersampled data, without requiring fully sampled training datasets or accurate precomputed sensitivity maps. Unlike existing methods that bin the acquired data into different motion states, with or without motion compensation, MOSAIC reconstructs a motion-resolved 3D LGE image from each heartbeat. The method was evaluated using digital phantoms with simulated myocardial scars and in vivo animal and human studies. Results: In phantom experiments, MOSAIC achieved higher peak signal-to-noise ratio and structural similarity index measure than low-rank deep image prior reconstruction and ablation variants of MOSAIC. In animal and human studies, MOSAIC achieved higher blinded expert image-quality scores than inline image-navigated compressed-sensing and low-rank deep image prior reconstructions. Conclusion: MOSAIC demonstrated the feasibility of motion-resolved free-breathing dual-echo 3D LGE MRI at acceleration factors exceeding 1,000, with improved detail preservation and artifact suppression relative to the state-of-the-art comparison methods.
Read the full abstract

All authors Muhammad A. Sultan, Yingmin Liu, Katherine Binzel, Katarzyna E. Gil, Karolina M. Zareba, Rizwan Ahmad

Image and Video Processing

Deep Learning for Automated Quantification of Tumor-Associated Macrophages from H&E-Stained Slides in Diffuse Large B-Cell Lymphoma

Anastasiia Studenikina, Svetlana Illarionova, Joaquim Carreras et al. (+7 more)

While M2-polarized tumor-associated macrophages (TAMs) have been established as indicators of disease aggressiveness in diffuse large B-cell lymphoma (DLBCL), traditional CD163 immunohistochemistry (IHC) remains resource-intensive. This study aims to investigate the feasibility of using deep learning to quantify TAMs directly from standard hematoxylin and eosin-stained (HE) tissue sections. Using a curated dataset of 52 patients with DLBCL, with high-resolution HE images and IHC-validated annotations (1,713 TAM instances), five architectures were evaluated: U-Net, Swin-U-Net, Cerberus-U-Net3+, YOLOv11, and HoVer-Net. High CD163 TAM density (>20.04%) was associated with significantly reduced overall survival (HR 2.73; 95% CI:1.21-6.16; p=0.012) and progression-free survival (HR 2.88; 95% CI: 1.23-6.76; p=0.011). In multivariate Cox proportional hazards analysis adjusting for IPI, molecular subtype (GCB/non-GCB per Hans algorithm), EBV status, and age, CD163 TAM density showed a prognostic trend for overall survival (HR 3.24; 95% CI: 0.94-11.15; p=0.062) and progression-free survival (HR 2.41; 95% CI: 0.80-7.32; p=0.120). Among the evaluated models, the domain-specific Cerberus-U-Net3+ achieved the highest sensitivity (Recall 0.656), while the Transformer-based Swin-U-Net demonstrated superior segmentation fidelity (Precision 0.694, F1-score 0.633). Additionally, the survival analysis based on the predicted Swin-U-Net CD163 level revealed a 20.8% cutoff point for patients with high and low CD163 levels near IHC, as well as a downward trend in overall survival among patients with higher predicted CD163 values of 2.63 (95% CI: 0.85-8.33; p=0.083). These findings suggest that deep learning architectures using shifted-window self-attention show potential as a candidate surrogate for IHC that is scalable and cost-effective for prognostic assessment of TAMs in DLBCL.
Read the full abstract

All authors Anastasiia Studenikina, Svetlana Illarionova, Joaquim Carreras, Daniil Sulimov, Olga Filimonova, Dmitry Zvezdin, Arsenii Galimov, Ivan Tyukin, Rifat Hamoudi, Maxim Sharaev

Image and Video Processing · Cross-listed from cs.CV

Simple, Safe, and Overlooked: Reclaiming Sustainable Domain Generalization with Statistical Color Matching

Sebastian Doerrich, Francesco Di Salvo, Shyam Nandan Rai et al. (+2 more)

Hardware shifts, color variations, and changing patient characteristics between development and deployment routinely break trained medical image classifiers. Existing remedies fall short: standard color jittering provides insufficient diversity, while deep generative style transfer algorithms hallucinate features, destroy clinically relevant structures, and waste massive compute resources. To address this, we revisit classical statistical color matching and repurpose it as Colorist, a highly efficient data augmentation strategy that applies global mean-standard deviation matching directly in the RGB color space. We demonstrate that this training-free, fully interpretable approach safely generates structurally intact domain variations, outperforming deep generative models in structural fidelity and color alignment. Across out-of-distribution histopathology, peripheral blood, dermatology, and retinal datasets, it improves balanced accuracy by up to +9% over state-of-the-art domain generalization regularizers and by +13% over an unaugmented baseline. Moreover, by avoiding neural networks in the augmentation loop, Colorist preserves anatomical structure, minimizes carbon footprint, and integrates seamlessly into standard dataloaders. Together, these findings establish statistical matching as a safe, interpretable, yet overlooked alternative to deep architectures for clinical robustness. Source code is available at https://github.com/sdoerrich97/colorist.
Read the full abstract

All authors Sebastian Doerrich, Francesco Di Salvo, Shyam Nandan Rai, Marco Lents, Christian Ledig

Tuesday 18 August

Image and Video Processing

Optic Disc Segmentation in Fundus Images: From Classical Image Processing and Deformable Models to Modern AI

Buket D. Barkana

Accurate localization and segmentation of the optic disc (OD) are important for retinal image analysis and glaucoma assessment, yet remain challenging due to variations in illumination, pathology, vascular interference, and poorly defined boundaries. This structured methodological review examines the evolution of OD segmentation from classical image-processing and deformable models to contemporary artificial intelligence (AI)-based approaches. A structured literature search and study-selection process was used to identify representative studies spanning major methodological developments. The review first summarizes commonly used fundus-image datasets, then organizes classical methods by principal mechanisms, including intensity and thresholding, histogram and entropy analysis, morphology, geometric and Hough-transform methods, filtering and feature operators, texture- and region-based approaches, and active-contour and level-set models. This paper pays particular attention to the assumptions, strengths, limitations, and complementary roles of these methods in OD localization and boundary delineation. Representative AI approaches are subsequently examined to illustrate the transition from handcrafted features and explicitly defined priors to learned representations, Transformer-based segmentation, boundary- and shape-aware learning, promptable segmentation, and retinal foundation models. Across these methodological generations, several core segmentation principles persist, including region-of-interest localization, multiscale representation, geometric and anatomical constraints, and boundary regularization, although their implementation has shifted from predefined operators to learned modules, losses, prompts, and pretrained representations. The review further identifies boundary ambiguity, anatomical variability, domain shift, and cross-dataset generalization as continuing challenges.
Read the full abstract

Image and Video Processing

QuARC-GS: Quantized Anchored Residual Coding for Compact Dynamic Scene Streaming with Gaussian Splatting

Vu Trung Nghia Nguyen, Yuchen Wang, Kyung Chul Lee et al. (+1 more)

3D scene representation techniques such as neural radiance fields (NeRFs) and Gaussian splatting have made substantial progress in novel view synthesis, achieving high-quality renderings from arbitrary view angles. More recently, such techniques have been extended to dynamic 3D scenes; however, achieving sustainable online free-viewpoint video (FVV) streaming remains challenging, especially for longer videos, due to significant storage demands of detailed scene representations and high reconstruction/rendering speed needs. To address these challenges, we propose Quantized Anchored Residual Coding Gaussian Streaming (QuARC-GS), a quantization-aware 4D scene optimization framework for online dynamic scene reconstruction that achieves ultra-high compression while maintaining reconstruction speed and quality. QuARC-GS represents a scene using a single canonical frame and highly compressed per-frame residuals. Specifically, we compress each residual through two complementary strategies targeting motion, appearance, and densification. We introduce quantization-aware anchor deformation, which suppresses insignificant motion updates while preserving meaningful deformations, maintaining reconstruction quality under low-storage streaming. Furthermore, we design a change-gated densification strategy that allocates new Gaussians only in regions exhibiting genuine temporal changes, effectively eliminating redundant appearance updates and reducing storage overhead. Extensive experiments on widely used datasets demonstrate that QuARC-GS enables competitive reconstruction quality and training speed while cutting per-frame storage by up to 11$\times$ compared to the state-of-the-art.
Read the full abstract

All authors Vu Trung Nghia Nguyen, Yuchen Wang, Kyung Chul Lee, Kevin C. Zhou

Image and Video Processing

Primitive Representation Learning for Unsupervised Dynamic Contrast Enhanced MRI Reconstruction

Veronika Spieker, Wenqi Huang, Cemre Ariyurek et al. (+5 more)

Reliable quantitative analysis of dynamic contrast-enhanced MRI requires high-quality spatiotemporal reconstructions at high undersampling rates. Scan-specific reconstructions using Gaussian and Gabor primitives have shown promising results without the need for large training datasets, but have not addressed the additional dimension of dynamic contrast. We propose a multi-dimensional, primitive based framework for dynamic contrast-enhanced MRI reconstruction that disentangles the underlying anatomy, the dynamic contrast enhancement, and residual motion into separate temporal basis functions, thereby enabling a geometrical interpretation of the representation. We show that this architecture achieves performance competitive with conventional reconstruction methods, both in reconstruction quality and in the accuracy of extracted aorta and kidney enhancement curves. The modular tier design extends naturally to additional dynamic factors and higher acceleration rates. Code available at https://github.com/compai-lab/2026-GaborDCE-spieker.
Read the full abstract

All authors Veronika Spieker, Wenqi Huang, Cemre Ariyurek, Liam Timms, Daniel Rueckert, Onur Afacan, Julia A. Schnabel, Sila Kurugol

Image and Video Processing

Harnessing Magnitude-Only and Complex Measurements for Improved Dynamic MRI Reconstruction with Learned Priors

Mahdi Saberi, Yaşar Utku Alçalar, Merve Gülle et al. (+2 more)

MRI reconstruction methods for undersampled k-space data naturally utilize complex-valued measurements. Parallel developments in sparse phase retrieval have shown that magnitude-only measurements may provide complementary information for signal recovery. However, their use in MRI reconstruction remains largely unexplored, due to lack of practical settings where informative magnitude measurements can be obtained without additional scan time. In this work, we investigate the use of auxiliary k-space magnitude information for accelerated steady-state dynamic MRI reconstruction, and demonstrate strong consistency of k-space magnitudes across time-frames. Building on this observation, we propose $\mathbb{C}+\text{Mag}$, a magnitude-informed physics-driven deep learning reconstruction method. The proposed method employs an ADMM-based unrolling framework with a novel magnitude-aware data-fidelity formulation, where quadratically smoothed optimization and momentum-based updates are introduced to address the non-differentiability and non-convexity of the magnitude constraints. Experiments on retrospectively undersampled cine MRI and phase-contrast flow MRI datasets, as well as prospectively undersampled real-time cine MRI acquisitions, demonstrate improved artifact suppression, sharper anatomical recovery, and better preservation of phase information compared to conventional PD-DL methods, which is further supported through blinded expert reader evaluations.
Read the full abstract

All authors Mahdi Saberi, Yaşar Utku Alçalar, Merve Gülle, Chetan Shenoy, Mehmet Akçakaya

Image and Video Processing · Cross-listed from cs.RO

Iterative Grasp Pose Refinement: A Deep Reinforcement Learning Approach for 2D Vision

Amir Arsalan Nematollahi, Shayan Ahmadi, Mehdi Tale Masouleh et al. (+1 more)

Developing robots capable of understanding and manipulating objects requires compact, interpretable, and generalizable representations. This work proposes a reinforcement learning-based framework for robotic grasp refinement, integrating keypoint-based object representations with a Deep Q-Network (DQN). Using 2D overhead images captured in a simulated environment, a geometric-based algorithm generates initial grasp candidates, which are iteratively refined by the proposed framework, transforming failed grasps into successful ones. Experiments conducted on 300 objects from the Dex-Net dataset using a UR5 manipulator demonstrate the framework's effectiveness, achieving a 100% success rate on objects previously deemed ungraspable by geometrical methods. The framework's sim-to-real transferability is further validated through physical experiments on a Delta parallel robot, where a refined grasp successfully manipulates an object that was previously ungraspable. The findings underscore the effectiveness of reinforcement learning in addressing challenges in robotic grasping, offering a scalable and adaptable solution for contact-rich manipulation tasks.
Read the full abstract

All authors Amir Arsalan Nematollahi, Shayan Ahmadi, Mehdi Tale Masouleh, Ahmad Kalhor

Monday 17 August

Image and Video Processing · Cross-listed from cs.CV

Rapid Debris-Volume Estimation from Post-Hurricane Aerial Imagery

Kooshan Amini, Jamie Ellen Padgett and Guha Balakrishnan

Hurricane debris removal is planned, contracted, and federally reimbursed on the basis of volume estimates, yet operational practice still relies on parametric forecasts with 41-90% documented over-estimation or on truck-load tallies that arrive only after hauling begins. We present DebrisHeightNet, a segmentation-conditioned monocular debris-height network that estimates spatially explicit debris volume from a single pass of post-event aerial RGB imagery, the kind of survey routinely flown within days of a hurricane landfall. We train only a lightweight 1.08 M-parameter head on top of two frozen vision foundation models. This head regresses height from a Depth Anything V2 backbone, conditioned on the debris segmentation of CLIPSeg-debris from our prior work. Because no post-hurricane debris-height ground truth exists, we synthesize the training target by confidence-weighted LiDAR-monocular fusion (CW-LMF), designed to suppress non-debris LiDAR returns. This fused target is a constructed supervision signal rather than ground truth, so we corroborate it against external references rather than claiming it as truth. A region-level power-law calibration, driven by each region's low-density debris fraction, converts model volume into an estimate of the reported hauled debris with quantified uncertainty. Across ten regions spanning five hurricanes and three states, the uncalibrated model agrees with an independent uncrewed-aerial-vehicle (UAV) survey of the training region at Spearman $\rho = 0.87$ and lands within 30% of the reported record where the Hazus and FEMA-hybrid parametric forecasts over-predict it by 2.7-4.8$\times$. Deployment requires no LiDAR, no ground access, and no second flight, so the method can produce spatially explicit volume estimates wherever single-pass post-event imagery is flown.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CL

Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models

Xiutian Zhao, Luqi Sun, Björn Schuller et al. (+1 more)

Modern multimodal foundation models (MFMs) have made rapid progress on tasks requiring integrated perception across speech, vision, and language, including emotion recognition. However, it remains unclear whether they recognize speech and facial emotion through shared affective functional units or modality-specific pathways. We explore emotion-sensitive neurons (ESNs), sparse decoder neurons selectively associated with emotion categories, in three MFMs: Gemma-4-12B-it, MiniCPM-o-4.5, and Qwen2.5-Omni-7B. Using speech emotion recognition and facial expression recognition as complementary probes, we identify acoustic and visual ESNs. Visual ESNs are causally meaningful: deactivating them selectively impairs recognition of the associated facial emotion, whereas steering their activations selectively enhances recognition of that emotion relative to other emotion categories. Acoustic and visual ESNs further show emotion-matched overlap and similar layer-wise distributions, indicating partial structural alignment between affective representations across speech and faces. Finally, cross-modal interventions reveal bidirectional causal transfer: ESNs identified from one modality produce emotion-specific effects when applied to the other. Our findings provide one of the first cross-modality activation-level analyses of affective functional units in MFMs, suggesting that speech and facial emotion recognition partially converge onto sparse decoder-level components that can be localized and manipulated without training.
Read the full abstract

All authors Xiutian Zhao, Luqi Sun, Björn Schuller, Berrak Sisman

Image and Video Processing · Cross-listed from cs.CV

AerialYield-B2D: A Greenhouse Blueberry Dataset with Five-Stage Ripeness Masks and Fruit Counts

Iyyakutti Iyappan Ganapathi, Afeefa Azam, Muhammad Owais et al. (+2 more)

Blueberry ripeness is judged by berry colour, cluster composition, and the distribution of maturity stages within a plant, however, public green house image resources with dense ripeness-stage masks remain limited. We present AerialYield-B2D, where B2D denotes BlueBerry Dataset, acurated real-image resource containing 514 RGB images and 30,195 annotated blueberry instances across five ripeness stages: green immature, pale pink, pink-turns-purple, fully ripe and over-ripe. The release provides class-specific binary masks, overall berry masks, semantic label maps, image-level count tables, SHA-256 hashes, source metadata, recommended train/validation/test splits and technical validations. AerialYield is the broader project name; this release does not provide harvest weight, fruit mass or per-area yield measurements, and the count labels should therefore be interpreted as image-level berry counts rather than yield estimates. The images include 424 smartphone greenhouse images, 67 video-derived frames, and 23 DJI Fly video-frame samples, providing a reproducible dataset for ripeness segmentation, berry counting, and class-imbalance analysis in controlled-environment blueberry production.
Read the full abstract

All authors Iyyakutti Iyappan Ganapathi, Afeefa Azam, Muhammad Owais, Irfan Hussain, Yusra Abdulrahman

Image and Video Processing

Technically Plausible but Clinically Misleading? Expert Evaluation of Patient-Personalized Synthetic Prostate MRI

Gabriel Paulo Maglalang Israel, Sol Gedde and Alvaro Fernandez-Quilez

Magnetic resonance imaging (MRI) is central to prostate cancer assessment, yet its acquisition is costly and time-consuming, making it a major bottleneck in the patient care pathway. Personalized MRI synthesis is a promising direction because it may allow the generation of clinically realistic images tailored to individual patients while reducing the dependence on scanner-acquired imaging. In this work, we in-vestigate whether patient-personalized MRI synthesis produces images that experts perceive as clinically plausible and how such personaliza-tion influences expert interpretation. We synthesize patient-personalized prostate MRI using a 3D diffusion model conditioned on routine pre-imaging clinical variables age and PSA. Using a large publicly available dataset, we first verify that synthesized images are technically compara-ble to scanner-acquired MRIs using standard image similarity metrics. We then conduct a pilot, single-blinded, randomized expert study on a subset of cases sampled to reflect variation in patient clinical pro-files. A radiologist compared personalized and unconditioned synthetic MRIs using paired assessments and Likert-scale ratings of clinical plausibility, confidence, and risk of misleading interpretation, supplemented by qualitative feedback. Although personalized synthetic MRIs appeared technically plausible, expert interpretation highlighted variability in con-fidence and potential risks of misleading cues when routine clinical data was embedded into image generation. These findings suggest that while personalized synthesis may be technically feasible, careful assessment is needed to understand how generated cues influence clinician interpreta-tion before such systems can be safely integrated into clinical workflows.
Read the full abstract

Image and Video Processing

Clinical Pathways Matter for Multimodal Deep Learning in Early Alzheimers Disease Detection

Yao Lu, Solveig Kristina Hammonds and Alvaro Fernandez-Quilez

Identifying individuals at risk of Alzheimer's disease (AD), particularly in the preclinical and early stages, remains challenging. Although deep learning approaches based on structural MRI show promise as a non-invasive biomarker, existing multimodal models require task-specific training and depend on biomarkers that are not routinely available in clinical practice. Here, we propose a zero-shot multimodal framework based on SigLIP that combines structural MRI embeddings with text embeddings of routinely collected clinical variables for early AD risk stratification in individuals at preclinical or mild cognitive impairment (MCI) stages. We evaluated the approach in 416 individuals from the ADNI cohort (age: 72.73 +- 6.7). SigLIP was used without fine-tuning to extract MRI and clinical text embeddings, which were combined into multimodal representations for individual-level AD risk prediction within 4 years. We further compared the model performance in a single-visit and two-visit settings to assess the value of longitudinal information and framework scalability. In the single-visit setting, combining MRI embeddings with MMSE, age, and sex achieved an AUC of 0.91 +- 0.02, outperforming both a CSF Ae̱ta}42-based model (AUC 0.73 +- 0.08) and an MMSE-based model (AUC 0.85 +- 0.22). In the two-visit setting, performance was maintained or improved, supporting the scalability of the approach to longitudinal data. These findings suggest that zero-shot multimodal fusion of structural MRI and routinely collected clinical variables may provide a practical and scalable strategy for early AD risk stratification without task-specific retraining.
Read the full abstract

Image and Video Processing

What Matters is the Prompt: Prompt Sensitivity and Prompt Generation in Foundation Models for Lung Nodule Segmentation

Jorge F. Lazo, Xixi Liu, Andreas Hallqvist et al. (+5 more)

Lung nodule segmentation in computed tomography is essential for extracting clinically relevant information for lung cancer assessment and treatment planning. Foundation models have shown notable segmentation capabilities, but state-of-the-art approaches often depend on input prompts, such as points or boxes, making their performance sensitive to prompt quality and placement. Understanding the limitations and constraints of prompt-based foundation models is therefore essential for designing reliable medical image segmentation solutions. In this work, we investigate how prompt quality affects foundation models performance for lung nodule segmentation. We further propose a synthetic prompt-generation model to test if the dependence on manually provided prompts can be mitigated by generating synthetic prompts that can also improve segmentation performance. Perturbation experiments show that bounding box prompts generally outperform point prompts, while latest specialized medical imaging models achieve better performance than general purpose ones. The proposed approach obtains a Dice coefficient of 0.85, suggesting that synthetic prompt generation as a promising strategy for lung nodule segmentation with foundation models.
Read the full abstract

All authors Jorge F. Lazo, Xixi Liu, Andreas Hallqvist, Mikael Johansson, Åse Johnsson, Jonas S. Andersson, Jennifer Alvén, Ida Häggström

Image and Video Processing

A cross-modal generative model for incomplete and degraded prostate MRI with multicentre clinical validation

Siyuan Ma, Liang He, Mengying Zhu et al. (+14 more)

Missing or degraded sequences can limit prostate multiparametric MRI. We developed MSCNet, a sequence-conditioned cross-modal generative framework for reconstructing unavailable contrasts and restoring degraded acquisitions. Across ten completion tasks, task-specific MSCNet achieved mean structural similarity of 0.818 versus 0.798 for the strongest task-matched comparators; matched-capacity analyses showed larger differences in lesion fidelity and boundary preservation. In a blinded 1,000-case reader study, overall image quality met the prespecified non-inferiority criterion for DWI, ADC and T2W completion, but not T1W. In a separate 200-case diagnostic assessment, AUCs for clinically significant cancer were 0.860 with acquired images, 0.841 with MSCNet and 0.797 with baseline-generated images. A locked 186-case three-hospital cohort supported multicentre transportability. These retrospective results support quality-controlled cross-modal reconstruction as an adjunct to acquired prostate MRI.
Read the full abstract

All authors Siyuan Ma, Liang He, Mengying Zhu, Yi Chai, Mengyao Lyu, Haowei Wang, Qizhen Lan, HaoBo Sun, Qixin Zhang, Jingli Chen, Xiaobing Wei, Jiaming Liu, Guiqin Liu, Qianwen Zhang, Yang Liu, Dacheng Tao, Guangyu Wu

Image and Video Processing

BiCRVC: An Efficient Bidirectional Neural Video Compression Framework via Coupled Representation Coding

Wei Jiang, Junru Li, Kai Zhang et al. (+1 more)

Neural video compression (NVC) has achieved strong compression performance, but practical random-access coding still faces two technical challenges: existing bidirectional NVCs (BVCs) usually require costly motion-first decoding, and reliable motion estimation is difficult under long-range bidirectional prediction. To address these issues, we present BiCRVC, an efficient bidirectional neural video compression framework based on coupled representation coding. Instead of coding motion and frame information with two separate codecs, BiCRVC transforms the motion representation and the current-frame latent into a unified latent representation for entropy coding. This design enables motion and frame information to be decoded from the same bitstream with one unified codec, while still reconstructing motion-aligned contexts for frame decoding. To improve motion accuracy, we introduce multi-candidate motion estimation (MCME), which combines multi-scale motion estimation and parallel accumulated motion estimation to better handle diverse and long-range motions. To reduce motion coding overhead, we further propose bidirectional motion feature propagation (BMFP), which reuses previously decoded motion features at both the encoder and decoder as temporal priors for conditional motion coding. In addition, coupled distortion training and random GOP structure training are used to encourage joint motion-frame coding and improve adaptation to hierarchical random-access structures. Experiments show that BiCRVC achieves better compression performance than state-of-the-art BVCs while providing about 30 times faster 1080p decoding than recent BVCs.
Read the full abstract

All authors Wei Jiang, Junru Li, Kai Zhang, Li Zhang