Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Monday 31 August

Image and Video Processing · Cross-listed from cs.CV

Real-Time Video Anomaly Detection Using YOLO Pose Estimation and CLIP-Based Semantic Scoring

Vanodhya G. Warnasooriya, Amir Hajian, Watchara Ruangsang et al. (+1 more)

We propose a lightweight two-stage framework for real-time video anomaly detection. The first stage employs YOLO v11n-pose to detect persons and extract seventeen skeletal keypoints in a single forward pass. The second stage encodes each cropped person region through CLIP ViT-B/32 and computes cosine similarity against predefined textual descriptions of anomalous behaviors. This architecture eliminates the need for optical flow, standalone pose estimators, and density-based scoring modules. Experiments on CUHK Avenue, ShanghaiTech Campus, and a custom indoor dataset collected at Chulalongkorn University demonstrate an end-to-end throughput of approximately 51 FPS on an NVIDIA Titan XP GPU, a 3.36x speedup over the multi-feature baseline, while maintaining frame-level AUROC values of 89.26%, 70.26%, and 84.13%, respectively.
Read the full abstract

All authors Vanodhya G. Warnasooriya, Amir Hajian, Watchara Ruangsang, Supavadee Aramvith

Image and Video Processing · Cross-listed from cs.CV

LISynSeg: Data-Centric Label-to-Image Synthesis for Cross-Modality Whole-Heart Segmentation

Jiacheng Wang, Ivana Isgum and Ipek Oguz

Whole-heart segmentation (WHS) in computed tomography (CT) and magnetic resonance imaging (MRI) is affected by acquisition shifts and heterogeneous cardiac annotations. Existing WHS systems combine architectural design, transfer learning, and generic spatial or intensity augmentation. We investigate whether changes to data augmentation and training supervision can improve cross-modality WHS while the segmentation architecture is held constant. We present LISynSeg, a data-centric approach that augments real-image nnU-Net training with label-to-image synthesis. Synthetic volumes are generated from cardiac label maps using contrast and acquisition perturbations calibrated to the training cohort, then mixed with real images to retain thoracic context absent from the labels (and thus the synthesized images). We model cardiac label variation through controlled changes in myocardial wall thickness and partial supervision of uncertain vessel endpoints. On the CARE Whole-Heart benchmark, synthetic-only training performs worse than the real-image nnU-Net baseline, whereas calibrated real-synthetic training improves cross-modality segmentation without changing the architecture; the improvement is larger for MRI than for CT. The results show that modifying the training data strategy can benefit model development for heterogeneous cardiac data. Code and trained weights will be released at https://github.com/MedICL-VU/Care26_LISynSeg.
Read the full abstract

Image and Video Processing

OmniRAS: Standardizing Foundation Model Training and Evaluation in Robot-Assisted Surgery

Leonardo Borgioli, Neil Getty, Wenli Xiu et al. (+8 more)

Few foundation models exist for robot-assisted surgery, partly because large robotic-surgery video corpora are difficult to assemble and existing models are evaluated mostly on laparoscopic benchmarks. Further, most existing models are evaluated on a small set of public benchmarks, mostly focused on laparoscopic surgery. We present OmniRAS, a family of 1B- and 2B-parameter V-JEPA-2.1 encoders for robot-assisted surgery, and detail their training. First, we release two densely annotated robotic-cholecystectomy datasets: OmniRAS-PR and a multi-label YT-Chole tool-verb-target task, the first triplet-style annotation for robotic cholecystectomy, together with splits, probe protocols, and an inter-rater study validating the shared phase ontology. Second, we document continued pretraining at up to 256 compute nodes with global batch 6,144 over 19 sources totaling approximately 2,650 hours of surgical video, 51% robotic, and analyze compute and data composition. Third, we evaluate against raw V-JEPA-2.1 and specialized surgical models on six tasks spanning triplet, phase, and step recognition, action segmentation, and detection, under frozen-encoder and final-four-block fine-tuning regimes. Across three seeds, this yields 254 downstream runs, including 109 with partial backbone fine-tuning. The best OmniRAS models achieve the strongest adapted results across all task families, while frozen differences are smaller.
Read the full abstract

All authors Leonardo Borgioli, Neil Getty, Wenli Xiu, Jessica Cassiani, Alvaro Ducas, Carlos Agustin Orda, Hira Waris, Fangfang Xia, Rick Stevens, Pier Cristoforo Giulianotti, Milos Zefran

Image and Video Processing

ReLViC: Loss-Resilient Learned Video Coding with Dispersed Packetization and Controllable Packet Dependencies

Xuyang Chen, Daquan Feng, Xianfu Chen et al. (+1 more)

Packet loss can severely impair learned video coding because missing latent tokens compromise both spatial reconstruction and temporal prediction. We present ReLViC, a loss-resilient learned video coding framework that jointly addresses latent coding and packet-loss recovery. ReLViC disperses spatially adjacent latent tokens across packets and employs a dual-purpose Transformer to estimate entropy-model parameters during coding and reconstruct missing latent tokens at the receiver. It controls packet dependencies through a periodic-reset packet-context topology parameterized by the segment length, thereby tuning the trade-off between compression efficiency and error-propagation range without retraining. A three-stage progressive training procedure establishes single-frame coding, learns temporal context for entropy modeling, and then optimizes the recovery of masked latent tokens under simulated packet loss. Experiments using burst-loss traces evaluate ReLViC against H.265 protected by Reed–Solomon forward error correction (FEC) and GRACE, a loss-resilient learned video codec. ReLViC delivers more stable reconstruction and outperforms both baselines under severe packet loss.
Read the full abstract

All authors Xuyang Chen, Daquan Feng, Xianfu Chen, Xiang-Gen Xia

Saturday 29 August

Image and Video Processing

The MYOSAIQ Challenge: Myocardial Segmentation with Automated Infarct Quantification

Olivier Bernard, William A. Romero R., Cyprien Bouton et al. (+24 more)

Late gadolinium enhancement (LGE) cardiac magnetic resonance (MR) imaging is the modality of choice to assess myocardial infarction (MI) lesions. Nowadays MI volume quantification is not performed routinely in clinical practice. Numerous deep learning (DL) methods have been developed to automate the segmentation of the myocardium and infarct regions. However, most studies rely on relatively small datasets which typically undergo pre-processing steps to standardize images and focus on a specific phase of myocardial infarction following reperfusion therapy. These limitations have impeded the development of models that are generalizable across diverse conditions and thus suitable for routine clinical use. To advance research and establish benchmarks in generalizable learning for myocardial infarct quantification, this paper presents findings from the Myocardial Segmentation with Automated Infarct Quantification (MYOSAIQ) challenge. The dataset set up for the challenge combines 439 CMR volumes from two multicenter clinical trials, with representative data acquired in acute and chronic phases after acute MI. Data were acquired in 16 centers using MRI scanners from three different vendors. Six teams participated until the end of the challenge, employing various baseline models, data augmentation techniques, and confidence strategies. To enhance the significance of this study, we compare the challengers' results with those of fine-tuned foundation models. Our results indicate that well-designed UNet-based techniques outperform fully automatic foundation models for LGE MR segmentation. While the best methods achieve high-quality and stable delineations of the left ventricle and myocardium under various conditions, they remain improvable in accurately segmenting infarct regions.
Read the full abstract

All authors Olivier Bernard, William A. Romero R., Cyprien Bouton, Celia Goujat, Hang Jung Ling, Pierre-Marc Jodoin, Fumin Guo, Calder Sheagren, Graham Wright, Abdul Qayyum, Moona Mazher, Steven A. Niederer, Hairui Wang, Xiaomei Wu, Franz Thaler, Gernot Plank, Martin Urschler, Ricardo M. Rosales, Esther Pueyo, Nicolas Duchateau, Frederic Cervenansky, Patrick Clarysse, Loic Belle, Thomas Bochaton, Nathan Mewton, Magalie Viallon, Pierre Croisille

Image and Video Processing · Cross-listed from cs.CV

Bayesian-Optimized Superpixel-GrabCut for Traceable Optic Disc Segmentation

Shraddha Changune, Vivek Noel Soren, Gautam Das et al. (+1 more)

Optic disc (OD) segmentation is essential for diagnosing ophthalmic pathologies from retinal fundus images. However, prevailing deep learning approaches operate as opaque black boxes, lacking the inference-stage mathematical traceability–a critical requirement for algorithmic auditing and failure analysis in clinical workflows. This paper presents a fully algorithmically traceable and trainable segmentation pipeline that jointly combines superpixel decomposition, hybrid brightness-proximity superpixel scoring, morphological regularization, iterative GrabCut refinement, and elliptical shape fitting. The hyperparameter optimization is formulated as an objective function and solved via Bayesian optimization to eliminate manual parameter tuning. A quantitative evaluation on the Drishti-GS dataset demonstrates that our method achieves a Dice coefficient of 0.9536, matching state-of-the-art performance. By maintaining explicit mathematical transparency across all processing stages, our framework offers a deterministic, traceable alternative to black-box architectures for medical review and debugging.
Read the full abstract

All authors Shraddha Changune, Vivek Noel Soren, Gautam Das, Tapan Kumar Gandhi

Image and Video Processing · Cross-listed from cs.CV

Feature-Spectral Fragility in Segmentation: Dataset Dependence, Architecture-Specific Localization, and Spectral Correlates

Subhash Kashyap

Robustness of segmentation models is commonly assessed through input-domain perturbations, while dependence on frequency content within learned feature representations remains less understood. We probe this dependence using targeted post-training low-pass interventions on internal representations of three segmentation architectures, ResNet50-UNet (CNN), VM-UNet (SSM), and Swin-UNETR (Transformer), across CVC-ClinicDB and ISIC2018, with headline evaluations performed on untouched held-out test sets. At cutoff $\rho=0.25$, feature-domain low-pass filtering causes severe degradation on CVC: Dice drops by 100%, 73.2%, and 30.9% for CNN, SSM, and Transformer, respectively, compared with 9.4%, 10.3%, and 0.6% on ISIC. The cross-dataset difference is statistically significant for every architecture. Single-stage interventions further show that the most sensitive stage depends on architecture and dataset: the CNN peaks at the 2nd to 3rd encoder block, whereas the SSM peaks at the 1st to 2nd encoder stage. Native feature-domain spectral measurements show an inverse association between high-frequency energy and fragility on CVC; the relationship is only partial on ISIC and is therefore treated as a candidate correlate rather than a proven mechanism. Finally, Fourier augmentation improves robustness to input-space low-pass filtering but leaves feature-domain degradation essentially unchanged. These results show that feature-spectral robustness is strongly dataset-dependent, architecture-specific, and distinct from input-domain spectral robustness.
Read the full abstract

Image and Video Processing

Manifold-Constrained PET Reconstruction with Learned Flow-Matching Priors

Hengjia Ran, Jie Luo, Yutao Zhu et al. (+3 more)

Image reconstruction for positron emission tomography (PET) is an ill-posed Poisson inverse problem that often suffers from severe noise amplification and artifacts. In this work, we introduce an unsupervised, optimization-based reconstruction framework that employs a flow-matching generative model as a learned manifold prior. We train the flow-matching model on high-quality PET images to learn a deterministic ordinary differential equation transport from a Gaussian latent distribution to the empirical PET image distribution, yielding a differentiable generator of anatomically plausible images. We incorporate this generator as an explicit manifold constraint into a regularized Poisson likelihood formulation. We solve the resulting optimization problem using an alternating direction method of multipliers algorithm, in which an expectation-maximization-type surrogate update enforces data consistency and a gradient-based latent-space projection enforces manifold proximity. We evaluate the proposed method on both simulated and real PET datasets, assessing dose-level robustness, lesion-insertion generalization, and cross-scanner transfer. Compared with conventional reconstruction methods and state-of-the-art deep learning baselines, the proposed method provides superior noise suppression, structural preservation, and quantitative accuracy, while maintaining high computational efficiency.
Read the full abstract

All authors Hengjia Ran, Jie Luo, Yutao Zhu, Rui Hu, Huafeng Liu, Bo Zhao

Friday 28 August

Image and Video Processing

Evidence-Driven Differential Diagnosis of Malignant Melanoma

Naren Akash, Anirudh Kaushik and Jayanthi Sivaswamy

We present a modular and multi-level framework for the differential diagnosis of malignant melanoma. Our framework integrates contextual information and evidence at the lesion, patient, and population levels, enabling decision-making at each level. We introduce an anatomic-site aware masked transformer, which effectively models the patient context by considering all lesions in a patient, which can be variable in count, and their site of incidence. Additionally, we incorporate patient metadata via learnable demographics embeddings to capture population statistics. Through extensive experiments, we explore the influence of specific information on the decision-making process and examine the tradeoff in metrics when considering different types of information. Validation results using the SIIM-ISIC 2020 dataset indicate including the lesion context with location and metadata improves specificity by 17.15% and 7.14%, respectively, while enhancing balanced accuracy. The code is available at https://github.com/narenakash/meldd.
Read the full abstract

Image and Video Processing

Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction

Jin Yang, Tian Liu, Jing Wang et al. (+4 more)

Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram-based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury. Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer (MM-DiT) framework that jointly estimates RP risk and characterizes the sources of predictive uncertainty. MM-DiT integrates planning computed tomography (CT) images and three-dimensional radiation dose distributions through self-supervised multimodal pre-training, reducing reliance on limited and potentially noisy toxicity labels. The resulting representations are further refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation. A learnable label-noise model is incorporated to explicitly account for uncertainty arising from imperfect toxicity annotations. Therefore, it provides individualized RP risk estimates with complementary measures of aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability at the individual-patient level. We evaluated MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty. The results demonstrate its potential to provide accurate RP risk estimates while quantifying clinically relevant sources of predictive uncertainty.
Read the full abstract

All authors Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom, Ming Chao

Image and Video Processing

Patch-Based Diffusion Reconstruction for Accelerated Cardiac Cine

Xuan Lei, Philip Schniter, Juliet Varghese et al. (+1 more)

Purpose: To develop and evaluate a diffusion-based reconstruction framework for highly accelerated 2D real-time (RT) cine cardiovascular magnetic resonance imaging (CMR). Methods: We trained an unconditional patch-based diffusion model and incorporated it into a reconstruction framework, termed CineDiff, using diffusion posterior sampling for data consistency. CineDiff was evaluated in four settings: (i) 30 retrospectively undersampled breath-held cine at 1.5T and 3T from healthy participants across multiple acceleration rates, (ii) 15 prospectively undersampled free-breathing RT cine at 1.5T and 3T from patients indicated for clinical CMR, and (iii) 10 prospectively undersampled mid-field (0.55T) free-breathing scans, including five from healthy subjects and five from porcine models. For retrospective undersampling, reconstruction quality was assessed using peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), learned perceptual image patch similarity (LPIPS), and deep image structure and texture similarity (DISTS). For prospective undersampling, image quality was evaluated by blinded expert scoring on a 5-point Likert scale. Results: In retrospectively undersampled breath-held cine data, CineDiff achieved higher PSNR and SSIM and lower LPIPS and DISTS than the comparison methods across all evaluated acceleration rates. In prospectively undersampled free-breathing RT cine data, CineDiff received higher expert image-quality scores. Qualitatively, CineDiff reduced block-like artifacts and preserved finer anatomical detail compared with traditional compressed sensing and a variational network method, termed CineVN. Conclusion: CineDiff enabled high-quality reconstruction of highly accelerated 2D RT cine CMR. The method also demonstrated robustness to out-of-distribution data, including mid-field and porcine acquisitions.
Read the full abstract

All authors Xuan Lei, Philip Schniter, Juliet Varghese, Rizwan Ahmad

Image and Video Processing

Generative Translation Priors: Bayesian Imaging with Cross-Modality Image Translation

Evan Bell, Jiaming Liu, Yifan Chen et al. (+1 more)

The ability to leverage images from co-available modalities to inform target-domain reconstruction is highly desirable in imaging algorithms. In this work, we introduce Generative Translation Priors (GTP)–a Bayesian framework that transforms diffusion-based image-to-image translation models into cross-modality image priors for ill-posed imaging inverse problems. GTP incorporates target-domain measurements through likelihood guidance, steering the translation process toward the desired posterior distribution. The framework is grounded in a theoretical analysis of the resulting posterior dynamics, which reveals an intrinsic bias introduced by likelihood guidance. We further characterize this bias and derive a ground-truth-free formulation for its estimation, enabling it to serve as a practical metric for assessing posterior sampling quality. Building on this analysis, we derive two discretized GTP algorithms based on gradient and proximal likelihood guidance, respectively. We validate GTP on computed tomography reconstruction with magnetic resonance side information, and on positron emission tomography reconstruction with computed tomography side information. Experiments demonstrate that GTP effectively incorporates complementary cross-modality information and achieves high-fidelity reconstruction even under severely undersampled measurements.
Read the full abstract

All authors Evan Bell, Jiaming Liu, Yifan Chen, Yu Sun

Image and Video Processing

Medical Foundation Model Features as Perceptual Loss for Brain MRI Contrast Dose Simulation

Changsheng Fang, Dayang Wang, T. Campbell Arnold et al. (+2 more)

Perceptual losses are widely used in medical image synthesis because they encourage agreement in high-level structure beyond voxel-wise intensity similarity. In practice, most perceptual losses are still computed with natural-image backbones such as VGG16 or ResNet50, even when the target domain is magnetic resonance imaging (MRI). This mismatch may weaken supervision for anatomy, contrast enhancement, and acquisition variability. We test whether medical foundation model features provide a more suitable perceptual loss for brain MRI contrast dose simulation. The study has two stages. First, we compare RadImageNet, SegVol, and BrainIAC with ImageNet-pretrained VGG16 and ResNet50 as frozen feature extractors on four public medical imaging benchmarks: thyroid ultrasound, breast ultrasound, anterior cruciate ligament knee MRI, and meniscus knee MRI. RadImageNet achieves the lowest mean rank across the Stage I representation suite and is selected as $\phi^\star$. Second, we replace only the VGG16 feature extractor in an existing iterative brain MRI dose simulation framework with $\phi^\star$. The generator, reconstruction loss, adversarial loss, auxiliary losses, optimization schedule, and loss weights are kept unchanged. Standard metrics change modestly, with PSNR increasing from 41.63 to 41.74, SSIM from 0.9739 to 0.9754, RMSE decreasing from 0.1384 to 0.1369, and residual-uptake CNR from 0.0085 to 0.0082. The visual results show the main effect: RadImageNet reduces residual enhancement in marked structures, follows a more faithful dose-reduction trajectory, and remains close to the acquired 10% low-dose target. These results support domain-aligned radiology features as a practical perceptual feature space for MRI dose simulation, while leaving clinical equivalence and larger-cohort validation as future work.
Read the full abstract

All authors Changsheng Fang, Dayang Wang, T. Campbell Arnold, Enhao Gong, Srivathsa Pasumarthi

Image and Video Processing

Efficient JPEG Restoration in the Wavelet Domain via Mean Flows

Stefan-Alexandru Asandei and Mihai-Alexandru Radu

Latest JPEG restoration systems achieve strong quality with large models, yet often remain too slow and expensive for efficient on-device deployment. We present a 65M-parameter generative restorer that attains the lowest LPIPS at QF 10 and 20 on LIVE-1, Urban100, and DIV2K-val while sustaining 8.05 images/s at $1024\times1024$ on a single RTX 3090, roughly $4.9\times$ the reported throughput of one-step SODiff at one-twentieth of its parameters. Trained from scratch, the model replaces the learned VAE encoder-decoder with an exactly invertible two-level Haar transform, predicts a clean wavelet-domain residual through a rank-enhanced linear-attention DiT that estimates compression severity internally, and is optimized with an improved MeanFlow objective that enables inference in one or two network evaluations without distillation. Large pretrained priors remain stronger under severe compression (QF 5), whereas our model prioritizes throughput for deployment-constrained restoration.
Read the full abstract

Image and Video Processing

Evaluating the Effects of Inter-Observer and Model Variability on Radiological Peritoneal Cancer Index Assessment

Savvas Saragiotis, Pieter C. Gort, Lotte J.S. Fleurkens-Ewals et al. (+5 more)

Deep learning segmentation models are often evaluated using geometric metrics such as Dice, HD95, and ASD, yet it remains unclear to what extent improvements in these metrics translate into clinically meaningful changes in downstream decision-making. The metric-to-decision gap is examined using radiological Peritoneal Cancer Index (rPCI) region segmentation on contrast-enhanced CT, where a consensus definition provides anatomically grounded 3D regions and the clinically used PCI 20 threshold enables decision-level evaluation. Inter-observer variability is quantified across four experts on ten abdominal CT scans, and a published nnU-Net based rPCI segmentation model is benchmarked against this human reference using Dice, HD95, and ASD across all 13 regions. To relate geometric differences to clinical impact, a probabilistic peritoneal metastasis simulation is implemented on majority-vote rPCI maps, propagating region-boundary variability into variability of derived (r)PCI scores and classification at the PCI 20 cutoff. Observers showed high agreement (mean Dice $0.87$), while the model matched human performance in most regions but deviated more in regions 4, 8, and the small-bowel regions (9-12). Across simulations, score differences were typically small (mean $\Delta$rPCI $\approx 0.3$-$0.6$) for both observers and the model, and decision flips occurred predominantly when the reference score was near 20. These results suggest that rPCI-derived scoring is generally robust to typical segmentation variability, while highlighting borderline cases as the main setting where expert review remains essential.
Read the full abstract

All authors Savvas Saragiotis, Pieter C. Gort, Lotte J.S. Fleurkens-Ewals, Anna F. van Herwijnen, Marion Tops-Welten, L.D. Kampmeijer, Joost Nederend, Fons van der Sommen

Image and Video Processing

SCoPE-Reg: Efficient Rigid Ultrasound Slice-to-Volume Registration via State-Space Correlation and Closed-Form Pose Estimation

Niklas Schwarz, Jens Kleesiek and Moritz Rempe

Ultrasound-guided interventions can require localization of an untracked 2D frame within a 3D anatomical reference. Rigid slice-to-volume registration (SVR) estimates this six-degree-of-freedom pose but remains challenging because of limited anatomical context, acoustic artifacts, and view-dependent appearance. Existing methods often use dense cross-attention, whose cost scales with the product of slice and volume token counts, or direct pose regression without explicit correspondence constraints. We introduce SCoPE-Reg, combining state-space slice–volume interaction, dense 3D coordinate prediction, and parameter-free weighted Kabsch estimation. On SVR tasks from CAMUS and $\mu$-RegPro, SCoPE-Reg yields mean target registration errors of $0.73$ mm and $2.27$ mm against $1.24$ mm and $2.63$ mm for the state of the art (SOTA), reduces peak error on CAMUS by 56% below SOTA ($12.5\!\to\!5.5$ mm), and registers $100\%$ and $80\%$ of frames within $3$ mm. On CAMUS at $128^2$ it retains the lowest error at increasing pose-perturbation magnitude. It holds $6.49$ M parameters independent of resolution, sustaining $51$ FPS at $512^2$. SCoPE-Reg establishes a SOTA in rigid ultrasound SVR: by coupling correspondence-based accuracy with bounded worst-case error and resolution-independent cost, it becomes viable at native acquisition resolution during intervention, where prior methods trade accuracy, reliability, or frame rate against one another. Supplementary code provided and will be open-sourced upon acceptance.
Read the full abstract

Image and Video Processing

Evaluating the Safety of Deep Learning-Based Brain MRI Reconstruction

Dat Tat Mai, Thai Viet Pham, Thu Nguyen Thi Dang et al. (+1 more)

Objective: Deep learning accelerates brain MRI four- to tenfold, but models can erase lesions or synthesize false tissue - failures pixel-averaged metrics like PSNR and SSIM miss. We review whether current evaluation practices detect this blind spot. Methods: Following PRISMA 2020, we searched seven databases without date limits, including 263 studies (1995-2026), appraised them using QUADAS-2 and matched instruments, and synthesized narratively. Categories were derived from titles, abstracts, and controlled vocabulary; reported prevalence figures represent floors. Duplicate screening achieved high agreement (Fleiss kappa = 0.877), as did appraisal (0.788; 0.390 where observable). Extraction is unaudited. Results: Only 18 of 263 studies (6.8%) recorded both a fidelity metric and reader assessment on identical data, leaving the central surrogate unmeasured. Reader studies mostly measured inter-reader agreement, which was weak: fastMRI 2020 concordance reached 0.457 and 0.386 (Kendall W), improving only where SSIM diverged. Erasing a 100 mm3 lacunar infarct shifts global PSNR by 0.03 dB under the stated error model. As the corpus grew fivefold, reader assessments dropped from 32% to 18%, recovering to 21%. Generative models - most associated with hallucination (39%) - were among the least reader-evaluated (11.3%), while self-supervised models reached 47% with zero reader evaluation. Only 5% released code and ran reader studies; none evaluated a model observer; no named dataset covered acute stroke or hemorrhage. Conclusions: On these floors, current evaluation practices cannot certify diagnostic safety. We derive five requirements safety-oriented evaluations must meet.
Read the full abstract

All authors Dat Tat Mai, Thai Viet Pham, Thu Nguyen Thi Dang, James Jin Kang

Image and Video Processing

Coronary Mask Guided Registration for Continuous Time 4D Cardiac CT Dataset Construction

Yuang Wang, Shuo Wang, Changyu Chen et al. (+12 more)

Objective: Clinical cardiac CT multiphase reconstructions generally provide acceptable image quality in end-diastole (ED) or end-systole (ES) phases, but in other phases may exhibit motion artifacts, especially in the right coronary artery (RCA). This limits ground-truth availability in 4D cardiac CT imaging research. We aim to construct a 4D cardiac CT dataset that is generally suitable to serve as pseudo ground truth. Methods: We propose Coronary Mask Guided Registration (CMGR) to produce a motion-preserved, artifact-reduced, and continuous-time 4D cardiac CT sequence from the clinical multiphase reconstruction of each patient. For artifact reduction, CMGR uses the ED or ES phase as the reference phase and warps the reference volume with deformation fields to produce the sequence. For motion preservation, CMGR registers the reference phase to each non-reference phase of the multiphase reconstruction. To capture the motion of both the RCA and other cardiac structures in each registration, CMGR regularizes RCA masks and incorporates them into image-domain registration. Time-continuity is achieved by interpolating the deformation fields for non-reference phases to arbitrary times. Results: CMGR outperformed representative image-domain registration methods in capturing RCA motion and providing reasonable RCA shape, and showed competitive performance in capturing whole-heart motion. Additionally, CMGR reduced motion artifacts from clinical multiphase reconstructions, and intermediate CMGR frames generally provided plausible transitions between discrete cardiac phases. Conclusion: CMGR provides an effective approach for constructing continuous-time 4D cardiac CT datasets. Significance: The dataset can be used in system design simulations and in reconstruction algorithm development, thereby facilitating advances in cardiac CT imaging.
Read the full abstract

All authors Yuang Wang, Shuo Wang, Changyu Chen, Dufan Wu, Pengfei Jin, Yunqiang An, Yang Gao, Bin Lu, Dongrui Dai, Muge Du, Yan Yan, Dong Li, Liang Li, Li Zhang, Zhiqiang Chen

Image and Video Processing · Cross-listed from cs.CV

Lossy Event Compression: From Event Stream Distortion to Task Performance

Zahra Rezaee, Catarina Brites and João Ascenso

Event cameras generate asynchronous, sparse data streams with microsecond temporal resolution, but in moderate-to-high motion scenes they can produce as many as hundreds of millions of events per second, creating significant bandwidth and storage challenges. Lossy compression is therefore essential for practical deployment, yet existing event stream distortion metrics fail to reliably predict compression-induced degradation at the task level, forcing codec optimization to rely on expensive task-specific evaluations. To address this gap, this paper introduces two fundamentally different event compression pipelines: i) an aggregation-based pipeline that converts the event stream into polarity-based histogram frames for compression with the conventional image codec JPEG 2000, and ii) a frame-free point cloud-based pipeline that codes events natively as 3D points using the octree-based codec G-PCC. Both pipelines are then assessed within a unified task-driven evaluation framework that relates event stream distortion to downstream application performance across four representative tasks: i) video reconstruction, ii) object detection, iii) optical flow estimation, and a delay-sensitive task iv) asynchronous feature tracking under a reference-relative protocol. Building on this framework, five classification-based distortion metrics are applied to event compression for the first time, to the best of the authors' knowledge, and benchmarked against existing event stream metrics. Experimental results demonstrate that the proposed metrics reliably predict compression-induced task degradation across different coding frameworks. This demonstrates that event stream distortion assessment can be an efficient alternative to repeated task-specific evaluation, providing direct guidance for the development and optimization of future event data coding solutions.
Read the full abstract

Image and Video Processing

CheXtriev: Anatomy-Centered Representation for Case-Based Retrieval of Chest Radiographs

Naren Akash, Arihanth Tadanki and Jayanthi Sivaswamy

We present CheXtriev, a graph-based, anatomy-aware framework for chest radiograph retrieval. Unlike prior methods focussed on global features, our method leverages graph transformers to extract informative features from specific anatomical regions. Furthermore, it captures spatial context and the interplay between anatomical location and findings. This contextualization, grounded in evidence-based anatomy, results in a richer anatomy-aware representation and leads to more accurate, effective and efficient retrieval, particularly for less prevalent findings. CheXtriv outperforms state-of-the-art global and local approaches by 18% to 26% in retrieval accuracy and 11% to 23% in ranking quality. The code is available at https://github.com/cvit-mip/chextriev.
Read the full abstract

Image and Video Processing

Do Medical Vision Models Reason About Anatomy? Probing the Spatial Inductive Biases of Learned Visual Representations

Naren Akash and Neeraja Ramanan

Interpreting a CT scan means comparing structures on either side, judging how far apart organs sit, and knowing where each one belongs. Medical vision encoders are evaluated on diagnostic accuracy, or through assembled multimodal systems where a failure is hard to attribute, so it remains unclear whether their representations support any of this. We construct SPAR-Bench, eight probes over multi-organ abdominal CT that separate coordinate localization, relational reasoning, and spatial queries, and apply them to five architectural configurations and three medical foundation models, frozen and finetuned. Probes that ask for a comparison within the slice stay at chance, and neither pretraining scale, finetuning, nor architecture closes the gap. Probes that appear solved in domain fall to chance under zero-shot transfer, indicating that their accuracy reflects recall of canonical anatomy rather than computation over the image. Reading the same frozen features with a pooled head rather than the full set of tokens moves relational recovery from 0.7% to 67.8%, so pooled probing understates what a representation holds. Questions the encoders answer well are answered at chance by four open-weight MLLMs. Our results suggest these encoders carry a map of where organs usually lie, and little of the machinery for comparing structures within a particular patient. Code and data will be available at https://spar-bench.github.io.
Read the full abstract

Image and Video Processing · Cross-listed from cs.IT

Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation

Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi et al. (+3 more)

Token Communications (TokenCom) has recently emerged as a new paradigm in which tokens serve as unified units for communication and computation, enabling efficient multimodal semantic and goal-oriented transmission. In this paper, we develop Ada-TokenCom, a rate-adaptive TokenCom framework based on large autoregressive models, which integrates next-token prediction with arithmetic coding to achieve ultra-low bitrate semantic communication at the token level. We propose a mixed reconstruction/generation scheme, where the transmitter encodes and transmits the highly informative tokens at the beginning of the token sequence leveraging a pre-trained autoregressive large model, while the receiver uses an identical model to predict the rest. Moreover, we design a Lyapunov-based algorithm to dynamically optimize both the source compression rate and the modulation and coding scheme, adapting to time-varying network conditions. Simulation results demonstrate that our proposed Ada-TokenCom framework outperforms both digital and deep joint source-channel coding-based semantic communication baselines.
Read the full abstract

All authors Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Mehdi Bennis, Kaibin Huang

Thursday 27 August

Image and Video Processing

Enhancing MRI Brain Tumor Edge Detection: A Hybrid Preprocessing Approach Utilizing CLAHE

Shahid-E-Kaiser Md. Tashrif, Munshi Md Arafat Hussain, Sheikh Nahian et al. (+1 more)

Accurate boundary delineation of brain tumors in Magnetic Resonance Imaging (MRI) is a critical yet formidable challenge in neuro-oncology due to inherent scanner noise, complex anatomical structures, and uneven illumination. Traditional edge detection algorithms, while computationally lightweight and mathematically interpretable, frequently fail to capture the diffuse, localized boundaries of edema when relying solely on global preprocessing and manual parameter tuning. To overcome these limitations, we propose a hybrid automated edge detection pipeline. Our approach integrates an optimally configured Contrast-Limited Adaptive Histogram Equalization (CLAHE) layer into a comprehensive morphological preprocessing framework, followed by a deterministic sequential parameter sweep to fully automate threshold selection. The proposed hybrid model demonstrated enhancement in detecting critical anatomical structures in a publicly available benchmark database from Kaggle. By intelligently amplifying localized gradients without overwhelming the image with background noise, our method achieved higher Recall (Sensitivity). Consequently, the overall F1-Score elevated, and the Structural Similarity Index (SSIM) improved, all while maintaining a highly efficient execution. This establishes our optimized pipeline as a highly practical and near real-time operational model for clinical diagnostics, offering a compelling alternative to computationally heavy deep learning approaches.
Read the full abstract

All authors Shahid-E-Kaiser Md. Tashrif, Munshi Md Arafat Hussain, Sheikh Nahian, Sumaiya Islam

Image and Video Processing

Is Deformable Image Registration Ready for Brain Metastasis Reirradiation Dose Accumulation? A Longitudinal MRI Benchmark of Registration Accuracy

Hengjie Liu, Manju Sharma, Xinyi Fu et al. (+2 more)

Dose accumulation is increasingly important in adaptive radiation therapy and reirradiation, but its clinical validity depends on the performance of deformable image registration (DIR). Reirradiation of brain metastases (BMs) with stereotactic radiosurgery (SRS) provides a controlled but clinically meaningful DIR test case: intra-subject brain deformation is usually limited after rigid alignment, yet recurrent lesions can undergo substantial local shape and volume changes that rigid registration cannot capture and can affect dose accumulation. We benchmarked a wide range of learning-based and optimization-based DIR methods on 87 manually screened longitudinal contrast-enhanced T1-weighted MRI lesion pairs from an institutional BM SRS retreatment cohort. Learning-based methods pretrained on healthy-brain MRI were evaluated zero-shot and after instance-specific optimization (ISO) or tumor-proximity target-specific optimization (TSO). Registration was assessed using lesion overlap (Dice), surface distance metrics (HD95 and sASD), target-volume recovery, and runtime and memory. Pretrained learning-based methods showed variable zero-shot performance, while ISO/TSO improved all tested learning-based families. However, optimization-based methods remained the best-performing approach while maintaining reasonable runtime. These findings suggest that even state-of-the-art DIR methods do not yet provide sufficiently accurate and consistent registration for unmonitored use in brain metastasis reirradiation dose accumulation. Because accurate registration is a prerequisite for deformable dose accumulation, clinical application will require case-level quality control and direct assessment of how registration uncertainty affects downstream dose metrics.
Read the full abstract

All authors Hengjie Liu, Manju Sharma, Xinyi Fu, Di Xu, Ke Sheng

Image and Video Processing

CARDINAL Predicts Cardiovascular Risk From Non-contrast Cardiac CT

Roy Gabriel, Nattakorn Kittisut, Jamshid Hassanpour et al. (+6 more)

Cardiovascular risk prediction remains limited by incomplete clinical data and imaging biomarkers that reduce computed tomography (CT) to a small number of handcrafted features. We developed CARDINAL (Cardiovascular Assessment via Representation learning from Deep Imaging with Nested Anatomical Latent embeddings), a clinically grounded framework that learns compact representations from routine non-contrast cardiac CT for major adverse cardiovascular event (MACE) prediction. In 17,659 patients, CARDINAL was evaluated for 1-, 3-, 5-, and 10-year MACE prediction against American Heart Association (AHA) pooled cohort equations (PCE), AHA predicting risk of cardiovascular disease events (PREVENT), coronary artery calcium (CAC), segmentation-derived CT biomarkers, and 70-feature structural radiomics. Gains were largest at longer horizons. At 10 years, CARDINAL (joint) achieved an area under the receiver operating characteristic curve (AUROC) of 0.866 $\pm$ 0.020 and an area under the precision-recall curve (AUPRC) of 0.890 $\pm$ 0.015, compared with an AUROC of 0.826 $\pm$ 0.023 and an AUPRC of 0.826 $\pm$ 0.022 for structural radiomics, the strongest baseline. CARDINAL also achieved the highest survival concordance index (C-index), 0.753 $\pm$ 0.015, and high-versus-low risk-tertile hazard ratio, 10.78 $\pm$ 3.16, with favorable reclassification and exploratory calibration. These findings suggest that non-contrast cardiac CT contains prognostic information beyond conventional risk equations, CAC scoring, and engineered imaging biomarkers.
Read the full abstract

All authors Roy Gabriel, Nattakorn Kittisut, Jamshid Hassanpour, Michael Galarnyk, Abanoub Abdelmalak, Marly van Assen, Carlo N. De Cecco, Arshed Quyyumi, Ali Adibi

Image and Video Processing

Destroy Me: Automatic Artifact Generation for Histopathology Images

Zuzanna Krawczyk-Borysiak, Adam Krawczyk, Mateusz Miller et al. (+4 more)

Deep learning's diagnostic utility in pathology is constrained by model vulnerability to real-world data imperfections. While current strategies favor "perfect data" by filtering low-quality regions, which can lead to the loss of valuable diagnostic context, we propose a paradigm shift: engineering models to thrive in imperfect environments using "Destroy Me", a hybrid framework for realistic artifact synthesis and robust data augmentation. Our approach combines Stable Diffusion, fine-tuned to preserve morphological continuity by realistically integrating artifacts with the underlying tissue architecture, with physics-based procedural modeling to synthesize six common artifact types: tissue folds, precipitates, blur, stitching errors, dust, and pen markers. Artifact fidelity is assessed using Kernel Inception Distance (KID) and color Wasserstein distance metrics. Validating this strategy on lung adenocarcinoma pattern classification with an nnU-Net, we confirm that models trained on "destroyed" patches consistently outperform baselines on independent real-world datasets. Specifically, we observed a 10.5% relative improvement in macro F1-score and a 15% relative increase in the Cohen's Kappa ($\kappa$) coefficient. Crucially, our results demonstrate that selective, impact-weighted augmentation is vital for balancing practical robustness with the preservation of subtle diagnostic features.
Read the full abstract

All authors Zuzanna Krawczyk-Borysiak, Adam Krawczyk, Mateusz Miller, Gabriela Kaczmarek, Sławomir Pakuło, Małgorzata Sokół, Żaneta Swiderska-Chadaj

Image and Video Processing · Cross-listed from cs.IT

Knowledge Distillation Driven Semantic NOMA with GAN Refinement for 6G Robotic Vehicle Networks

Qifei Wang, Zhen Gao, Li Qiao et al. (+4 more)

To achieve sustainable intelligent mobility, 6G-empowered robotic vehicles (RVs) require high-fidelity visual perception under stringent bandwidth and energy constraints. Semantic communication offers a spectral-efficient solution but suffers from severe interference in uplink non-orthogonal multiple access (NOMA) RV networks. To address this, we propose a knowledge distillation-driven and generative models-enhanced NOMA framework for robust and green RV communications, named KDG-SemNOMA. First, we develop a ConvNeXt-based deep joint source-channel coding (DeepJSCC) architecture with an enhanced attention feature (AF) module for dynamic channel adaptation. Second, to mitigate interference without inference overhead, an orthogonal transmission teacher model guides the NOMA student model via a two-stage knowledge distillation strategy. Finally, to address the over-smoothing artifacts of pixel-wise optimization, we introduce a channel-conditional GAN (cGAN). By explicitly taking the Stage-I initial reconstruction and channel states as conditional inputs, this module refines coarse outputs into high-fidelity images with realistic textures. Experiments on FFHQ-256 demonstrate that KDG-SemNOMA significantly outperforms state-of-the-art methods in both pixel-level accuracy and perceptual fidelity.
Read the full abstract

All authors Qifei Wang, Zhen Gao, Li Qiao, Ziwei Wan, De Mi, Dapeng Li, Ying Sun

Image and Video Processing · Cross-listed from cs.CV

Anatomy-Guided Foundation Model Adaptation with Within-Case Prototype Supervision for Standard Plane Detection in Fetal Ultrasound Blind Sweeps

Yuzhe Zhao

Detecting the fetal abdominal circumference standard plane in low-cost obstetric blind sweeps is a highly imbalanced frame-classification problem: positive frames account for under 3% of a sequence, form short contiguous segments, and are poorly handled by off-the-shelf ultrasound and vision foundation models. We propose AnatoProto, a lightweight sequence-level framework that adapts a frozen BiomedCLIP encoder to fetal blind sweeps through four components: (i) anatomy-weighted spatial pooling that uses nnU-Net abdominal-region probabilities as a spatial prior to reweight BiomedCLIP patch tokens, so frozen semantic features are aggregated onto anatomically meaningful regions; (ii) a within-case prototype loss that pulls each frame embedding toward the mean of positive frames of the same sweep, exploiting case-level structure unavailable at the frame level; (iii) a three-stage cascade refinement (frame->segment->case-level rejecter) that lifts the prediction unit from noisy frames to structurally-constrained segments; and (iv) a hybrid prediction head that jointly models per-frame stability and inter-frame boundary transitions to suppress boundary false positives. On the ACOUSLIC-AI benchmark, AnatoProto reaches a test F1 of 67.72, outperforming the strongest foundation-model baseline (FetalCLIP + PRS, F1 = 54.52) by +13.20 F1 and the strongest video temporal-action-detection baseline (TriDet + PRS) by +15.76 F1. A synergy study, backed by embedding geometry and paired-bootstrap confidence intervals, shows that the prototype loss and anatomy-weighted pooling are not additive: applied alone the prototype loss reduces recall by 12 points, but combined with anatomy-weighted pooling it increases recall by 6.5 points – a sign-flip we trace to the accuracy of the within-case prototype.
Read the full abstract

Image and Video Processing · Cross-listed from eess.SP

DP-JMRNet: A Deep Unfolding Network for Differential Phase Preservation in Sparse Bitemporal SAR Reconstruction

Juncheng Bao, Zhen Zhang and George P. Petropoulos

Complex SAR imagery is usually visualized and evaluated mainly through its magnitude. Phase is retained in the complex data but is rarely treated as a direct image-quality objective. Existing sparse reconstruction methods typically focus on magnitude fidelity and single-epoch complex reconstruction accuracy. However, the phase difference between two acquisitions is what drives line-of-sight deformation retrieval in InSAR, from ground subsidence monitoring to earthquake deformation mapping. This paper proposes the Differential-Phase-Oriented Joint Masked Reconstruction Network (DP-JMRNet), which uses deep unfolding to reconstruct the two epochs jointly from masked observations under a differential-phase objective. An exchange-equivariant interaction module makes the reconstruction independent of epoch ordering. A coherence-aware gate opens cross-epoch sharing in coherent regions and closes it where the two epochs disagree. On simulated bitemporal SAR data, DP-JMRNet attains the lowest differential-phase RMSE at 30%, 40%, and 50% sampling rate, while maintaining competitive amplitude and complex-image fidelity. This corresponds to a 47.5%–51.3% reduction over the best baseline, achieved with one third of its parameters. The same trend is validated on three Sentinel-1 scenes. A systematic study of acquisition design further shows that sharing the same aperture support across epochs is necessary for phase fidelity, whereas optimizing the sampling mask does not improve the differential phase. The code and data are available at https://github.com/JasonBao05/coherent-sar-unfolding.
Read the full abstract

Wednesday 26 August

Image and Video Processing · Cross-listed from cs.CV

UltraPIPS: Improving model perception in B-mode ultrasound with foundation models

Tal Grutman and Tali Ilovitsh

In medical imaging, it is common to use learned perceptual image patch similarity (LPIPS) to compare images semantically in feature space. Although backbones pretrained on natural images are widely used for LPIPS computation, B-mode ultrasound images possess distinct speckle patterns and acoustic-specific image statistics that are fundamentally different from natural images and even from other images in radiology. Consequently, we propose that domain-specific models are needed to measure perceptual similarity in ultrasound data, a finding which is not necessarily the case for other imaging modalities. We compare LPIPS metrics across downstream tasks like classification, segmentation and reconstruction using natural image, medical generalist and ultrasound backbone models and show that selection of LPIPS backbone is a non-trivial design choice. In particular, the ultrasound backbone models were more correlated with downstream performance of supervised models than classical and natural image models, and optimization of the LPIPS loss with an ultrasound backbone achieved a strong balance between reconstruction quality and realism. Our code is available at https://github.com/talg2324/UltraPIPS and introduces the UltraPIPS library, a set of LPIPS metrics based on the open-source foundation models analyzed in this paper.
Read the full abstract