Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Monday 10 August

Image and Video Processing · Cross-listed from cs.CV

Structure-Enhanced Features and Quality-Aware Dynamic Anchor Scoring for Robust Lane Detection

Weize Cai, Yongqi Dong, Zhida Shao et al. (+2 more)

Lane detection requires recovering thin, elongated, and frequently occluded lane structures under challenging driving conditions. While anchor-based detectors provide efficient candidate generation, their performance is limited by two coupled issues: backbone features often lose structural continuity along partially visible lanes, and classification confidence may decouple from line-level localization quality, allowing inaccurate anchors to persist before non-maximum suppression (NMS). We propose a structure-enhanced and quality-aware framework that improves lane representation and dynamic-anchor scoring while preserving the inference pipeline of the Anchor Decomposition Network (ADNet). Specifically, a Gated Horizontal-Vertical Token (GHVT) module enhances mid- and high-level backbone features via lightweight directional token interactions with a learnable residual gate. In parallel, Line-Quality-Aware Dynamic Anchor Scoring (LQAS) calibrates existing classification logits using quality supervision, hard-negative suppression, and pairwise ranking without adding inference branches. On the VIL-100 dataset, our method improves ADNet-R34 from 89.97 to 91.28 in F1 score at the 0.5 intersection-over-union threshold (F1@50), reducing both false positives and false negatives. Additional experiments on CULane and TuSimple datasets, extensive ablations, score-distribution diagnostics, and runtime analysis confirm complementary structural and ranking improvements with minimal computational overhead.
Read the full abstract

All authors Weize Cai, Yongqi Dong, Zhida Shao, Yichen Liu, Zixin Fu

Image and Video Processing · Cross-listed from cs.CV

Deep Learning based Detection of Fishing Vessels and Fishing Monitoring using Nightlight Images

Shantakar Mohanty, Prasun Kumar Gupta and Raian Vargas Maretto

The demand for maritime surveillance has given rise to the need for monitoring fishing vessel activities, particularly in addressing the challenge of "dark vessels" that operate without Automatic Identification System (AIS) transmission. This study presents a novel approach for detecting small-scale fishing vessels using nighttime light (NTL) imagery from the SDGSAT-1 satellite, combined with deep learning techniques to enhance fishing monitoring awareness along the western coast of India. A dual-branch YOLO11 architecture was developed to exploit both the 10-meter panchromatic and 40-meter RGB imagery from SDGSAT-1. The custom model architecture was specifically optimized for small object detection in NTL imagery, featuring parallel convolutional backbones that process both modalities before concatenation for enhanced feature extraction. The dual-branch YOLO11 model demonstrated optimal performance with a precision of 0.99, recall of 0.93, F1-score of 0.96, and mAP@50 of 0.96, significantly outperforming single-branch implementations of YOLOv5s, YOLOv8s, and standard YOLO11s architectures. When applied to the western coast of India, the model detected 31525 vessel instances across the temporal dataset spanning 2022-23. Cross-matching analysis with AIS data revealed that only 7146 (22.7%) of detected vessels had corresponding AIS transmissions, while 24379 (77.3%) were identified as potential dark vessels. Spatio-temporal analysis showed peak fishing activity during January-April, with a primary activity corridor parallel to the coastline within 50-100 km, corresponding to productive continental shelf areas. This research contributes to maritime surveillance capabilities by highlighting the effectiveness of nighttime lights satellite imagery for fishing vessel detection and provides valuable insights into fishing patterns and potential regulatory compliance issues in Indian waters.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

Bright-Channel Retinex Enhancement with a Conditional Overdispered-Noise Analysis

Jongpil Jeong

I present a training-free low-light enhancement method that combines local bright-channel illumination estimation, Retinex division, and edge-preserving denoising. For a fixed illumination estimate, a conditional Negative -Binominal psueduo-count method characterises the heteroscedastic noise amplified by division. The unconstrained reflectance ratio is the pixelwise maximum-likelihood estimate, with a boundary solution for zero-valued observations; the implemented estimate additionally applies illumination filtering and range clipping. The NB model is a diagnostic noise analysis rather than a calibrated sensor model, and the final fixed-bandwidth bilateral filter is an empirical approximation rather than the exact Bayesian solution. On the LOL-v1 dataset, the methodobtains mean PSNR/SSIM of 17.74dB/0.739, the highest values among the evaluated with conventional methods. A 400X600 image is processed at approximately 43 FPS on an Apple M2 Pro CPU.
Read the full abstract

Sunday 9 August

Image and Video Processing · Cross-listed from eess.SP

One-Bin Fourier Challenges for Dimension-Free Reconstruction Certification

Milad Bafarassat

Partial-Fourier measurement underlies computational imaging, and its reconstructions increasingly come from iterative or learned solvers whose recovery guarantees are conditional on a signal model, a sampling law, and solver accuracy. None of those guarantees transfers to a particular committed output: it can satisfy every acquired coefficient while remaining badly wrong in the unmeasured nullspace. Native Fourier holdout does not repair this, since a spectrally concentrated error is missed unless its bin is drawn, so uniform worst-case risk needs a number of hidden bins proportional to the dimension. We propose a post-commit acceptance test: secret finite-phase masks turn one fixed Fourier readout into dense randomized residual checks, and a median-of-means threshold handles calibrated readout noise. We prove that errors above $E_{\rm rej}$ are rejected and errors below $E_{\rm acc}$ are accepted with confidence $1-\delta$ once $q\ge C(1+N\sigma^2/E_{\rm rej})^2\log(1/\delta)$, explicitly exposing the one-bin resolution floor $N\sigma^2/\sqrt q$; in the zero-noise limit, QPSK gives $4^{-q}$ soundness versus $97.9\%$ minimax false acceptance for a worst-case ordinary-bin error. At a fixed noise-to-threshold ratio, measured screening false acceptance fell from about $21\%$ at 16 checks to $5\%$ at 64; a data-consistent learned nullspace failure with NMSE $0.200$ was rejected by $96.9\%$ of challenge banks while 16 ordinary bins missed it $98.9\%$ of the time; and error statistics at 16 checks stayed stable from 128 through $65{,}536$ dimensions. The test supports solver acceptance, model selection, and stopping certification, with a certified fresh-bank error interval.
Read the full abstract

2609.25019 · 9 Aug · The author withdraws this paper because the coded-diffraction implementation claimed in Section 3.3 is not established. The certification procedure requires complex-valued linear Fourier measurements, whereas intensity-only acquisition does not directly provide them. The manuscript does not establish the additional acquisition procedure needed to bridge this gap

Abstract PDF $FOUR ×4

Image and Video Processing

MD2G-Cast: Relay-Coordinated Multicast for Scalable Volumetric Streaming over MoQ

Ruonan Chai, Yisu Wang, Zili Meng et al. (+1 more)

Volumetric streaming remains difficult to scale because receivers with overlapping fields of view are often served independently, causing repeated transmission of shared content. We present MD2G-Cast, a relay-coordinated multicast framework over Media over QUIC with an application-aware control layer for scalable multi-user volumetric delivery. MD2G-Cast jointly uses viewing overlap, receiver capability, and bandwidth conditions to form reusable multicast groups, share common Base content, and selectively admit Enhanced delivery. We formulate grouping and Enhanced admission as a sequential control problem, realize it with Proximal Policy Optimization (PPO), and train a compact relay model with teacher guidance for Enhanced admission. We implement MD2G-Cast with real MoQ processes and evaluate it with real access and 6DoF viewing traces for up to 100 users. At 20 and 100 users, MD2G-Cast keeps the receiver-side $P_{99}$ delivery interval below 40 ms across all seven access profiles, while Rolling reaches the 500 ms reporting cap in most cases. Across the evaluated user scales, MD2G-Cast achieves the highest or tied-highest mean system utility under homogeneous access and the highest mean utility under heterogeneous access, while reducing aggregate link load by about 27% relative to Clustering at 100 users. A matched relay-control ablation separates the control structure from its optimizer, showing that random feasible actions reduce utility while deterministic control remains competitive with PPO. Together, the results support relay coordination and selective Enhanced admission, rather than a particular policy optimizer, as the central design contribution.
Read the full abstract

All authors Ruonan Chai, Yisu Wang, Zili Meng, Dirk Kutscher

Saturday 8 August

Image and Video Processing

Retrieval-Augmented Generation-Based Color Restoration for Low-Light Image Enhancement

Li-Wei Lu and Shaou-Gang Miaou

Recent low-light image enhancement (LLIE) methods have driven brightness and structural fidelity close to that of normally-exposed images, yet their outputs still exhibit systematic color shifts such as greenish skies, yellowish faces, and warm-tinted white objects. We attribute this to end-to-end LLIE training coupling brightness, structure, and color within a single network, leaving the color channels weakly supervised. We recast color restoration as an independent sub-problem and decouple it from brightness enhancement, realizing it as a general-purpose post-processing module built on retrieval-augmented generation (RAG). Rather than relying solely on parametric color priors learned during training, the module dynamically retrieves a reference image from an external high-quality color knowledge base and injects its color distribution into a color-restoration network to correct residual bias. The design has three components: (i) a dual-index FAISS retriever built on intermediate VGG19 features, capturing textural and structural similarity through global mean and variance statistics; (ii) GlobalSPHistAdaIN, which reduces the reference spatial-preserving color histogram to a global color vector and modulates network features via adaptive instance normalization, removing dependence on pixel-level correspondence; and (iii) a residual formulation that predicts a color correction over the front-end output. Across LOLv1, LOLv2-Real, and LOLv2-Synthetic, the module consistently improves color-specific metrics, and it remains effective when the front end is swapped among CPGA-Net++, LLFormer, FLIGHTNet, and IAT, confirming cross-front-end generality. Ablations show that a VGG19 dual index outperforms CLIP-based retrieval, indicating that color restoration depends on textural and structural similarity rather than high-level semantics.
Read the full abstract

Image and Video Processing

$\texttt{DisMorph}$: learning to disentangle technical distortions from true biological change

Jingru Fu, Kathleen E. Larson, Douglas N. Greve et al. (+2 more)

Longitudinal MRI enables sensitive measurement of structural brain change for studying aging and neurodegenerative disease. Deformable image registration is a key tool for estimating such change by computing a dense deformation that captures geometric differences between longitudinal scans. However, MRI scanners introduce geometric distortions that vary across acquisition systems and protocols, such as gradient non-linearity (GNL) distortion. Existing registration methods estimate a single field that conflates biological and technical effects, potentially biasing downstream morphometric measurements if distortions remain (partially) uncorrected. We propose $\texttt{DisMorph}$, a registration framework trained entirely on synthetic data that explicitly decomposes longitudinal deformation into technical and anatomical transforms. It predicts two dense deformations, each encoding one effect. During training, a novel generative model synthesizes both effects separately to provide disentanglement supervision, while domain randomization promotes generalization across imaging protocols. We evaluate our method in three complementary settings. On simulated data with known ground truth, our method detects anatomical change more accurately and consistently than conventional registration. On real image pairs that differ only by GNL distortion, our method assigns most geometric change to the distortion field, demonstrating specificity in the absence of anatomical change. On longitudinal Alzheimer's disease (AD) pairs, our method detects anatomical change in AD-related brain structures while identifying residual distortion left after standard correction. By disentangling MRI-induced distortion from biological change in the longitudinal deformation, our method paves the way for more accurate longitudinal morphometry in clinical settings where maintaining acquisition consistency is challenging.
Read the full abstract

All authors Jingru Fu, Kathleen E. Larson, Douglas N. Greve, Bruce Fischl, Malte Hoffmann

Image and Video Processing · Cross-listed from econ.EM

Beyond the Grid: Cost, Carbon, and Capital Requirements of On-Site Power Technologies for AI Data Centers

Eliseo Curcio

Interconnection queues, not electricity prices, now govern where data centers can be built, and the standard levelized-cost comparison answers a question no developer faces: it assumes a load profile, freezes the grid price while modeling the demand that moves it, and quotes busbar costs a facility cannot buy. This paper evaluates nine on-site supply technologies against a delivered grid whose price is endogenous to projected data-center demand, on a complete-site basis that retains standby charges, with measured GPU training load, delivered fuel prices, production-pathway carbon, and statutory 45V and 48E incentive mechanics. Nothing beats the wire: gas combined cycle produces at 47 USD/MWh but costs about 114 USD per megawatt-hour of complete site energy against a 92 USD grid; four-hour storage is physically capped near 18 percent of annual energy and, charged at the margin, dirtier than the grid; hydrogen from grid-priced power fails on cost and carbon together. An investment inversion converts these findings into capital terms: conversion-hardware learning buys nothing, because free hardware still exceeds the grid for every low-carbon arm, while global electrolyser deployment on sited sub-20 USD/MWh power brings PEM hydrogen power to about 2.2 times the grid at 300 billion USD and 1.9 times at 1 trillion USD (2.7 and 2.3 for the hydrogen engine), with a carbon reduction of roughly 85 percent (6.8-fold) against grid-power production. Grid parity is not purchasable at any budget. On-site supply is an access and depth product; most current investment targets the wrong term.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

EvBS: Event-guided Blur Synthesis for Domain-adaptive Motion Deblurring

Junsik Jung, Seokryun Choi, Yoonki Cho et al. (+3 more)

Motion deblurring has achieved remarkable progress with deep learning, yet pre-trained deblurring models often suffer from performance degradation in real-world scenarios due to the domain shift between training and testing distributions. To remedy this, we propose EvBS, an event-guided blur synthesis framework that generates diverse training pairs for calibrating pre-trained models to the target domain. While existing methods are constrained by the inherent entanglement between motion and visual content, our method leverages the high temporal resolution of event cameras to effectively decouple them. This enables us to utilize not only the intrinsic motion that is inherent to the given content but also extrinsic motion transferred from different sources within the target domain, thereby facilitating effective adaptation via fine-tuning. Specifically, EvBS comprises two complementary strategies: Intrinsic-Blur Synthesis, which blurs sharp contents with their own motion patterns, and Extrinsic-Blur Synthesis, which transfers motion from blurry patches to distinct sharp content. This approach generates a diverse set of training pairs that break the inherent constraints of naturally coupled motion and content, resulting in enhanced domain-adaptive deblurring performance. Extensive experiments on multiple benchmarks demonstrate that EvBS effectively enhances the robustness of existing deblurring models on unseen testing datasets.
Read the full abstract

All authors Junsik Jung, Seokryun Choi, Yoonki Cho, Woo Jae Kim, Andrew Jeong, Sung-Eui Yoon

Image and Video Processing · Cross-listed from cs.CV

Low, High, and Semantic-Level Vision-Language Learning for AI-Generated Image Detection

Qian Yao, Luming Yang, Jun-Jie Huang et al. (+3 more)

Driven by advances in diffusion models and autoregressive models, the fidelity and resolution of AI-generated images now rival those of real images. However, existing AI-generated image detection methods often downsample the images,inevitably overlooking critical low-level texture details in high-resolution AI-generated images, thereby limiting their detection performance. In addition, the continuous emergence of unknown generative models makes large-scale pre-training datasets inaccessible. To address these challenges, we propose a novel AI-generated image detector, termed LHSDet (Low-, High-, and Semantic-level Detection). Specifically, we formulate the AI-generated image detection task as a Visual Question Answering problem, leveraging a fine-tuned vision-language framework to fully exploit the complementary information between visual and textual modalities. Recognizing that the default visual encoder of existing vision-language models is not tailored for AI-generated image detection, we redesign a visual encoder to better capture both the low-level and high-level artifacts inherent in AI-generated images. Furthermore, we incorporate a semantic-level textual branch to enable multi-modal feature fusion and detection. Consequently, LHSDet employs a triple-branch architecture to extract complementary multi-modal features: a low-level visual branch that aggregates non-overlapping patches for local texture cues, a high-level visual branch based on SigLIP2 for global perception feature extraction,and a semantic-level textual branch that generates captions using BLIP-2. Extensive experimental results demonstrate that LHSDet achieves strong detection accuracy and robust performance across diverse generative models.
Read the full abstract

All authors Qian Yao, Luming Yang, Jun-Jie Huang, Yongjun Wang, Xinwang Liu, Meng Wang

Friday 7 August

Image and Video Processing

LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3

Yang Zhou, Edoardo Occhipinti, Banboye Kidzeru Elvis et al. (+11 more)

Hierarchical Phase-Contrast Tomography (HiP-CT) is a synchrotron based X-ray imaging technique that enables non-destructive, volumetric imaging of intact organs with multi-resolutions bridging 20 $\mu m$/voxel for whole organs to near-cellular resolution ($\sim$0.8 $\mu m$/voxel) in local regions. This offers the opportunity to bring volumetric whole-organ context to histology. However, nonlinear registration between H&E histology and HiP-CT volumes is challenging due to the differences in feature representations of different colour spaces. Synthesis-before-registration methods have shown strong results in histology-to-MRI and histology-to-CT alignment. However, existing approaches either rely on manual anatomical contours or are trained from scratch without semantic constraints, limiting their generalisability to soft tissue organs and novel modalities. We propose LoRCA (LoRA Cycle Adaptation), a cycle consistent style translation framework built on a shared frozen DINOv3 with modality-specific LoRA adapters, learning modality-specific representations that are decoded and adversarially trained. LoRCA enables structure-preserving translation without requiring paired training data. The frozen backbone is intended to be a structural anchor that prevents content drift by preserving pretrained semantic-extraction capability. We evaluate translation quality using Fréchet Inception Distance (FID) and structural fidelity via mutual information and Canny edge preservation. LoRCA outperforms CycleGAN in both translation quality and structural consistency. As a preliminary indicator of downstream registration utility, we find that style-translated images yield increased feature correspondences under MatchAnything on manually aligned HiP-CT and histology test pairs, suggesting that LoRCA-style translation is a promising step towards 2D histological sections to 3D HiP-CT volumes registration.
Read the full abstract

All authors Yang Zhou, Edoardo Occhipinti, Banboye Kidzeru Elvis, Jishizhan Chen, Stathis Megas, Joseph Brunet, Joanna Purzycka, Theresa Urban, Hector Dejea, Sarah Amalia Teichmann, Menna R Clatworthy, Paul Tafforeau, Peter D Lee, Claire L Walsh

Image and Video Processing

Implicit representations are dead. Long live explicit primitives!

Nil Stolt-Ansó, Maik Dannecker, Wenqi Huang et al. (+2 more)

Continuous parameterization of medical data has emerged as a powerful paradigm for resolution-independent image representation. While Implicit Neural Representations offer high fidelity and compact storage, their reliance on global Multi-Layer Perceptrons incurs sizeable computational costs, large memory requirements, and extensive optimization times. As medical imaging trends towards ever-more detailed, high-resolution volumes, these costs impose significant bottlenecks in the applicability of implicit approaches. Recently, explicit Gaussian-based primitives have revolutionized the representation learning paradigm by trading deep network evaluations for localized, rasterization-friendly primitives. In this paper, we present a comprehensive, cross-dimensional evaluation of Gaussian representations against implicit approaches for medical imaging applications. First, we outline a theoretical overview on the mathematical properties offered by explicit primitives beyond what is capable under the implicit neural paradigm. Subsequently, we benchmark the computational performance on two demanding image datasets: 2D microscopy histology and 3D lung Computed Tomography (CT). Our experiments demonstrate that Gaussian representations consistently match or surpass reconstruction metrics compared to implicit methods across all compression factors, while displaying significantly lower optimization times, and memory requirements. Together with the compelling mathematical properties offered by explicit primitives, these findings motivate the wider adoption of Gaussian representations and position them as an attractive direction for future research in medical imaging.
Read the full abstract

All authors Nil Stolt-Ansó, Maik Dannecker, Wenqi Huang, Andras Jakab, Daniel Rueckert

Image and Video Processing

Pre- to Post-Contrast Synthesis of Breast DCE-MRI using Latent Bridge Matching

Sina Amirrajab, Zohaib Salahuddin, Henry C Woodruff et al. (+1 more)

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is central to breast cancer imaging, but gadolinium administration increases scan burden and motivates contrast-reduced alternatives, including synthetic contrast generation. We propose a latent bridge matching (LBM) framework for synthesizing peak-enhanced breast DCE-MRI from pre-contrast images in the MAMA-SYNTH challenge setting. Instead of starting from Gaussian noise as in conventional latent diffusion models (LDMs), the proposed model learns a conditional bridge between paired pre-contrast and peak-enhanced VAE latents. A latent UNet predicts the remaining correction from intermediate bridge states to the peak-enhanced latent, enabling iterative refinement while keeping the trajectory anchored to patient-specific anatomy. We evaluated two LBM conditioning variants on 91 DUKE validation cases. For the tumor-conditioned variant, tumor masks were used as conditioning inputs. Tumor-conditioning improved performance compared with pre-contrast conditioning, reducing MSE from 1.023 to 0.940 and FRD from 7.523 to 4.716, while increasing tumor SSIM from 0.355 to 0.429. The tumor-conditioned LBM also outperformed the evaluated LDM baseline on this validation cohort. These results suggest that latent bridge matching is a promising pre-contrast-anchored formulation for virtual contrast enhancement, while further work is needed to validate generalization and remove dependence on ground-truth tumor masks at inference.
Read the full abstract

All authors Sina Amirrajab, Zohaib Salahuddin, Henry C Woodruff, Philippe Lambin

Image and Video Processing

Robustness of transferability estimation metrics for medical imaging

Niclas Claßen, Théo Sourget, Dovile Juodelyte et al. (+2 more)

In transfer learning, the choice of source model largely influences the performance on a target dataset. Still, selecting a fitting source remains a challenging task, especially in medical imaging where one has to decide between models pre-trained on off-the-shelf options, such as ImageNet, and domain specific datasets. Transferability estimation (TE) metrics address this problem by aiming to predict the best performing source model in a computationally cost effective way. However, previous work has reported conflicting TE metric performances due to differences in experimental setups. Moreover, most TE metrics are designed for and evaluated on natural images, while being optimized for accuracy, whereas in medical imaging metrics that are more robust to class imbalance are typically used. We study the impact of varying the target dataset as an isolated factor, by constructing miniature populations of different sample sizes and random seeds. In addition, we investigate the influence of the evaluation metric used to obtain the reference ranking. We find that small modifications to the target dataset change the rankings. Furthermore, we show that the choice of evaluation metric affects the reference rankings and therefore the evaluation of TE metrics. Overall, we observe a low agreement between rankings from TE metrics and reference. The code, model checkpoints and data splits used in this work are available through https://github.com/niclasclassen/robustness-of-transferability-estimation-metrics-for-medical-imaging.
Read the full abstract

All authors Niclas Claßen, Théo Sourget, Dovile Juodelyte, Rob van der Goot, Veronika Cheplygina

Image and Video Processing

Energy and Performance Benchmarking of Deep Learning Models for Breast Cancer Detection

Samar Garrab and Ghada Achour

Recent advances in machine learning have greatly improved breast cancer detection, enabling more accurate and timely diagnosis. Deep learning (DL) models show strong potential for medical image analysis; however, as their architectural complexity increases, their environmental impacts are becoming a growing concern. In this paper, we present a comparative analysis of seven DL models for breast cancer detection on two medical datasets: Breast Ultrasound and BreakHis 400X. The evaluated architectures range from Convolutional Neural Networks (CNNs) and transformers to hybrid models. In addition to performance metrics, we assess CO2 emissions during both training and inference. Our results show that EfficientNet and ResNet consistently deliver strong performance, although with higher CO2 emissions. The selected transformers, such as DeiT-Tiny, perform competitively on both datasets, whereas DenseNet121 achieves lower accuracy. On the Breast Ultrasound Dataset, DeiT provides the most favourable balance between accuracy and energy consumption, whereas on the BreakHis dataset, the ViT and Swin models achieve the best results. Overall, our findings indicate that no single architecture category from the evaluated ones consistently dominates across the two selected datasets. Our results highlight the importance of jointly considering performance, emissions, and dataset characteristics when selecting models for medical applications.
Read the full abstract

Image and Video Processing

Structural Guidance for Unified Joint Demosaicing and Denoising

Qixin Zheng, Ping Chen, Qiangqiang Shen et al. (+1 more)

Joint demosaicing and denoising is a fundamental step in camera image signal processing, yet remains challenging because different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content. Existing unified restoration networks explicitly model CFA geometry but are still driven primarily by pixel-level supervision, making them prone to structural degradation around edges, repetitive textures, and moiré patterns where local evidence is unreliable. We attribute this limitation partly to the absence of explicit structural guidance beyond pixel-level reconstruction supervision. Motivated by this observation, we propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration. Our model receives a unified five-channel observation consisting of the raw mosaic, CFA masks, and a noise-level map. A SwinIR restoration branch reconstructs pixel details under CFA-conditioned modulation, while a parallel structural reasoning branch extracts complementary structural cues from a sparse pseudo-RGB observation. To bridge the substantial domain gap between sparse noisy sensor data and the natural-image pretraining domain of the structural encoder, we introduce a lightweight trainable adapter before residually fusing structural and restoration features. A shared decoder jointly predicts the restored RGB image and an auxiliary clean mosaic, providing supervision in both image and sensor domains. Extensive experiments across multiple CFA patterns and noise levels demonstrate consistent improvements over state-of-the-art unified and CFA-specific methods, indicating that adapted structural priors can enhance robust camera image restoration. The source codes and dataset are provided in the supplementary material.
Read the full abstract

All authors Qixin Zheng, Ping Chen, Qiangqiang Shen, Haijin Zeng

Image and Video Processing

SpecF2M: A Spectral-Aware Multi-task Network Estimating Axial Length and Refractive Error from Pediatric Fundus Photographs

Mengxian He, Xinyue Liu, Yunyun Sun et al. (+5 more)

Spherical Equivalent Refraction (SER) and Axial Length (AL) are core indicators for pediatric myopia screening, yet their measurements require dedicated biometry and cycloplegic refraction. Fundus photography offers an accessible imaging modality, as myopia-related posterior-pole changes are visible in 45$^\circ$ fundus images. However, these cues are often low-contrast, spatially diffuse, and multi-scale. Moreover, AL, Sphere (SPH), and Cylinder (CYL) share partially overlapping but non-identical anatomical correlates. We propose SpecF2M, a spectral-aware multi-task network for estimating AL and SER components from pediatric fundus photographs. SpecF2M integrates a deterministic anatomy-guided enhancement module, a hybrid spatial–spectral backbone combining MixCNN and Hybrid Spectral Learning (HSL) blocks, and an expert-routing head for component-level estimation of AL, SPH, and CYL. On a pediatric cohort of 4,359 eligible child visits and 6,966 fundus images, SpecF2M outperforms controlled CNN/ViT baselines for AL and SPH estimation, achieving MAEs of 0.5347 mm and 0.7062 D, respectively. Component-level analysis further reveals asymmetric task coupling, where CYL exhibits weaker association with fundus-derived myopic patterns than AL/SPH. These results support fundus-based, screening-oriented estimation of pediatric myopia indicators, while external validation remains necessary before deployment.
Read the full abstract

All authors Mengxian He, Xinyue Liu, Yunyun Sun, Wei Hao, Minqing Zhang, Lichun Wang, Shunyi Zhang, Wu Yuan

Image and Video Processing

APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT–IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction

Xincan Zheng, Yaqi Wang, Zhi Li et al. (+4 more)

Registration between cone-beam computed tomography (CBCT) and intraoral scans (IOS) is essential for patient-specific surgical planning. However, disparate imaging modalities, limited overlap, and large pose offsets make automated registration unreliable. Consequently, clinical registration remains dependent on conventional geometry pipelines and manual clinician adjustment. To address these challenges, we propose APCReg, an anatomical-prior-guided coarse-to-fine framework for global registration and reliability-controlled residual correction. Specifically, multi-view anatomical coarse registration (MACR) performs ordered orthogonal projection alignment (buccal, proximal, and occlusal) to decompose the six-degree-of-freedom search before three-dimensional refinement. Overlap-aware residual registration (OARR) combines shared KPConv features, a folded arch-length cue, overlap-gated cross-attention, and Sinkhorn matching. Finally, dental-arch-structured hypothesis selection evaluates diverse poses on held-out reliable correspondences, while a ground-truth-free coarse-retention guard conditionally retains a geometrically reliable coarse pose. On 60 held-out jaw pairs, APCReg achieves a submillimeter mean Chamfer distance of 0.87 mm and a Hausdorff distance of 2.92 mm under this evaluation protocol, and ranks first across the six reported metrics among the evaluated open-source baselines.
Read the full abstract

All authors Xincan Zheng, Yaqi Wang, Zhi Li, Jiahao Bao, Lan Feng, Yiru Xia, Shuai Wang

Image and Video Processing

Knowledge-Guided 3D CT Generation: A Conditioning-Centric Taxonomy

Francesca Pia Panaccione, Eugenio Lomurno and Matteo Matteucci

Controllable generation guided by external knowledge is a key requirement in modern generative deep learning applications, enabling the synthesis of samples with explicit constraints on semantic content, structural properties, and variability. In 3D Computed Tomography (CT), such control is essential for clinical applications, including data augmentation, privacy-preserving data sharing, and the simulation of specific anatomical or pathological scenarios. While research on conditional 3D CT generation has expanded rapidly, the diversity of existing approaches makes systematic comparison difficult and obscures fundamental design choices. In this survey, we propose a conditioning-centric taxonomy that organizes the literature along three orthogonal dimensions: the type of external knowledge (K), the knowledge integration paradigm (I), and the generative architecture (A). This factorization defines an explicit design space (K x I x A) that provides a unified perspective on prior work. Using this framework, we systematize existing methods, identify dominant trends and recurring design patterns, and highlight underexplored regions of the design space that point toward promising directions for future research.
Read the full abstract

Image and Video Processing

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang et al. (+3 more)

Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging during treatment could support treatment response assessment. Many existing imaging-based approaches rely on a single static timepoint, which fails to capture changes that occur during treatment. In this work, we present a longitudinal framework that combines a frozen 3D foundation encoder (Pillar-0) with our Temporal Dynamics Network (TDN) to predict treatment response from serial Dynamic Contrast-Enhanced (DCE) MRI acquired across four clinical timepoints from pre-treatment to pre-surgery. The TDN combines time-aware volumetric embeddings with clinical and treatment data to predict pCR. Evaluated on 982 patients from the combined I-SPY2 and ACRIN-6698 cohort, the proposed model achieves strong performance across all reported metrics when longitudinal 3D imaging is fused with clinical data (test AUROC: 73.6%, balanced accuracy: 69.1%). While clinical variables provide the strongest individual predictive signal, longitudinal 3D imaging contributes complementary information when fused with clinical data, improving pCR prediction. Our source code is available at: https://github.com/omarftt/longitudinal_temporal_pillar.
Read the full abstract

All authors Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang, Pegah Khosravi, Yu Tian, Chen Chen

Image and Video Processing

Integrating spectral and morphological plant features with decision-tree models for early-season cotton biomass and nitrogen status estimation from multi-year UAV data

Vaishali Swaminathan, Nithya Rajan, J Alex Thomasson et al. (+4 more)

Precision nitrogen (N) management (PNM) for cotton requires in-season monitoring of crop growth parameters and N status indicators to decide fertilizer timing, placement, and application rates for optimal canopy development and yield. This study developed remote sensing and machine learning-based methods to estimate cotton dry biomass weight (DBW), plant N uptake (PNU), plant N concentration (PNC), critical N dilution (Nc), and nitrogen nutrition index (NNI) to support PNM. To achieve this, a three-year field-based N-management study was conducted and unmanned aerial vehicle (UAV)-based multispectral images were acquired between early vegetative growth and flowering stages, critical for fertilizer applications. Spatiotemporally consistent spectral and morphological plant features, including plant height (PH) and fractional canopy cover (FCC), provided reliable model training inputs. DBW, PNU, and PNC estimates from simple regression using vegetation indices (VIs), multiple linear regression (MLR) combining VIs, PH, and FCC, and decision-tree models, random forest regression (RFR) and extreme gradient boosting (XGB), combining spectral reflectance, PH, and FCC were evaluated using trial-held-out (THO) and leave-one-year-out (LOYO) validation methods. The best validation accuracies were from RFRTHO (R2 = 0.88 and MAPE = 23.14% for DBW; R2 = 0.84 and MAPE = 20.61% for PNU; R2 = 0.85 and MAPE = 7.82% for PNC) and XGBTHO (R2 = 0.87 and MAPE = 21.91% for DBW; R2 = 0.81 and MAPE = 21.40% for PNU; R2 = 0.86 and MAPE = 7.66% for PNC). Nc was calculated from model estimated DBW and PNC for high-yielding, medium-to-tall cotton varieties grown in the Texas Coastal Plains and validated using ground-truth biomass measurements. NNI derived from XGBTHO outputs performed marginally better than NNI from RFRTHO in identifying N-deficient plots and multi-level N-stress categorization.
Read the full abstract

All authors Vaishali Swaminathan, Nithya Rajan, J Alex Thomasson, Amrit Shrestha, Karem Meza Capcha, Robert Hardin, Pramod Pokhrel

Image and Video Processing

Bit Allocation Transfer for Perceptual Quality Enhancement of Traditional Video Codecs

Runyu Yang and Ivan V. Bajić

Traditional block-based video codecs, such as H.264/AVC, H.265/HEVC and H.266/VVC, rely on hand-crafted Rate-Distortion Optimization (RDO) processes that primarily minimize Mean Squared Error (MSE), which correlates poorly with human perceptual quality. While neural video compression methods can easily optimize perceptually aligned metrics like MS-SSIM, their high computational complexity limits practical deployment. This paper proposes a novel bit allocation transfer framework that bridges these two paradigms to enhance the perceptual quality of conventional video codecs. Specifically, we train a quantization step generation model using a perceptual loss within a neural video compression framework (DCVC-FM). The model takes the original frame and a motion-compensated prediction as input and outputs a quantization step map. We then derive a block-wise bit ratio from this map and convert it into a Quantization Parameter (QP) map for a traditional video codec. Experimental results on the HEVC B$\sim$D dataset demonstrate that our method achieves 20.20%, 8.25%, and 8.37% bitrate savings in terms of MS-SSIM compared with the standard reference software JM-19.0, HM-16.20, and VTM-23.0, respectively, with additional gains when utilizing predicted frames. Our approach effectively transfers the implicit perceptual importance learned by neural video compression models to guide block-level bit allocation in traditional video codecs without modifying their core decoding syntax.
Read the full abstract

Image and Video Processing · Cross-listed from q-bio.QM

JUMP-lite: Compact, reproducible benchmarking of cell representations

Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen et al. (+2 more)

Image-based profiling captures rich phenotypic signatures for drug discovery and functional genomics. Large public datasets like JUMP Cell Painting now provide millions of images for systematic study. However, JUMP alone occupies 115 TB, and fragmented evaluation practices make systematic comparisons of representation methods impractical for many researchers. Here we present Nahual, an open-source framework for reproducible model deployment, and JUMP-lite, a 92.0 GB subset of JUMP that is approximately 1,250-fold smaller, selected to cover genetic modalities and compound annotations and reduced via lossy JPEG XL compression. Using these resources, we benchmark five representation methods, including classical features (CellProfiler) and deep learning models (MorphEM, OpenPhenom, SubCell, DINOv2). Moderate compression broadly retains signal relative to uncompressed images. Standardized phenotypic activity and consistency metrics reveal meaningful performance differences across methods. Together, JUMP-lite and Nahual provide a foundation for accessible, reproducible benchmarking of image-based cell representations.
Read the full abstract

All authors Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen, Anne E. Carpenter, Shantanu Singh

Image and Video Processing

IceHorizon: A Dataset for Horizon Detection in Ice-Covered Maritime Environments and Comparative Evaluation of Detection Methods

Alisa Pesotskaia and Emin Zerman

Horizon detection in images of ice-covered waters is a challenging problem for maritime navigation due to low contrast between water and sky, cluttered ice structures, and varying illumination conditions. This paper presents a comparative evaluation of six horizon detection algorithms, including four classical computer vision methods and two hybrid approaches combining deep learning with classical line detection. A new bespoke IceHorizon dataset consisting of 30 ship-based and 8 drone-based videos is used to evaluate detection accuracy, horizon coverage, and computational performance. The results show that hybrid methods achieve the highest accuracy and most reliable horizon estimates. In contrast, purely classical methods exhibit reduced robustness, particularly in visually ambiguous scenes. Performance on ship-based imagery was consistently higher than on drone-based imagery, indicating a strong dependency on acquisition characteristics. The created dataset and codes used in this study are made publicly available to support further research on this topic. The code is available at https://github.com/allythe/HorizonDetection. The dataset is available at https://doi.org/10.5281/zenodo.20411867
Read the full abstract

Image and Video Processing

Spatial Masked-Set Learning for Sparse Multi-Shell Diffusion MRI Signal Synthesis

Yousef Sadegheih, Pratibha Kumari and Dorit Merhof

Dense multi-shell diffusion MRI provides rich q-space information but requires long acquisition times. We propose a spatial masked-set framework for sparse multi-shell diffusion MRI signal synthesis. The model treats observed measurements as an unordered set, uses a local $3 \times 3 \times 3$ neighborhood for spatial context, and predicts radial-order-6 SHORE coefficients for the center voxel. The coefficients can then be decoded analytically to synthesize signals at arbitrary q-space locations. Training combines shell-wise gradient dropping, dense signal supervision, and rotation-consistent SHORE targets so that sparse input signals remain aligned with their coefficient supervision under augmentation. We evaluate on held-out HCP100 white-matter voxels by retaining limited subsets of measured diffusion-weighted signals from the reference acquisition. The proposed method achieves lower signal NMSE than both analytical q-space models and a state-of-the-art continuous dMRI signal synthesis model designed for arbitrary input and output q-space sampling. In the $b=1000$ setting with 10 input gradients, it achieves $2.70\%$ NMSE, a $22.4\%$ relative reduction over this continuous model. Fractional anisotropy on reconstructed $b=1000$ signals provides a complementary tensor-derived endpoint, with analytical models remaining competitive for FA despite higher dense-signal NMSE across the evaluated q-space. The implementation is available on \href{https://github.com/xmindflow/SHOREPred}{https://github.com/xmindflow/SHOREPred}.
Read the full abstract

Thursday 6 August

Image and Video Processing

Algorithmic statistics of retinal images

Loan Huynh, Ronald Zambrano, Layton Aho et al. (+4 more)

There has been a tremendous amount of image processing and machine learning research to measure and classify disease progression from live optical coherence tomography (OCT) imaging of the retina. The images considered here are large, complex, three-dimensional (3-D) and difficult to visualize effectively. Many current supervised machine learning approaches, e.g. neural networks, are non-metric meaning that any features or measurements generated can introduce systematic distortion that may be correlated with underlying non-meaningful physiological differences. Here we present a metric learning approach using the normalized compression distance (NCD) combined with anisotropic structure-enhancing filters to quantify and visualize the principal differences among a collection of 3-D retinal images. We validate the NCD-measured structural differences between pairs of images against the physician-measured change in visual field function, achieving a prediction error of $\sim$ 0.5 dB, more accurate than non-metric deep learning approaches. The normalized compression vectors (NCV) are proposed as a feature set measuring visual differences among a collection of 3-D microscopy images. The utility of the NCV for visualizing and measuring patterns of change is demonstrated for a human with moderate non-progressing glaucoma and for a non-human primate model using intraocular pressure setting manipulation. We conclude with a brief simulation of non-metric embedding features, e.g. from neural networks, introducing class-correlated statistical distortion.
Read the full abstract

All authors Loan Huynh, Ronald Zambrano, Layton Aho, Fabio Lavinsky, Gadi Wollstein, Joel S. Schuman, Andrew R. Cohen

Image and Video Processing · Cross-listed from cs.RO

Enhanced Real-Time 6-DOF Extended Reality Catheter Tracking for Evaluating Potential Improvement in Efficiency, Precision, and Depth Perception for Cardiac Interventions

Mohsen Annabestani, Sandhya Sriram, Andrew Kuzemczak et al. (+3 more)

Despite advances in 3D ultrasound, most percutaneous cardiac interventions still rely on 2D visualization, limiting depth perception and spatial understanding. To address this challenge, we developed an Extended Reality (XR)-based platform that enables real-time six-degree-of-freedom (6-DOF) catheter tracking and visualization within a patient-specific 3D heart model. The system combines a custom machine-vision algorithm for 5-DOF catheter tracking with a 3D-printed electromechanical encoder that measures catheter roll, providing complete 6-DOF motion reconstruction. In a proof-of-concept study, 20 novice medical students navigated an intracardiac echocardiography (ICE) catheter to six anatomical targets using either immersive 3D visualization or a conventional 2D cathlab-style view. Participants in the 3D condition completed the task in 54.6 seconds and traveled 1,939 mm on average, compared with 267.5 seconds and 7,854 mm in the 2D condition. Therefore, the XR-based 3D system was more than 5x faster and required ~5x less catheter travel. The 3D mode also improved targeting precision and reduced performance variability. Participants consistently rated immersive visualization higher for accuracy, speed, usability, and clinical value. Kinematic analysis showed smoother depth-axis navigation in 3D, whereas 2D users relied on repeated corrective movements. These findings demonstrate that XR-based visualization can substantially improve procedural training efficiency, precision, and motor control.
Read the full abstract

All authors Mohsen Annabestani, Sandhya Sriram, Andrew Kuzemczak, S. Chiu Wong, Alexandros Sigaras, Bobak Mosadegh

Image and Video Processing · Cross-listed from cs.RO

AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion

Xirui Liang, Jiaqi Liang, Jingkai Xu et al. (+6 more)

Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback, a capability that remains challenging to replicate in robotic systems. Existing robotic grasping approaches predominantly rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact, limiting robustness and generalization. To address this challenge, we propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement. At its core, our method introduces an efficient visuo-tactile representation that tightly fuses object geometry with tactile feedback by associating tactile signals with finger identities. This unified representation supports contact-aware grasp pose generation during planning and tactile-guided refinement after contact, enabling the system to reason about fine-grained finger-object interactions and adjust grasps dynamically. Comprehensive experiments in both simulation and real-world environments demonstrate that our approach significantly enhances grasp success rates and generalization across diverse objects.
Read the full abstract

All authors Xirui Liang, Jiaqi Liang, Jingkai Xu, Yuran Wang, Ruochong Li, Yuanpei Chen, Masayoshi Tomizuka, Wei Zhan, Ruihai Wu

Image and Video Processing

Implicit Neural Speckle Denoising

Matthew R. Ziemann, Casey J. Pellizzari, Tyler J. Hardy et al. (+1 more)

Speckle fundamentally limits coherent imaging by introducing multiplicative, spatially correlated noise that obscures scene structure. Removing speckle noise from dynamic scenes–that do not benefit from conventional speckle averaging–is particularly challenging. We introduce a training-free framework that combines a spatiotemporal implicit neural representation with an aperture-aware maximum-likelihood formulation to recover dynamic, speckle-free imagery directly from noisy observations. The coherent likelihood explicitly models the aperture-dependent spatial covariance of speckle, enabling adaptation to arbitrary pupil geometries without retraining. A matrix-free implementation based on FFT-accelerated operators, stochastic approximations, and conjugate gradients makes optimization practical for realistic image sizes. Meanwhile, a blind holdout criterion provides automatic early stopping without clean reference data. Simulated and laboratory results demonstrate improved spatial fidelity, temporal consistency, and robustness to varying speckle statistics relative to classical, unsupervised, and supervised baselines.
Read the full abstract

All authors Matthew R. Ziemann, Casey J. Pellizzari, Tyler J. Hardy, Christopher A. Metzler

Image and Video Processing · Cross-listed from cs.CV

TLNM: Externally Validated Tooth Detection, Numbering and Segmentation from Smartphone Photographs Using Mask R-CNN

Arash Nedaei, Henna Tiensuu, Elina Väyrynen et al. (+2 more)

Oral health issues affect billions of people globally, but the cost and limited access to professional dental care hinder preventive oral healthcare. Research relies on clinical-grade sensors, unavailable for public self-screening. This study introduces a tooth localisation and numbering model for smartphone photographs. We developed a customised Mask Region-based Convolutional Neural Network pipeline trained on 1,272 annotated smartphone images. To address variability in patient-generated health data, the pipeline incorporates two domain-informed mechanisms: a masked gray-world white-balancing algorithm to mitigate artificial colour casts and an anatomically constrained detection layer to enforce structural validity and suppress false positives. The system was evaluated using internal and external testing, descriptive ablation study and training stability analysis. On the internal test set, the model achieved an instance-mask AP50 of 0.818, class-aware PQ of 0.780, and operational F1 of 0.884. On the external dataset, the model achieved an instance-mask AP50 of 0.901, class-aware PQ of 0.832, and operational F1 of 0.928 despite differences in population, sensors, and acquisition protocols. The inference pipeline is available as an open-source, containerised API. These results demonstrate that consumer-grade smartphone imagery can support automated tooth-level anatomical mapping, offering a scalable, potentially low-cost foundation for remote screening and tele-dentistry in resource-constrained environments.
Read the full abstract

All authors Arash Nedaei, Henna Tiensuu, Elina Väyrynen, Saujanya Karki, Jaakko Suutala