Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing

Monday 14 September

Image and Video Processing · Cross-listed from q-bio.TO

Rare Disease Classification in Video Capsule Endoscopy

Amit Kumar Patel and Debesh Jha

Video capsule endoscopy offers a non-invasive way to examine the small bowel, however, the rare but clinically important findings that appear only in less than 1% of labelled frames, provide very less signal for detection. As a result, while models can achieve high overall performance, they almost completely miss these rare findings. We condition each frame's decision on evidence from the rest of that patient's video, not on the frame's own posterior alone, supplying information a single frozen frame can never contain. Building upon Expected Gain Arbitration, a per-frame, macro-F1-optimal correction over the confusion matrix, we further compose it with video-disease-presence conditional resulting in our decoding method Hierarchical Conditional-Expected-gain (HCE). We evaluate HCE on two independent VCE datasets, Kvasir-Capsule and GALAR, against a static temporal-smoothing baseline and an optimal-transport (OT) frame-level correction. HCE significantly improves rare-class recall over a temporal-smoothing baseline on GALAR (2.7–3.1x; 95% CI excludes zero in both scopes) and shows a similar, though not statistically confirmed, gain on Kvasir-Capsule (up to 3.4x). Against OT, HCE still improves rare-class recall in all four evaluation settings (+0.016 to +0.029), and additionally improves macro-F1 and MCC.
Read the full abstract

Image and Video Processing

Semantic-Aware Neural Video Codec for Error-Resilient Low-Latency Transmission

Matin Mortaheb, Homa Esfahanizadeh, Jinfeng Du et al. (+1 more)

Emerging physical AI systems require low-latency, task-oriented video communication over unreliable channels. We propose a semantic-aware multi-level neural video coding method for robust low-latency video transmission over unreliable channels that are abstracted as multi-level packet erasure channels. Built upon the real-time DCVC-RT neural video codec, the proposed framework introduces a semantic- and feature-aware coding strategy that partitions encoded representations into packets carrying different levels of semantic and latent-feature importance and assigns these packets to different streams, each associated with a priority level when transmitted over unreliable communication channels. We also developed an error-resilient entropy model that removes inter-packet dependencies, allowing each packet to be decoded independently under packet losses. The complete system is trained end-to-end over the abstracted multi-level packet erasure channels, enabling learning of channel-aware representations together with importance-aware packet assignment while facilitating the network for differentiated packet prioritization. Experiments show that the proposed framework significantly improves robustness over baseline DCVC-RT under packet erasures, achieving graceful degradation in less important regions while better preserving task-relevant visual content.
Read the full abstract

All authors Matin Mortaheb, Homa Esfahanizadeh, Jinfeng Du, Harish Viswanathan

Image and Video Processing

A Sentinel-2 benchmark dataset for deep-learning active-fire segmentation across 25 California wildfires

Shreyan Mitra, Mohammadreza Narimani and Parastoo Farajpoor

This article describes an open image dataset for developing and evaluating active-fire segmentation methods in satellite imagery. The dataset contains 2,148 image-mask pairs from 25 California wildfires, with acquisitions spanning July 2020 to August 2026. Each image is a 512x512-pixel, three-channel composite derived from Sentinel-2 Level-2A bands B12, B11 and B8A at 20 m spatial sampling. A fixed linear rendering is applied throughout the dataset. Corresponding masks distinguish background, SWIR-rule active fire and invalid observations. The masks were generated from shortwave-infrared brightness and near-infrared contrast, followed by constrained neighborhood growth. The release includes chip-level metadata and an incident-disjoint partition containing 18 training, three validation and four test fires. Among the image pairs, 841 contain active-fire labels; these labels occupy 0.0766% of all grid cells. A mask-blind analyst review covers 233 test chips and provides a separate assessment of the rule-generated labels at chip and connected-component levels. Reference training and evaluation code accompanies the data, including a ResNet-34 U-Net implementation with validation-based checkpoint and threshold selection. The archived images, masks, metadata and review annotations support research on rare-class segmentation, learning from algorithmic labels and transfer across fire incidents. The versioned dataset is deposited on Zenodo, with preparation and reuse software maintained in a public GitHub repository.
Read the full abstract

Image and Video Processing

Multisource Remote Sensing and Geospatial Analysis of Vineyard Wildfire Impacts and Resilience: The 2019 Kincade Fire

Parastoo Farajpoor, Mahla Ardebili Pour, Mohammad Bagher Ghiasi et al. (+1 more)

Working agricultural landscapes are often treated as background to wildfire disasters, even though they are managed fuel mosaics, productive assets, and parts of regional infrastructure systems. We examine vineyard wildfire resilience during the electrically initiated 2019 Kincade Fire in Sonoma County, California, using an open, event-anchored geospatial framework spanning 4,581 vineyard fields (8,813.2 ha), wildland vegetation, surveyed structures, roads, overhead smoke, and post-fire greenness. Sentinel-2, OpenET, gridMET, soils, terrain, NOAA smoke polygons, an ignition-date OpenStreetMap network, and three-dimensional data inventories were analyzed at native decision scales. Vineyard pixels showed substantially lower descriptive dNBR than wildland pixels inside the perimeter (means 0.130 and 0.337). This contrast did not identify a universal vineyard firebreak effect: a segment-clustered boundary model gave a small negative contrast at 100 m (tau = -0.0166) but changed across bandwidths, failed slope continuity, disappeared in a 100 m donut, and produced a wrong-signed placebo. A 250 m spatial GAM reversed the unconditional pattern: after conditioning on location, terrain, and water use, vineyard fraction was positively associated with dNBR, while residual Moran's I remained 0.519. Beyond spectral impact, all mapped vineyards intersected overhead smoke on at least one day (mean 7.78 potential smoke-days per field), 34.2% of road-network nodes were dead ends, and inside-perimeter vineyards showed a larger greenness deficit through 2021 (recovery ratios 0.815 inside and 0.854 outside). Lower immediate spectral impact therefore did not imply complete resilience. The study provides a reproducible urban-rural informatics template separating descriptive contrasts, conditional associations, exposure indicators, and recovery evidence for decisions in working landscapes.
Read the full abstract

All authors Parastoo Farajpoor, Mahla Ardebili Pour, Mohammad Bagher Ghiasi, Mohammadreza Narimani

Image and Video Processing · Cross-listed from cs.RO

Goal-Oriented Communications for Physical AI: Design and Testbed

Shutong Chen, Wenkai Zhang, Adnan Aijaz et al. (+2 more)

Physical AI relies on frequently-updated, latency-sensitive video stream to perceive, reason, and interact with the physical world, resulting in strict latency requirements with much higher data volumes that existing 5G networks cannot support. Goal-oriented communication (GoC) offers as a promising approach to solve this challenge by transmitting only task-relevant semantic representations. However, existing GoC frameworks were mainly evaluated in the simulations while their effectiveness has never been validated in a practical deployment of physical AI application. In this work, we develop an end-to-end GoC testbed for Physical AI, which connects a PiPER robot arm equipped with an RGB-D camera and a 5G modem to an NVIDIA Jetson AGX Orin edge server through a 5G OpenAirInterface network. We propose and implement three GoC frameworks that transmit 3D bounding boxes, 2D scene graphs, and 3D scene graphs, as three types of semantic representations, respectively. They share the common functional modules designed for closed-loop Physical AI applications, including semantic extraction, full stack 5G transmission, language model inference, digital twin validation, and robotic control. Extensive experiments on our testbed show that our GoC frameworks reduce the task completion time by up to 52.6% and improve task success probability by up to 45%, compared to the traditional framework that periodically transmits the raw image data. These results validate the practical effectiveness of our GoC framework and pave the way for efficient and reliable Physical AI applications over future 6G networks. Project website: https://sites.google.com/view/goc-physical-ai-testbed.
Read the full abstract

All authors Shutong Chen, Wenkai Zhang, Adnan Aijaz, Miao Guo, Yansha Deng

Image and Video Processing

OASIS: Online Adaptive Video Compression via Closed-loop Feedback Control

Rentao Wan, Jinho Park and Mingoo Seok

Computer vision systems are a key building block in an autonomous vehicle, responsible for a range of perception tasks. However, they incur massive data transmission over long communication links from multiple cameras, creating a critical bandwidth and energy bottleneck. Although conventional codecs such as H.264 can reduce data rates, they are ill-suited for real-time vision systems due to high processing latency and energy consumption, as well as their reliance on static user-defined compression settings. In light of these challenges, we propose OASIS, an adaptive video compression framework that integrates lightweight in-sensor compression with task-aware compression ratio control. Based on the real-time task performance, it dynamically updates the optimal compression ratio. Experimental results demonstrate that OASIS generalizes across multiple vision tasks, achieving on average a 6x data compression, 5.8x reduction in link power consumption, and 2.5x reduction in link latency, with at most 1.5% performance degradation.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

Assessing nnU-Net Generalization across Brain Tumor Populations in BraTS-GoAT 2026

Tristan Kirscher, Vivian Metzger, Philippe Meyer et al. (+1 more)

BraTS-GoAT evaluates tumor segmentation across heterogeneous populations. We trained a conventional 3D nnU-Net on 1,351 labeled cases using five-fold cross-validation and 1,000 epochs per fold. The final predictor averaged all folds and applied test-time mirroring. On pooled official validation, global DSC values were 0.7805, 0.8288, and 0.8854 for enhancing tumor (ET), tumor core (TC), and whole tumor (WT). Under matched fold-0 inference, mean regional Dice decreased from 0.9058 on source out-of-fold (OOF) cases to 0.8310 on pooled validation (difference–0.0747). Mirroring gave small single-fold gains but no clear ensemble benefit; a residual-encoder alternative reached 0.8282 mean Dice. In labeled OOF predictions, failure cases had substantially smaller reference ET volumes; after adjustment for ET and WT volume, lower Dice remained associated with more disconnected ET components and a smaller fraction of ET contained in the largest component.
Read the full abstract

All authors Tristan Kirscher, Vivian Metzger, Philippe Meyer, Xavier Coubez

Image and Video Processing · Cross-listed from eess.SP

Thinking in Tokens, Talking in Bits: A Practical Interface for Token Communication

Chanho Park, Bumsu Park, Soonhee Kwon et al. (+2 more)

Advanced artificial intelligence models think in tokens; contemporary communication systems carry bits. The direct way to bridge this gap is to transmit tokens, but that makes a model-specific representation part of the air interface, coupling the endpoints through a shared tokenizer, codebook, and often a neural transceiver. We take a different route: keep bits in the payload and let tokens control how those bits are generated and protected. The resulting token-bit interface transition aligns task-side tokens with source- and channel-coding units, translates token relevance into codec controls, and preserves the induced priority order across the coding chain. We instantiate it for image classification, where a vision transformer scores the task relevance of each image region from its attention maps: those scores steer block-wise JPEG rate allocation, then group the compressed bits for protection at different polar-code rates. The payload remains an explicit, reconstructable bitstream recovered by a correspondingly configured decoder. Over-the-air experiments on a software-defined radio testbed show improved accuracy–latency tradeoffs over separate source-channel coding, performance competitive with far more memory-intensive neural joint source-channel coding, and graceful degradation under channel mismatch. Token communication, then, need not transmit tokens explicitly; what it needs is an interface through which tokens determine how bits are communicated.
Read the full abstract

All authors Chanho Park, Bumsu Park, Soonhee Kwon, Sangrim Lee, Namyoon Lee

Sunday 13 September

Image and Video Processing · Cross-listed from cs.CV

Complementary rPPG-Derived and Lip-Region Frequency Cues for Talking-Face Deepfake Detection

Othmane Harraq and Tamer Aldwairi

Talking-face (TF) deepfakes are detected unevenly by rPPG-based methods across generators. We study two lightweight visual-only cues, rPPG-derived waveforms extracted by RhythmFormer and lip-region discrete cosine transform (DCT) coefficients, on the seven TF methods of Celeb-DF++ under a subject-independent protocol. In-domain, lip-region DCT matches or exceeds the rPPG-derived 1D ResNet on every method except SadTalker, and Concat fusion reaches AUC 0.891 against 0.824 and 0.827 for the unimodal baselines. Under leave-one-generator-out evaluation the cues split: each transfers clearly better to three held-out methods, and IP-LAP is near chance for both. Concat averages 0.798 but falls below rPPG alone where DCT transfers poorly, so static fusion only partly exploits this complementarity. Lip-region DCT outperforms full-face DCT on six of seven methods. We treat the rPPG-derived signal as an empirical cue and do not claim it is cardiac in origin.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification

Leon Fernando, C Dombawala, P. Hettigoda et al. (+3 more)

Person re-identification (ReID) is essential for multi-camera surveillance and tracking, yet remains difficult due to viewpoint and illumination changes, occlusion, background clutter, and low resolution imagery. We propose a generative AI integrated multimodal ReID framework designed explicitly for robustness under missing cues and low latency deployment. The key idea is a cost aware early-exit cascade that prioritizes inexpensive, high confidence evidence and only triggers expensive modalities for ambiguous cases. Our system integrates (i) global visual embeddings from segmented person regions, (ii) automatically generated fine grained semantic attribute descriptions generated by vision-language models (VLMs), and (iii) optional facial embeddings when face observations are reliable. To optimize the balance between accuracy and latency, we use a cost aware early-exit cascade instead of fusing all modalities. Specifically, we first inspect the top-k retrieval results to determine whether the query is unambiguous. If the best match is clearly separated from the remaining candidates, we stop early and return the result to minimize latency; in ambiguous cases, we keep multiple hypotheses and invoke additional modalities (face/semantic) with adaptive reliability weighting to refine the decision. We report person re-identification performance using mAP and Rank-1 accuracy on the Market-1501 and DukeMTMC-reID benchmarks. The proposed adaptive early-exit cascade resolves 60.7% of DukeMTMC-reID queries and 68.4% of Market-1501 queries without invoking semantic reasoning, reducing computational overhead while maintaining competitive retrieval performance.
Read the full abstract

All authors Leon Fernando, C Dombawala, P. Hettigoda, Vanodhya G. Warnasooriya, Ishara Neranjana, Rashmika Nawaratne

Image and Video Processing · Cross-listed from cs.CV

What Input Resolution Is Required for Bird Species Identification, and What Is Its Latency Cost on an Edge Device? A Study of 14 Input Resolutions and Six Architectures with On-Device Measurements

Takeshi Nishikawa

Bird-strike mitigation at wind farms requires identifying distant birds that span only tens of pixels, so the classifier's input resolution N is a design variable, not a fixed specification. We study it with a factorial design over 14 side lengths N (16 to 224), six architectures, two training and evaluation regimes and 30 random seeds – 2,520 checkpoints and 5,040 evaluations – plus latency measured on an NVIDIA Jetson Orin Nano. Four results. (1) The selected N depends on the target: 0.90 is met on validation by ResNet50 at N=112 in an estimated 1.85 ms (0.8980 on test) and 0.95 by DINOv2-L at N=144 in 12.70 ms; changing the model buys more accuracy than raising N (+5.93 versus +2.33 points at N=112). (2) The benefit of lowering N depends on the assumed preprocessing path: N=224 -> 80 saves 13.5% when each individual is decoded from its own file but 46.2% when the detector decodes the 4K frame once; the Pareto set grows from 20 to 23 configurations. (3) Accuracy must be measured on the deployed engine: half precision costs ViT-S/16 alone 4 to 7 points at N>=96 while the CNNs stay within 0.1 points, and with selection held at validation the choice differs at 26 of 176 targets. A broken FP16 engine can run faster than a correct one, undetectable from latency; admitting 14 ViT-S/16 FP32 configurations moves the recommendation over the 0.931-0.938 band and under the 10 ms budget. (4) ViT-L-scale models fit this device, but activations exceed the FP16 range; splitting the graph at transformer-block boundaries confines FP32 to the affected segments, making the deployed DINOv2-L chain 1.85x faster than the single-engine build. We also quantify how the regime-difference sign stabilises with seed count; a sensitivity split removing some forms of group sharing preserves all 14 non-trivial signs at the selection boundary.
Read the full abstract

Saturday 12 September

Image and Video Processing

Deformable 2D Gaussian Splatting for Efficient 4K Video Compression

Chenhao Zhang and Fengqing Zhu

Ultra-High-Definition (UHD) video presents significant challenges for efficient storage and real-time decoding. Learning-based methods, such as Neural Video Compression (NVC) and Implicit Neural Representations (INR), achieve competitive rate-distortion performance but suffer from high decoding latency and excessive memory usage. Meanwhile, Gaussian Splatting has recently attracted attention in the computer graphics community due to its ultra-fast rendering and high-fidelity visual quality. Despite these advantages, its application in video compression remains largely unexplored. To bridge this gap, we propose a real-time video compression framework that represents and compresses a Group of Pictures (GOP) using a coarse-to-fine multi-scale 2D Gaussian Splatting (2DGS) structure coupled with a lightweight deformation network. Experiments demonstrate that our method delivers rate-distortion performance in LPIPS that surpasses H.265 and other state-of-the-art learning-based video compression methods. Our work demonstrates the potential of Gaussian Splatting as a practical solution for efficient high-resolution video compression.
Read the full abstract

Image and Video Processing

SONAR: A Structure-Consistent Neural Operator for Null-Space-Aware Sparse View CT Reconstruction

Song Ni, Haijun Yu, Haodong Li et al. (+4 more)

Sparse-view computed tomography (CT) reduces radiation dose and acquisition time but remains severely ill-posed because incomplete projections poorly constrain null-space information. Existing learning-based methods often estimate this information in high-dimensional image space, conflate physical measurement errors with prediction errors, and depend on fixed discretizations. We propose SONAR, a Structure-Consistent Neural Operator for Null-Space-Aware Reconstruction. Instead of recovering the full null-space component, SONAR predicts a low-dimensional null-space-aware representation from the acquired projections as pseudo-measurements. It separates measurement and pseudo-measurement residuals, lifts them into the image domain through physics operators, and applies independent neural operators to constrain their structural effects, thereby accommodating admissible errors while suppressing unsupported structures. To support cross-discretization reconstruction, an anisotropic U-shaped neural operator models the periodic angular and nonperiodic detector dimensions using direction-dependent continuous supports, while image-domain neural operators re-discretize continuous kernels on target grids. These components form an optimization-inspired unrolled network. Experiments on simulated AAPM and clinical MARS photon-counting CT data demonstrate consistent improvements across seen and unseen view settings and unseen image resolutions. On AAPM dataset, SONAR improves PSNR by 1.87~dB at 62 views and by 7.63~dB under zero-shot transfer to a $512\times512$ grid over the strongest competing methods. SONAR also achieves the best overall performance in all clinical settings evaluated, demonstrating accurate, structurally reliable, and discretization-robust sparse-view CT reconstruction.
Read the full abstract

All authors Song Ni, Haijun Yu, Haodong Li, Changsheng Fang, Shuyi Fan, Yixing Huang, Hengyong Yu

Friday 11 September

Image and Video Processing

Anatomy-Change-Aware Bidirectional Selective State-Space Memory for Clinically Deployed Thoracic Radiotherapy Auto-Contouring

Galib Ahmed, Istiak Ahmed, Aritra Islam Saswato et al. (+15 more)

We developed DAMM-Net++, a 2.5D architecture for thoracic OAR and target volume segmentation that addresses three persistent challenges in radiotherapy auto-contouring: inter-slice surface incoherence, systematic failure on small low-contrast targets, and the absence of per-case reliability signals. The central component is an anatomy-change-aware bidirectional selective state-space memory that models through-plane anatomical change and selectively propagates context along the axial slice sequence. A boundary-aware decoder sharpens near-surface predictions, and an uncertainty head provides calibrated per-voxel confidence for clinical triage. We evaluated 2,146 patients across four centers, an independent external cohort of 112 patients, and a multicenter reader study involving 17 radiation oncologists on 305 cases. The model achieves a mean Dice of 0.955 and HD95 of 3.78 mm, with the largest gains on low-contrast organs-at-risk (OARs) and target volumes where through-plane context is most critical. The uncertainty head is well-calibrated and supports case-level triage. In the reader study, AI assistance reduced contouring time by 75-80% across experience levels and raised junior-reader IoU from 0.861 to 0.925, matching the unedited model. External validation showed a modest internal-to-external drop (< 5%) with calibrated uncertainty transferring without recalibration. The complete deployment pipeline from DICOM ingestion to TPS-compatible RTSTRUCT export has been integrated into the clinical workflow at a partner hospital, where it is used to assist with contouring. These results suggest that anatomically motivated inter-slice memory, paired with uncertainty-guided review, offers a clinically viable path for thoracic auto-contouring.
Read the full abstract

All authors Galib Ahmed, Istiak Ahmed, Aritra Islam Saswato, Asib Mostakim Fony, Kazi Shahriar Sanjid, Md. Tanzim Hossain, Md. Anwarul Islam, Md. Nishan Khan, Md. Misbah Khan, Labiba Faiza Karim, Jobaer Rahman, S M Hasibul Hoque, Rahnuma Shahrin Rista, Kamruzzaman Rumman, Md Arifur Rahman, Syed Md. Akram Hussain, Mohammad Ashrafuzzaman Khan, M. Monir Uddin

Image and Video Processing

Automated Distinction of Intimal and Medial Intracranial Arterial Calcification from CT Head

Benjamin Jin, Maria del C. Valdés Hernández, Richard Bortsov et al. (+3 more)

Intracranial arterial calcifications (IACs) are a common finding on clinical non-contrast enhanced head CT scans and are associated with neurovascular disease. Calcifications can occur in the intimal or medial layer of the arterial wall, subtypes that differ in aetiology and may have distinct clinical relevance. These subtypes can be visually distinguished by radiologists based on the shape of the calcifications. We investigate three automated approaches for subtype classification of IAC from head CT-derived segmentation masks: (1) an automated adaptation of the established radiological visual score, (2) a sphericity-based method, and (3) a method based on shape embeddings extracted by a medical shape foundation model. All approaches use the same lightweight classification pipeline on top of the features they compute and are evaluated using 5-fold cross-validation. The three methods achieved comparable performance, with the embedding-based approach yielding the best overall results with a weighted F1 (mean $\pm$ SD) of up to 71.5 $\pm$ 3.7 for a single artery and 59.8 $\pm$ 1.7 for the joint artery classification. Performance was largely preserved when using automated instead of manual IAC segmentation masks, and we found the difference in weighted F1 not significant. Our results show that fully automated IAC subtype quantification from head CT is feasible and remains robust to the use of manual and automated IAC segmentation masks. Code at https://github.com/bjin96/iac-subtyping.
Read the full abstract

All authors Benjamin Jin, Maria del C. Valdés Hernández, Richard Bortsov, Joanna M. Wardlaw, Daniel Bos, Grant Mair

Image and Video Processing

LM-PCVMNet: Pediatric Cervical Vertebral Maturation Analysis with Deep Fusion of Landmarks and Metadata

Peng Wang, Wanzhen Song, Anli Wang et al. (+3 more)

Cervical vertebral maturation (CVM) assessment plays a pivotal role in orthodontic diagnosis and determining the optimal timing of treatment, especially for pediatric patients. In this paper, we propose LM-PCVMNet, a novel deep learning framework for automatic pediatric CVM staging. Specifically, our method integrates vertebral anatomical landmark information, heatmap-guided feature modulation, and metadata-informed similarity modeling into a unified learning framework. We introduce a heatmap-guided feature modulation module that enhances feature extraction by leveraging landmark-centered heatmaps to highlight morphologically relevant vertebral regions. A vertebral landmark-prompting block is designed to incorporate anatomical geometry into the representation learning process. Furthermore, we develop a learnable metadata supervised contrastive loss that adaptively modulates positive-pair similarity based on metadata similarity, enabling the model to learn more biologically consistent and discriminative features. To facilitate further research in pediatric orthodontic treatment, we additionally release PCVM+. It contains 1800 lateral cephalometric radiographs from real-world patients aged 3-15 years, with expert-annotated CVM stages, 13 vertebral anatomical landmarks, and corresponding metadata. We perform comprehensive experiments on two datasets, and the results show that our method achieves state-of-the-art performance, effectively improving landmark localization and classification accuracy over existing models. Code and dataset will be available at github.com/ybupengwang/LM-PCVMNet.
Read the full abstract

All authors Peng Wang, Wanzhen Song, Anli Wang, Xueshuo Xie, Xiaohang Guan, Tao Li

Image and Video Processing

Conditioning noise is a free regularizer for LoRA fine-tuning: no pathology encoder required for diffusion-based artifact detection in histopathology

Konstantinos Moutselos and Ilias Maglogiannis

Diffusion-based artifact detectors score whole-slide image patches by reconstruction error under a model fine-tuned on clean tissue. We show that conditioning this fine-tuning on random Gaussian embeddings – resampled at every step from approx. 200 KB of precomputed embedding statistics, with no encoder, no cache, and no change to inference – consistently widens the clean/artifact separation. A four-step ablation chain shows the benefit requires neither content (shuffled real embeddings), provenance (synthetic Gaussians), a tuned intensity (flat across an 8x variance range), nor per-patch identity (fresh per-step noise); a LoRA-dropout control shows the conditioning pathway specifically, not generic weight perturbation, carries the effect. Patch-level gains of +0.25-0.48 Cohen's d replicate across nine trainings; honest leave-one-slide-out evaluation clears a pre-registered bar in 2/2 seeds; and two pre-registered external endpoints on a 281-case set confirm pooled Delta F1 = +0.0073 (95% CI) and +0.0129 (97.5% CI, two-look corrected). We release the full evaluation protocol, including measured seed noise and selection-optimism pricing.
Read the full abstract

Image and Video Processing

A deep dictionary network-based foundation model for ultra-low-dose CT denoising

Baoshun Shi, Shuangyi Yang, Ke Jiang et al. (+3 more)

Ultra-low-dose computed tomography (ULDCT) reduces radiation exposure but suffers from severe noise that degrades diagnostic image quality. Existing deep learning-based denoising methods are typically trained in an organ-specific fashion, resulting in limited generalization across heterogeneous multi?organ imaging scenarios. Foundation models present a promising all-in-one paradigm for unified multi-organ denoising. However, their architectures suffer from poor interpretability and rely on heuristic training strategies. To address these limitations, we propose an architecture?interpretable foundation model based on the deep dictionary network (DDN) for unified multi-organ ULDCT denoising. Inspired by multilayer sparse representation theory, DDN cascades convolutional sparse coding layers with iterative soft-thresholding, providing inherent architectural interpretability. Furthermore, a dynamic dictionary module and a threshold generation module are embedded within each layer to enhance representation ability. We conduct DDN pre-training on more than one million multi-organ normal-dose CT images by recovering clean images from Gaussian-noised inputs. Sparse regularization is additionally imposed on latent feature representations, guiding the network to learn compact and noise-robust priors. The complete architecture is jointly fine-tuned on multi-organ ULDCT datasets, enabling a single unified model to perform denoising across diverse anatomical regions. Extensive experiments validate that our proposed method achieves state-of-the?art performance and consistently surpasses competing ULDCT methods across all mul
Read the full abstract

All authors Baoshun Shi, Shuangyi Yang, Ke Jiang, Bin Zhu, Zhanli Hu, Huazhu Fu

Image and Video Processing · Cross-listed from physics.optics

Versatile Sleeve-and-Bulk Fabrication of Inversely Designed Nanophotonic Structures with High Pattern Transfer Fidelity

Lan Hoang Mai, Radheya Sham Sarode, William G. Eshbaugh et al. (+6 more)

Inverse design is a powerful computational tool for generating nanophotonic devices that should outperform their traditionally designed counterparts. However, translating these theoretical gains into physical devices is challenging because the irregular and non-intuitive features that make inversely designed structures so effective also make them exceptionally difficult to fabricate, typically necessitating substantial process optimization unique to each device design. We describe a versatile two-step nanofabrication process that achieves pattern transfer fidelity in excess of 90% for each of three distinct inverse device designs with no device-specific process optimization.
Read the full abstract

All authors Lan Hoang Mai, Radheya Sham Sarode, William G. Eshbaugh, Sulaiman Al Ghadani, Tyler Manfre, Nazifa Tasnim Arony, Joshua M. O. Zide, Matthew F. Doty, Edward B. Flagg

Image and Video Processing

Tile-Based Phase Unwrapping with Reliability-Ordered Joining and Residual-Weighted Multipath Averaging

Antoine Moevus and Max Mignotte

Phase unwrapping recovers a continuous phase map from measurements known only modulo 2{\pi}, in the presence of noise and discontinuities. We propose a tile-based method that combines local spectral reconstruction, reliability-ordered tile joining, and residual-weighted multipath averaging. Each tile is reconstructed by a discrete cosine transform (DCT) least-squares solve, and robust measurements along tile borders guide their joining along a spanning tree. We repeat this reconstruction on shifted tile grids and on rotated and reflected copies of the image, which changes tile borders and joining paths. A residual-weighted average combines the resulting reconstructions, which are computed in parallel and averaged on the fly without being stored. We also introduce the corrected circular rewrapping error, a congruence measure: after removing a global phase offset, it measures how far the rewrapped reconstruction is from the observed phase. It applies to least-squares and path-based methods, even without an unwrapped reference. Used with reference-based errors, it separates agreement with the measurement from recovery of the correct number of cycles. Experiments on radar-interferometric, magnetic-resonance, holographic, and constructed phase images compare our method with seven reference algorithms. Among the methods that complete every test image, ours has the lowest mean fraction of pixels with an error above {\pi}. On the images all methods complete, it lies on several speed-accuracy Pareto fronts. Error maps also reveal isolated cycle errors that average scores or congruence alone can hide.
Read the full abstract

Image and Video Processing

PiMiX 2.02: Toward AI-Driven Data Fusion in Radiographic Imaging and Tomography

Zhehui Wang, Shanny Lin, Nicholas Amano et al. (+7 more)

PiMiX (Physics-informed Meta-instrument for eXperiments) was introduced for multi-instrument, multi-experiment, and simulation-experiment data fusion in radiographic imaging and tomography (RadIT). Here we present PiMiX 2.02 as an evolving AI-enhanced cyber-physical meta-instrument integrating imaging sensors, near-sensor computing, data fusion, physics-informed inference, and human-supervised AI workflows across X-ray, neutron, and other modalities. Demonstrated capabilities include multimodal CMOS radiation imaging, simulation-assisted sub-pixel neutron localization, and edge-deployed optical-neural-network (ONN) inference; GPU and ONN implementations achieved greater than 96% precision for neutron-event detection with sub-micron localization. A further advance is human-in-the-loop agentic-AI co-analysis of X-ray and neutron images from inertial-confinement-fusion experiments. Beyond conventional preprocessing, the workflow generates competing feature hypotheses, ranks contours using physics-informed evidence, estimates confidence, and presents alternatives for human review. The same architecture adapts to different physics: X-ray analysis emphasizes dark, nonuniform ring structures using deformable closed paths and multi-scale evidence, whereas neutron analysis targets bright emission envelopes using fractional-emission levels, intensity gradients, and cross-filter persistence. We also highlight automated comparison of an as-designed stereolithography model with an X-ray CT reconstruction of an additively manufactured metal lattice. Together, these examples show progression from AI assistance in specific processing tasks to multi-task, multi-domain scientific co-analysis. PiMiX 2.0 further provides a pathway toward PRISM, a RadIT scientific foundation model, and tighter integration of diagnostics, digital representations, inference, and experimental control.
Read the full abstract

All authors Zhehui Wang, Shanny Lin, Nicholas Amano, Ramya Gurunathan, Katie Liu, Nathan E. Peterson, Michelle A. Espy, Adam Thompson, Amy J. Clarke, Ray T. Chen

Image and Video Processing

Recurrent Dynamic Range Extension

Sebastian Dille, Keru Fu, S. Mahdi H. Miangoleh et al. (+1 more)

We present an approach to progressively extend the highlights of an image. Instead of reconstructing the full dynamic range of a complex scene directly, we learn a simpler task first: We extend the dynamic range of an input image by a single exposure value. Once this is mastered, we retrieve the full HDR image for the scene by executing our network recurrently, progressively increasing the dynamic range of the input. Our formulation is agnostic to the input dynamic range and targets a bounded output domain. This enables us to use widely available RAW images for the reconstruction task and adapt adversarial losses to construct realistic images. By incorporating Memory Replay for backpropagation, we can train our network recurrently over multiple inference stages and reduce reconstruction errors. As a consequence, our system reconstructs challenging long-tailed HDR scenes robustly and shows powerful recovery of bright light sources and highlights.
Read the full abstract

All authors Sebastian Dille, Keru Fu, S. Mahdi H. Miangoleh, Yağ{\i}z Aksoy

Image and Video Processing

Physics-informed denoising method for image reconstruction in quantitative low-field MRI

Catarina Redshaw Kranich, Claudia Prieto, Christoph Kolbitsch et al. (+1 more)

Low-field magnetic resonance imaging (MRI) is becoming increasingly important for medical imaging because it can reduce healthcare costs while ensuring high diagnostic output. Nevertheless, quantitative imaging in low-field MRI faces challenges, such as low signal-to-noise ratio and long scan durations. Deep learning approaches have been proposed for image reconstruction to overcome these challenges. Still, deep learning often requires large high-quality training datasets which are usually not available for low-field applications. Here we propose a modular unrolled end-to-end deep learning method for the denoised reconstruction of quantitative parameter maps directly from k-space data for low-field MRI. It consists of three sub-networks that are iteratively applied. They are used for the regularization of the quantitative parameter estimation, as well as for the signal estimation that is based on simulated signal curves. It generalises well and can be applied to different field strengths and even different quantitative MR sequences without the need for new training data. We applied the presented method to noisy data of knees acquired at 0.55 T for the reconstruction of $T_2$-maps and compared it to other classical and deep learning methods. We also applied the proposed approach to $T_1$-mapping of knees at 72 mT and $T_2$-mapping of brains at 0.6 T. The presented approach outperforms the other reconstruction methods with a median difference below 4 ms to the ground truth $T_2$-map. Even though the network was trained with $T_2$-maps acquired at 0.55 T, it successfully denoised data acquired at different field strengths, sequences, and of different anatomies. As a result, the proposed network and its underlying method offer an efficient and flexible solution to denoise low-field MR data and make quantitative low-field MRI a feasible diagnostic tool for clinical applications.
Read the full abstract

All authors Catarina Redshaw Kranich, Claudia Prieto, Christoph Kolbitsch, Felix Frederik Zimmermann

Image and Video Processing · Cross-listed from cs.CV

Aligned Radiometric RGB-Thermal Fusion for UAV Facade Anomaly Screening

Yuan Yang, Shulei Li and Haobo Liang

Unmanned aerial vehicle facade inspection can combine red, green, and blue (RGB) imagery with thermal measurements to screen surface and subsurface anomalies. However, geometric discrepancies between the sensors and thermal image rendering can obscure spatial correspondence and weak temperature contrasts. This article presents a sensor-level pipeline comprising per-sensor correction, RGB-to-thermal registration, common-support cropping, and signed local contrast encoding of 16-bit radiometric measurements. The encoding preserves the distinction between locally hotter and colder regions and supplies the fourth input channel of a compact single-stream detector. We introduce M3T, a dataset of 674 paired RGB and radiometric thermal samples from five facade-inspection projects covering eight component and anomaly categories. The median residual registration error is 3.384 pixels, and a controlled-displacement analysis characterizes how the local contrast response changes under controlled displacement. Project-grouped four-fold evaluation yields mean average precision of 0.168 over intersection-over-union thresholds from 0.5 to 0.95, using 28.50 billion floating-point operations per image. A separate single-split ablation shows improved delamination detection over RGB-only and alternative thermal inputs, although aggregate accuracy does not improve over RGB alone. Evaluation on RGBT-Tiny shows mixed performance with rendered thermal imagery. These results characterize the category-specific benefits and limitations of aligned radiometric contrast for compact facade screening.
Read the full abstract

Thursday 10 September

Image and Video Processing · Cross-listed from cs.CV

UBone3D: Physics-Rectified Conditional Flow Matching for Anatomical 3D Shape Completion from Ultrasound

Weiying Chen, Yuchong Gao, Siyuan Li et al. (+3 more)

Three-dimensional ultrasound (US) is a safe, radiation-free complementary modality to CT and X-rays for longitudinal monitoring, yet its segmentation-derived partial point clouds are extremely artifact-laden. Consequently, it is challenging to recover a clean and complete anatomical structure from such US point clouds. In this paper, we present UBone3D, a novel framework based on physics-rectified conditional flow matching (CFM) that performs point cloud completion directly from partial US observations. UBone3D models deterministic physics artifacts (e.g., surface thickening, streaking, dropouts) via a simulated physics proxy, and introduces test-time physics rectification to steer the shape completion. At inference, the completion is jointly steered by two decoupled forces: (1) anatomical plausibility enforced by a CT-trained generative shape prior, BoneFM, and (2) physics consistency enforced by USimNet in the ultrasound formation space. Extensive experiments on simulated and in-vivo data demonstrate significant improvements in reconstruction accuracy and anatomical fidelity over existing baselines.
Read the full abstract

All authors Weiying Chen, Yuchong Gao, Siyuan Li, Marek Reformat, Rui Zheng, Edmond Lou

Image and Video Processing · Cross-listed from cs.CV

Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models

Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang et al. (+5 more)

We address few-shot object detection with vision-language models (VLMs) in out-of-domain settings such as aerial, industrial, and medical imagery, using only ten annotated images for supervision. Existing adaptation methods are discrete prompt optimization and LoRA fine-tuning. We revisit a third option: soft prompting, where a small number of continuous prompt tokens are optimized while the pretrained backbone remains frozen. We identify two key design choices. First, placing prompt tokens at the cross-modal boundary between visual and text tokens outperforms other placements (10.0 vs. 8.4 mAP). Second, initializing prompts from the empty space token outperforms semantic and random initialization. With these choices, one to three learned tokens (7,168 parameters on average) match the best LoRA configuration on Roboflow20-VL (14.2 mAP, 10-shot) while training over 20,000x fewer parameters. Soft prompting remains harder to optimize, exhibiting higher variance across random seeds. Unlike LoRA, however, it causes no forgetting: the LoRA rank matching our accuracy reduces NaturalBench VQA accuracy by 35% relative, rising to 56% at the largest rank, whereas soft prompting leaves pretrained performance unchanged. The learned tokens behave like prompts rather than weights. They transfer to a newer model without retraining (+0.8 mAP on Qwen3.5-9B) and can be verbalized into readable prompts competitive with prompt-search methods (matching DetPO and outperforming GEPA). The approach also extends beyond detection. On RoboCasa manipulation tasks, the frozen $\pi_{0.5}$ vision-language-action policy benefits from soft prompting, matching the LoRA baseline on two of three tasks when tokens are placed at the gradient bottleneck. These results suggest modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask.
Read the full abstract

All authors Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang, Cesar Daniel Hernandez, Wei Zhao, Wolfgang M. Pauli, John Galeotti, Deva Ramanan

Image and Video Processing

Exponential Pixelating Integral transform with dual fractal features for enhanced chest X-ray abnormality detection

Naveenraj Kamalakannan, Sri Ram Macharla, M Kanimozhi et al. (+1 more)

The heightened prevalence of respiratory disorders, particularly exacerbated by a significant upswing in fatalities due to the novel coronavirus, underscores the critical need for early detection and timely intervention. This imperative is paramount, possessing the potential to profoundly impact and safeguard numerous lives. Medically, chest radiography stands out as an essential and economically viable medical imaging approach for diagnosing and assessing the severity of diverse Respiratory Disorders. However, their detection in Chest X-Rays is a cumbersome task even for well-trained radiologists owing to low contrast issues, overlapping of the tissue structures, subjective variability, and the presence of noise. To address these issues, a novel analytical model termed Exponential Pixelating Integral is introduced for the automatic detection of infections in Chest X-Rays in this work. Initially, the presented Exponential Pixelating Integral enhances the pixel intensities to overcome the low-contrast issues that are then polar-transformed followed by their representation using the locally invariant Mandelbrot and Julia fractal geometries for effective distinction of structural features. The collated features labeled Exponential Pixelating Integral with dually characterized fractal features are then classified by the non-parametric multivariate adaptive regression splines to establish an ensemble model between each pair of classes for effective diagnosis of diverse diseases. Rigorous analysis of the proposed classification framework on large medical benchmarked datasets showcases its superiority over its peers by registering a higher classification accuracy and F1 scores ranging from 98.46 to 99.45% and 96.53-98.10% respectively, making it a precise and interpretable automated system for diagnosing respiratory disorders.
Read the full abstract

All authors Naveenraj Kamalakannan, Sri Ram Macharla, M Kanimozhi, M S Sudhakar

Wednesday 9 September

Image and Video Processing

Seamless Whole Slide Label-Free Virtual Staining

Dou Hoon Kwark, Kianoush Falahkheirkhah, Ji-hun Oh et al. (+3 more)

Label-free virtual staining offers a compelling, non-destructive alternative to standard histopathology; however, its clinical adoption is hindered by the computational bottlenecks inherent to processing gigapixel Whole Slide Images (WSIs). Current deep learning approaches require patch-based inference to avoid memory constraints, which disrupts global tissue continuity and introduces tiling artifacts–displaying visible seams and color shifts. To address this, we introduce the Consistency Memory Bank (COMB), a novel label-free virtual staining framework that enforces spatial and channel consistency across tiles without memory bottlenecks. COMB decouples context storage from computation, utilizing a dynamic retrieval mechanism to fetch feature representations from adjacent tiles. This enables a retrieval-based context integration strategy that adopts local padding to resolve spatial discontinuities and neighbor-aware channel attention to stabilize statistical drift. Further optimized with a sliding window schedule to ensure minimal memory overhead, our method demonstrates superior performance over state-of-the-art baselines, achieving significant improvements in both perceptual fidelity and tiling consistency, while suggesting its downstream utility in tumor segmentation. Code is available at https://github.com/dou0000/COMB.
Read the full abstract

All authors Dou Hoon Kwark, Kianoush Falahkheirkhah, Ji-hun Oh, Shirui Luo, Volodymyr Kindratenko, Rohit Bhargava

Image and Video Processing

Annotating anatomy and pathology in the National Lung Screening Trial computed tomography images

Deepa Krishnaswamy, Vamsi Thiriveedhi, Suraj Pai et al. (+6 more)

Large-scale public medical imaging datasets contribute critically to translational research. When accompanied by rich clinical and multi-omics data, they can stimulate exploratory research and enable secondary analyses. Expert annotations of such imaging collections can support the development of new image analysis tools. Continuous enrichment of images with image-derived data makes them more usable for researchers without expertise in image analysis or access to large-scale computational resources. The National Lung Screening Trial (NLST) released a rich longitudinal dataset that includes Computed Tomography (CT) images for over 26,000 patients. We introduce three Digital Imaging and Communications in Medicine (DICOM) formatted datasets, complementing NLST CT images, shared as analysis results in the National Cancer Institute Imaging Data Commons (IDC). Two of those (IDC NLSTSeg and IDC NLSTSybil) contain DICOM-harmonized annotations and extracted measurements (for 581 and 601 NLST patients, respectively) shared earlier using research formats (Sybil and NLSTseg). The third one (TotalSegmentator-CT-Segmentations) contains volumetric segmentations generated using TotalSegmentator and radiomics features for each segment for 26,194 NLST patients.
Read the full abstract

All authors Deepa Krishnaswamy, Vamsi Thiriveedhi, Suraj Pai, David Clunie, Igor Octaviano, Christopher P. Bridge, Steve Pieper, Ron Kikinis, Andrey Fedorov

Image and Video Processing

Scale-Aware 3D Deep Learning for Robust Brain Metastasis Detection in Multimodal MRI

Sylvain Jaume, Hongming Wang and Simon K. Warfield

Detecting brain metastases in magnetic resonance imaging (MRI) remains challenging because lesions vary widely in size and appearance, with very small metastases occupying only a minute fraction of a three-dimensional input. We investigate whether combining different spatial fields of view (FOVs) improves lesion detection in multimodal MRI and present a scale-aware 3D deep-learning framework. The method uses independently trained $96^3$ and $64^3$ 3D U-Nets whose whole-volume probability maps are combined by weighted late fusion. This design allows us to study the effect of spatial context separately from image resolution and modality choice. On a 97-patient development cohort, cross-FOV fusion improved lesion-level precision and F1 while substantially reducing false positives relative to the individual models. A same-FOV ensemble control showed that these gains were not explained solely by averaging independently trained networks, supporting a contribution from complementary spatial context. An exploratory cross-FOV agreement filter reduced false positives but did not improve overall F1. These results support cross-FOV probability fusion as a simple and computationally practical strategy for improving the precision-false-positive trade-off in 3D brain-metastasis detection.
Read the full abstract