Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science Image and Video Processing

eess.IV Watch Image and Video Processing (sign in to use)

Tuesday 25 August

Image and Video Processing · Cross-listed from cs.CV

Lowering the Barrier to AI-Driven Inspection: A No-Code Workflow for Automated Structural Defect Detection

Structural health monitoring (SHM) is essential in modern engineering, providing data for condition-based maintenance, lifecycle assessment, and predictive decision-making. Traditionally, SHM relied on visual inspection to detect defects such as cracks and deformations. Early computer vision (CV) methods, including thresholding, edge detection, and handcrafted features, aimed to automate this process but were highly sensitive to noise, imaging variations, and multiscale defects, limiting their reliability. Recent advances in machine learning, particularly convolutional neural networks (CNNs) and You Only Look Once (YOLO), have improved defect detection accuracy and enabled real-time analysis. However, adoption in SHM remains limited due to technical barriers such as data labeling, model training, and deployment, which typically require programming expertise. To address this gap, we introduce YOLOEZ, an open-source, GUI-based tool for end-to-end YOLO model application. YOLOEZ integrates data labeling, training, and inference into a single interface, enabling high-performance model development without code while supporting reproducible workflows. Evaluation against existing software and classical image processing demonstrates that YOLOEZ not only outperforms traditional methods across most detection metrics, but also lowers adoption barriers present in other modern CV tools. By combining accuracy with accessibility, YOLOEZ facilitates wider use of AI-driven monitoring for predictive maintenance, digital twins, and intelligent structural systems.
Read the full abstract

All authors Michael Holm, Tanner McElroy, Xinghang Zhang, Guang Lin

Image and Video Processing

Learning spatially varying regularisation parameters of low regularity for image reconstruction

In this chapter, we review and discuss the regularity properties of spatially adaptive regularisation weight functions used in variational image reconstruction. Incorporating such weights into classical model-based regularisers, such as Total Variation (TV) and Total Generalised Variation (TGV), allows the regularisation strength to vary across the image and adapt to local image content. When appropriately estimated, these weights can thus significantly improve edge and detail preservation in the reconstructions. We review the existing theoretical literature on this topic for different regularity classes, including constant, continuous, and piecewise constant functions. Our discussion is motivated by recent work on hybrid image reconstruction methods that combine model-based regularisation with deep neural networks to learn highly adaptive regularisation weights. In particular, we discuss how the structural properties of these weights influence the reconstruction from both theoretical and practical perspectives. Through representative examples in image denoising and magnetic resonance imaging (MRI) reconstruction, we demonstrate that the learned weights are often of low regularity and can adapt not only to the image structure but also to the specific noise realisation. We conclude by highlighting several directions for future research on this topic.
Read the full abstract

Image and Video Processing

Improving Cross-Site Whole-Heart Segmentation

Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis, but remains challenging under multi-center and multi-modality distribution shift. In the CARE whole-heart segmentation task, models must generalize from limited labeled sites to unseen acquisition distributions, where variation in spacing, intensity, reconstruction texture, and anatomy can degrade out-of-distribution performance. We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-initialized nnU-Netv2 models with site-characterized, label-preserving appearance augmentation. We first characterize the available sites using measurable image properties and use this analysis to motivate candidate data-space generalization routes. The final retained recipe applies Bias Field + Bezier appearance augmentation, combining smooth spatial intensity perturbation with nonlinear intensity remapping, followed by lightweight class-wise largest-connected-component cleanup. On the primary held-out-site validation splits, the final configuration improves CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830, while also reducing HD95. These results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation. Our code can be found in https://github.com/Purdue-M2/Improving-Cross-Site-Whole-Heart-Segmentation
Read the full abstract

All authors Tanish Mudaliar, Justin Li, Daniel Lin, Julianna Vo, Kaitao Liao, Xin Wang, Shu Hu

Image and Video Processing · Cross-listed from physics.optics

Effects of Quantum Noise and Source Blurring on Dark-Field Signal Retrieval in X-ray Speckle-Based Imaging

X-ray speckle-based dark-field imaging offers high sensitivity to sub-pixel structural features, yet its quantitative reliability in clinical and preclinical settings remains constrained by low photon flux and finite focal spot sizes. However, how hardware-induced noise and source blurring propagate through retrieval algorithms to degrade signal integrity is not fully understood. Here, we systematically evaluate algorithm robustness quantified by signal linearity, sensitivity, and bias under photon starvation and source blurring across two mathematically distinct frameworks: differential-based intrinsic tracking (Low-Coherence System, LCS) and patch-wise explicit tracking (X-ray Speckle-Tracking Speckle-Vector-Tracking, XST-XSVT). Our experimental results demonstrate that input speckle pattern distortions propagate through retrieval algorithms in fundamentally different ways depending on algorithm architecture. As an example, using our setup, under severe photon starvation, derivative noise amplification in LCS causes its dark-field signal linearity and sensitivity to drop precipitously, while sharply elevating baseline bias. In contrast, XST-XSVT restricts these losses for sensitivity while maintaining a stable baseline, as its patch-wise variance calculation inherently suppresses stochastic noise. Similarly, under blur-limited conditions , source blurring washes out the speckle pattern, directly reducing dark-field sensitivity for both LCS and XST-XSVT. This characterization establishes operational boundaries for low-power and low-coherence X-ray systems, guiding algorithm selection and framework optimization to realize quantitative dark-field imaging in preclinical and clinical applications.
Read the full abstract

Image and Video Processing

Score-Based Ideal Observer Approximation via Denoising Score Matching for Signal-Known-Exactly Detection Tasks

The Bayesian Ideal Observer (IO) establishes the theoretical upper bound on task performance for binary detection tasks. However, analytical computation of the IO test statistic is generally intractable. Numerical approaches based on Markov-chain Monte Carlo (MCMC) methods, including their recent deep generative model-based extensions, typically require extensive posterior sampling for each test image. Supervised learning has also been investigated to approximate the IO performance. However, such methods are typically trained for a specific detection task and signal and may require retraining when the task or signal changes. The score function, defined as the gradient of the log probability density, encodes the local geometry of the data distribution and is a fundamental quantity in modern score-based generative modeling. This work reformulates the IO test statistic in terms of the score function and introduces a score-based ideal observer (SIO). The proposed SIO uses a denoising convolutional neural network trained exclusively on signal-absent images to estimate the signal-absent score function. Once trained, the resulting score model can be used to approximate the IO test statistic for detection tasks involving arbitrary additive signals, without per-image posterior sampling or signal-specific retraining. Numerical studies consider a signal-known-exactly (SKE) detection task with a stochastic lumpy-background model. The results demonstrate that the proposed SIO can closely approximate the IO performance.
Read the full abstract

Image and Video Processing

Unified-protocol voxel-level pulmonary embolism annotations for three public CT angiography datasets

Reliable clot-volume quantification and subsequent risk assessment in pulmonary embolism depend on precise segmentation of emboli on computed tomography pulmonary angiography. Deep learning models for this task must be trained on accurate voxel-level labels. The three public datasets that provide such labels were annotated under different protocols, and some of their studies contain unlabeled emboli or labels that are discontinuous across slices. This Data Descriptor presents voxel-level pulmonary embolism annotations for 149 of the 166 studies in these datasets. A primary rater drew all annotations under a single protocol. A thoracic radiologist with more than 20 years of experience reviewed and revised them. Three raters at three different centers independently annotated a subset of 15 studies. The subset was selected by source dataset and embolus location. Technical validation quantifies volumetric agreement with the source annotations, changes in within-mask attenuation, and inter-rater agreement on the subset. The dataset is intended to allow segmentation models to be developed and compared under a common reference standard.
Read the full abstract

All authors Qihang Sun, Zhongxiao Liu, Bailiang Jian, Shenman Qiu, Jingyuan Wang, Lei Zhang, Lixiang Xie, Jiazhen Pan, Christian Wachinger

Monday 24 August

Image and Video Processing

Native-Space 3D CarveMix for Multi-Site T1w Stroke Segmentation

Segmenting ischemic stroke lesions on T1-weighted (T1w) MRI acquired across different scanners and protocols without intensity standardization is difficult because lesions are subtle and share intensity characteristics with cerebrospinal fluid. Standard deep learning architectures trained across multiple centers plateau around Dice 0.66, with acute lesions ($\le 7$ days post-stroke) performing substantially worse due to severe sample scarcity. We combine a MedNeXt-L ($k=5$) backbone with on-the-fly 3D CarveMix augmentation that pastes real lesion patches into healthy brain regions during training. By generating synthetic lesion placements dynamically within each fold with subject-level split isolation, the model sees more diverse lesion patterns without requiring pre-generated copies on disk. We evaluate on 1,453 native T1w scans from 55 clinical centers in the ISLES 2026 challenge. Our method achieves a mean 5-fold cross-validation Dice of 0.648 at 500 epochs, a +0.018 improvement over the MedNeXt-L backbone at a matched training budget (0.630)
Read the full abstract

Image and Video Processing

Spatiotemporal Distillation via Recurrent Bottlenecks for Aortic Tracking

Cardiac cine-MRI serves as a direct visual indicator of cardiovascular hemodynamics by capturing the continuous wall motion of the aorta. Quantifying these dynamic structural changes across the cardiac cycle is essential for measuring aortic distensibility, a primary marker of arterial stiffness. However, standard 2D segmentation networks focus on each frame independently. Consequently, when rapid systolic flow temporarily obscures the aorta's boundaries, this lack of continuous context results in frame-to-frame tracking dropouts and boundary inconsistencies. Spatiotemporal ($2\text{D}+t$) networks can enforce temporal consistency across the sequence but suffer from a scarcity of expert annotations. To address this, we present a semi-supervised spatiotemporal ($2\text{D}$ to $2\text{D}+t$) knowledge distillation framework exploiting the cardiac cycle. The framework distills a spatial teacher's expertise into a spatiotemporal student network by executing a dynamic latent interception, pairing a recurrent spatiotemporal bottleneck with a residual spatial bypass. Our model selection strategy applies a baseline validation threshold ($\text{DSC} \ge 0.50$) prior to selecting the epoch that maximizes anatomical consistency. This strategy enables the spatiotemporal student model to achieve superior surface tracking accuracy ($\text{NSD@1mm} = 92.3\% \pm 0.2\%$) and high structural reliability ($\text{Frac}_{2\text{CC}} = 99.2\% \pm 0.6\%$), reducing population-wide structural anomalies by over 56% compared to a 2D nnU-Net baseline.
Read the full abstract

Image and Video Processing

Multi-Stage Prompt-Guided Feature Modulation for Generalizable Brain Tumor Segmentation

Accurate brain tumor segmentation from magnetic resonance imaging (MRI) is essential for diagnosis, treatment planning, surgical guidance, and disease monitoring. However, developing automated segmentation models that generalize across diverse tumor characteristics, imaging protocols, acquisition sites, and patient populations remains challenging. Variations in tumor morphology and imaging distributions can substantially degrade performance outside the training domain. Consequently, improving the robustness and generalization of deep learning-based segmentation models has become a key objective in medical image analysis. To improve segmentation robustness, we propose Multi-Stage Dynamic Prompt nnU-Net, a prompt-conditioned extension of nnU-Net. Three independent dynamic prompt modules are inserted into the deepest encoder stages. Each module contains a learnable bank of ten 256-dimensional prompt vectors and uses globally pooled encoder features to generate image-specific prompt representations. These representations are projected into feature-wise scaling $(\gamma)$ and shifting $(\beta)$ parameters that modulate encoder feature maps through Feature-wise Linear Modulation (FiLM), enabling adaptive feature conditioning at multiple semantic levels. Evaluation on the BraTS GOAT validation dataset demonstrated that the proposed Multi-Stage Dynamic Prompt nnU-Net outperformed the baseline nnU Net across the majority of evaluated metrics and tumor subregions. The proposed model achieved average lesion-wise Dice scores of 76.16% (ET), 80.04% (TC), and 86.42% (WT), compared with 74.38%, 78.14% and 84.01% for the baseline model. The results demonstrate that multi-stage dynamic prompt conditioning improves segmentation accuracy and boundary delineation for brain tumor segmentation.
Read the full abstract

Image and Video Processing

An Energy-Proportional Multimodal and Context-Aware Vision IoT Node

While recent advancements in TinyML have significantly reduced the computational complexity of on-device vision pipelines, image acquisition remains a dominant contributor to system-level energy consumption and memory footprint. In vision-enabled IoT platforms, the image sensor consumes energy comparable to the inference engine, thereby offsetting algorithmic efficiency gains. Consequently, current designs face a fundamental trade-off: continuous and always-on sensing incurs prohibitive energy consumption, whereas aggressive duty cycling increases latency and risks missing transient events. This work presents an energy-proportional, context-aware vision IoT node that addresses this challenge through a heterogeneous multimodal dual-camera architecture. Detection and recognition are decoupled by combining an event-based imager operating asynchronously in an energy-efficient always-on wake-on-motion mode together with an RGB imager. Deployed on a low-power microcontroller, a novel TinyissimoYOLOv12 is introduced for efficient and accurate object detection. By activating the high-power image acquisition and processing stages only upon sparse visual triggers, the proposed architecture improves efficiency and latency, eliminating redundant sensing while maintaining continuous monitoring coverage. Experimental results demonstrate an energy consumption of only 222$\mu$Wh. Upon a motion trigger, the system completes a full sense-to-report cycle-RGB acquisition, object detection across 80 classes, and LoRa telemetry-with a total energy consumption of 28.7mJ. The network achieves up to 32.3% mAP with a model size of 1 million parameters. At a 1% daily activity ratio, the platform achieves a three-month operational lifetime with a 1.85Wh battery, enabling always-on visual monitoring in a place-and-forget scenario through autonomous edge intelligence.
Read the full abstract

All authors Julian Moosmann, Philipp Mayer, Luca Benini, Michele Magno

Image and Video Processing

Interpretable Landsat-to-Hyperspectral Dual Super-Resolution Without Large Matrix Inversion

Direct acquisition of global hyperspectral images (HSIs) is infeasible given contemporary hardware facilities and limited resources, while global hyperspectral monitoring is critical for remote sensing applications. A more economical approach is to interpretably convert global Landsat-8/9 multispectral images into NASA's AVIRIS-level HSIs. This conversion involves both spatial super-resolution (SpaSR, 30-m to 15-m) and the highly ill-posed spectral super-resolution (SpeSR, 7-band to 172-band), collectively referred to as dual super-resolution (DualSR), whose duality between SpaSR and SpeSR has recently been established. Existing SpeSR methods, mostly designed to reconstruct CAVE-level HSIs with only 31 visible bands, are not applicable to the AVIRIS-level task involving 172 visible, near-infrared, and shortwave-infrared bands. This motivates us to customize an interpretable alternating direction method of multipliers network (ADMM-Net) using the Woodbury W-Lemma and a spectral continuity prior. Unlike conventional generative SpaSR, we employ a panchromatic sharpening strategy to recover physically grounded spatial details. However, this strategy induces very large matrix inversions (LMIs), with dimensionality proportional to the number of pixels, even after applying the W-Lemma. We resolve this issue by designing an LMI-free proximal gradient descent network (PGD-Net). Consequently, the proposed PGD-ADMM interpretable network (PAINT) achieves substantial improvements in both computational complexity and reconstruction performance. Beyond state-of-the-art reconstruction performance, PAINT improves Landsat classification from 78.98% accuracy and 76.06% kappa to 92.16% accuracy and 90.98% kappa.
Read the full abstract

All authors Chia-Hsiang Lin, Jian-Kai Huang, Si-Sheng Young, Wei-Cheng Zheng

Sunday 23 August

Image and Video Processing

Local SVD-Entropy Maps as a Complementary Structural Representation for Full-Reference and No-Reference Image Quality Assessment

We investigate a local spectral-complexity representation for perceptual image quality assessment (IQA) based on Shannon entropy of singular values computed directly from two-dimensional image patches. For each $3\times3$-pixel grayscale patch, SVD is applied directly and the normalized singular-value entropy defines one HSVD-map value. The construction requires neither flattening nor delay embedding, uses no boundary padding, and is invariant to $90^{\circ}$ rotations and mirror reflections at the local-descriptor level. A nested salt-and-pepper experiment on Lena separates absolute similarity to a clean reference from sensitivity to an additional degradation step. HSVD-SSIM responds more strongly to local corruption and retains a larger neighboring-state response at severe noise levels. Validation on all 10,125 distorted KADID-10k images shows that HSVD-SSIM is weaker than conventional SSIM as a standalone full-reference metric (SRCC $0.450$ vs. $0.619$), but complementary when combined with it: grouped cross-validation increases SRCC from $0.618$ to $0.659$, with a bootstrap 95% confidence interval of $[0.036,0.046]$ for the gain. In a no-reference experiment, adding HSVD-derived single-image descriptors improves the best nonlinear model from SRCC $0.528$ to $0.575$ (95% CI $[0.033,0.061]$) and also improves prediction of quality changes between neighboring distortion states. These results support direct local SVD entropy as an interpretable structural channel that complements conventional image-domain similarity and remains informative without a pristine reference.
Read the full abstract

Image and Video Processing

Modality Contribution Score - A Per-Patient Framework for Quantifying the Relative Diagnostic Contribution of Structural MRI and Amyloid PET in Alzheimer's Disease

Multimodal neuroimaging combining structural MRI and positron emission tomography (PET) captures complementary structure-function relationships across the Alzheimer's disease (AD) continuum, yet existing artificial intelligence systems produce a single diagnostic label without quantifying which imaging modality drove that decision for a specific patient. We introduce the Modality Contribution Network (MCNet) and the Modality Contribution Score (MCS), the first per-patient attribution framework quantifying the shift in modality dominance from structural atrophy to amyloid and metabolic dysfunction across the cognitively normal to MCI to AD continuum. MCS is normalised to unity per subject via modality ablation (MCS_MRI_i + MCS_PET_i = 1.0 for every subject i), providing an interpretable, clinically actionable score that fluid biomarkers cannot supply. Applied to 327 ADNI-3 participants balanced across cognitively normal, mild cognitive impairment, and AD groups, MCNet achieved competitive three-class staging performance (AUC=0.881). The MCS revealed a statistically significant monotonic gradient (Kruskal-Wallis p<0.0001), with increasing PET dominance from cognitively normal (MCS_PET 0.412+/-0.229) through MCI (0.489+/-0.289) to AD (0.671+/-0.426), validated against amyloid SUVR (r=0.172, p=0.006) and FDG metabolic biomarkers (r=-0.287, p=0.0005) from separate imaging pipelines. External replication in 1,073 independent OASIS-3 subjects confirmed cross-cohort generalisability (H=166.99, p<0.0001, eta^2=0.156). A mechanistic comparison with SHAP demonstrated that ablation-based MCS captures clinically meaningful modality dependence that deviation-based methods cannot. These findings position MCNet as a foundation for personalised imaging decisions, clinical trial stratification, and trustworthy AI in dementia care.
Read the full abstract

All authors Dawa Chyophel Lepcha, Aaliya Ali, Sophie A. Martin, Deepika Koundal, Pierrick Coupe, Shabbir Syed-Abdul

Image and Video Processing

GET: Generative Embedding Translation for Medical Image Segmentation

Generative segmentation provides an alternative to direct pixel-wise prediction by operating on learned latent representations, but effective image-to-mask translation must preserve target structure while remaining computationally efficient. We propose Generative Embedding Translation (GET), a structured embedding-translation framework that progressively transforms image embeddings into mask embeddings within the frozen latent space of a Stable Diffusion VAE. GET uses a U-Net-style Embedding Translation Network with 1.07M trainable parameters, combining Mobile Bottleneck Convolutions, Subsampled Self-Attention, and Multi-scale Feature Enrichment for local modeling, global context, and multi-scale refinement. Across five medical segmentation datasets, GET outperforms generative, CNN, and Transformer baselines. Compared with the strongest generative baseline, GMS, GET improves average Dice and IoU by 0.93% and 1.26%, reduces HD95 by 0.81 pixels, and uses 31.41% fewer trainable parameters. Under bidirectional BUS-BUSI domain shift, GET further improves Dice and IoU by 3.51% and 3.39%, while reducing HD95 by 27.37 pixels. Our code is available at: https://github.com/maklachur/GET.
Read the full abstract

All authors Md Maklachur Rahman, Md Hasan Al Banna, Saraf Anjum, Mahmudul Hasan, Tracy Hammond

Image and Video Processing

Multispectral Household Plastic Classification for Recycling Using a Camera Array

Plastic pollution has become a persistent problem in natural ecosystems. Driven by insufficient waste management, limited recycling efficiency, and high costs for recycling, plastic waste in the environment poses significant ecological and health risks. Recycling requires accurate identification of polymer types, but existing optical sorting systems often struggle to distinguish common household plastics. In this work, we present a novel classification approach based on a multispectral imaging system consisting of nine cameras equipped with near-infrared bandpass filters. The system is designed to discriminate the seven most common household plastics. From the resulting multispectral images, we extract the spectral fingerprints and derive features such as intensity differences between specific wavelength pairs and their slopes, as well as false-color image representations. A dedicated preprocessing pipeline aligns and normalizes the data before classification. We recorded a multispectral household plastic database (https://github.com/FAU-LMS/MHPM) and trained four different classifiers Gradient Boosting, Extreme Gradient Boosting, Light Gradient Boosting Machine, and CatBoost. The best-performing model achieves a classification accuracy of 86.7%. The computational runtime is 2.603 {\mu}s per pixel, enabling efficient processing of high-resolution images. The entire setup is built from off-the-shelf hardware components, which makes replication straightforward and allows direct integration into industrial sorting pipelines.
Read the full abstract

Image and Video Processing · Cross-listed from physics.med-ph

Pulse-Echo Ultrasound Methods for Speed-of-Sound Estimation: A Review

Speed of sound (SoS) is fundamental to ultrasound beamforming and a promising quantitative biomarker for tissue characterization. Although transmission- and reflector-based methods have demonstrated the diagnostic value of SoS, their need for dedicated hardware or constrained acquisition geometries limits clinical usability. Pulse-echo SoS imaging addresses these limitations by estimating SoS with conventional ultrasound probes and data acquisition paradigms. This review provides a structured overview of pulse-echo SoS estimation methods, categorized by the source of information used, the estimated SoS representation, and the reconstruction strategy. We review methods by their locality, e.g., global, layer-wise, and spatially-resolved predictions, as well as in terms of prediction strategies including gradient-free search, optimization-based solutions, analytical closed-form approximations, and deep learning-based methods. Finally, we summarize emerging diagnostic applications in the breast, liver, and muscle, while discussing challenges and possibilities with SoS imaging.
Read the full abstract

Image and Video Processing · Cross-listed from cs.AR

Precision-Aware Variable Bit Processing Elements for Hardware-Efficient Systolic Array Designs

Systolic arrays (SAs) have emerged as prominent hardware accelerators for matrix operations in deep learning, while floating point number formats enable precision control across computational domains. This research investigates approximate computing techniques for floating point (FP) multipliers in Weight Stationary Systolic Arrays, focusing on IEEE 754 (FP32), TensorFloat-32 (TF32), and Brain Floating point (BF16) formats. By integrating partial product matrix (PPM) column truncation with positive and negative compressors in the FP multiplier architecture, we optimize the trade-off between computational efficiency and accuracy. NSGA-II optimization algorithm was employed to explore the vast design space for evolving FP multiplier designs, towards achieving substantial hardware improvements while maintaining acceptable output quality. Substantial hardware benefits were observed in the FP multiplier designs across various applications, while preserving output quality. The FP approximated Processing Elements designed in the SA was found to offer comparable CNN accuracy for models trained on MNIST, F-MNIST, and CIFAR-10 dataset. The FP approximated SA designs that fall in the top 10 CNN performance offered substantial hardware gains in the range of 66% to 92% footprint savings, 60% to 93% of power benefits with 21% to 54% improvement in the delay when compared with the corresponding exact implementations mentioned in the literature for running the model trained on CIFAR-10 dataset. The TF32 and BF16 approximated SA designs also achieved substantial gains while maintaining comparable CNN accuracy. Our findings confirm that targeted approximation in FP multiplier design significantly improves the efficiency of hardware accelerators for error-tolerant applications, establishing an effective approach to hardware resource optimization in contemporary computing architectures.
Read the full abstract

Image and Video Processing · Cross-listed from physics.geo-ph

The spatial anatomy of urban wildfire vulnerability: a spatially validated GeoAI framework reveals the roles of building density and vegetation moisture in structure loss during the 2025 Palisades Fire

Urban wildfire resilience depends on interactions among built form, vegetation condition, and extreme fire weather, yet city-scale risk models often overlook whether predictive skill transfers across neighborhoods. We developed a spatially validated GeoAI workflow for the January 2025 Palisades Fire, linking 12,081 CAL FIRE damage inspections to pre-fire Sentinel-2 vegetation indices, Landsat surface temperature, LANDFIRE fuels, terrain, and OpenStreetMap buildings and roads. Among 9,883 inspected residential structures, 5,566 were destroyed. Random cross-validation yielded ROC-AUC 0.92 for the integrated XGBoost model, but 1 km spatial block validation reduced performance to 0.75; logistic regression performed similarly and was better calibrated. Building count within 100 m was the strongest predictor, with destruction odds increasing 4.12-fold per standard deviation. Vegetation moisture and greenness showed opposing conditional associations: NDMI at 100-300 m was protective (OR 0.52), whereas NDVI at 30-100 m was positively associated with destruction after accounting for moisture (OR 1.74). Predictive information was concentrated at the 100-300 m neighborhood scale. A separate post-fire track mapped burn severity and vegetation recovery without leakage. The results support neighborhood-scale susceptibility screening, moisture-aware vegetation management, and spatial block validation as a minimum standard for single-event urban wildfire modeling.
Read the full abstract

Image and Video Processing

CiUNet: A Hybrid Swin-CNN UNet for Medical Image Segmentation

Medical image segmentation requires high accuracy and robustness, yet practical commercial deployment also demands privacy preservation and computational efficiency. In this context, the U-Net architecture, which can be inherently decoupled into independent encoder and decoder components, serves as a natural commercial choice. However, pure Transformer-based variants like Swin-UNet often suffer from insufficient local detail capture and limited interpretability. In this paper, we propose a lightweight hybrid architecture built upon the Swin-UNet framework. Our model integrates a parallel CNN encoder to complement the shallow layer reasoning of Swin Transformers with local texture features. To bridge the semantic gap and enhance fine-grained spatial detail recovery, we design an asymmetric feature fusion strategy and introduce cross-layer skip (XSkip) connections that explicitly propagate shallow CNN features into the decoder. We further incorporate novel loss functions and an auxiliary supervision head (Aux-Head) to strengthen training stability, boundary delineation, and intermediate feature interpretability. Extensive experiments on the Synapse multi-organ segmentation dataset demonstrate that our approach achieves state-of-the-art competitive Dice scores and Hausdorff distances, offering an accurate, efficient, and interpretable solution for clinical deployment.
Read the full abstract

Saturday 22 August

Image and Video Processing

Pretreatment DCE-MRI Resolves Response Quality Within Pathologic Endpoints in Neoadjuvant Breast Cancer

Pathologic complete response (pCR) is a strong neoadjuvant endpoint, yet 5-15% of complete responders recur and clinical/genomic variables do not reliably identify them. We tested whether pretreatment dynamic contrast-enhanced MRI entropy - intratumoral enhancement heterogeneity - resolves response quality hidden within pCR and residual cancer burden (RCB). Across four cohorts (1,200 patients), a prespecified entropy threshold defined favorable and adverse structural states. Crossing structure with pathology yielded a four-tier framework spanning 4.1-fold recurrence in I-SPY1 and 7.7-fold at response extremes. In I-SPY2, 55 of 219 complete responders (25.1%) were structurally adverse, pretreatment. In an external HER2-positive responder synthesis (I-SPY1 pathology-confirmed pCR plus UCSF best-response proxy; n = 33, 10 events), adverse structure was associated with higher recurrence risk (HR = 2.87, 95% CI 1.38-5.96) capturing 7 of 10 recurrences, enriching rather than determining risk. In a HER2-positive RCB-0 subset, recurrence was 12.5% with favorable and 80.0% with adverse structure; Firth Cox regression preserved the association (HR = 8.13, 95% CI 1.71-49.21; n = 21, 6 events). In Duke (n = 908; 76 events), favorable structure remained independently associated with lower distant-recurrence risk (adjusted HR = 0.61, 95% CI 0.41-0.91). RNA linked favorable structure to a directionally reproduced immune-architecture program among non-overlapping patients within ISPY2; EMT-pathway enrichment was favorable-side, while the adverse tier contained a broadly immune-depleted substate. Yet full-cohort RNA models weakly discriminated structural state and did not recover continuous entropy. Pretreatment MRI therefore does not replace pCR or RCB; it reveals response-quality differences that these endpoints compress and identifies a recurrence-enriched group for prospective validation.
Read the full abstract

Image and Video Processing

SweepLSD: A One-Pass, O(width)-Memory Line Segment Detector with an Integer-Only Streaming Core and a Real-Time FPGA Realization

We present SweepLSD, a line segment detector that reads the image exactly once and emits each segment within a few rows of its last pixel passing the scan line. Every stage, including connected-component labeling and the final line test, processes the image as a row stream: intermediate memory is O(width) rather than O(pixels), and the per-pixel core is integer-only. We give the first complete description of the algorithm, designed in the author's 2014 master's thesis but never published, together with an open-source C++17 implementation and an FPGA realization – held bit-exact against the software in its hardware configuration – detecting segments in live 1080p30 video on 2009-era silicon without frame buffer or external memory. On structure-rich public 4K photographs downscaled to Full-HD, one CPU thread detects segments in ~11 ms – 4.6x/5.2x/25x faster than the original authors' implementations of ELSED, EDLines, and LSD – with the tightest frame-time distribution and the best per-segment direction accuracy of the four detectors, and curve rejection by design, while trailing ELSED in F-score on synthetic ground truth. A Manhattan-frame vanishing-point study on York Urban and NYU-VP scores every detector under a selection/evaluation-separated best-estimator-per-detector protocol, under which SweepLSD leads on NYU-VP by ~0.3 degrees and trails by 0.1 degrees on York Urban, with the fastest end-to-end pipeline of the four detectors on both. A single-frame camera-attitude application, evaluated on synthetic scenes with exact ground truth and on EuRoC and TUM-VI, matches the baselines' accuracy at a fraction of their memory, and drives a 4K horizon lock to 0.06 degrees median attitude error at 32 ms median per frame.
Read the full abstract

Image and Video Processing

CRS-Bench: A Reference-Relative Reliability Benchmark for Medical Image Encoders

Pretrained image encoders are central to medical image classification, where expert annotation is costly and task-specific cohorts are often limited. As the model space expands from general-purpose to broad-medical and specialty-specific encoders, selecting the representation becomes a substantive modeling decision. Clean-test discrimination alone is insufficient for this purpose: encoders with similar AUROC can differ in calibration, label efficiency, and stability under acquisition perturbations or distribution shift. We introduce CRS-Bench, a controlled benchmark for multi-objective medical encoder selection. CRS-Bench evaluates 15 pretrained encoder families across dermatology, ophthalmology, and radiology using ISIC 2019, APTOS 2019, and CheXpert, with CheXpert-to-MIMIC-CXR as an observed institutional shift, yielding 17,575 controlled run records and 3,515 seed-aggregated metric rows. Each encoder is characterized along four operational reliability dimensions: discrimination, calibration, label efficiency, and robustness. We summarize these dimensions using the Clinical Reliability Score (CRS), a Pareto-aware, reference-relative score combining dominance, profile balance, and worst-axis performance. AUROC and CRS are positively associated but not decision-equivalent: 21 of 105 pairwise orderings reverse, with a mean absolute rank displacement of 1.87. Paired-seed bootstrap analysis identifies PanDerm, MedSigLIP, and MedGemma as a stable leading reliability tier rather than a statistically resolved single leader. CRS-Bench provides a controlled framework for selecting medical image encoders from multi-axis reliability profiles rather than clean-test AUROC alone.
Read the full abstract

All authors Xingtao Lin, Hangqi Ren, Caiwan Sun, You Chen

Image and Video Processing · Cross-listed from physics.med-ph

A scattering correction method for CT reconstruction based on the Wavelet Adaptive Material-dependent Boltzmann Transport Equation (WAM-BTE)

X-ray computed tomography (CT) is an essential imaging technology in clinical diagnosis. However, scattered photons can reduce image contrast and introduce CT value bias, which severely degrades image quality. Recently, scatter correction methods based on the Boltzmann transport equation (BTE) have attracted increasing attention due to their high physical accuracy and flexibility. Nevertheless, existing BTE-based methods usually employ single-material models, which cannot accurately describe the nonlinear energy dependence of photon interaction cross-sections in different materials. In this work, a scatter correction method based on the wavelet adaptive material-dependent BTE (WAM-BTE) is proposed. The conventional single-material model is extended to a multi-material model by introducing material-dependent scattering distributions. Furthermore, an adaptive multi-scale framework is established through wavelet decomposition. The low-frequency wavelet coefficients are used for coarse-scale scatter estimation to reduce computational complexity, while the high-frequency wavelet energy of high attenuation materials is utilized for adaptive local refinement. Theoretical analysis demonstrates that, when using the Haar basis function, the low-frequency wavelet coefficients at the $w$-th level are mathematically equivalent to block-average downsampling with a scale factor of $2^w$, except for a deterministic normalization factor. Experimental results show that the proposed WAM-BTE method achieves comparable accuracy to the Monte Carlo method while preserving the computational efficiency of coarse-scale estimation. The scatter calculation time for a single projection view is reduced to the millisecond level.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices

Event-based vision has emerged as a promising paradigm for energy-aware artificial intelligence (AI), offering sparse, low-latency visual signals that reduce redundant data processing and support sustainable edge computing. However, the asynchronous and noise-prone nature of event streams creates challenges for conventional deep learning models, which are often too computationally intensive for low-power embedded platforms. This work presents a compact and configurable event-driven autoencoder that efficiently compresses neuromorphic data while preserving essential spatiotemporal structure for downstream inference. The architecture integrates lightweight convolutional encoding with robust performance under adaptive event thresholding and a minimal classifier head, enabling substantial reductions in computational cost without degrading recognition fidelity. Extensive evaluations on the Smart Event Face Dataset (SEFD) and Event-Based Crossing Dataset (EBCD) show that the proposed framework achieves competitive or superior accuracy compared to YOLOv9 while requiring up to 35.6$\times$ fewer parameters. To assess real-world sustainability, the model is deployed on resource-constrained hardware: a Raspberry Pi 4B and a NVIDIA Jetson Nano. On NVIDIA Jetson Nano, it delivers real-time throughput of 44.8 FPS. On a Raspberry Pi 4B CPU, the 50% autoencoder classifier consumes 16.19 J for the evaluated inference workload, corresponding to approximately 726.3$\times$ lower energy consumption than YOLOv9 under the same evaluation protocol. These results demonstrate the potential of compact event-driven models to advance environmentally conscious, low-power AI systems for high-speed perception in autonomous, mobile, and embedded computing environments.
Read the full abstract

All authors Riadul Islam, Joey Mule, Dhandeep Challagundla, Shahmir Rizvi, Sean Carson, Rachit Saini

Image and Video Processing

Single-Model Adaptive Wireless Image Transmission via Feature Sparsity Regularization

Learned joint source-channel coding (JSCC) enables robust wireless image transmission by jointly optimizing the transmitter and receiver over differentiable channel models. For bandwidth-limited and time-varying visual links, a single model should support user-adjustable transmission rate and adapt to changing wireless channel conditions, while also dynamically allocating resources according to spatial content. Existing content-adaptive or dynamic allocation schemes often rely on entropy coding, context/probability prediction, explicit rate maps or masks, or auxiliary allocation networks, complicating the encoder-decoder pipeline and increasing side-information overhead. We propose TS-JSCC, a single-model adaptive JSCC framework with tail-structured sparsification. First, an L1-based tail-structured sparsification objective encourages each token to retain an active feature-channel prefix while suppressing trailing ones. This enables content-adaptive feature-channel allocation with compact side information through active-prefix transmission. Second, lightweight stage-wise neural regulating modules use a normalized sparsity-control coefficient and the channel signal-to-noise ratio (SNR) to rescale intermediate features for single-model transmission rate and SNR adaptation. Experiments on CIFAR-10, Kodak, and CLIC2021 under additive white Gaussian noise (AWGN) and Rayleigh fading show that TS-JSCC achieves strong rate-distortion performance against the latest learned-JSCC baselines and remains competitive with the considered idealized separation baselines, while retaining a simple one-shot encoder-decoder without extra structures or computations.
Read the full abstract

All authors Xianghao Cui, Li Lan, Qi He, Bo Che, Chenyuan Feng, Zhi Chen, Tony Q. S. Quek

Friday 21 August

Image and Video Processing

Image Denoising Using Lower Semi-Frames

A blind image denoising framework based on an infinite directional lower semi-frame (DLSF) is proposed for additive white Gaussian noise. The model employs scale-dependent directional analysis with resolvent regularization of the unbounded semi-frame operator. Noise variance is estimated directly in the DLSF domain by modeling the joint covariance of four directional difference channels and applying covariance whitening to obtain a chi-square statistic. A lower-tail moment estimator provides blind noise estimation without median absolute deviation. The estimated noise level is incorporated into channel-wise Wiener-type shrinkage and canonical-dual synthesis, followed by a data-consistent iterative reconstruction with automatic stopping. Experiments on three standard grayscale images at noise levels 15–30 yield a mean relative noise-estimation error of 3.28%, with average improvements of 7.45 dB in PSNR and 0.367 in SSIM. At 30/255 noise, the estimation error decreases to 1.73%, with a mean PSNR gain of 8.31 dB. Results demonstrate effective noise suppression and structural preservation, with the strongest performance on smooth and edge-dominated images.
Read the full abstract

Image and Video Processing

FlowSem: Flow Matching for Adaptive Wireless Image Transmission in Semantic Communication

Wireless image transmission becomes challenging under poor channel conditions and stringent bandwidth constraints, as the receiver needs to preserve both pixel-level fidelity and meaningful visual structure. Classical separation-based systems, such as better portable graphics with low-density parity-check coding (BPG+LDPC), may suffer from cliff-effect behavior. While deep joint source-channel coding (DeepJSCC) provides graceful degradation as channel conditions worsen, its reconstructions may lose fine details under strong compression and severe channel distortion. To address this limitation, this paper proposes a two-stage flow matching-based semantic communication framework, termed FlowSem. In the first stage, a signal-to-noise ratio (SNR)-adaptive DeepJSCC model maps the source image into channel symbols and produces a coarse reconstruction at the receiver. In the second stage, a conditional flow matching model generates the final image from Gaussian noise conditioned on the DeepJSCC reconstruction and channel SNR. The proposed framework is evaluated on the Cityscapes dataset under additive white Gaussian noise (AWGN) and Rayleigh fading channels using a fixed channel-symbol budget. The baselines include a rate-matched BPG+LDPC system, DeepJSCC, a denoising diffusion probabilistic model (DDPM), and a denoising diffusion implicit model (DDIM). Results show that FlowSem achieves competitive pixel-level fidelity and improved structural and perceptual reconstruction quality over the considered generative baselines across different channel conditions. FlowSem provides up to 60% lower Fréchet Inception Distance (FID) than the diffusion baselines at low SNRs. Moreover, FlowSem reaches high reconstruction quality using only a few ODE integration steps, providing a favorable quality-latency tradeoff compared with standard DDPM and accelerated DDIM sampling.
Read the full abstract

Image and Video Processing

CHIMERA Challenge: Biochemical Recurrence Prediction in Prostate Cancer Patients using multimodal datasets

Biochemical recurrence (BCR), defined as any detectable prostate-specific antigen level after prostatectomy with confirmatory elevation, is widely used as a surrogate endpoint and typically assessed using clinical and pathological variables. Currently, no standardized benchmark exists for multimodal prognostic modeling in urological cancers, partly because curating heterogeneous multimodal data remains challenging. We developed the CHIMERA Challenge, a multimodal benchmark integrating preoperative mpMRI, post-prostatectomy histopathology, patient characteristics, and clinician-derived variables from 267 patients across two institutions. The dataset comprises 801 MRI sequences, 13 clinical variables per case, and 942 WSIs. Training (n=95), validation (n=23), and test (n=149) splits were established and hosted on the Grand Challenge platform. Baseline clinical and pathological characteristics did not differ significantly across splits. Models were evaluated on predicting time to BCR using the C-index. Post-challenge analyses tested how each model type performed when clinician-derived variables were withheld or randomized. Unimodal clinical models achieved the highest test C-index of 0.7402 but proved sensitive to the integrity of these variables, with performance collapsing toward chance (C approximately 0.50) when they were randomized. Multimodal models retained near-baseline performance when these variables were withheld (delta C at most 0.04), indicating their ability to recover prognostic signal directly from imaging data. CHIMERA is the first public, standardized multimodal benchmark for prostate cancer prognosis. Although models using only patient characteristics and clinician-derived variables yielded the highest leaderboard performance, multimodal models demonstrated greater robustness in clinically realistic scenarios where complete expert annotation is not guaranteed.
Read the full abstract

All authors Robert N. Spaans, Catherine Chia, Tongjie Wang, Adam Kowalewski, Parandzem Khachatryan, Domingos Oliveira, Khrystyna Faryna, Jean-Paul A. van Basten, Geert Litjens, Nadieh Khalili

Image and Video Processing

MDFI: A Multi-Domain Features Integration for Compressed Video Quality Enhancement

The latest video coding standard, H.266/VVC, has demonstrated significant improvements in compression efficiency compared to H.265/HEVC. Despite its advanced coding techniques, H.266/VVC still faces challenges in meeting the increasing demand for higher perceptual quality and enhanced compression performance. To address these limitations, we propose MDFI (Multi-Domain Features Integration), a compressed video quality enhancement approach that features a novel Frame-Prediction Feature Transform (FPFT) module to process prediction information. Moreover, MDFI integrates a multi-domain feature fusion strategy that effectively combines spatiotemporal characteristics, cross-frequency representations, and compressed-domain prediction information to enhance decoded video quality. Additionally, we introduce a comprehensive dataset that encompasses uncompressed video sequences, corresponding reconstructed versions at multiple QP levels, and predicted frames generated from H.266/VVC compressed bitstreams, providing essential resources for developing and benchmarking video enhancement approaches. Extensive experiments demonstrate that our MDFI approach achieves superior performance to state-of-the-art methods in both objective metrics and visual quality, effectively mitigating video compression artifacts. The code is available at: https://github.com/dangdinh17/MDFI.git.
Read the full abstract

All authors Sang NguyenQuang, Hieu Bui Minh, Dang BuiDinh, Xiem HoangVan

Image and Video Processing

Reliability- and Anatomy-Consistency-Aware Multimodal Learning for Robust Fracture Classification from Bangladeshi Radiographs

Background: Multimodal fracture classifiers may benefit from patient and anatomical metadata, but they can also become brittle when contextual information is missing or mismatched. Methods: We studied 1493 radiographs from the Bangladeshi OrthoFrac-XR dataset using leakage-safe age, sex, bone type, and laterality. A ConvNeXt image encoder was combined with a clinical multilayer perceptron through concatenation, late fusion, reliability-gated residual fusion, and a hierarchical state-location formulation. We additionally introduced an anatomy-consistency gate that attenuates metadata corrections when an image-side anatomical prediction disagrees with the reported bone type. Results: Across five folds and three seeds, hierarchical residual fusion achieved a macro-F1 of 0.6046 +/- 0.0279, compared with 0.5727 +/- 0.0270 for image-only learning, while improving the Brier score from 0.5239 to 0.4948. In a five-fold robustness experiment, anatomy-consistency fusion reduced the macro-F1 loss under shuffled metadata from 0.0567 to 0.0203 relative to ordinary residual fusion, although its clean-data macro-F1 was lower. Without bone type at inference, auxiliary anatomy supervision improved macro-F1 from 0.5620 +/- 0.0330 to 0.5899 +/- 0.0289. Conclusions: Structured context improves fracture classification, and consistency-aware gating limits harm from mismatched metadata. The observed clean-performance-robustness trade-off and the absence of patient-level identifiers motivate external and prospective validation.
Read the full abstract