Computer Vision · Cross-listed from physics.ao-ph
A curated UK rain radar data set for training and benchmarking nowcasting models
Viv Atureta, Rifki Priansyah Jasin and Stefan Siegert
This paper documents a data set of UK rain radar image sequences for use in statistical modeling and machine learning methods for nowcasting. The main
dataset contains 1,000 randomly sampled sequences of length 20 steps (15-minute increments) of 2D radar intensity fields of dimension 40x40 (at 5km spatial resolution). Spatially stratified sampling ensures spatial homogeneity despite removal of clear-sky cases by threshold-based truncation. For each radar sequence, additional atmospheric and geographic features are made available, including date, location, mean elevation, mean wind direction and speed and prevailing storm type. New R functions to extract data from the binary "Nimrod" radar data format are provided. A case study is presented to train and evaluate a simple convolutional neural network for radar nowcasting, including self-contained R code.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
SepsisSuite: Beyond Risk Stratification – A Comparative Analysis of Deep Fusion vs. Expert Stacking for Prescriptive Sepsis AI
Ryan Cartularo
Sepsis accounts for nearly 20% of global ICU admissions, yet conventional prediction models often fail to effectively integrate heterogeneous data streams, remaining either siloed by modality or reliant on brittle early fusion. In this work, we present a rigorous architectural comparison between End-to-End Deep Fusion and Context-Aware Stacking for sepsis tasks. We initially hypothesized that a novel Quad-Modal Hierarchical Gated
Attention Network – termed SepsisFusionFormer – would resolve complex cross-modal interactions between vitals, text, and imaging. However, experiments on MIMIC-IV revealed that SepsisFusionFormer suffered from "
attention starvation" in the small antibiotic cohort ($N \approx 2,100$), resulting in overfitting (AUC 0.66). This counterintuitive result informed the design of SepsisLateFusion, a "leaner" Context-Aware Mixture-of-Experts (MoE) architecture. By treating modalities as orthogonal experts – the "Historian" (Static), the "Monitor" (Temporal), and the "Reader" (NLP) – and dynamically gating them via a CatBoost meta-learner, we achieved State-of-the-Art (SOTA) performance: 0.915 AUC for prediction 4 hours prior to clinical onset. By calibrating the decision threshold for clinical safety, we reduced missed cases by 48% relative to the default operating point, thus opening a true preventative window for timely intervention over reactive alerts. Furthermore, for the novel prescriptive task of multi-class antibiotic selection, we demonstrate that a Quad-Modal Ensemble achieved the highest performance (0.72 AUC). These models are integrated into SepsisSuite, a deployment-ready Python framework for clinical decision support. SepsisSuite is available for free at: https://github.com/RyanCartularo/SepsisSuite-Info
Read the full abstract ▾
Computer Vision
Demo: Generative AI helps Radiotherapy Planning with User Preference
Riqiang Gao, Simon Arberet, Martin Kraus et al. (+5 more)
Radiotherapy planning is a highly complex process that often varies significantly across institutions and individual planners. Most existing deep learning approaches for 3D dose prediction rely on reference plans as ground truth during training, which can inadvertently bias models toward specific planning styles or institutional preferences. In this study, we introduce a novel generative model that predicts 3D dose distributions based solely on user-defined preference
flavors. These customizable preferences enable planners to prioritize specific trade-offs between organs-at-risk (OARs) and planning target volumes (PTVs), offering greater flexibility and personalization. Designed for seamless integration with clinical treatment planning systems, our approach assists users in generating high-quality plans efficiently. Comparative evaluations demonstrate that our method can surpasses the Varian RapidPlan model in both adaptability and plan quality in some scenarios.
Read the full abstract ▾
All authors Riqiang Gao, Simon Arberet, Martin Kraus, Han Liu, Wilko F.A.R. Verbakel, Dorin Comaniciu, Florin-Cristian Ghesu, Ali Kamen
Computer Vision · Cross-listed from eess.IV
Enhanced Chest Disease Classification Using an Improved CheXNet Framework with EfficientNetV2-M and Optimization-Driven Learning
Ali M. Bahram, Saman Muhammad Omer, Hardi M. Mohammed et al. (+1 more)
The interpretation of Chest X-ray is an important diagnostic issue in clinical practice and especially in the resource-limited setting where the shortage of radiologists plays a role in delayed diagnosis and poor patient outcomes. Although the original CheXNet architecture has shown potential in automated analysis of chest radiographs, DenseNet-121 backbone is computationally inefficient and poorly single-label classifier. To eliminate such shortcomings, we suggest a better classification framework of chest disease that relies on EfficientNetV2-M and incorporates superior training approaches such as Automatic Mixed Precision training, AdamW, Cosine Annealing learning rate scheduling, and Exponential Moving Average regularization. We prepared a
dataset of 18,080 chest X-ray images of three source materials of high authority and representing five key clinically significant disease categories which included Cardiomegaly, COVID-19, Normal, Pneumonia, and Tuberculosis. To achieve statistical reliability and reproducibility, nine independent experimental runs were run. The suggested architecture showed significant gains with mean test accuracy of 96.45 percent compared to 95.30 percent at baseline (p less than 0.001) and macro-averaged F1-score increased to 91.08 percent (p less than 0.001). Critical infectious diseases showed near-perfect classification performance with COVID-19 detection having 99.95 percent accuracy and Tuberculosis detection having 99.97 percent accuracy. Although 6.8 times more parameters are included, the training time was reduced by 11.4 percent and performance stability was increased by 22.7 percent. This framework presents itself as a decision-support tool that can be used to respond to a pandemic, screen tuberculosis, and assess thoracic disease regularly in various healthcare facilities.
Read the full abstract ▾
All authors Ali M. Bahram, Saman Muhammad Omer, Hardi M. Mohammed, Sirwan Abdolwahed Aula
Computer Vision
Action-Guided Reachability for Deterministic World Models of Vision-Based Systems
Yuang Geng, Zhongzheng Zhang, Chengzhen Jiang et al. (+4 more)
End-to-end
vision-based controllers map raw camera images directly to control actions, yet verifying their closed-loop behavior remains challenging: no analytical model maps a physical state to a camera image, and propagating a set of states through high-dimensional images yields loose reachable sets. We address both challenges with two techniques. First, we propose a Deterministic World Model (DWM), a latent-free decoder that generates camera images only from the physical state, trained with a saliency-weighted reconstruction loss and a control-consistency loss. The DWM replaces the camera in the closed loop, so a sound and tight reachable tube of the surrogate closed loop can be computed without bounding any latent variables. Second, we propose an action-guided partition-and-verify scheme that certifies as many initial sets as possible within a fixed computation budget. A Gaussian process learned from cheap simulations orders the
cells and splits those that lie across the property boundary, while every other unknown
cell is split along the state dimension that most changes the controller's actions along the simulated closed loop, i.e., the one that most reduces the pixel uncertainty reaching the controller. On a CARLA braking system and three Gym
benchmarks, the DWM is more accurate to the real camera than a cGAN surrogate and yields 1.9x to 69x narrower robustness intervals. The
vision part causes most of the remaining overapproximation, and our splitting reduces the unknown fraction of the four-dimensional CartPole initial set from 21.88% to 8.18% within the budget.
Read the full abstract ▾
All authors Yuang Geng, Zhongzheng Zhang, Chengzhen Jiang, Yanru Li, Xinyang Wang, Zhuoyang Zhou, Ivan Ruchkin
Computer Vision · Cross-listed from eess.IV
Agreement Disagreement Guided Knowledge Transfer for Cross-Scene Hyperspectral Imaging
Lu Huo, Haimin Zhang and Min Xu
Knowledge transfer plays a crucial role in cross-scene hyperspectral imaging (HSI). However, existing studies often overlook the challenges of gradient conflicts and dominant gradients that arise during the optimization of shared parameters. Moreover, many current approaches fail to simultaneously capture both agreement and disagreement information, relying only on a limited shared subset of target features and consequently missing the rich, diverse patterns present in the target scene. To address these issues, we propose an Agreement Disagreement Guided Knowledge Transfer (ADGKT) framework that integrates both mechanisms to enhance cross-scene transfer. The agreement component includes GradVac, which aligns gradient directions to mitigate conflicts between source and target domains, and LogitNorm, which regulates logit magnitudes to prevent domination by a single gradient source. The disagreement component consists of a Disagreement Restriction (DiR) and an ensemble strategy, which capture diverse predictive target features and mitigate the loss of critical target information. Extensive experiments demonstrate the effectiveness and superiority of the proposed method in achieving robust and balanced knowledge transfer across heterogeneous HSI scenes.
Read the full abstract ▾
Computer Vision
Enhancing Knowledge Transfer in Hyperspectral Image Classification via Cross-scene Knowledge Integration
Lu Huo, Wenjian Huang, Jianguo Zhang et al. (+2 more)
Knowledge transfer has strong potential to improve hyperspectral image (HSI) classification, yet two inherent challenges fundamentally restrict effective cross-domain transfer: spectral variations caused by different sensors and semantic inconsistencies across heterogeneous scenes. Existing methods are limited by transfer settings that assume homogeneous domains or heterogeneous scenarios with only co-occurring categories. When label spaces do not overlap, they further rely on complete source-domain coverage and therefore overlook critical target-private information. To overcome these limitations and enable knowledge transfer in fully heterogeneous settings, we propose Cross-scene Knowledge Integration (CKI), a framework that explicitly incorporates target-private knowledge during transfer. CKI includes: (1)
Alignment of Spectral Characteristics (ASC) to reduce spectral discrepancies through domain-agnostic projection; (2) Cross-scene Knowledge Sharing Preference (CKSP), which resolves semantic mismatch via a Source Similarity Mechanism (SSM); and (3) Complementary Information Integration (CII) to maximize the use of target-specific complementary cues. Extensive experiments verify that CKI achieves state-of-the-art performance with strong stability across diverse cross-scene HSI scenarios.
Read the full abstract ▾
All authors Lu Huo, Wenjian Huang, Jianguo Zhang, Min Xu, Haimin Zhang
Computer Vision · Cross-listed from math.NA
Generalizations of the Normalized Radon Cumulative Distribution Transform for Limited Data Recognition
Matthias Beckmann, Robert Beinert and Jonas Bresch
The Radon cumulative distribution transform (R-CDT) exploits one-dimensional Wasserstein transport and the Radon transform to represent prominent features in images. It is closely related to the sliced Wasserstein distance and facilitates classification tasks, especially in the small data regime, like the recognition of watermarks in filigranology. Here, a typical issue is that the given data may be subject to affine transformations caused by the measuring process. To make the R-CDT invariant under arbitrary affine transformations, a two-step normalization of the R-CDT has been proposed in our earlier works. The aim of this paper is twofold. First, we propose a family of generalized normalizations to enhance flexibility for applications. Second, we study multi-dimensional and non-Euclidean settings by making use of generalized Radon transforms. We prove that our novel feature representations are invariant under certain transformations and allow for linear separation in feature space. Our theoretical results are supported by numerical experiments based on 2d images, 3d shapes and 3d rotation
matrices, showing near perfect classification accuracies and clustering results.
Read the full abstract ▾
Computer Vision
Identification of Deforestation Areas in the Amazon Rainforest Using Change Detection Models
Christian Massao Konishi and Helio Pedrini
The preservation of the Amazon Rainforest is one of the global priorities in combating
climate change, protecting biodiversity, and safeguarding indigenous cultures. The Satellite-based Monitoring Project of Deforestation in the Brazilian Legal Amazon (PRODES), a project of the National Institute for Space Research (INPE), stands out as a fundamental initiative in this effort, annually monitoring deforested areas not only in the Amazon but also in other Brazilian biomes. Recently, machine learning models have been developed using PRODES data to support this effort through the comparative analysis of multitemporal satellite images, treating deforestation detection as a change detection problem. However, existing approaches present significant limitations: models evaluated in the literature still show unsatisfactory effectiveness, many do not incorporate modern architectures, such as those based on self-
attention mechanisms, and there is a lack of methodological standardization that allows direct comparisons between different studies. In this work, we address these gaps by evaluating various change detection models in a unified
dataset, including fully convolutional models and networks incorporating self-
attention mechanisms based on
Transformers. We investigate the impact of different pre- and post-processing techniques, such as filtering deforested areas predicted by the models based on the size of connected components, texture replacement, and image enhancements; we demonstrate that such approaches can significantly improve individual model effectiveness. Additionally, we test different strategies for combining the evaluated models to achieve results superior to those obtained individually, reaching an F1-score of 80.41%, a value comparable to other recent works in the literature.
Read the full abstract ▾
Computer Vision
Family Matters: A Systematic Study of Spatial vs. Frequency Masking for Continual Test-Time Adaptation
Chandler Timm C. Doloriel, Yunbei Zhang, Yeonguk Yu et al. (+6 more)
Recent continual test-time adaptation (CTTA) methods adopt masked image modeling to stabilize learning under distribution shift, yet each treats its masking family F as a fixed design choice and innovates exclusively along the selection strategy S, leaving the family axis underexplored. We present a systematic empirical study that isolates this axis. Using a controlled CTTA instantiation, Mask to Adapt (M2A), that fixes S=random and standard losses, we vary only F across spatial (patch, pixel) and frequency (all-band, low-band, high-band) families while keeping every other component identical. The study's contributions are the design guidance it extracts for the CTTA settings we evaluated: (1) the masking family determines whether adaptation compounds useful structure or compounds errors, on patch-tokenized architectures, spatial masking accumulates stable representations over long streams while frequency masking collapses catastrophically. We characterize this instability through a structural-preservation account, where spatial coherence maintains the broad-spectrum redundancy needed to avoid terminally overlapping with a corruption's spectral signature; (2) the optimal family depends on architecture-task
alignment, on CNNs, whose overlapping receptive fields dilute patch occlusion, the family gap vanishes, whereas on fine-grained tasks with global cues and large-capacity ViTs, frequency masking becomes competitive. In confounded system-level comparisons, where baselines also differ in losses and auxiliary components, M2A's random selection performs comparably to heuristic strategies, though we treat this observation as suggestive context rather than a controlled quantification of S's relative importance.
Read the full abstract ▾
All authors Chandler Timm C. Doloriel, Yunbei Zhang, Yeonguk Yu, Taki Hasan Rafi, Muhammad salman siddiqui, Tor Kristian Stevik, Fadi Al Machot, Kristian Hovde Liland, Habib Ullah
Computer Vision
Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking
Chandler Timm C. Doloriel, Habib Ullah, Kristian Hovde Liland et al. (+2 more)
Universal deepfake detection aims to identify AI-generated images across a broad range of generative models, including unseen ones. This requires robust generalization to new and unseen deepfakes, which emerge frequently, while minimizing computational overhead to enable large-scale deepfake screening, a critical objective in the era of Green AI. In this work, we explore frequency-domain masking as a training strategy for deepfake detectors. Unlike traditional methods that rely heavily on spatial features or large-scale pretrained models, our approach introduces random masking and
geometric transformations, with a focus on frequency masking due to its superior generalization properties. We demonstrate that frequency masking not only enhances detection accuracy across diverse generators but also maintains performance under significant model
pruning, offering a scalable and resource-conscious solution. Our method achieves state-of-the-art generalization on GAN- and
diffusion-generated image
datasets and exhibits consistent robustness under structured
pruning. These results highlight the potential of frequency-based masking as a practical step toward sustainable and generalizable deepfake detection. Code and models are available at https://github.com/chandlerbing65nm/FakeImageDetection.
Read the full abstract ▾
All authors Chandler Timm C. Doloriel, Habib Ullah, Kristian Hovde Liland, Fadi Al Machot, Ngai-Man Cheung
Computer Vision
Youngjoon Jang, Liliane Momeni, Zifan Jiang et al. (+3 more)
Our aim is to develop a unified model for sign
language understanding, that performs sign
language translation (SLT) and sign-subtitle
alignment (SSA). Together, these two tasks enable the conversion of continuous signing
videos into spoken
language text and also the temporal
alignment of signing with subtitles – both essential for practical communication, large-scale corpus construction, and educational applications. To achieve this, our approach is built upon three components: (i) a lightweight visual backbone that captures manual and non-manual cues from human keypoints and lip-region images while preserving signer
privacy; (ii) a Sliding Perceiver mapping network that aggregates consecutive visual features into word-level
embeddings to bridge the
vision-text gap; and (iii) a multi-task scalable training strategy that jointly optimises SLT and SSA, reinforcing both linguistic and temporal
alignment. To promote cross-linguistic generalisation, we pretrain our model on large-scale sign-text corpora covering British Sign
Language (BSL) and American Sign
Language (ASL) from the BOBSL and YouTube-SL-25
datasets. With this
multilingual pretraining and strong model design, we achieve state-of-the-art results on the challenging BOBSL (BSL)
dataset for both SLT and SSA. Our model also demonstrates robust zero-shot generalisation and finetuned SLT performance on How2Sign (ASL), highlighting the potential of scalable translation across different sign
languages.
Read the full abstract ▾
All authors Youngjoon Jang, Liliane Momeni, Zifan Jiang, Joon Son Chung, Gül Varol, Andrew Zisserman
Computer Vision
SSplain: Sparse and Smooth Explainer for Retinopathy of Prematurity Classification
Elifnur Sunger, Tales Imbiriba, Peter Campbell et al. (+3 more)
Neural networks are frequently used in medical diagnosis. However, due to their black-box nature, model explainers are used to help clinicians understand better and trust model outputs. This paper introduces an explainer method for classifying Retinopathy of Prematurity (ROP) from fundus images. Previous methods fail to generate explanations that preserve input image structures such as smoothness and sparsity. We introduce Sparse and Smooth Explainer (SSplain), a method that generates pixel-wise explanations while preserving image structures by enforcing smoothness and sparsity. This results in realistic explanations to enhance the understanding of the given black-box model. To achieve this goal, we define an optimization problem with combinatorial constraints and solve it using the Alternating Direction Method of Multipliers (ADMM). Experimental results show that SSplain outperforms commonly used explainers in terms of both post-hoc accuracy and smoothness analyses. Additionally, SSplain identifies features that are consistent with domain-understandable features that clinicians consider as discriminative factors for ROP. We also show SSplain's generalization by applying it to additional publicly available
datasets. Code is available at https://github.com/neu-spiral/SSplain.
Read the full abstract ▾
All authors Elifnur Sunger, Tales Imbiriba, Peter Campbell, Deniz Erdogmus, Stratis Ioannidis, Jennifer Dy
Computer Vision · Cross-listed from cs.LG
CLARITY: Medical World Model for Guiding Treatment Decisions by Simulating Context-Aware Disease Trajectories
Tianxingjian Ding, Yuanhao Zou, Chen Chen et al. (+2 more)
Clinical decision-making in oncology requires
forecasting how disease evolves under treatment, yet most AI systems remain static predictors that cannot model longitudinal, treatment-conditioned progression. Although generative and world models have demonstrated strong capabilities in general domains, their adaptation to medicine remains limited and insufficient for capturing complex, treatment-induced physiological dynamics across temporal scales. To address these gaps, we introduce CLARITY, a medical world model that enables counterfactual simulation of treatment-conditioned disease trajectories for clinical decision-making. By jointly encoding imaging-derived latent states, temporal intervals that capture irregular follow-ups, and patient-specific clinical context, CLARITY learns smooth and interpretable representations of disease progression, allowing the model to simulate how alternative treatments reshape future disease dynamics. Because treatment optimization is inherently sequential and uncertain, requiring evaluation of long-term outcomes across multiple possible interventions, we further propose an
entropy-regularized, computationally efficient long-horizon prediction-to-decision framework that plans treatment strategies over imagined disease trajectories and iteratively refines therapy proposals through survival-aware feedback, forming a closed-loop simulation-to-decision framework for treatment planning. CLARITY achieves state-of-the-art performance in treatment planning and survival prediction across three cancer
datasets, including two
brain tumor cohorts (MU-Glioma-Post and zero-shot on UCSF-ALPTDG) and one breast cancer
dataset (ISPY-2), demonstrating strong generalization across cancer types while consistently outperforming prior generative methods and medical-domain large
language model baselines.
Read the full abstract ▾
All authors Tianxingjian Ding, Yuanhao Zou, Chen Chen, Mubarak Shah, Yu Tian
Computer Vision
FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Models
Jiyoon Pyo, Yuankun Jiao, Dongwon Jung et al. (+11 more)
Cartographic
reasoning is the skill of interpreting geographic relationships by aligning legends, map scales, compass directions, map texts, and geometries across one or more map images. Although essential as a concrete cognitive capability and for critical tasks such as disaster response and urban planning, it remains largely unevaluated. Building on progress in chart and infographic understanding, recent large
vision language model studies on map visual question-answering often treat maps as a special case of charts. In contrast, map VQA demands comprehension of layered symbology (e.g., symbols, geometries, and text labels) as well as spatial relations tied to orientation and distance that often span multiple maps and are not captured by chart-style evaluations. To address this gap, we introduce FRIEDA, a
benchmark for testing complex open-ended cartographic
reasoning in LVLMs. FRIEDA sources real map images from documents and reports in various domains and geographical areas. Following classifications in Geographic Information System (GIS) literature, FRIEDA targets all three categories of spatial relations: topological (border, equal, intersect, within), metric (distance), and directional (orientation). All questions require multi-step inference, and many require cross-map grounding and
reasoning. We evaluate eleven state-of-the-art LVLMs under two settings: (1) the direct setting, where we provide the maps relevant to the question, and (2) the contextual setting, where the model may have to identify the maps relevant to the question before
reasoning. Even the strongest models, Gemini-2.5-Pro and GPT-5-Think, achieve only 38.20% and 37.20% accuracy, respectively, far below human performance of 84.87%. These results reveal a persistent gap in multi-step cartographic
reasoning, positioning FRIEDA as a rigorous
benchmark to drive progress on spatial intelligence in LVLMs.
Read the full abstract ▾
All authors Jiyoon Pyo, Yuankun Jiao, Dongwon Jung, Zekun Li, Leeje Jang, Sofia Kirsanova, Jina Kim, Yijun Lin, Qin Liu, Junyi Xie, Hadi Askari, Nan Xu, Muhao Chen, Yao-Yi Chiang
Computer Vision · Cross-listed from cs.RO
DIJIT: A Robotic Head for an Active Observer
Mostafa Kamali Tabrizi, Mingshi Chi, Bir Bikram Dey et al. (+5 more)
We present DIJIT, a novel binocular robotic head expressly designed for mobile
agents that behave as active observers. DIJIT's unique breadth of functionality enables active
vision research and the study of human-like eye and head-neck motions, their interrelationships, and how each contributes to visual ability. DIJIT is also being used to explore the differences between how human
vision employs eye/head movements to solve visual tasks and current computer
vision methods. DIJIT's design features nine mechanical degrees of freedom, while the cameras and lenses provide an additional four optical degrees of freedom. The ranges and speeds of the mechanical design are comparable to human performance. DIJIT attains 85% of the peak human saccade speed. Our design includes the ranges of motion required for convergent stereo, namely, vergence, version, and cyclotorsion. Here, we present DIJIT and some aspects of its performance. We also present a novel method for saccadic camera movements, using a direct relationship between camera orientation and motor values. The resulting saccadic camera movements are close to human movements in terms of their accuracy, with 1.17$^\circ$ and 1.14$^\circ$ mean error for the left and right cameras, respectively.
Read the full abstract ▾
All authors Mostafa Kamali Tabrizi, Mingshi Chi, Bir Bikram Dey, Kelly Yuan, Markus D. Solbach, Yiqian Liu, Michael Jenkin, John K. Tsotsos
Computer Vision
Restrictive Hierarchical Semantic Segmentation for Stratified Tooth Layer Detection
Ryan Banks, Camila Lindoni Azevedo, Hongying Tang et al. (+1 more)
Accurate understanding of anatomical structures is essential for reliably staging certain dental diseases. A way of introducing this within semantic
segmentation models is by utilising hierarchy-aware methodologies. However, existing hierarchy-aware
segmentation methods largely encode anatomical structure through the loss functions, providing weak and indirect supervision. We introduce a general framework that embeds an explicit anatomical hierarchy into semantic
segmentation by coupling a recurrent, level-wise prediction scheme with restrictive output heads and top-down feature conditioning. At each depth of the class tree, the backbone is re-run on the original image concatenated with logits from the previous level. Child class features are conditioned using Feature-wise Linear Modulation of their parent class probabilities, to modulate child feature spaces for fine grained detection. A probabilistic composition rule enforces consistency between parent and descendant classes. Hierarchical loss combines per-level class weighted Dice and cross
entropy loss and a consistency term loss, ensuring parent predictions are the sum of their children. We validate our approach on our proposed
dataset, TL-pano, containing 194 panoramic radiographs with dense instance and semantic
segmentation annotations, of tooth layers and alveolar bone. Utilising UNet and HRNet as donor models across a 5-fold cross validation scheme, the hierarchical variants consistently increase IoU, Dice, and recall, particularly for fine-grained anatomies, and produce more anatomically coherent masks. However, hierarchical variants also demonstrated increased recall over precision, implying increased false positives. The results demonstrate that explicit hierarchical structuring improves both performance and clinical plausibility, especially in low data dental imaging regimes.
Read the full abstract ▾
All authors Ryan Banks, Camila Lindoni Azevedo, Hongying Tang, Yunpeng Li
Computer Vision · Cross-listed from cs.LG
CIP-Net: Continual Interpretable Prototype-based Network
Federico Di Valerio, Michela Proietti, Alessio Ragno et al. (+1 more)
Continual learning constrains models to learn new tasks over time without forgetting what they have already learned. A key challenge in this setting is catastrophic forgetting, where learning new information causes the model to lose its performance on previous tasks. Recently, explainable AI has been proposed as a promising way to better understand and reduce forgetting. In particular, self-explainable models are useful because they generate explanations during prediction, which can help preserve knowledge. However, most existing explainable approaches use post-hoc explanations or require additional
memory for each new task, resulting in limited scalability. In this work, we introduce CIP-Net, an exemplar-free self-explainable prototype-based model designed for continual learning. CIP-Net avoids storing past examples and maintains a simple architecture, while still providing useful explanations and strong performance. We demonstrate that CIPNet achieves state-of-the-art performances compared to previous exemplar-free and self-explainable methods in both task- and class-incremental settings, while bearing significantly lower
memory-related overhead. This makes it a practical and interpretable solution for continual learning.
Read the full abstract ▾
All authors Federico Di Valerio, Michela Proietti, Alessio Ragno, Roberto Capobianco
Computer Vision · Cross-listed from cs.RO
VLD: Visual Learned Distance as an Interface for Reinforcement Learning Navigation
Lazar Milikic, Manthan Patel and Jonas Frey
Training end-to-end policies from image data to directly predict
navigation actions for robotic systems has proven inherently difficult. Existing approaches often suffer from either the sim-to-real gap during policy transfer or limited action-labeled training data. To address this problem, we introduce Visual Learned Distance (VLD), a scalable framework for image-goal
navigation that decouples perception learning from policy learning. We first train a self-supervised distance-to-goal predictor on about 3,000 hours of internet-scale and simulated
video, providing a distance signal that a reinforcement learning (RL) policy can minimize. The RL policy is trained separately in simulation using privileged
geometric distances perturbed by GeoNoise, a learned noise model that reproduces the viewpoint- and overlap-dependent errors of the trained distance predictor. At deployment, the policy consumes VLD predictions zero-shot, without retraining, inheriting “where to go” from large-scale visual training while retaining the robust
navigation behaviors learned in simulation. This error modeling is key to the transfer: with VLD inputs, GeoNoise-trained policies reach $58.7\%$ success on HM3D and $76.2\%$ on Gibson, whereas policies trained with generic temporally correlated noise collapse to $4.9\%$ and $11.5\%$. On a physical
robot, the VLD-driven policy succeeds in $88\%$ of trials, compared with $60\%$ for a strong end-to-end image-goal baseline that outperforms it in simulation. Our decoupled design thus provides an alternative and scalable path toward reliable
navigation policies that transfer from simulation to the real world.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Sparse Variable Projection in Robotic Perception: Exploiting Separable Structure for Efficient Nonlinear Optimization
Alan Papalia, Nikolas Sanderson, Haoyu Han et al. (+3 more)
Robotic perception often requires solving large nonlinear least-squares (NLS) problems. While sparsity has been well-exploited to scale solvers, a complementary and underexploited structure is separability – where some variables (e.g., visual landmarks) appear linearly in the residuals and, for any estimate of the remaining variables (e.g., poses), have a closed-form solution. Variable projection (VarPro) methods are a family of techniques that exploit this structure by analytically eliminating the linear variables and presenting a reduced problem in the remaining variables that has favorable properties. However, VarPro has seen limited use in robotic perception; a major challenge arises from gauge
symmetries (e.g., cost invariance to global shifts and rotations), which are common in perception and induce specific computational challenges in standard VarPro approaches. We present a VarPro scheme designed for problems with gauge
symmetries that jointly exploits separability and sparsity. Our method can be applied as a one-time preprocessing step to construct a
matrix-free Schur complement operator. This operator allows efficient evaluation of costs, gradients, and Hessian-vector products of the reduced problem and readily integrates with standard iterative NLS solvers. We provide precise conditions under which our method applies, and describe extensions when these conditions are only partially met. Across synthetic and real
benchmarks in SLAM, SNL, and SfM, our approach achieves up to \textbf{2$\times$–35$\times$ faster runtimes} than state-of-the-art methods while maintaining accuracy. We release an open-source C++ implementation and all
datasets from our experiments.
Read the full abstract ▾
All authors Alan Papalia, Nikolas Sanderson, Haoyu Han, Heng Yang, Hanumant Singh, Michael Everett
Computer Vision
Preserving Source Video Realism: High-Fidelity Face Swapping for Cinematic Quality
Zekai Luo, Zongze Du, Zhouhang Zhu et al. (+7 more)
Video face swapping is crucial in film and entertainment production, where achieving high fidelity and temporal consistency over long and complex
video sequences remains a significant challenge. Inspired by recent advances in reference-guided image editing, we explore whether rich visual attributes from source
videos can be similarly leveraged to enhance both fidelity and temporal coherence in
video face swapping. Building on this insight, this work presents LivingSwap, the first
video reference guided face swapping model. Our approach employs keyframes as conditioning signals to inject the target identity, enabling flexible and controllable editing. By combining keyframe conditioning with
video reference guidance, the model performs temporal stitching to ensure stable identity preservation and high-fidelity reconstruction across long
video sequences. To address the scarcity of data for reference-guided training, we construct a paired face-swapping
dataset, Face2Face, and further reverse the data pairs to ensure reliable ground-truth supervision. Extensive experiments demonstrate that our method achieves state-of-the-art results, seamlessly integrating the target identity with the source
video's expressions, lighting, and motion, while significantly reducing manual effort in production workflows. Project webpage: https://aim-uofa.github.io/LivingSwap
Read the full abstract ▾
All authors Zekai Luo, Zongze Du, Zhouhang Zhu, Hao Zhong, Muzhi Zhu, Wen Wang, Yuling Xi, Chenchen Jing, Hao Chen, Chunhua Shen
Computer Vision
Near–Real-Time Conflict-Related Fire Detection in Sudan Using Unsupervised Deep Learning
Kuldip Singh Atwal, Dieter Pfoser and Daniel Rothbart
Ongoing armed conflict in Sudan highlights the need for rapid monitoring of conflict-related fire-affected areas. Recent advances in deep learning and high-frequency satellite imagery enable near–real-time assessment of active fires and burn scars in war zones. This study presents a near–real-time monitoring approach using a lightweight Variational Auto-Encoder (VAE)–based model integrated with 4-band
Planet Labs imagery at 3 m spatial resolution. We demonstrate that these impacted regions can be detected within approximately 24 to 30 hours under favorable observational conditions using accessible, commercially available satellite data. To achieve this, we adapt a VAE–based model, originally designed for 10-band imagery, to operate effectively on high-resolution 4-band inputs. The model is trained in an unsupervised manner to learn compact latent representations of nominal land-surface conditions and identify burn signatures by quantifying changes between temporally paired latent
embeddings. Performance is evaluated across five case studies in Sudan and compared against cosine distance, CVA, and IR-MAD using precision, recall, F1-score, and the area under the precision-recall curve (AUPRC) computed between temporally paired image tiles. Results show that the proposed approach consistently outperforms the other methods, achieving higher recall and F1-scores while maintaining viable precision in highly imbalanced fire-detection scenarios. Experiments with 8-band imagery and temporal image sequences yield only marginal performance gains over single 4-band inputs, underscoring the effectiveness of the proposed lightweight approach for scalable, near–real-time conflict monitoring.
Read the full abstract ▾