Machine Learning · Cross-listed from cs.DC
RAPID-Serve: Resource-efficient and Accelerated P/D Intra-GPU Disaggregation
Amna Masood, Pratishtha Gaur and Nuwan Jayasena
Two widely adopted techniques for LLM inference serving systems today are hybrid batching and disaggregated serving. A hybrid batch combines prefill and decode
tokens of different requests in the same batch to improve resource utilization and throughput at the cost of increased latency per
token. In contrast, disaggregated serving decouples compute-bound prefill and bandwidth-bound decode phases to optimize for service level objectives (SLOs) at the cost of resource under-utilization and KV-cache transfer overheads. To address the limitations of these techniques, we propose RAPID-Serve: a technique to concurrently execute prefill and decode on the same GPU(s) to meet latency SLOs while maintaining high throughput and efficient resource utilization. Furthermore, we propose Adaptive Resource Management for runtime compute resource allocation, optionally leveraging CU masking (a fine-grained Compute Unit partitioning feature on AMD Instinct\textsuperscript{TM} GPUs). RAPID-Serve provides up to 4.1x (average 1.7x) unconstrained throughput improvement and 32x and higher (average 4.9x) throughput improvement under SLO constraints, showing it as an effective strategy compared to the state-of-the-art approaches, particularly in resource-constrained environments.
Read the full abstract ▾
Machine Learning
Shapelets-Enriched Selective Forecasting using Time Series Foundation Models
Shivani Tomar, Seshu Tirupathi, Elizabeth Daly et al. (+1 more)
Time series foundation models have recently gained a lot of
attention due to their ability to model complex time series data encompassing different domains including traffic, energy, and weather. Although they exhibit strong average zero-shot performance on
forecasting tasks, their predictions on certain critical regions of the data are not always reliable, limiting their usability in real-world applications, especially when data exhibits unique trends. In this paper, we propose a selective
forecasting framework to identify these critical segments of time series using shapelets. We learn shapelets using shift-invariant dictionary learning on the validation split of the target domain
dataset. Utilizing distance-based similarity to these shapelets, we facilitate the user to selectively discard unreliable predictions and be informed of the model's realistic capabilities. Empirical results on diverse
benchmark time series
datasets demonstrate that our approach leveraging both zero-shot and full-shot fine-tuned models reduces the overall error by an average of 22.17% for zero-shot and 22.62% for full-shot fine-tuned model. Furthermore, our approach using zero-shot and full-shot fine-tuned models, also outperforms its random selection counterparts by up to 21.41% and 21.43% on one of the
datasets.
Read the full abstract ▾
All authors Shivani Tomar, Seshu Tirupathi, Elizabeth Daly, Ivana Dusparic
Machine Learning
Physics-Constrained Denoising Autoencoders for Data-Scarce Wildfire UAV Sensing
Abdelrahman Ramadan, Zahra Dorbeigi Namaghi, Emily Taylor et al. (+5 more)
Wildfire monitoring requires high-resolution atmospheric measurements, yet low-cost sensors on Unmanned Aerial Vehicles (
UAVs) exhibit baseline drift, cross-sensitivity, and response lag that corrupt concentration estimates. Traditional deep learning denoising approaches demand large
datasets impractical to obtain from limited
UAV flight campaigns. We present PC$^2$DAE, a physics-informed denoising autoencoder that addresses data scarcity by
embedding physical constraints directly into the network architecture. Non-negative concentration estimates are enforced via softplus activations and physically plausible temporal smoothing, ensuring outputs are physically admissible by construction rather than relying on loss function penalties. The architecture employs hierarchical decoder heads for Black Carbon, Gas, and CO$_2$ sensor families, with two variants: PC$^2$DAE-Lean (21k parameters) for edge deployment and PC$^2$DAE-Wide (204k parameters) for offline processing. We evaluate on 7,894 synchronized 1 Hz samples collected from
UAV flights during prescribed burns in Saskatchewan, Canada (approximately 2.2 hours of flight data), two orders of magnitude below typical deep learning requirements. PC$^2$DAE-Lean achieves 67.3% smoothness improvement and 90.7% high-frequency noise reduction with zero physics violations. Five baselines (LSTM-AE, U-Net,
Transformer, CBDAE, DeSpaWN) produce 15–23% negative outputs. The lean variant outperforms wide (+5.6% smoothness), suggesting reduced capacity with strong inductive bias prevents overfitting in data-scarce regimes. Training completes in under 65 seconds on consumer hardware.
Read the full abstract ▾
All authors Abdelrahman Ramadan, Zahra Dorbeigi Namaghi, Emily Taylor, Lucas Edwards, Xan Giuliani, David S. McLagan, Sidney Givigi, Melissa Greeff
Machine Learning · Cross-listed from stat.ML
Gradient-based Active Learning with Gaussian Processes for Global Sensitivity Analysis
Guerlain Lambert, Céline Helbert and Claire Lauvernet
Global sensitivity analysis of complex numerical simulators is often limited by the small number of model evaluations that can be afforded. In such settings, surrogate models built from a limited set of simulations can substantially reduce the computational burden, provided that the design of computer experiments is enriched efficiently. In this context, we propose an active learning approach that, for a fixed evaluation budget, targets the most informative regions of the input space to improve sensitivity analysis accuracy. More specifically, our method builds on recent advances in active learning for sensitivity analysis (Sobol' indices and derivative-based global sensitivity measures, DGSM) that exploit derivatives obtained from a Gaussian process (GP) surrogate. By leveraging the joint posterior distribution of the GP gradient, we develop acquisition functions that better account for correlations between partial derivatives and their impact on the response surface, leading to a more comprehensive and robust methodology than existing DGSM-oriented criteria. The proposed approach is first compared to state-of-the-art methods on standard
benchmark functions, and is then applied to a real environmental model of pesticide transfers.
Read the full abstract ▾
Machine Learning
Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
Shenyang Deng, Boyao Liao, Zhuoli Ouyang et al. (+3 more)
This paper explores the suspicious
alignment phenomenon in
stochastic gradient descent (SGD) under ill-conditioned optimization, where the Hessian spectrum splits into dominant and bulk subspaces. This phenomenon describes the behavior of gradient
alignment in SGD updates. Specifically, during the initial phase of SGD updates, the
alignment between the gradient and the dominant subspace tends to decrease. Subsequently, it enters a rising phase and eventually stabilizes in a high-
alignment phase. The
alignment is considered “suspicious” because, paradoxically, the projected gradient update along this highly-aligned dominant subspace proves ineffective at reducing the loss. The focus of this work is to give a fine-grained analysis in a high-dimensional quadratic setup about how step size selection produces this phenomenon. Our main contribution can be summarized as follows: We propose a step-size condition revealing that in low-
alignment regimes, an adaptive critical step size $\eta_t^*$ separates
alignment-decreasing ($\eta_t < \eta_t^*$) from
alignment-increasing ($\eta_t > \eta_t^*$) regimes, whereas in high-
alignment regimes, the
alignment is self-correcting and decreases regardless of the step size. We further show that under sufficient ill-conditioning, a step size interval exists where projecting the SGD updates to the bulk space decreases the loss while projecting them to the dominant space increases the loss, which explains a recent empirical observation that projecting gradient updates to the dominant subspace is ineffective. Finally, based on this adaptive step-size theory, we prove that for a constant step size and large initialization, SGD exhibits this distinct two-phase behavior: an initial
alignment-decreasing phase, followed by stabilization at high
alignment.
Read the full abstract ▾
All authors Shenyang Deng, Boyao Liao, Zhuoli Ouyang, Tianyu Pang, Minhak Song, Yaoqing Yang
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Quantum Kernel Machine Learning for Autonomous Materials Science
Felix Adams, Daiwei Zhu, David W. Steuerman et al. (+2 more)
Autonomous materials science, where active learning is used to navigate large compositional phase space, has emerged as a powerful vehicle to rapidly explore new materials. A crucial aspect of autonomous materials science is exploring new materials using as little data as possible. Gaussian process-based active learning allows effective charting of multi-dimensional parameter space with a limited number of training data, and thus is a common algorithmic choice for autonomous materials science. An integral part of the autonomous workflow is the application of kernel functions for quantifying similarities among measured data points. A recent theoretical breakthrough has shown that
quantum kernel models can achieve similar performance with less training data than classical models. This signals the possible advantage of applying
quantum kernel machine learning to autonomous materials discovery. In this work, we compare
quantum and classical kernels for their utility in sequential phase space
navigation for autonomous materials science. Specifically, we compute a
quantum kernel and several classical kernels for x-ray diffraction patterns taken from an Fe-Ga-Pd ternary composition spread library. We conduct our study on both IonQ's Aria trapped ion
quantum computer hardware and the corresponding classical noisy simulator. We experimentally verify that a
quantum kernel model can outperform some classical kernel models. The results highlight the potential of
quantum kernel machine learning methods for accelerating materials discovery and suggest complex x-ray diffraction data is a candidate for robust
quantum kernel model advantage.
Read the full abstract ▾
All authors Felix Adams, Daiwei Zhu, David W. Steuerman, A. Gilad Kusne, Ichiro Takeuchi
Machine Learning · Cross-listed from eess.AS
Lightweight Self-Supervised Detection of Fundamental Frequency and Accurate Probability of Voicing in Monophonic Music
Venkat Suprabath Bitra and Homayoon Beigi
Reliable fundamental frequency (F 0) and voicing estimation is essential for neural synthesis, yet many pitch extractors depend on large labeled corpora and degrade under realistic recording artifacts. We propose a lightweight, fully self-supervised framework for joint F 0 estimation and voicing inference, designed for rapid single-instrument training from limited audio. Using transposition-equivariant learning on CQT features, we introduce an EM-style iterative reweighting scheme that uses Shift Cross-
Entropy (SCE) consistency as a reliability signal to suppress uninformative noisy/unvoiced frames. The resulting weights provide confidence scores that enable pseudo-labeling for a separate lightweight voicing classifier without manual annotations. Trained on MedleyDB and evaluated on MDB-stem-synth ground truth, our method achieves competitive cross-corpus performance (RPA 95.84, RCA 96.24) and demonstrates cross-instrument generalization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Industry-Aligned Granular Topic Modeling
Sae Young Moon, Myeongjun Erik Jang, Haoyan Luo et al. (+3 more)
Topic modeling has extensive applications in text mining and data analysis across various industrial sectors. Although the concept of granularity holds significant value for business applications by providing deeper insights, the capability of topic modeling methods to produce granular topics has not been thoroughly explored. In this context, this paper introduces a framework called TIDE, which primarily provides a novel granular topic modeling method based on large
language models (LLMs) as a core feature, along with other useful functionalities for business applications, such as summarizing long documents, topic parenting, and
distillation. Through extensive experiments on a variety of public and real-world business
datasets, we demonstrate that TIDE's topic modeling approach outperforms modern topic modeling methods, and our auxiliary components provide valuable support for dealing with industrial business scenarios. The TIDE framework is currently undergoing the process of being open sourced.
Read the full abstract ▾
All authors Sae Young Moon, Myeongjun Erik Jang, Haoyan Luo, Chunyang Xiao, Antonios Georgiadis, Fran Silavong
Machine Learning · Cross-listed from cs.CL
Early Linguistic Pattern of Anxiety from Social Media Using Interpretable Linguistic Features: A Multi-Faceted Validation Study with Author-Disjoint Evaluation
Arnab Das Utsa
Anxiety affects hundreds of millions of individuals globally, yet large-scale screening remains limited. Social media
language provides an opportunity for scalable detection, but current models often lack interpretability, keyword-robustness validation, and rigorous user-level data integrity. This work presents a transparent approach to social media-based anxiety detection through linguistically interpretable feature-grounded modeling and cross-domain validation. Using a substantial
dataset of Reddit posts, we trained a logistic regression classifier on carefully curated subreddits for training, validation, and test splits. Comprehensive evaluation included feature ablation, keyword masking experiments, and varying-density difference analyses comparing anxious and control groups, along with external validation using clinically interviewed participants with diagnosed anxiety disorders. The model achieved strong performance while maintaining high accuracy even after sentiment removal or keyword masking. Early detection using minimal post history significantly outperformed random classification, and cross-domain analysis demonstrated strong consistency with clinical interview data. Results indicate that transparent linguistic features can support reliable, generalizable, and keyword-robust anxiety detection. The proposed framework provides a reproducible baseline for interpretable mental health screening across diverse online contexts.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
LIME-LLM: Probing Models with Fluent Counterfactuals, Not Broken Text
George Mihaila, Suleyman Olcay Polat, Poli Nemkova et al. (+3 more)
Local explanation methods such as LIME (Ribeiro et al., 2016) remain fundamental to trustworthy AI, yet their application to NLP is limited by a reliance on random
token masking. These heuristic perturbations frequently generate semantically invalid, out-of-distribution inputs that weaken the fidelity of local surrogate models. While recent generative approaches such as LLiMe (Angiulli et al., 2025b) attempt to mitigate this by employing Large
Language Models for neighborhood generation, they rely on unconstrained paraphrasing that introduces confounding variables, making it difficult to isolate specific feature contributions. We introduce LIME-LLM, a framework that replaces random noise with hypothesis-driven, controlled perturbations. By enforcing a strict "Single Mask-Single Sample" protocol and employing distinct neutral infill and boundary infill strategies, LIME-LLM constructs fluent, on-
manifold neighborhoods that rigorously isolate feature effects. We evaluate our method against established baselines (LIME, SHAP, Integrated Gradients) and the generative LLiMe baseline across three diverse
benchmarks: CoLA, SST-2, and HateXplain using human-annotated rationales as ground truth. Empirical results demonstrate that LIME-LLM establishes a new
benchmark for black-box NLP explainability, achieving significant improvements in local explanation fidelity compared to both traditional perturbation-based methods and recent generative alternatives.
Read the full abstract ▾
All authors George Mihaila, Suleyman Olcay Polat, Poli Nemkova, Himanshu Sharma, Namratha V. Urs, Mark V. Albert
Machine Learning · Cross-listed from cs.CV
SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models
Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl et al. (+2 more)
Visual Foundation Models (VFMs), such as DINO and CLIP, excel in semantic understanding of images but exhibit limited spatial
reasoning capabilities, which limits their applicability to
embodied systems. As a result, recent work incorporates some 3D tasks (such as depth estimation) into VFM training. However, VFM performance remains inconsistent across other spatial tasks, raising the question of whether these models truly have spatial awareness or overfit to specific 3D objectives. To address this question, we introduce the Spatial Relation Recognition Task (SpaRRTa)
benchmark, which evaluates the ability of VFMs to identify relative positions of objects in the image. Unlike traditional 3D objectives that focus on precise metric prediction (e.g., surface normal estimation), SpaRRTa probes a fundamental capability underpinning more advanced forms of human-like spatial understanding. SpaRRTa generates an arbitrary number of photorealistic images with diverse scenes and fully controllable object arrangements, along with freely accessible spatial annotations. Evaluating a range of state-of-the-art VFMs, we reveal significant disparities between their spatial
reasoning abilities. Through our analysis, we provide insights into the mechanisms that support or hinder spatial awareness in modern VFMs. We hope that SpaRRTa will serve as a useful tool for guiding the development of future spatially aware visual models.
Read the full abstract ▾
All authors Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl, Bartosz Zieliński, Marcin Przewięźlikowski
Machine Learning · Cross-listed from cs.ET
A Proof of Concept for a Digital Twin of an Ultrasonic Fermentation System
Francesco Saverio Sconocchia Pisoni, Andrea Vitaletti, Davide Appolloni et al. (+4 more)
This paper presents the design and implementation of a proof of concept digital twin for an innovative ultrasonic-enhanced beer-fermentation system, developed to enable intelligent monitoring, prediction, and actuation in yeast-growth environments. A traditional fermentation tank is equipped with a piezoelectric transducer able to irradiate the tank with ultrasonic waves, providing an external abiotic stimulus to enhance the growth of yeast and accelerate the fermentation process. At its core, the digital twin incorporates a predictive model that estimates yeast's culture density over time based on the surrounding environmental conditions. To this end, we implement, tailor and extend the model proposed in Palacios et al., allowing us to effectively handle the limited number of available training samples by using temperature, ultrasonic frequency, and duty cycle as inputs. The results obtained along with the assessment of model performance demonstrate the feasibility of the proposed approach.
Read the full abstract ▾
All authors Francesco Saverio Sconocchia Pisoni, Andrea Vitaletti, Davide Appolloni, Federico Ortenzi, Blasco Morozzo della Rocca, Mariano José Guillén, Alessandro Contaldo
Machine Learning
jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation
Ho Fung Tsoi and Dylan Rankin
Self-supervised learning, in the context of foundation model training, is a powerful pre-training method for learning feature representations without labels, which often capture generic underlying semantics from the data and can later be fine-tuned for downstream tasks. In this work, we introduce jBOT, a pre-training method based on self-
distillation for jet data from the CERN Large
Hadron Collider, which combines local particle-level
distillation with global jet-level
distillation to learn jet representations that support downstream tasks such as anomaly detection and classification. We observe that pre-training on unlabeled jets leads to emergent semantic class clustering in the representation space. The clustering in the frozen
embedding, when pre-trained on background jets only, enables anomaly detection via simple distance-based metrics, and the learned
embedding can be fine-tuned for classification with improved performance compared to supervised models trained from scratch.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.ins-det
AllShowers: One model for all calorimeter showers
Thorsten Buss, Henry Day-Hall, Frank Gaede et al. (+2 more)
Accurate and efficient detector simulation is essential for modern
collider experiments. To reduce the high computational cost, various fast machine learning surrogate models have been proposed. Traditional surrogate models for calorimeter shower modeling train separate networks for each particle species, limiting scalability and reuse. We introduce AllShowers, a unified generative model that simulates calorimeter showers across multiple particle types using a single generative model. AllShowers is a continuous normalizing flow model with a
Transformer architecture, enabling it to generate complex spatial and energy correlations in variable-length point cloud representations of showers. Trained on a diverse
dataset of simulated showers in the highly granular ILD detector, the model demonstrates the ability to generate realistic showers for electrons,
photons, and charged and neutral
hadrons across a wide range of incident energies and angles without retraining. In addition to unifying shower generation for multiple particle types, AllShowers surpasses the fidelity of previous single-particle-type models for hadronic showers. Key innovations include the use of a layer
embedding, allowing the model to learn all relevant calorimeter layer properties; a custom
attention masking scheme to reduce computational demands and introduce a helpful inductive bias; and a shower- and layer-wise optimal transport mapping to improve training convergence and sample quality. AllShowers marks a significant step towards a universal model for calorimeter shower simulations in
collider experiments.
Read the full abstract ▾
All authors Thorsten Buss, Henry Day-Hall, Frank Gaede, Gregor Kasieczka, Katja Krüger
Machine Learning · Cross-listed from eess.SP
Inter-Cell Interference Rejection Based on Ultrawideband Walsh-Domain Wireless Autoencoding
Rodney Martinez Alonso, Cel Thys, Cedric Dehos et al. (+2 more)
This paper proposes a novel technique for rejecting partial-in-band inter-
cell interference (ICI) in ultrawideband communication systems. We present the design of an end-to-end wireless autoencoder architecture that jointly optimizes the transmitter and receiver encoding/decoding in the Walsh domain to mitigate interference from coexisting narrower-band 5G base stations. By exploiting the orthogonality and self-inverse properties of Walsh functions, the system distributes and learns to encode bit-words across parallel Walsh branches. Through analytical modeling and simulation, we characterize how 5G CPOFDM interference maps into the Walsh domain and identify optimal ratios of transmission frequencies and sampling rate where the end-to-end autoencoder achieves the highest rejection. Experimental results show that the proposed autoencoder achieves up to 12 dB of ICI rejection while maintaining a low block error rate (BLER) for the same baseline channel noise, i.e., baseline Signal-to-Noise-Ratio (SNR) without the interference.
Read the full abstract ▾
All authors Rodney Martinez Alonso, Cel Thys, Cedric Dehos, Yuneisy Esthela Garcia Guzman, Sofie Pollin
Machine Learning · Cross-listed from cs.CV
Telling Human and Machine Handwriting Apart
Luis A. Leiva, Moises Diaz, Nuwan T. Attygalle et al. (+2 more)
Handwriting movements can be leveraged as a unique form of behavioral biometrics, to verify whether a real user is operating a device or application. This task can be framed as a reverse Turing test in which a computer has to detect if an input instance has been generated by a human or artificially. To tackle this task, we study ten public
datasets of handwritten symbols (isolated characters, digits, gestures, pointing traces, and signatures) that are artificially reproduced using seven different synthesizers, including, among others, the Kinematic Theory (Sigma h model), generative
adversarial networks,
Transformers, and
Diffusion models. We train a shallow recurrent neural network that achieves excellent performance (98.3 percent Area Under the ROC Curve (AUC) score and 1.4 percent equal error rate on average across all synthesizers and
datasets) using nonfeaturized trajectory data as input. In few-shot settings, we show that our classifier achieves such an excellent performance when trained on just 10 percent of the data, as evaluated on the remaining 90% of the data as a test set. We further challenge our classifier in out-of-domain settings, and observe very competitive results as well. Our work has implications for computerized systems that need to verify human presence, and adds an additional layer of security to keep attackers at bay.
Read the full abstract ▾
All authors Luis A. Leiva, Moises Diaz, Nuwan T. Attygalle, Miguel A. Ferrer, Rejean Plamondon
Machine Learning · Cross-listed from eess.IV
Anisotropic Tensor Deconvolution of Hyperspectral Images
Xinjue Wang, Xiuheng Wang, Esa Ollila et al. (+1 more)
Hyperspectral image (HSI) deconvolution is a challenging ill-posed inverse problem, made difficult by the data's high dimensionality.We propose a parameter-parsimonious framework based on a low-rank Canonical Polyadic Decomposition (CPD) of the entire latent HSI $\mathbf{\mathcal{X}} \in \mathbb{R}^{P\times Q \times N}$.This approach recasts the problem from recovering a large-scale image with $PQN$ variables to estimating the CPD factors with $(P+Q+N)R$ variables.This model also enables a structure-aware, anisotropic Total Variation (TV) regularization applied only to the spatial factors, preserving the smooth spectral signatures.An efficient
algorithm based on the Proximal Alternating Linearized Minimization (PALM) framework is developed to solve the resulting non-convex optimization problem.Experiments confirm the model's efficiency, showing a numerous parameter reduction of over two orders of magnitude and a compelling trade-off between model compactness and reconstruction accuracy.
Read the full abstract ▾
All authors Xinjue Wang, Xiuheng Wang, Esa Ollila, Sergiy A. Vorobyov
Machine Learning · Cross-listed from eess.IV
Explainable histomorphology-based survival prediction of glioblastoma, IDH-wildtype
Jan-Philipp Redlich, Friedrich Feuerhake, Stefan Nikolin et al. (+12 more)
Glioblastoma, IDH-wildtype (GBM-IDHwt) is the most common malignant
brain tumor. While histomorphology is a crucial component of GBM-IDHwt diagnosis, it is not further considered for prognosis. Here, we present an explainable artificial intelligence (AI) framework to identify and interpret histomorphological features associated with patient survival. The framework combines an explainable multiple instance learning (MIL) architecture that directly identifies prognostically relevant image tiles with a sparse autoencoder (SAE) that maps these tiles to interpretable visual patterns. The MIL model was trained and evaluated on a new real-world
dataset of 720 GBM-IDHwt cases from three hospitals and four cancer registries across Germany. The SAE was trained on 1,878 whole-slide images from five independent public glioblastoma collections. Despite the many factors influencing survival time, our method showed some ability to discriminate between patients living less than 180 days or more than 360 days solely based on histomorphology (AUC: 0.67; 95% CI: 0.63-0.72). Cox proportional hazards regression confirmed a significant survival difference between predicted groups after adjustment for established prognostic factors (hazard ratio: 1.47; 95% CI: 1.26-1.72). Three neuropathologists categorized the identified visual patterns into seven distinct histomorphological groups, revealing both established prognostic features and unexpected associations, the latter being potentially attributable to surgery-related confounders. The presented explainable AI framework facilitates prognostic biomarker discovery in GBM-IDHwt and beyond, highlighting promising histomorphological features for further analysis and exposing potential confounders that would be hidden in black-box models.
Read the full abstract ▾
All authors Jan-Philipp Redlich, Friedrich Feuerhake, Stefan Nikolin, Nadine Sarah Schaadt, Sarah Teuber-Hanselmann, Joachim Weis, Sabine Luttmann, Andrea Eberle, Christoph Buck, Timm Intemann, Pascal Birnstill, Klaus Kraywinkel, Jonas Ort, Peter Boor, André Homeyer
Machine Learning
Proof of Concept: Multi-Target Wildfire Risk Prediction and Large Language Model Synthesis
Nicolas Caron, Christophe Guyeux, Hassan Noura et al. (+1 more)
Current state-of-the-art approaches to wildfire risk assessment often overlook operational needs, limiting their practical value for first responders and firefighting services. Effective wildfire management requires a multi-target analysis that captures the diverse dimensions of wildfire risk, including meteorological danger, ignition activity, intervention complexity, and resource mobilization, rather than relying on a single predictive indicator. In this proof of concept, we propose the development of a hybrid framework that combines predictive models for each risk dimension with large
language models (LLMs) to synthesize heterogeneous outputs into structured, actionable reports.
Read the full abstract ▾
All authors Nicolas Caron, Christophe Guyeux, Hassan Noura, Benjamin Aynes
Machine Learning
CoVar: Confidence-Variance-Guided Pseudo-Label Selection for Semi-Supervised Learning
Jinshi Liu, Lei He and Pan Liu
Pseudo-label selection in semi-supervised learning is commonly driven by maximum-confidence thresholds, yet confidence alone can be unreliable under model overconfidence and class imbalance. We propose CoVar, a confidence–variance framework that assesses pseudo-label reliability by jointly modeling Maximum Confidence (MC) and Residual-Class Variance (RCV). Starting from
entropy minimization, we derive a second-order cross-
entropy approximation showing that low-loss pseudo-labels are favored when MC is high and RCV is low, with a confidence-dependent penalty that becomes stronger for near-certain predictions. Based on this criterion, CoVar embeds predictions into a two-dimensional confidence–variance space and uses SVD-based spectral relaxation to separate reliable and unreliable predictions without hand-tuned confidence thresholds. Cluster-wise Gaussian weighting then converts this separation into per-sample training weights. The resulting weights can be integrated into existing semi-supervised
segmentation and classification pipelines during training and introduce no inference-time overhead. Experiments on PASCAL VOC 2012, Cityscapes, CIFAR-10, CIFAR-100, SVHN, and STL-10 show clear gains on VOC and Cityscapes under matched backbones, as well as competitive or improved error rates on standard classification
benchmarks. These results indicate that residual-class dispersion provides a useful signal complementary to confidence for robust pseudo-label selection.
Read the full abstract ▾
Machine Learning
IPEC: Test-Time Incremental Prototype Enhancement Classifier for Few-Shot Learning
Wenwen Liao, Hang Ruan, Jianbo Yu et al. (+3 more)
Metric-based few-shot approaches have gained significant popularity due to their relatively straightforward implementation, high interpret ability, and computational efficiency. However, stemming from the batch-independence assumption during testing, which prevents the model from leveraging valuable knowledge accumulated from previous batches. To address these challenges, we propose a novel test-time method called Incremental Prototype Enhancement Classifier (IPEC), a test-time method that optimizes prototype estimation by leveraging information from previous query samples. IPEC maintains a dynamic auxiliary set by selectively incorporating query samples that are classified with high confidence. To ensure sample quality, we design a robust dual-filtering mechanism that assesses each query sample based on both global prediction confidence and local discriminative ability. By aggregating this auxiliary set with the support set in subsequent tasks, IPEC builds progressively more stable and representative prototypes, effectively reducing its reliance on the initial support set. We ground this approach in a
Bayesian interpretation, conceptualizing the support set as a prior and the auxiliary set as a data-driven posterior, which in turn motivates the design of a practical "warm-up and test" two-stage inference protocol. Extensive empirical results validate the superior performance of our proposed method across multiple few-shot classification tasks.
Read the full abstract ▾
All authors Wenwen Liao, Hang Ruan, Jianbo Yu, Xiaofeng Yang, Qingchao Jiang, Xuefeng Yan
Machine Learning
Distill-then-Replace: Efficient Task-Specific Hybrid Attention Model Construction
Xiaojie Xia, Huigang Zhang, Chaoliang Zhong et al. (+2 more)
Transformer architectures deliver state-of-the-art accuracy via dense full-
attention, but their quadratic time and
memory complexity with respect to sequence length limits practical deployment. Linear
attention mechanisms offer linear or near-linear scaling yet often incur performance degradation. Hybrid models that integrate full and linear
attention layers promise a balance between efficiency and expressiveness, but face two major challenges: training such hybrid models from scratch is computationally expensive, and manually designing the optimal placement of
attention types is highly nontrivial. We propose DtR (Distill-then-Replace), which first transfers weights from the pretrained full-
attention modules to its linear
attention counterparts through blockwise local
distillation, and then applies a greedy layer replacement strategy that iteratively substitutes full
attention blocks with linear ones while monitoring validation performance on the target task. DtR yields a task-specific hybrid model in a single efficient pass, without costly re-training or neural architecture search, and can be applied to any pretrained full-
attention backbone for diverse downstream tasks.
Read the full abstract ▾
All authors Xiaojie Xia, Huigang Zhang, Chaoliang Zhong, Jun Sun, Yusuke Oishi
Machine Learning
Building Production-Ready Probes For Gemini
János Kramár, Joshua Engels, Zheng Wang et al. (+4 more)
Frontier
language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful systems. Prior work has shown that activation probes may be a promising misuse mitigation technique, but we identify a key remaining challenge: probes fail to generalize under important production distribution shifts. In particular, we find that the shift from short-context to long-context inputs is difficult for existing probe architectures. We propose several new probe architectures that handle this long-context distribution shift. We evaluate these probes in the cyber-offensive domain, testing their robustness against various production-relevant distribution shifts, including multi-turn conversations, long context
prompts, and adaptive red teaming. Our results demonstrate that while our novel architectures address context length, a combination of architecture choice and training on diverse distributions is required for broad generalization. Additionally, we show that pairing probes with prompted classifiers achieves optimal accuracy at a low cost due to the computational efficiency of probes. These findings have informed the successful deployment of misuse mitigation probes in user-facing instances of Gemini, Google's frontier
language model. Finally, we find early positive results using AlphaEvolve to automate improvements in both probe architecture search and adaptive red teaming, showing that automating some AI safety research is already possible.
Read the full abstract ▾
All authors János Kramár, Joshua Engels, Zheng Wang, Bilal Chughtai, Rohin Shah, Neel Nanda, Arthur Conmy
Machine Learning · Cross-listed from cs.CV
ShapeR: Robust Conditional 3D Shape Generation from Casual Captures
Yawar Siddiqui, Duncan Frost, Samir Aroudj et al. (+9 more)
Recent advances in 3D shape generation have achieved impressive results, but most existing methods rely on clean, unoccluded, and well-segmented inputs. Such conditions are rarely met in real-world scenarios. We present ShapeR, a novel approach for conditional 3D object shape generation from casually captured sequences. Given an image sequence, we leverage off-the-shelf visual-inertial SLAM, 3D detection
algorithms, and
vision-
language models to extract, for each object, a set of sparse SLAM points, posed multi-view images, and machine-generated captions. A rectified flow
transformer trained to effectively condition on these modalities then generates high-fidelity metric 3D shapes. To ensure robustness to the challenges of casually captured data, we employ a range of techniques including on-the-fly compositional augmentations, a curriculum training scheme spanning object- and scene-level
datasets, and strategies to handle background clutter. Additionally, we introduce a new evaluation
benchmark comprising 178 in-the-wild objects across 7 real-world scenes with geometry annotations. Experiments show that ShapeR significantly outperforms existing approaches in this challenging setting, achieving an improvement of 2.7x in Chamfer distance compared to state of the art.
Read the full abstract ▾
All authors Yawar Siddiqui, Duncan Frost, Samir Aroudj, Armen Avetisyan, Henry Howard-Jenkins, Daniel DeTone, Pierre Moulon, Qirui Wu, Zhengqin Li, Julian Straub, Richard Newcombe, Jakob Engel
Machine Learning
MetaboNet: The Largest Publicly Available Consolidated Dataset for Type 1 Diabetes Management
Miriam K. Wolff, Peter Calhoun, Eleonora Maria Aiello et al. (+2 more)
Progress in Type 1 Diabetes (T1D)
algorithm development is limited by the fragmentation and lack of standardization across existing T1D management
datasets. Current
datasets differ substantially in structure and are time-consuming to access and process, which impedes data integration and reduces the comparability and generalizability of algorithmic developments. This work aims to establish a unified and accessible data resource for T1D
algorithm development. Multiple publicly available T1D
datasets were consolidated into a unified resource, termed the MetaboNet
dataset. Inclusion required the availability of both continuous glucose monitoring (CGM) data and corresponding insulin pump dosing records. Additionally, auxiliary information such as reported carbohydrate intake and physical activity was retained when present. The MetaboNet
dataset comprises 3135 subjects and 1228 patient-years of overlapping CGM and insulin data, making it substantially larger than existing standalone
benchmark datasets. The resource is distributed as a fully public subset available for immediate download at https://metabo-net.org/ , and with a Data Use Agreement (DUA)-restricted subset accessible through their respective application processes. For the
datasets in the latter subset, processing pipelines are provided to automatically convert the data into the standardized MetaboNet format. A consolidated public
dataset for T1D research is presented, and the access pathways for both its unrestricted and DUA-governed components are described. The resulting
dataset covers a broad range of glycemic profiles and demographics and thus can yield more generalizable algorithmic performance than individual
datasets.
Read the full abstract ▾
All authors Miriam K. Wolff, Peter Calhoun, Eleonora Maria Aiello, Yao Qin, Sam F. Royston
Machine Learning
QUPID: A Partitioned Quantum Neural Network for Anomaly Detection in Smart Grid
Hoang M. Ngo, Tre' R. Jeter, Jung Taek Seo et al. (+1 more)
Smart grid infrastructures have revolutionized energy distribution, but their day-to-day operations require robust anomaly detection methods to counter risks associated with cyber-physical threats and system faults potentially caused by natural disasters, equipment malfunctions, and cyber attacks. Conventional machine learning (ML) models are effective in several domains, yet they struggle to represent the complexities observed in smart grid systems. Furthermore, traditional ML models are highly susceptible to
adversarial manipulations, making them increasingly unreliable for real-world deployment.
Quantum ML (QML) provides a unique advantage, utilizing
quantum-enhanced feature representations to model the intricacies of the high-dimensional nature of smart grid systems while demonstrating greater resilience to
adversarial manipulation. In this work, we propose QUPID, a partitioned
quantum neural network (PQNN) that outperforms traditional state-of-the-art ML models in anomaly detection. We extend our model to R-QUPID that even maintains its performance when including differential
privacy (DP) for enhanced robustness. Moreover, our partitioning framework addresses a significant scalability problem in QML by efficiently distributing computational workloads, making
quantum-enhanced anomaly detection practical in large-scale smart grid environments. Our experimental results across various scenarios exemplifies the efficacy of QUPID and R-QUPID to significantly improve anomaly detection capabilities and robustness compared to traditional ML approaches.
Read the full abstract ▾
All authors Hoang M. Ngo, Tre' R. Jeter, Jung Taek Seo, My T. Thai
Machine Learning · Cross-listed from cs.NE
On the Probability of First Success in Differential Evolution: Hazard Identities and Tail Bounds
Dimitar Nedanovski, Svetoslav Nenov and Dimitar Pilev
We study first-hitting times in Differential
Evolution (DE) through a conditional hazard frame work. Instead of analyzing convergence via Markov-chain transition kernels or drift arguments, we ex press the survival probability of a measurable target set $A$ as a product of conditional first-hit probabilities (hazards) $p_t=\Prob(E_t\mid\mathcal F_{t-1})$. This yields distribution-free identities for survival and explicit tail bounds whenever deterministic lower bounds on the hazard hold on the survival event. For the L-SHADE
algorithm with current-to-$p$best/1 mutation, we construct a checkable algorithmic witness event $\mathcal L_t$ under which the conditional hazard admits an explicit lower bound depending only on sampling rules, population size, and crossover statistics. This separates theoretical constants from empirical event frequencies and explains why worst-case constant-hazard bounds are typically conservative. We complement the theory with a Kaplan–Meier survival analysis on the CEC2017
benchmark suite . Across functions and budgets, we identify three distinct empirical regimes: (i) strongly clustered success, where hitting times concentrate in short bursts; (ii) approximately
geometric tails, where a constant-hazard model is accurate; and (iii) intractable cases with no observed hits within the evaluation horizon. The results show that while constant-hazard bounds provide valid tail envelopes, the practical behavior of L-SHADE is governed by burst-like transitions rather than homogeneous per-generati on success probabilities.
Read the full abstract ▾
Machine Learning
Extractive summarization on a CMOS Ising machine
Ziqing Zeng, Abhimanyu Kumar, Ahmet Efe et al. (+4 more)
Extractive summarization (ES) aims to generate a concise summary by selecting a subset of sentences from a document while maximizing relevance and minimizing redundancy. Although modern ES systems achieve high accuracy using powerful neural models, their deployment typically relies on CPU or GPU infrastructures that are energy-intensive and poorly suited for real-time inference in resource-constrained environments. In this work, we explore the feasibility of implementing McDonald-style extractive summarization on a low-power CMOS coupled oscillator-based Ising machine (COBI) that supports integer-valued, all-to-all
spin couplings. We first propose a hardware-aware Ising formulation that reduces the scale imbalance between local fields and coupling terms, thereby improving robustness to coefficient quantization: this method can be applied to any problem formulation that requires k of n variables to be chosen. We then develop a complete ES pipeline including (i)
stochastic rounding and iterative refinement to compensate for precision loss, and (ii) a decomposition strategy that partitions a large ES problem into smaller Ising subproblems that can be efficiently solved on COBI and later combined. Experimental results on the CNN/DailyMail
dataset show that our pipeline can produce high-quality summaries using only integer-coupled Ising hardware with limited precision. COBI achieves 3-4.5x runtime speedups compared to a brute-force method, which is comparable to software Tabu search, and two to three orders of magnitude reductions in energy, while maintaining competitive summary quality. These results highlight the potential of deploying CMOS Ising solvers for real-time, low-energy text summarization on edge devices.
Read the full abstract ▾
All authors Ziqing Zeng, Abhimanyu Kumar, Ahmet Efe, Ruihong Yin, Chris H. Kim, Ulya R. Karpuzcu, Sachin S. Sapatnekar
Machine Learning · Cross-listed from math.OC
A Probabilistic Approach to Trajectory-Based Optimal Experimental Design
Ahmed Attia
We present a novel probabilistic approach for optimal experimental path design. In this approach a discrete path optimization problem is defined on a static
navigation mesh, and trajectories are modeled as random variables governed by a parametric Markov policy. The discrete path optimization problem is then replaced with an equivalent
stochastic optimization problem over the policy parameters, resulting in an optimal probability model that samples estimates of the optimal discrete path. This approach enables exploration of the utility function's distribution tail and treats the utility function of the design as a black box, making it applicable to linear and nonlinear inverse problems and beyond experimental design. Numerical verification and analysis are carried out by using a parameter identification problem widely used in model-based optimal experimental design, namely a two-dimensional time-dependent advection
diffusion problem in which the initial condition is the inference target. Experiments use both coarse and fine
navigation meshes, with either a single moving sensor or a group of seven coordinated sensors, and the proposed approach is evaluated under D-, A-, and E-optimality criteria.
Read the full abstract ▾
Machine Learning
Low-Rank Key Value Attention
James O'Neill, Robert Clancy, Mariia Matskevichus et al. (+1 more)
The key-value (KV) cache is a primary
memory bottleneck in
Transformers. We propose Low-Rank Key-Value (LRKV)
attention, which reduces KV cache
memory by exploiting redundancy across
attention heads, while being compute efficient. Each layer uses a shared full-rank KV projection augmented with low-rank, head-specific residuals, providing a continuous trade-off between complete sharing and full independence. After pretraining models of size 128M to 6.3B parameters, LRKV consistently achieves the lowest test loss among standard MHA, MQA/GQA, and MLA while using only 45-53% of MHA's KV cache. LRKV reaches equivalent baseline quality 18-25% faster (measured in training steps). After supervised midtraining, LRKV achieves the highest downstream task performance across ARC-Easy, ARC-Challenge, MMLU, GSM8K, and HumanEval
benchmarks.
Read the full abstract ▾
All authors James O'Neill, Robert Clancy, Mariia Matskevichus, Fergal Reid