Machine Learning
Induction Heads Interpolate N-Grams
Francesco D'Angelo, Oguz Kaan Yuksel, Swathi Shree Narashiman et al. (+1 more)
Induction heads are
attention circuits believed to underlie in-context learning in
transformers, yet a precise characterization of the
estimators they implement remains elusive. We study
transformers trained on order-$k$ Markov chains and identify two complementary smoothing mechanisms. First, at finite
attention-weight scale, the
circuit implements a soft context-matching
estimator: it aggregates contributions from exact and partial context matches, weighted exponentially by their overlap, and induces a data-dependent interpolation across context orders analogous to Jelinek-Mercer smoothing. Second, a beginning-of-sequence (BOS)
token induces additive pseudo-counts, recovering Dirichlet-style smoothing. We construct a disentangled
transformer implementing both mechanisms and show that trained
transformers recover the predicted
attention patterns. Across settings where pseudo-count smoothing is optimal or lower-order contexts provide structured evidence, trained
transformers match or outperform classical count-based baselines. Our results bridge mechanistic interpretability of induction heads with classical statistical smoothing, revealing that
transformers learn to regularize in-context estimation rather than simply count.
Read the full abstract ▾
All authors Francesco D'Angelo, Oguz Kaan Yuksel, Swathi Shree Narashiman, Nicolas Flammarion
Machine Learning
Safe Inference-Time Alignment via Lagrangian Reward Augmentation
Yaswanth Chittepu, Ativ Joshi, Sohini Chintala et al. (+1 more)
Inference-time
alignment steers a frozen
language model during decoding using auxiliary reward signals, avoiding the cost of repeated weight updates. However, existing inference-time
alignment methods typically optimize a single scalar score, so explicit safety constraints must either be ignored or encoded through manually tuned penalties. We propose Lagrangian Reward Augmentation (LARA), a general inference-time
alignment framework under safety constraints. Starting from a KL-regularized constrained objective with a reward model and a cost model, LARA dualizes the constraint and reduces the optimization problem to a one-dimensional convex problem over a nonnegative dual variable. Estimated on a small calibration set, this dual variable defines an augmented reward that can be used as a drop-in scoring signal within existing inference-time
alignment methods. For sequence-level sampling methods, such as Best-of-N reranking, the calibrated dual variable corresponds to the solution of the expected-cost constrained problem. For
token-level reward-guided decoding methods, the same construction yields a principled dual-calibrated heuristic rather than an exact constrained-policy guarantee. We evaluate LARA on both sequence-level and
token-level inference-time
alignment methods, and find that LARA improves the helpfulness-harmlessness tradeoff, with Best-of-N achieving the best performance among inference-time methods, approaching finetuning-based direct
alignment baselines.
Read the full abstract ▾
All authors Yaswanth Chittepu, Ativ Joshi, Sohini Chintala, Scott Niekum
Machine Learning · Cross-listed from eess.IV
Pretreatment MRI reveals a latent, molecular-subtype-independent structural phenotype that organizes treatment trajectories and recurrence risk
Dattatreya Kantha and Murray H. Loew
Pathologic complete response and tumor shrinkage measure whether breast cancer responds to neoadjuvant therapy, but not whether that response was structurally favorable, persistent, or hidden beneath volume loss. We built an outcome-blind longitudinal DCE-MRI
manifold from I-SPY2 trajectories to test whether pretreatment imaging carries a structural response phenotype missed by conventional descriptors. The dominant axis of response geometry was not recoverable from the full clinical and genomic stack – age, receptor subtype, MammaPrint, PAM50, treatment arm, and tumor burden – but became strongly recoverable once baseline structural
entropy was added. A constrained representation mapping recovered the same axes as unconstrained decomposition, establishing the structure as intrinsic rather than a post-hoc interpretation. The phenotype persisted through therapy, and as treatment proceeded the volumetric signal faded while
entropy stayed separated – a crossover from burden to structural persistence. Among complete responders, structurally disordered tumors could shrink more early yet remain structurally disordered, a volumetric deception invisible to endpoint labels. External analyses in UCSF, I-SPY1, and Duke established recurrence relevance under representation-dependent boundaries, and a representation-family commensurability assessment showed why feature-name matching is insufficient: the same label can fail, transport, or entangle with extraction geometry. Pretreatment MRI therefore exposes a structural response phenotype that endpoint-based
language leaves invisible – including, among complete responders, a pretreatment imaging signal of structurally distinct response states that awaits prospective validation.
Read the full abstract ▾
Machine Learning
Out-of-Distribution Generalization of Risk Aversion in Language Models
Kristina Zhang, Junior Chinomso Okoroafor, Benjamin Maltbie et al. (+3 more)
Training AIs to be risk-averse in resources could offer a failsafe in the event that AIs turn out misaligned. Misaligned but risk-averse AIs would tend to prefer low-risk, low-reward strategies like cooperation over high-risk, high-reward strategies like rebellion, limiting the downsides of any misalignment. But we can only feasibly train AIs to be risk-averse on low-stakes gambles, and we will only be safe if their risk aversion generalizes to astronomically-high-stakes gambles. Will it? To shed light on this question, we introduce RiskAverseOOD: a
benchmark for measuring how well risk aversion generalizes out of distribution. We then offer some initial results. Using a variety of methods to make Qwen3-8B choose risk-aversely when the stakes are low, we find that we can induce substantial risk aversion when the stakes are astronomically high. From a baseline 2% rate of choosing a safe `Cooperate' option, we see rates around 70% (SFT and tie training) and 52% (DPO). Activation steering scores 78% but hurts capabilities and makes the model excessively risk-averse. We observe similar effects at different scales (Qwen3-1.7B and Qwen3-14B) and across model families (Gemma-3-12B-IT and Llama-3.1-8B-Instruct). Overall, we find that risk aversion learned at low stakes can generalize OOD to astronomically high stakes, though not yet consistently enough to serve as a reliable failsafe. Achieving that level of consistency is an open problem.
Read the full abstract ▾
All authors Kristina Zhang, Junior Chinomso Okoroafor, Benjamin Maltbie, Andrew Lin, Abhitej Bokka, Elliott Thornley
Machine Learning · Cross-listed from physics.comp-ph
CodeJeNN: A simple C++ neural network generator for physics applications
Jay Arcities, Pavel Popov, Eric J Ching et al. (+2 more)
Machine learning has shown promise for physics applications, but integrating Python-based libraries into high-performance C++ solvers can create performance bottlenecks. We present CodeJeNN, which addresses this gap by auto-generating self-contained C++ code from trained Keras models for inference. This eliminates external dependencies through inlined functions, allowing seamless integration into existing frameworks. We describe the Keras-to-C++ workflow, supported architectures, and limitations. CodeJeNN is demonstrated through inference
benchmarks against Keras (eager and optimized) and a computational fluid dynamics (CFD) simulation of a hydrogen-air mixing layer in which viscosity is modeled using a neural network, showing that the generated models integrate directly into a solver while remaining efficient and accurate.
Read the full abstract ▾
All authors Jay Arcities, Pavel Popov, Eric J Ching, Kamal Viswanath, Ryan F Johnson
Machine Learning · Cross-listed from math.OC
Optimality-Informed Neural Networks for Lunar Landing Trajectory Optimization
Zhenbo Wang
This paper develops an Optimality-Informed Neural Network (OINN) approach for the energy-optimal, free-final-time powered descent of a lunar lander from any initial position, velocity, and mass within a bounded operating envelope to a fixed landing site with zero terminal velocity. Building on a recent framework that jointly embeds Pontryagin's minimum principle and the Hamilton-Jacobi-Bellman equation for general nonlinear optimal control, the proposed OINN approach specializes that idea to a lunar landing problem with free time of flight and fixed terminal state. Every boundary and transversality condition is hard-encoded into the network architecture by construction, the closed-form Pontryagin-optimal thrust magnitude and direction law is substituted directly rather than learned, and the remaining state, costate, and an auxiliary value-function output are trained against a physics-residual loss formed entirely from the necessary conditions of optimality, with no precomputed optimal trajectories required. A preliminary theoretical analysis is explored, establishing a
stochastic-optimization stationarity guarantee for the offline training procedure, an explicit bound translating the achieved training residual into bounds on touchdown position, touchdown velocity, and flight-time error, and a fixed, input-independent onboard computational and
memory cost suitable for real-time deployment. Numerical simulations evaluate the trained policy, with no retraining, against an independently solved indirect-method boundary-value problem at six representative initial states spanning the operating envelope and against eighty additional Monte Carlo simulation runs, demonstrating close agreement with the indirect-method solution and consistently small dynamics and transversality residuals throughout the envelope.
Read the full abstract ▾
Machine Learning
Weighted Conformal Prediction for Lab-to-Track Thermal Transfer in EV Motorsport Powertrains
Varshith Roy Kotla
Predicting thermal
volatility in high-performance EV powertrains is difficult as internal temperatures are rarely observable outside the lab, and models calibrated on lab drive cycles fail when deployed against real-world loads. We study this lab-to-track transfer problem using conformal prediction, offering distribution-free uncertainty bounds. We implement Ensemble Batch Prediction Intervals (EnbPI; Xu & Xie, 2021), a leave-one-out bootstrap-ensemble conformal method for autocorrelated time series, and calibrate it on real CALCE lithium-ion cycler data (A123 SP20
cells, FUDS profile). We evaluate it under a genuine, measured
covariate shift: a second real CALCE test condition (US06 Highway Driving Schedule at 45{\deg}C). The unweighted EnbPI bound, achieving its nominal 95% coverage in-distribution (measured: 95.00%), degrades to 70.13% empirical coverage under this real shift. We introduce a weighted EnbPI procedure combining EnbPI's ensemble residuals with density-ratio weighting (Tibshirani et al., 2019), estimating the density ratio via a probabilistic domain classifier. This recovers coverage to 72.42%, a modest, honestly-reported improvement, not a complete fix. We additionally apply the calibrated model to real 2023 Formula 1 telemetry (Monza and Silverstone, driver VER) as an unsupervised out-of-distribution diagnostic. Because no internal thermal channel exists in public trackside telemetry, we report only unsupervised flag rates (65.6% at Monza, 58.0% at Silverstone, well above the 5% in-distribution base rate) and note inconsistent associations between flags and braking/DRS zones. We conclude that conformal domain adaptation is a promising but only partially solved tool for this problem, detailing exactly where it falls short.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Provable Pruning for Efficient 3D Gaussian Splatting via Coresets
Waseem Mousa and Alaa Maalouf
3D Gaussian Splatting (3DGS) enables high-quality real-time novel-view synthesis, but practical scenes often contain millions of Gaussians, making compression essential for deployment on limited hardware. Existing reduction methods are effective but mostly heuristic: they provide no multiplicative approximation guarantee for the rendered objective, and thus rely heavily on costly post-
pruning finetuning to recover quality. We ask a basic question: can a 3DGS scene be provably replaced by a much smaller weighted subset (coreset) while preserving the objective of interest? We first show that, in the unrestricted setting, no non-trivial multiplicative 3DGS coreset exists. We then show that multiplicative guarantees are not impossible, but resolution-dependent. For a prescribed rendering resolution, such as representative views or grids of views/rays, we provide the first weighted coreset construction
theorem for 3DGS. The construction samples Gaussians by sensitivity: provable importance scores measuring each Gaussian's role in the full-scene objective. Finally, under explicit validity and log-transmittance stability assumptions, we turn this objective guarantee into a rendering guarantee. Empirically, our method is strongest where deployment needs it most: aggressive compression with no or minimal recovery compute. In prune-only and very short finetuning regimes, it achieves state-of-the-art performance, showing that principled importance estimation can be both theoretically meaningful and practically useful. Open-source code is available at https://github.com/waseem-m/3dgs_provable_coresets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models
Stanislav Panev, Minhyek Jeon, Vaishnavi Khindkar et al. (+5 more)
Recent advances in large-scale image generative models enable photorealistic scene synthesis with controllable attributes. Beyond data augmentation, their potential as diagnostic tools for trained
vision systems remains unexplored in the aerial and remote sensing domains. We introduce a synthetic diagnostic framework for aerial-view vehicle detection that combines text-guided generation, attribute-controlled editing, and automated attribute verification to construct a controllable synthetic testbed. This enables fine-grained evaluation of pretrained detectors under diverse scene types and environmental conditions that are difficult to isolate in real
datasets. Across three detection architectures and three real aerial
datasets, synthetic scene-wise performance trends closely match real-world weaknesses. Guided by these diagnostics, targeted supplementation with small real
datasets from the identified weak categories yields improvements of up to 13% AP50 while requiring substantially fewer additional samples than non-targeted augmentation. Our results show that controlled synthetic probing can predict real-domain performance gaps and guide efficient data collection. The proposed diagnostic framework is modular and can incorporate alternative generative or
vision-
language models as capabilities evolve. Our code and
datasets are available here: https://humansensinglab.github.io/AVODDiag/
Read the full abstract ▾
All authors Stanislav Panev, Minhyek Jeon, Vaishnavi Khindkar, Ahish Deshpande, Celso M de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre
Machine Learning
LiNO: Lifting based multiresolution neural operator
Himanshu Pandey, Subham Patel and Ratikanta Behera
Recently, neural operators have shown promising outcomes for learning solution operators of differential equations directly from data. This framework learns a functional mapping from the parameter field to the solution field, enabling the prediction of an entire class of solutions rather than a specific instance. However, existing operators often struggle to capture both global dynamics and fine-scale structure simultaneously. To design an effective operator capable of representing multiscale features, a hierarchical multiscale decomposition framework is required. In this study, we develop the Lifting Neural Operator (LiNO), a multiresolution operator built on the second-generation wavelet lifting scheme. LiNO learns a multiresolution decomposition directly from data by parameterizing the lifting transform. This lifting transformation is adaptive to the underlying solution function and exactly invertible by construction, enabling information-preserving multiscale operator learning. In the lifted multiresolution space, the operator evolves coarse and directional detail coefficients separately, resulting in scale-aware modeling of the underlying physics. We evaluate LiNO on several
benchmarks, including Darcy flow, the Poisson equation, the Allen-Cahn equation, the compressible Navier-Stokes equation, and the Gray-Scott reaction-
diffusion system. Together, these
benchmarks cover a wide range of physical behaviors, including multiscale phenomena, transport-dominated dynamics, and chaotic systems. LiNO demonstrates strong performance on these challenging
benchmarks compared with state-of-the-art neural operators. These results suggest that adaptive multiresolution operators provide a promising direction for scientific machine learning.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Property-Constrained 3D Porous Media Reconstruction from 2D Images via Conditional Generative Adversarial Networks
Ali Sadeghkhani, Brandon Bennett and Arash Rabbani
This study presents a conditional Generative
Adversarial Network (cGAN) framework for generating 3D porous media volumes with controlled porosity, trained exclusively on 2D thin section images. The key innovation lies in combining property-conditioned generation with 2D-to-3D reconstruction, eliminating the need for expensive 3D training data while maintaining control over petrophysical properties. The framework employs a hybrid architecture with a 3D generator and 2D discriminator, where multi-axis slice extraction enables learning 3D-consistent structures from 2D training data. Porosity labels are extracted using an Enhanced U-Net
segmentation model. The methodology was demonstrated on two carbonate samples with different lithologies: dolomite-anhydrite and pure dolomite. Results show that the framework successfully generates realistic 3D volumes capturing lithological features such as anhydrite inclusions and fine crystalline textures. Porosity control achieved an $R^2$ of 0.93, with mean absolute errors of 0.019 and 0.010 for the heterogeneous and homogeneous samples, respectively.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
RES-DARE: Failure-Aware Expert Adaptation and Rollback-Safe Self-Repair for Intrusion Detection
Rahil Aftab, Anyash Prasad, Soumya Mazumdar et al. (+2 more)
Intrusion detection systems are often trained under static
benchmark conditions, although deployed network environments are affected by traffic drift, sensor noise, changing workloads, and evolving attack behaviour. Under such distribution shifts, static detectors may produce confident but incorrect predictions, leading to silent and unsafe failure modes. In this paper, RES-DARE (Recursive Evolving Specialists-Digital Adaptive
Reasoning Engine) is proposed as a failure-aware continual intrusion detection framework with rollback-safe self-repair. Difficult, uncertain, and misclassified samples are treated as failure signals for expert specialisation rather than being discarded as noise. A supervised contrastive encoder, two-pass expert router, failure-buffer mechanism, HDBSCAN-based failure-region discovery, and trust-risk monitor are integrated to support adaptive IDS behaviour. AEHM-v2 is introduced as a rollback-safe repair mechanism, where candidate adaptations are provisionally activated and committed only when macro-F1 is preserved or improved while trust risk remains stable. Otherwise, the system is rolled back to its last validated state. RES-DARE is evaluated on CICIDS2017, UNSW-NB15, and TON\_IoT, achieving macro-F1 scores of 0.9850, 0.9736, and 0.9691, respectively. Under Gaussian feature corruption at strength 0.10, RES-DARE retains an Attack-F1 of 0.7920 on CICIDS2017 and achieves near-zero catastrophic forgetting with F = 0.0015. The results show that RES-DARE improves robustness, warning capability, and deployment safety under degraded conditions.
Read the full abstract ▾
All authors Rahil Aftab, Anyash Prasad, Soumya Mazumdar, Vineet Kumar Rakesh, Tapas Samanta
Machine Learning · Cross-listed from cs.AI
ASK in the Dark: Uncertainty-Gated LLM Assistance under Partial Observability
Juarez Monteiro, Nathan Gavenski, Guilherme Lima et al. (+3 more)
Reinforcement learning
agents operating under partial observability must act on incomplete information, making them natural candidates for guidance from small
language models (SLMs) that carry broad
reasoning priors. Yet integrating SLM guidance into this setting has proven difficult: across all test environments, vanilla uncertainty-gated approaches achieve an overwrite rate at or near zero, meaning the SLM almost never contributes an independent action. We trace this failure to the bare
egocentric prompt, which provides insufficient context for genuine
reasoning, and identify it as a context problem rather than a capacity problem. We propose ASK+, which supplies the SLM with trajectory-aware context (a partially revealed map, visited positions, and action history) and structured chain-of-thought
reasoning, converting it from a passive redundancy check into a more informative consultant that occasionally corrects the policy. We further establish that the predictive
entropy signal used for selective querying measures action uncertainty rather than state uncertainty and remains informative in POMDPs, making uncertainty-gated assistance viable beyond fully observable settings. The stateful
prompt drives substantial gains: on DoorKey, where vanilla ASK matches PPO (both 89%), ASK+ reaches 93% success; on FourRooms, success climbs from 53% to 70%; on HigherLower, accuracy reaches 73.7%, matching the SLM-only upper bound. Across all environments, Qwen3.5-2B matches or exceeds Qwen3.5-4B, confirming that
prompt design and selective gating dominate the impact of model scale, enabling guidance without large models.
Read the full abstract ▾
All authors Juarez Monteiro, Nathan Gavenski, Guilherme Lima, Francisco Galuppo, Odinaldo Rodrigues, Adriano Veloso
Machine Learning · Cross-listed from stat.ML
Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms
Ye Tian, Mengchu Li and Marco Avella Medina
Integrating information across related tasks can improve estimation and prediction in transfer, multi-task, and
federated learning, but contamination and heterogeneity make robust borrowing challenging. We study a contaminated multi-task empirical risk minimization (ERM) framework in which an $\epsilon$ fraction of $K$ tasks, each with sample size $n$, may be arbitrarily contaminated while the remaining tasks are heterogeneous. Our goal is to estimate both the global minimizer of the average risk and the clean task-specific minimizers, thereby combining robustness and personalization. In the Gaussian mean model, we show that several common paradigms, including adaptive and robust regularization around a shared center, global
matrix regularization, decomposition-based regularization, and score-based outlier-task detection, all suffer from a worst-case contamination error of order $\epsilon\sqrt{d/n}$, which is suboptimal compared to the lower bound $\epsilon/\sqrt{n}$. This identifies a dimension-dependent barrier for these approaches. We then establish minimax lower bounds for a general heterogeneous ERM setting and propose a computationally efficient filtering-based robust multi-task gradient descent method. Under local strong convexity, smoothness, and sub-Gaussian gradient assumptions, the proposed method attains high-probability upper bounds matching the minimax rates up to logarithmic factors over a broad regime. In particular, it removes the extra $\sqrt{d}$ contamination dependence of many regularization-based methods and score-based outlier detection, while achieving personalization to local tasks under strong heterogeneity. Simulations and a real-data analysis demonstrate strong robustness and personalization relative to a broad range of
benchmark methods.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Benign Overfitting Does Not Occur in Diffusion Models
Tyler Farghly, Benjamin Dupuis, Alain Durmus et al. (+1 more)
Benign overfitting and double descent have come to shape our understanding of generalization in deep learning, establishing that overfitting is not only compatible with good generalization but can actively benefit it.
Diffusion models share much of the machinery of standard deep learning, so it is natural to assume that they also exhibit these properties. In this work, we show that this assumption is largely incorrect. We first establish fundamental impossibility results showing that, unless the sample size grows exponentially with the data dimension, overfitting and good generalization cannot occur simultaneously. Consequently, the population loss follows a classical U-shaped curve in model complexity rather than exhibiting double descent. Analyzing a simplified setting, we identify a key difference between regression and score matching: regression benefits from an
alignment between the target and the empirical covariance; score matching admits no such
alignment, leaving overfitting irreparably harmful. We further identify implicit regularization stemming from time-smoothness of the score and early stopping during training as mechanisms that prevent such overfitting and verify our findings with high-dimensional image generation experiments. Our results reveal that generalization in
diffusion models is governed by mechanisms distinct from those of traditional regression, motivating the development of new theory.
Read the full abstract ▾
All authors Tyler Farghly, Benjamin Dupuis, Alain Durmus, Umut Simsekli
Machine Learning
A Granularity-Aware EEG Feature Framework for Psychopathology Dimension Prediction
Haofan Cheng, Jingjing Hu, Jingrong Pei et al. (+6 more)
Electroencephalography (
EEG) offers a noninvasive approach for examining neurophysiological correlates of dimensional psychopathology, yet systematic evidence across
EEG paradigms and feature granularities remains limited. Here, we develop a granularity-aware
EEG feature pipeline that organizes multi-scale descriptors into global, regional, and channel levels. Using the Healthy
Brain Network (HBN) cohort, we evaluate the prediction of four psychopathology dimensions: p-factor, internalizing, externalizing, and
attention problems, across four
EEG paradigms. Given the heterogeneity of pediatric psychopathology and the moderate reliability of questionnaire-derived scores, this setting represents a challenging feasibility test rather than a clinical screening scenario. Tree-based models and granularity-balanced feature selection showed promising improvements over conventional approaches in selected conditions, although effect sizes remained modest. Visualization of selected markers revealed dimension-specific spatial and spectral patterns that were broadly aligned with existing neurophysiological knowledge. An exploratory cross-
dataset sanity check on the independent PEARL cohort suggested that the proposed selection principle remains technically feasible under protocol shifts, without claiming cross-
dataset generalizability. Overall, multi-scale
EEG features contain weak but detectable signals related to dimensional psychopathology, and granularity-aware selection may serve as a useful feature-reduction strategy for future
EEG-based phenotyping studies.
Read the full abstract ▾
All authors Haofan Cheng, Jingjing Hu, Jingrong Pei, Shuaiqi Fu, Meilun Shen, Shuai Fang, Meng Wang, Dan Guo, Jie Zhang
Machine Learning
Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splitting
Disheng Liu, Tuo Liang, Chaoda Song et al. (+1 more)
Recent generative models can produce high-quality synthetic images, offering scalable training training data for data-hungry models. Existing approaches to exploiting this potential typically involve 1) training or fine-tuning generators, or 2) using lightweight post-hoc adaptation like
prompt engineering or inference-time guidance, making them generator-specific and expertise-intensive. We study a complementary question: given a fixed pool of generated images, can downstream utility be improved purely by selecting an informative subset? The answer is yes. We show that effective selection must counter a structural bias of modern generators: they tend to over-produce canonical modes of each class while underrepresenting intra-class variation. Building on this insight, we split each real class into a canonical Homogeneous (HO) subset and a non-redundant Heterogeneous (HE) subset, then score synthetic images by a fidelity-diversity criterion that rewards semantic
alignment while penalizing canonical redundancy. The method is generator-agnostic and requires no retraining. Across multiple
benchmarks, it consistently outperforms state-of-the-art data selection baselines and matches the real-data performance with up to 40% fewer synthetic samples. The same criterion remains effective when applied on top of stronger task-tuned generators, with gains on both classification and
segmentation tasks. Post-generation selection is therefore not a substitute for better generators, but a complementary mechanism for improving the utility of synthetic data.
Read the full abstract ▾
All authors Disheng Liu, Tuo Liang, Chaoda Song, Yu Yin
Machine Learning
Federated Learning for Object Detection: Enabling Collaborative Drone Learning Without Centralizing Data
Daniel M. Jimenez-Gutierrez, Enrique Zuazua, Georgios Kellaris et al. (+3 more)
Object detection is a fundamental capability for AI-driven perception in safety-critical drone and edge-
vision systems, including disaster response, operational security environments, infrastructure monitoring and defense applications. Robust model performance in such environments depends on large, continuously updated
datasets. However, training high-performing detectors typically requires centralizing aerial imagery, which raises
privacy, regulatory, storage, and bandwidth challenges. This is especially problematic in distributed drone deployments, where visual data is generated onboard and is often impractical or undesirable to transfer to a centralized infrastructure. In this work, we apply
Federated Learning (FL) for object detection, enabling drones to improve a shared model while keeping image data local and private. We implement a
federated object detection pipeline using the Sherpa.ai FL platform on the KIIT-MiTA
dataset, and compare it with Single-drone and Centralized baselines using mean Average Precision (mAP) at IoU thresholds of 0.50 and 0.50-0.95. In our experiments, the proposed FL approach remains close to Centralized training while dramatically improving over Single-drone training, with the best lightweight model (YOLO26 nano), suitable for deployment even on very limited edge infrastructure, achieving relative gains of 52.89% and 67.80% in mAP@0.50 and mAP@0.50:0.95, respectively. These results show that FL enables scalable, high-performing, and
privacy-preserving object detection across distributed drone fleets without data centralization.
Read the full abstract ▾
All authors Daniel M. Jimenez-Gutierrez, Enrique Zuazua, Georgios Kellaris, Joaquin del Rio, Oleksii Sliusarenko, Xabi Uribe-Etxebarria
Machine Learning
GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speech
Antonis Asonitis, Francesco Verdini, Aref Farhadipour et al. (+4 more)
We present GRAFT, a per-word pronunciation conditioning mechanism for text-to-
speech neural codec
language modeling. Existing systems reach high intelligibility and naturalness but inherit the ambiguity of text and mispronounce rare proper nouns, loanwords and technical terms. Even phoneme-conditioned models offer no direct acoustic handle for per-word pronunciation. GRAFT controls the pronunciation of a chosen word from a short spoken sample of it, encoded with the model's own
speech tokenizer and bound to the word's position in the
prompt. Voice conversion during training-data construction disentangles the hint speaker from the target speaker, so the hint may come from any voice while the output stays in the target voice. In a blind English listening study, human raters rank GRAFT first by a clear margin, judging its rendering of the difficult word closest to a reference recording of that word. On a five-
language objective
benchmark, GRAFT reduces target-word phoneme error rate by 22-39% over the identical text-only backbone and outperforms competitive open-source zero-shot systems, both phoneme- and text-conditioned, on target-word pronunciation, while preserving speaker similarity and naturalness.
Read the full abstract ▾
All authors Antonis Asonitis, Francesco Verdini, Aref Farhadipour, Vijeta Avijeet, Pierre-Edouard Honnet, Marzieh Razavi, Juan Pablo Zuluaga Gomez
Machine Learning
QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
Shah Nawaz Haider, Steve Austin, Arnab Barua et al. (+2 more)
Time-series
forecasting supports decisions in finance, en-ergy, transportation, public health, and industrial monitoring. Recent foundation models improve transfer across forecast-ing tasks, but many depend on centralized data and Trans-former
attention, which restricts their use for long, high-di-mensional, and
privacy-sensitive signals. This paper presents QuantFlow, a probabilistic
forecasting framework that com-bines inverted sequence
embedding, bidirectional Mamba state-space decoders,
quantile regression, and
federated learning. Each variable is embedded over the complete ob-servation window, processed in forward and reverse direc-tions, and projected to five conditional
quantiles. TSMixup expands temporal diversity through Dirichlet-weighted inter-polation while preserving sequence structure. Experiments cover cryptocurrency, traffic, electricity, Electricity Trans-former Temperature, influenza, and weather data. QuantFlow obtains mean squared errors of 0.2834 on ETTm1 and 0.2218 on Weather, and a 20-client non-IID deployment retains use-ful accuracy after three communication rounds without cen-tralizing raw records. The results indicate that selective state-space modelling is a promising basis for scalable, uncer-tainty-aware, and
privacy-conscious time-series prediction, while also revealing limitations on irregular epidemiological signals and long-horizon generalization.
Read the full abstract ▾
All authors Shah Nawaz Haider, Steve Austin, Arnab Barua, Sarowar Morshed Shawon, Hadaate Ullah
Machine Learning · Cross-listed from cs.CV
SE-UNet: Singular Equivariant Imaging for Real-World Constrained Generation
Kanishk Awadhiya
While
diffusion models have revolutionized image synthesis, their application to real-world inverse problems is often hampered by the need for massive
datasets and the difficulty of imposing strict physical constraints. In this work, we introduce SE-UNet (Singular Equivariant UNet), a framework designed to solve ill-posed imaging tasks without extensive pre-training. By treating generation as an optimization problem constrained by
geometric equivariance ($D_4$ group) and singular value gating, SE-UNet effectively standardizes the solution space. We demonstrate that these strong inductive biases allow for state-of-the-art zero-shot inpainting results (80% missing pixels) on CIFAR-10. Our method surpasses Deep Image Prior (DIP) baselines by over 4 dB in PSNR and exhibits a characteristic "singular snap" convergence – rapidly locking into the signal
manifold. SE-UNet thus offers a data-efficient pathway for constrained generation, aligning with the ReALM-GEN goal of bridging theoretical priors with practical deployment.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
Schedulable Job-Level Dependencies for Cause-Effect Chains via Graph Neural Networks
Silviu S. Craciunas, Christian Hakert, Jian-Jia Chen et al. (+2 more)
Modern automotive software architectures comprise large sets of mixed-criticality functions executing on shared multi-core platforms with strict real-time and end-to-end timing requirements. Sensor-to-actuator data propagation in such systems is typically expressed via cause-effect chains with worst-case data-age budgets. Job-level dependencies (JLDs) have been introduced to provide a schedule-agnostic mechanism for bounding the data age independently of the underlying scheduler. The state-of-the-art methods for synthesizing JLDs, however, do not check whether the produced JLDs are enforceable under a concrete scheduling policy or jointly schedulable at the system level. In this paper we propose the first machine-learning-based JLD synthesis method, built around a two-level Graph Neural Network with temperature-controlled sampling that learns the structural patterns connecting cause-effect chain configurations to their JLD solutions. Since learned outputs may not be correct by construction, we embed the GNN in a novel Generate-and-Verify architecture in which a safe DP data-age checker, together with a per-chain EDF feasibility checker and a system-level demand-bound test, accept or reject each candidate. We show that the ML-based generator substantially outperforms the original greedy heuristic while achieving orders-of-magnitude lower synthesis time, demonstrating that learned structural priors can effectively replace exponential propagation-tree enumeration on this class of real-time scheduling problems.
Read the full abstract ▾
All authors Silviu S. Craciunas, Christian Hakert, Jian-Jia Chen, Zdeněk Hanzálek, Paul Pop
Machine Learning
Evaluating Time Series Foundation Models for Electricity Price Forecasting: Contamination Risk, Distributional Shifts, and Covariate Dependence
Zhenghua Pan and Ahmed Aziz Ezzat
Time series foundation models (TSFMs) have shown strong zero-shot
forecasting performance, but their generalization in
covariate-driven, non-stationary settings is underexplored. Electricity price
forecasting (EPF) presents a challenging testbed due to complex temporal dependencies, distributional shifts, and strong reliance on structural and contextual information. We propose a two-
dataset-benchmarking framework for EPF to mitigate contamination risk and enable fair evaluation of TSFMs. We examine key aspects of EPF including point and probabilistic
forecasting performance, tail behavior, price spikes, and comparisons against domain-specific methods. We find that TSFMs are highly competitive and often outperform general-purpose baselines. Yet, their performance depends critically on
covariate support, and they do not consistently surpass domain-specific methods tailored to EPF. Interestingly, simple ensembles of TSFMs and domain-specific methods appear to have significant potential, suggesting that the two approaches capture complementary predictive information.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning
Matteo Boglioni, Thibault Rousset, Siva Reddy et al. (+2 more)
LLMs memorize sensitive training data, including personally identifiable information (PII), creating a pressing need for reliable post hoc removal methods. Unlearning has emerged as a promising solution, with state-of-the-art (SOTA) methods often following a localize-first, unlearn-second paradigm that targets specific model parameters. However, existing
benchmarks evaluate unlearning solely at the output level, leaving open the question of whether unlearning truly erases knowledge from a model's parameters or merely obfuscates it, a concern reinforced by the success of resurfacing attacks. To bridge this gap, we introduce LACUNA: the first unlearning testbed with ground-truth parameter-level localization. LACUNA injects PII of synthetic individuals into predefined parameters of 1B and 7B OLMo-based models via masked continual pretraining, enabling direct evaluation of whether unlearning targets the weights responsible for knowledge storage. We use LACUNA to
benchmark current SOTA unlearning methods and find that, despite strong output-level performance, existing methods are highly imprecise and susceptible to resurfacing attacks. We further show that when localization is successful, even a simple gradient-based unlearning method achieves strong erasure and robustness to resurfacing attacks, highlighting the importance of precise unlearning. We release LACUNA to complement behavioral evaluations and drive further advances in robust, localization-based unlearning.
Read the full abstract ▾
All authors Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers
Machine Learning
Program-as-Weights: A Programming Paradigm for Fuzzy Functions
Wentao Zhang, Liliana Hotsko, Woojeong Kim et al. (+3 more)
Many everyday programming tasks resist clean rule-based implementation, such as alerting on important log lines, repairing malformed JSON, or ranking search results by intent, and are increasingly outsourced to large
language model APIs at the cost of locality, reproducibility, and price. We propose fuzzy-function programming: compiling such a function from a natural-
language specification into a compact, locally-executable neural artifact. We instantiate this paradigm with Program-as-Weights (PAW), in which a 4B compiler trained on FuzzyBench, a 10M-example
dataset we release, emits parameter-efficient adapters for a frozen, lightweight interpreter. A 0.6B Qwen3 interpreter executing PAW programs matches the performance of direct prompting of Qwen3-32B, while using roughly one fiftieth of the inference
memory and running at 30
tokens/s on a MacBook M3. PAW reframes the foundation model from a per-input problem solver into a tool builder: invoked once per function definition, it produces a small reusable artifact whose subsequent calls per function application are cheap and offline.
Read the full abstract ▾
All authors Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng
Machine Learning · Cross-listed from cs.AI
Online Safety Monitoring for LLMs
Mona Schirmer, Metod Jazbec, Alexander Timans et al. (+3 more)
Despite
alignment training, LLMs remain prone to generating unsafe outputs at deployment time. Monitoring outputs online and raising an alarm when safety can no longer be assumed is therefore critical. We study a simple real-time monitor that turns a verifier signal from an external model into an alarm decision by thresholding, with the threshold calibrated via risk control. In experiments on mathematical
reasoning and red teaming
datasets, we show that this simple design is competitive with more advanced monitors based on sequential hypothesis testing.
Read the full abstract ▾
All authors Mona Schirmer, Metod Jazbec, Alexander Timans, Christian Naesseth, Maja Waldron, Eric Nalisnick
Machine Learning · Cross-listed from cs.AI
What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates
Arman Ghaffarizadeh, Danyal Mohaddes, Aliakbar Izadkhah et al. (+1 more)
LLM
agents will increasingly act in socially structured settings where role, audience, and relational context can shape what is advantageous or costly to say. We study whether such social structure, without any explicit objective in the
prompt, changes what an
agent expresses publicly relative to an off-the-record (OTR) channel elicited under the same condition. We introduce a dual-channel debate framework in which
agents produce public utterances that enter the shared history alongside OTR responses that are recorded but never shown to the other participant. Across 10 models, 3 scenarios, and 5 variations within each scenario,
alignment-inducing settings produce systematic public-OTR divergence in the targeted
agent, with its decision divergence rising from a $\sim$3% baseline to roughly 40%. The effect is consistent across four aggregate analyses: stance, semantic similarity, natural
language inference, and survey responses. In some cases, the OTR response explicitly attributes public accommodation to relational pressures, such as career risk or sponsorship obligation. The findings suggest that
agent evaluation should extend beyond explicit goals and detect emergent objectives. We present a dual-channel evaluation framework and complementary behavioral measures that operationalize this assessment.
Read the full abstract ▾
All authors Arman Ghaffarizadeh, Danyal Mohaddes, Aliakbar Izadkhah, Shahriar Noroozizadeh
Machine Learning
DemoPSD: Disagreement-Modulated Policy Self-Distillation
Yunhe Li, Hao Shi, Wenhao Liu et al. (+5 more)
On-policy self-
distillation (OPSD) has emerged as a practical method for training large
language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access. However, recent studies have found that the teacher's dense
token-level supervision, conditioned on privileged information, can lead to overfitting to in-domain patterns, suppress exploration, and hurt cross-domain generalization, while also introducing a more fundamental issue: *privileged information leakage*, where the student encodes answer-dependent shortcuts that are unavailable at test time. We introduce **DemoPSD**, a novel framework that resolves such problems through the idea of *selective adoption of teacher guidance*. Instead of fitting the full teacher distribution, DemoPSD steers the student toward a *reverse-KL barycenter target*, a weighted
geometric combination of the teacher and student distributions, that naturally balances learning from the teacher with preserving the student's own
reasoning capacity. We measure the difference between their distributions and use such a discrepancy to adaptively control the blending at each
token position. We provably show that DemoPSD achieves **(1)** *leakage attenuation*, i.e., effective mitigation of privileged information leakage; and **(2)** *exploration preservation*, i.e., preservation of exploration capacity under dense
token-level
distillation. Extensive experiments on SciKnowEval across four scientific fields show that DemoPSD outperforms both GRPO and SDPO while maintaining higher training
entropy and robustly generalizing to out-of-distribution GPQA
benchmarks.
Read the full abstract ▾
All authors Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, Linqi Song
Machine Learning
Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials
Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng et al. (+4 more)
Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and
datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed
matrix-structured optimizers, including
Muon, SOAP, and the hybrid SOAP-
Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-
Muon emerge as robust and consistently strong methods, while
Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.
Read the full abstract ▾
All authors Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, Boris Kozinsky
Machine Learning · Cross-listed from cs.RO
Controllable Sim Agents with Behavior Latents
Juanwu Lu, Junyu Zhu and Ziran Wang
Realistic traffic simulation requires
agents that imitate logged behavior and can also be steered along interpretable axes. Such controllability enables engineers to isolate variables, reproduce specific edge cases, and test autonomous systems without real-world risk. We introduce Controllable Neural Variational
Agents (CNeVA), a controllable simulated-
agent framework that learns to infer a per-
agent Gaussian behavior latent from per-channel discounted returns via a closed-form conjugate variational update, conditioning a rectified-flow trajectory generator trained on a mixed channel-mask curriculum for classifier-free guidance. To tackle scarcity in reward signals, we propose soft eligibility gates that replace hard binary thresholds with smooth exponential decay, preserving the gradient signal for near-threshold
agents. On the Waymo Open Motion
Dataset, CNeVA attains competitive realism on the
benchmark while exposing per-channel controllability that the higher-ranked imitation models lack. Speed- and acceleration-based steering produces monotone responses without stall-induced reward hacking. Safety controllability is monotone and substantial with the introduction of soft eligibility. We manage to achieve steerable map compliance under a context-residual return measure. Furthermore, our experiment demonstrates that steering metrics must be read alongside physical-plausibility guardrails to avoid reward-hacking confounds.
Read the full abstract ▾