Machine Learning
Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization
Luca Della Libera, Cem Subakan and Mirco Ravanelli
Neural audio codecs are at the core of modern conversational
speech technologies, converting continuous
speech into sequences of discrete
tokens that can be processed by LLMs. However, existing codecs typically operate at fixed frame rates, allocating
tokens uniformly in time and producing unnecessarily long sequences. In this work, we introduce DyCAST, a Dynamic Character-Aligned
Speech Tokenizer that enables variable-frame-rate tokenization through soft character-level
alignment and explicit duration modeling. DyCAST learns to associate
tokens with character-level linguistic units during training and supports
alignment-free inference with direct control over
token durations at decoding time. To improve
speech resynthesis quality at low frame rates, we further introduce a
retrieval-augmented decoding mechanism that enhances reconstruction fidelity without increasing bitrate. Experiments show that DyCAST achieves competitive
speech resynthesis quality and downstream performance while using significantly fewer
tokens than fixed-frame-rate codecs. Code and checkpoints will be released publicly at https://github.com/lucadellalib/dycast.
Read the full abstract ▾
Machine Learning
Names Don't Matter: Symbol-Invariant Transformer for Open-Vocabulary Learning
İlker Iş{\i}k and Wenchao Li
Current neural architectures lack a principled way to handle interchangeable
tokens, i.e., symbols that are semantically equivalent yet distinguishable, such as bound variables. As a result, models trained on fixed vocabularies often struggle to generalize to unseen symbols, even when the underlying semantics remain unchanged. We propose a novel
Transformer-based mechanism that is provably invariant to the renaming of interchangeable
tokens. Our approach employs parallel
embedding streams to isolate the contribution of each interchangeable
token in the input, combined with an aggregated
attention mechanism that enables structured information sharing across streams. Experimental results confirm the theoretical guarantees of our method and demonstrate substantial performance gains on open-vocabulary tasks that require generalization to novel symbols. Project page: https://bu-depend-lab.github.io/Symbol-Invariant-
Transformer/
Read the full abstract ▾
Machine Learning
Stochastic Linear Bandits with Parameter Noise
Daniel Ezer, Alon Peled-Cohen and Yishay Mansour
We study the
stochastic linear bandits with parameter noise model, in which the reward of action $a$ is $a^\top \theta$ where $\theta$ is sampled i.i.d. We show a regret upper bound of $\widetilde{O} (\sqrt{d T \log (K/\delta) \sigma^2_{\max})}$ for a horizon $T$, general action set of size $K$ of dimension $d$, and where $\sigma^2_{\max}$ is the maximal variance of the reward for any action. We further provide a lower bound of $\widetilde{\Omega} (d \sqrt{T \sigma^2_{\max}})$ which is tight (up to logarithmic factors) whenever $\log (K) \approx d$. For more specific action sets, $\ell_p$ unit balls with $p \leq 2$ and dual norm $q$, we show that the minimax regret is $\widetilde{\Theta} (\sqrt{dT \sigma^2_q)}$, where $\sigma^2_q$ is a variance-dependent quantity that is always at most $4$. This is in contrast to the minimax regret attainable for such sets in the classic additive noise model, where the regret is of order $d \sqrt{T}$. Surprisingly, we show that this optimal (up to logarithmic factors) regret bound is attainable using a very simple explore-exploit
algorithm.
Read the full abstract ▾
Machine Learning
Probing the Trajectories of Reasoning Traces in Large Language Models
Marthe Ballon, Brecht F. Verbeken, Vincent Ginis et al. (+1 more)
Large
language models (LLMs) increasingly solve difficult problems by producing
reasoning traces before emitting a final response. However, it remains unclear how accuracy and decision commitment evolve along a
reasoning trajectory, and whether intermediate trace segments provide answer-relevant information beyond generic length or stylistic effects. Here, we propose a trajectory-probing protocol for evaluating how a
reasoning model's predicted answer evolves along a partial
reasoning trace. The protocol (1) generates a model's full
reasoning trace, (2) truncates it at fixed
token-percentiles, and (3) injects each partial trace back into the model, measuring the model's induced answer distribution. We apply the protocol to seven open-source
reasoning models (Qwen3-4B/8B/14B, gpt-oss-20b/120b, DeepSeek-R1-Distill-Llama-8B and Gemma-4-31B-it) across four
benchmarks (GPQA Diamond, MMLU-Pro, Omni-MATH-2 and CompMath). The protocol reveals that accuracy and decision commitment generally increase as the percentage of provided
reasoning tokens grows, and length, form, and
token-identity controls confirm that gains stem from instance-specific semantic content rather than context length or generic
reasoning style effects. Furthermore, cross-model probing shows that even stronger models frequently repeat the source model's errors when given its partial
reasoning trace, while allowing further
reasoning substantially improves recovery. Together, these results provide a reproducible, model-agnostic measurement of
reasoning trace dynamics that generalizes across answer formats and can reveal model-specific failure modes invisible to aggregate accuracy.
Read the full abstract ▾
All authors Marthe Ballon, Brecht F. Verbeken, Vincent Ginis, Andres Algaba
Machine Learning · Cross-listed from cs.CR
No More, No Less: Least-Privilege Language Models
Paulius Rauba, Dominykas Seputis, Patrikas Vanagas et al. (+1 more)
Least privilege is a core security principle: grant each request only the minimum access needed to achieve its goal. Deployed
language models almost never follow it, instead being exposed through a single API endpoint that serves all users and requests. This gap exists not because least privilege would be unhelpful; deployments would benefit greatly from reducing unnecessary capability exposure. The real obstacle is definitional and mechanistic: what does "access" mean inside a
language model, and how can we enforce it without retraining or deploying multiple models? We take inspiration from least privilege in computer systems and define a class of models called least-privilege
language models, where privilege is reachable internal computation during the forward pass. In this view, lowering privilege literally shrinks the model's accessible function class, as opposed to denying access via learned policies. We formalize deployment-time control as a monitor-allocator-enforcer stack, separating (i) request-time signals, (ii) a decision rule that allocates privilege, and (iii) an inference-time mechanism that selects privilege. We then propose Nested Least-Privilege Networks, a shape-preserving, rank-indexed intervention that provides a smooth, reversible control knob. We show that this knob yields policy-usable privilege-utility frontiers and enables selective suppression of targeted capabilities with limited collateral degradation across various policies. Most importantly, we argue for a new deployment paradigm that challenges the premise that
language models can only be controlled at the output level.
Read the full abstract ▾
All authors Paulius Rauba, Dominykas Seputis, Patrikas Vanagas, Mihaela van der Schaar
Machine Learning
Unsupervised Hierarchical Skill Discovery
Damion Harvey, Geraud Nangue Tasse, Benjamin Rosman et al. (+2 more)
We consider the problem of unsupervised skill
segmentation and hierarchical structure discovery in reinforcement learning. While recent approaches have sought to segment trajectories into reusable skills or options, most rely on action labels, rewards, or handcrafted annotations, limiting their applicability. We propose a method that segments unlabelled trajectories into skills and induces a hierarchical structure over them using a grammar-based approach. The resulting hierarchy captures both low-level behaviours and their composition into higher-level skills. We evaluate our approach in high-dimensional, pixel-based environments, including Craftax and the full, unmodified version of Minecraft. Using metrics for skill
segmentation, reuse, and hierarchy quality, we find that our method consistently produces more structured and semantically meaningful hierarchies than existing baselines. Furthermore, as a proof of concept, we demonstrate that these discovered hierarchies accelerate and stabilise learning on downstream reinforcement learning tasks.
Read the full abstract ▾
All authors Damion Harvey, Geraud Nangue Tasse, Benjamin Rosman, Branden Ingram, Steven James
Machine Learning
SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training
Powei Chang, Jinpeng Zhang, Bowen Chen et al. (+9 more)
Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling greedy
algorithms to achieve a $(1-1/e)$ approximation under a cardinality budget. In practice, however, we identify alleviating gradient conflicts, misalignment between per-sample gradients, is a key factor that slows down the decay of marginal log-determinant information gains, thereby preventing significant loss of information. We formalize this via an $\varepsilon$-decomposition that quantifies the deviation from ideal submodularity as a function of conflict statistics, yielding data-dependent approximation factors that tighten as conflicts diminish. Guided by this analysis, we propose SPICE, a conflict-aware selector that maximizes information while penalizing misalignment, and that supports early stopping and proxy models for efficiency. Empirically, SPICE selects subsets with higher log-determinant information than original criteria, and these informational gains translate into performance improvements: across 8
benchmarks with LLaMA2-7B and Qwen2-7B, SPICE uses only 10% of the data, yet matches or exceeds 6 methods including full-data tuning. This achieves performance improvements with substantially lower training cost.
Read the full abstract ▾
All authors Powei Chang, Jinpeng Zhang, Bowen Chen, Chenyu Wang, Chenlu Guo, Yixing Zhang, Yukang Gao, JianXiang Xiang, Yue Gao, Chaoqun Sun, Yiyi Chen, Dongying Kong
Machine Learning
On Safer Reinforcement Learning for Sedation and Analgesia in Intensive Care
Joel Romero-Hernandez and Oscar Camara
Pain management in intensive care usually involves complex trade-offs, since both inadequate and excessive treatment can compromise patient safety. Prior work on reinforcement learning for sedation and analgesia has explored how to optimize these interventions, but has not considered patient survival or partial observability. To investigate the risks of these design choices, we developed an offline deep reinforcement learning framework that suggests hourly medication doses based on recurrent state representations. Using retrospective data from 47,144 ICU stays in the MIMIC-IV database, we trained and evaluated behavior-regularized actor-critic models that prescribe continuous doses of opioids, propofol, benzodiazepines, and dexmedetomidine according to two goals: reduce pain or jointly reduce pain and 30-day post-discharge mortality. Although the two resulting policies were associated with lower pain, clinician agreement with the pain-only policy was positively correlated with mortality ($\rho$=0.119, p<0.0001), while agreement with the joint policy was negatively correlated ($\rho$=-0.316, p<0.0001). We found that such divergence arose from a different response to high levels of comorbidity. This suggests that valuing post-discharge outcomes could be critical for learning safer treatment policies, even if a short-term goal remains the primary objective.
Read the full abstract ▾
Machine Learning
Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data
Eugenia Iofinova and Dan Alistarh
As artificial neural networks, and specifically large
language models, have improved rapidly in capabilities and quality, they have increasingly been deployed in real-world applications, from customer service to Google search, despite the fact that they frequently make factually incorrect or undesirable statements. This trend has inspired practical and academic interest in model editing, that is, in adjusting the weights of the model to modify its likely outputs for queries relating to a specific fact or set of facts. This may be done either to amend a fact or set of facts, for instance, to fix a frequent error in the training data, or to suppress a fact or set of facts entirely, for instance, in case of dangerous knowledge. Multiple methods have been proposed to do such edits. However, at the same time, it has been shown that such model editing can be brittle and incomplete. Moreover the effectiveness of any model editing method necessarily depends on the data on which the model is trained, and, therefore, a good understanding of the interaction of the training data distribution and the way it is stored in the network is necessary and helpful to reliably perform model editing. However, working with large
language models trained on real-world data does not allow us to understand this relationship or fully measure the effects of model editing. We therefore propose Behemoth, a fully synthetic data generation framework. To demonstrate the practical insights from the framework, we explore model editing in the context of simple tabular data, demonstrating surprising findings that, in some cases, echo real-world results, for instance, that in some cases restricting the update rank results in a more effective update. The code is available at https://github.com/IST-DASLab/behemoth.git.
Read the full abstract ▾
Machine Learning
Manifold-Aware Perturbations for Constrained Generative Modeling
Katherine Keegan and Lars Ruthotto
Generative models have enjoyed widespread success in a variety of applications. However, they encounter inherent mathematical limitations in modeling distributions where samples are constrained by equalities, as is frequently the setting in scientific domains. In this work, we develop a computationally cheap, mathematically justified, and highly flexible distributional modification for combating known pitfalls in equality-constrained generative models. We propose perturbing the data distribution in a constraint-aware way such that the new distribution has support matching the ambient space dimension while still implicitly incorporating underlying
manifold geometry. Through theoretical analyses and empirical evidence on several representative tasks, we illustrate that our approach consistently enables data distribution recovery and stable sampling with both
diffusion models and normalizing flows.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Compressed BC-LISTA via Low-Rank Convolutional Decomposition
Han Wang, Yhonatan Kvich, Eduardo Pérez et al. (+2 more)
We study Sparse Signal Recovery (SSR) methods for multichannel imaging with compressed {forward and backward} operators that preserve reconstruction accuracy. We propose a Compressed Block-Convolutional (C-BC) measurement model based on a low-rank Convolutional Neural Network (CNN) decomposition that is analytically initialized from a low-rank factorization of physics-derived forward/backward operators in time delay-based measurements. We use Orthogonal Matching Pursuit (OMP) to select a compact set of basis filters from the analytic model and compute linear mixing coefficients to approximate the full model. We consider the Learned Iterative Shrinkage-Thresholding
Algorithm (LISTA) network as a representative example for which the C-BC-LISTA extension is presented. In simulated multichannel ultrasound imaging across multiple Signal-to-Noise Ratios (SNRs), C-BC-LISTA requires substantially fewer parameters and smaller model size than other state-of-the-art (SOTA) methods while improving reconstruction accuracy. In ablations over OMP, Singular Value Decomposition (SVD)-based, and random initializations, OMP-initialized structured compression performs best, yielding the most efficient training and the best performance.
Read the full abstract ▾
All authors Han Wang, Yhonatan Kvich, Eduardo Pérez, Florian Römer, Yonina C. Eldar
Machine Learning
Securing Time Integrity in Energy IoT Against Clock Drift and Y2K38 Failures
Saeid Jamshidi, Foutse Khomh, Carol Fung et al. (+2 more)
Time integrity across distributed Internet of Things (IoT) devices is fundamental to reliable sensing, control, and security in energy cyber-physical systems. However, operational energy IoT systems remain vulnerable to clock-drift escalation, time-synchronization manipulation, and catastrophic timestamp discontinuities, e.g., the Year 2038 (Y2K38) Unix epoch overflow. These failures violate timestamp monotonicity, distort temporal ordering, and introduce structured inconsistencies in system observations. Conventional anomaly detection models, which typically assume reliable and uniformly ordered timestamps, are therefore ill-equipped to capture timing-layer failures. This paper introduces STGAT (Spatio-Temporal Graph
Attention Network), a clock-dynamics-aware anomaly detection solution that jointly models temporal distortion and inter-device consistency in energy IoT systems. STGAT integrates drift-aware temporal
embeddings and temporal self-
attention to capture non-uniform and corrupted time
evolution within individual device streams, while graph
attention models the spatial propagation of timing inconsistencies across interconnected nodes. A Jacobian-regularized latent representation further promotes
geometric separation between nominal clock
evolution and anomalous temporal deformation caused by drift escalation, synchronization offsets, jitter accumulation, and epoch-overflow events. Experimental evaluation on energy IoT telemetry augmented with controlled timing-layer perturbations shows that STGAT achieves 95.7% accuracy, 94.0% precision, 92.0% recall, 93.0% F1-score, and 0.97 AUC under the primary test setting. STGAT also reduces detection delay to 2.3 time steps, corresponding to a 26% improvement over the closest baseline, while maintaining stable performance under overflow-induced discontinuities, stealthy drift escalation, and temporally induced physical inconsistencies.
Read the full abstract ▾
All authors Saeid Jamshidi, Foutse Khomh, Carol Fung, Omar Abdul Wahab, Rolando Herrero
Machine Learning
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
Cheng Ge, Caitlyn Heqi Yin, Hao Liang et al. (+1 more)
Reinforcement learning (RL) has become a key driver of
language model
reasoning. Among RL
algorithms, Group Relative Policy Optimization (GRPO) is the de facto standard, avoiding the need for a critic by using per-
prompt baselines and variance normalization. Yet why and when this normalization helps remains unclear. In this work, we answer both questions through the lens of local
curvature of the sequence-level policy gradient: standard deviation normalization implements an adaptive gradient. Theoretically, we prove that under mild conditions GRPO attains an improved convergence rate over unnormalized REINFORCE, with gains characterized by the average within-
prompt reward standard deviation across
prompts and iterations. We further introduce IS-GRPO, an importance-sampling variant of GRPO whose expected update remains aligned with the full gradient, and prove a convergence guarantee of the same form as REINFORCE's, which is tighter in practice during real training runs by a factor we measure. Empirically, on GSM8K and MATH we validate the
curvature–variance link and measure the quantities that govern our bounds along real training runs. At 1.5B scale, per-
prompt normalization outperforms both unnormalized and globally normalized baselines, with gains that emerge in a middle phase of training where per-
prompt variances become heterogeneous. At 7B scale, the normalization schemes become statistically indistinguishable while IS-GRPO retains a small lead, in line with the theory's prediction that the gains require heterogeneity.
Read the full abstract ▾
All authors Cheng Ge, Caitlyn Heqi Yin, Hao Liang, Jiawei Zhang
Machine Learning · Cross-listed from cs.AR
Machine Learning for Energy-Performance-aware Scheduling
Zheyuan Hu and Yifei Shi
In the post-Dennard era, optimizing embedded systems requires navigating complex trade-offs between energy efficiency and latency. Traditional heuristic tuning is often inefficient in such high-dimensional, non-smooth landscapes. In this work, we propose a
Bayesian Optimization framework using Gaussian Processes to automate the search for optimal scheduling configurations on heterogeneous multi-core architectures. We explicitly address the multi-objective nature of the problem by approximating the Pareto Frontier between energy and time. Furthermore, by incorporating Sensitivity Analysis (fANOVA) and comparing different covariance kernels (e.g., Matérn vs. RBF), we provide physical interpretability to the black-box model, revealing the dominant hardware parameters driving system performance.
Read the full abstract ▾
Machine Learning
Regularisation in neural networks: a survey and empirical analysis of approaches
Christiaan P. Opperman, Anna S. Bosman and Katherine M. Malan
Despite huge successes on a wide range of tasks, neural networks are known to sometimes struggle to generalise to unseen data. Many approaches have been proposed over the years to promote the generalisation ability of neural networks, collectively known as regularisation techniques. These are used as common practice under the assumption that any regularisation added to the pipeline would result in a performance improvement. In this study, we investigate whether this assumption holds in practice. First, we provide a broad review of regularisation techniques, including modern theories such as double descent. We propose a taxonomy of methods under four broad categories, namely: (1) data-based strategies, (2) architecture strategies, (3) training strategies, and (4) loss function strategies. Notably, we highlight the contradictions and correspondences between the approaches in these broad classes. Further, we perform an empirical comparison of the various regularisation techniques on classification tasks for ten numerical and image
datasets applied to the multi-layer perceptron and convolutional neural network architectures. Results show that the efficacy of regularisation is
dataset-dependent. For example, the use of a regularisation term only improved performance on numeric
datasets, whereas batch normalisation improved performance on image
datasets only. Generalisation is crucial to machine learning; thus, understanding the effects of applying regularisation techniques, and considering the connections between them is essential to the appropriate use of these methods in practice.
Read the full abstract ▾
Machine Learning
Distribution-informed Efficient Conformal Prediction for Full Ranking
Wenbo Liao, Huipeng Huang, Chen Jia et al. (+3 more)
Quantifying uncertainty is critical for the safe deployment of ranking models in real-world applications. Recent work offers a rigorous solution using conformal prediction in a full ranking scenario, which aims to construct prediction sets for the absolute ranks of test items based on the relative ranks of calibration items. However, relying on upper bounds of non-conformity scores renders the method overly conservative, resulting in substantially large prediction sets. To address this, we propose Distribution-informed Conformal Ranking (DCR), which produces efficient prediction sets by deriving the exact distribution of non-conformity scores. In particular, we find that the absolute ranks of calibration items follow Negative Hypergeometric distributions, conditional on their relative ranks. DCR thus uses the rank distribution to derive non-conformity score distribution and determine conformal thresholds. We provide theoretical guarantees that DCR achieves improved efficiency over the baseline while ensuring valid coverage under mild assumptions. Extensive experiments demonstrate the superiority of DCR, reducing average prediction set size by up to 36%, while maintaining valid coverage.
Read the full abstract ▾
All authors Wenbo Liao, Huipeng Huang, Chen Jia, Huajun Xi, Hao Zeng, Hongxin Wei
Machine Learning
To See Far, Look Close: Evolutionary Forecasting for Long-term Time Series
Jiaming Ma, Siyuan Mu, Ruilin Tang et al. (+6 more)
The prevailing Direct
Forecasting (DF) paradigm dominates Long-term Time Series
Forecasting (LTSF) by forcing models to predict the entire future horizon in a single forward pass. While efficient, this rigid coupling of output and evaluation horizons necessitates computationally prohibitive re-training for every target horizon. In this work, we uncover a counter-intuitive optimization anomaly: models trained on short horizons-when coupled with our proposed Evolutionary
Forecasting (EF) paradigm-significantly outperform those trained directly on long horizons. We attribute this success to the mitigation of a fundamental optimization pathology inherent in DF, where conflicting gradients from distant futures cripple the learning of local dynamics. We establish EF as a unified generative framework, proving that DF is merely a degenerate special case of EF. Extensive experiments demonstrate that a singular EF model surpasses task-specific DF ensembles across standard
benchmarks and exhibits robust asymptotic stability in extreme extrapolation. This work propels a paradigm shift in LTSF: moving from passive Static Mapping to autonomous Evolutionary
Reasoning.
Read the full abstract ▾
All authors Jiaming Ma, Siyuan Mu, Ruilin Tang, Haofeng Ma, Qihe Huang, Zhengyang Zhou, Pengkun Wang, Binwu Wang, Yang Wang
Machine Learning
CATTO: Balancing Preferences and Confidence in Language Models
Nisarg Parikh, Ananya Sai, Pannaga Shivaswamy et al. (+2 more)
Large
language models (LLMs) often make accurate next
token predictions but their confidence in these predictions can be poorly calibrated: high-confidence predictions are frequently wrong, and low-confidence predictions may be correct. This miscalibration is exacerbated by preference-based
alignment methods breaking the link between predictive probability and correctness. We introduce a Calibration Aware
Token-level Training Objective (CATTO), a calibration-aware objective that aligns predicted confidence with empirical prediction correctness, which can be combined with the original preference optimization objectives. Empirically, CATTO reduces Expected Calibration Error (ECE) by 2.22%-7.61% in-distribution and 1.46%-10.44% out-of-distribution compared to direct preference optimization (DPO), and by 0.22%-1.24% in-distribution and 1.23%-5.07% out-of-distribution compared to the strongest DPO baseline. This improvement in confidence does not come at a cost of losing task accuracy, where CATTO maintains or slightly improves multiple-choice question-answering accuracy on five
datasets. We also introduce Confidence@k, a test-time scaling mechanism leveraging calibrated
token probabilities for Bayes-optimal selection of output
tokens.
Read the full abstract ▾
All authors Nisarg Parikh, Ananya Sai, Pannaga Shivaswamy, Kunjal Panchal, Andrew Lan
Machine Learning · Cross-listed from cs.CL
Safer Policy Compliance with Dynamic Epistemic Fallback
Joseph Marvin Imperial and Harish Tayyar Madabushi
Humans develop a series of cognitive defenses, known as epistemic vigilance, to combat risks of deception and misinformation from everyday interactions. Developing safeguards for LLMs inspired by this mechanism might be particularly helpful for their application in high-stakes tasks such as automating compliance with data
privacy laws. In this paper, we introduce Dynamic Epistemic Fallback (DEF), a dynamic safety protocol for improving an LLM's inference-time defenses against deceptive attacks that make use of maliciously perturbed policy texts. Through various levels of one-sentence textual cues, DEF nudges LLMs to flag inconsistencies, refuse compliance, and fallback to their parametric knowledge upon encountering perturbed policy texts. Using globally recognized legal policies such as HIPAA and GDPR, our empirical evaluations report that DEF effectively improves the capability of frontier LLMs to detect and refuse perturbed versions of policies, with DeepSeek-R1 achieving a 100% detection rate in one setting. This work encourages further efforts to develop cognitively inspired defenses to improve LLM robustness against forms of harm and deception that exploit legal artifacts.
Read the full abstract ▾
Machine Learning
RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning
Yuexin Bian, Jie Feng, Tao Wang et al. (+3 more)
On-policy Reinforcement Learning (RL) remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy, and policy updates must be conservative. In this paper, we revisit actor policy representation as a first-class design choice for on-policy RL. We study discretized categorical actors, which represent each action dimension as a distribution over discrete bins and induce a policy objective analogous to classification cross-
entropy loss. Building on architectural advances from supervised learning, we further pair discretized categorical actors with regularized networks, yielding RN-D. Across diverse continuous-control
benchmarks, we show that simply replacing the standard Gaussian actor with our proposed actor substantially improves performance, achieving state-of-the-art results within on-policy RL. We release our code at https://github.com/alwaysbyx/RND-RL.
Read the full abstract ▾
All authors Yuexin Bian, Jie Feng, Tao Wang, Yijiang Li, Sicun Gao, Yuanyuan Shi
Machine Learning
SplineFlow: Flow Matching for Dynamical Systems with B-Spline Interpolants
Santanu Subhash Rathod, Pietro Liò and Xiao Zhang
Flow matching is a scalable generative framework for characterizing continuous normalizing flows with wide-range applications. However, current state-of-the-art methods are not well-suited for modeling dynamical systems, as they construct conditional paths using linear interpolants that may not capture the underlying state
evolution, especially when learning higher-order dynamics from irregular sampled observations. Constructing unified paths that satisfy multi-marginal constraints across observations is challenging, since naïve higher-order
polynomials tend to be unstable and oscillatory. We introduce SplineFlow, a theoretically grounded flow matching
algorithm that jointly models conditional paths across observations via B-spline interpolation. Specifically, SplineFlow exploits the smoothness and stability of B-spline bases to learn the complex underlying dynamics in a structured manner while ensuring the multi-marginal requirements are met. Comprehensive experiments across various deterministic and
stochastic dynamical systems of varying complexity, as well as on cellular trajectory inference tasks, demonstrate the strong improvement of SplineFlow over existing baselines. Our code is available at: https://github.com/santanurathod/SplineFlow.
Read the full abstract ▾
Machine Learning
ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations
Joao Fonseca and Julia Stoyanovich
Computing the importance of features in supervised classification tasks is critical for model interpretability. Shapley values are a widely used approach for explaining model predictions, but require direct access to the underlying model, an assumption frequently violated in real-world deployments. We investigate whether meaningful feature attributions can be obtained in a zero-shot setting, using only the input data distribution and no evaluations of the target model. Because multiple models can produce identical predictions yet yield different Shapley decompositions, the mapping from data to attributions is not uniquely identifiable. We therefore target attributions that are "true to the data" rather than "true to the model", learning a posterior mean attribution under a meta-training prior. To this end, we introduce ExplainerPFN, a tabular foundation model built on TabPFN, pretrained on synthetic structural causal
datasets supervised with exact or near-exact Shapley values, that predicts feature attributions for unseen tabular
datasets without model access, gradients, or example explanations. Our contributions are fourfold: (1) we show that few-shot surrogate explainers achieve high SHAP fidelity with as few as two reference observations; (2) we propose ExplainerPFN, the first zero-shot method for estimating Shapley-value-style feature attributions without access to the underlying model or reference explanations, providing a principled attribution where no existing explainer can be applied; (3) we release an open-source implementation including the full training pipeline and synthetic data generator; and (4) through extensive experiments on real and synthetic
datasets, we show that ExplainerPFN achieves performance competitive with few-shot surrogate explainers that rely on 2-10 SHAP examples.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Study
Antonio Vitale, Emanuela Guglielmi, Simone Scalabrino et al. (+1 more)
Large
Language Models (LLMs) are increasingly relevant in Software Engineering research and practice, with Automated Bug Fixing (ABF) being one of their key applications. ABF involves transforming a buggy method into its fixed equivalent. A common preprocessing step in ABF involves removing comments from code prior to training. However, we hypothesize that comments may play a critical role in fixing certain types of bugs by providing valuable design and implementation insights. In this study, we investigate how the presence or absence of comments, both during training and at inference time, impacts the bug-fixing capabilities of LLMs. We conduct an empirical evaluation comparing two model families, each evaluated under all combinations of training and inference conditions (with and without comments), and thereby revisiting the common practice of removing comments during training. To address the limited availability of comments in state-of-the-art
datasets, we use an LLM to automatically generate comments for methods lacking them. Our findings show that comments improve ABF accuracy by up to threefold when present in both phases, while training with comments does not degrade performance when instances lack them. Additionally, an interpretability analysis identifies that comments detailing method implementation are particularly effective in aiding LLMs to fix bugs accurately.
Read the full abstract ▾
All authors Antonio Vitale, Emanuela Guglielmi, Simone Scalabrino, Rocco Oliveto
Machine Learning
From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning
Wenzhe Niu, Wei He, Zongxia Xie et al. (+10 more)
Reinforcement learning has become a cornerstone for enhancing the
reasoning capabilities of Large
Language Models, where group-based approaches such as GRPO have emerged as efficient paradigms that optimize policies by leveraging intra-group performance differences. However, these methods typically rely on absolute numerical rewards, introducing intrinsic limitations. In verifiable tasks, identical group evaluations often result in sparse supervision, while in open-ended scenarios, the score range instability of reward models undermines advantage estimation based on group means. To address these limitations, we propose Reinforcement Learning with Relative Rewards (RLRR), a framework that shifts reward shaping from absolute scoring to relative ranking. Complementing this framework, we introduce the Ranking Reward Model, a listwise preference model tailored for group-based optimization to directly generate relative rankings. By transforming raw evaluations into robust relative signals, RLRR effectively mitigates signal sparsity and reward instability. Experimental results demonstrate that RLRR yields consistent performance improvements over standard group-based baselines across
reasoning benchmarks and open-ended generation tasks.
Read the full abstract ▾
All authors Wenzhe Niu, Wei He, Zongxia Xie, Jinpeng Ou, Huichuan Fan, Yuchen Ge, Yanru Sun, Ziyin Wang, Yizhao Sun, Chengshun Shi, Jiuchong Gao, Jinghua Hao, Renqing He
Machine Learning
Adaptive Edge Learning for Density-Aware Graph Generation
Seyedeh Ava Razi Razavi, James Sargant, Sheridan Houghten et al. (+1 more)
Generating realistic graph-structured data is challenging due to discrete structures, variable sizes, and class-specific connectivity patterns that resist conventional generative modelling. While recent graph generation methods employ generative
adversarial network (GAN) frameworks to handle permutation invariance and irregular
topologies, they typically rely on random edge sampling with fixed probabilities, limiting their capacity to capture complex structural dependencies between nodes. We propose a density-aware conditional graph generation framework using Wasserstein GANs (WGAN) that replaces random sampling with a learnable distance-based edge predictor. Our approach embeds nodes into a latent space where proximity correlates with edge likelihood, enabling the generator to learn meaningful connectivity patterns. A differentiable edge predictor determines pairwise relationships directly from node
embeddings, while a density-aware selection mechanism adaptively controls edge density to match class-specific sparsity distributions observed in real graphs. We train the model using a WGAN with gradient penalty, employing a GCN-based critic to ensure generated graphs exhibit realistic
topology and align with target class distributions. Experiments on
benchmark datasets demonstrate that our method produces graphs with superior structural coherence and class-consistent connectivity compared to existing baselines. The learned edge predictor captures complex relational patterns beyond simple heuristics, generating graphs whose density and
topology closely match real structural distributions. Our results show improved training stability and controllable synthesis, making the framework effective for realistic graph generation and data augmentation. Source code is publicly available at https://github.com/ava-12/Density_Aware_WGAN.git.
Read the full abstract ▾
All authors Seyedeh Ava Razi Razavi, James Sargant, Sheridan Houghten, Renata Dividino
Machine Learning
Avoiding Premature Collapse: Adaptive Annealing for Entropy-Regularized Structural Inference
Yizhi Liu
Differentiable matching layers and residual connection paradigms, often implemented via
entropy-regularized Optimal Transport (OT), serve as critical mechanisms in structural prediction and architectural scaling. However, recovering discrete permutations or maintaining identity mappings via annealing $\epsilon \to 0$ is notoriously unstable. In this work, we identify a fundamental mechanism for this failure: Premature Mode Collapse. By analyzing the non-normal dynamics of the Sinkhorn fixed-point map, we reveal a theoretical thermodynamic speed limit: standard exponential cooling outpaces the contraction rate of the inference operator, which degrades as $O(1/\epsilon)$. To address this, we propose Efficient Piecewise Hybrid Adaptive Stability Control (EPH-ASC), an adaptive scheduling
algorithm that monitors the stability of the inference process. We demonstrate that EPH-ASC is essential for stabilizing
Manifold-Constrained Hyper-Connections (mHC) during large-scale training on the FineWeb-Edu
dataset, effectively preventing late-stage gradient explosions by enforcing a linear stability law.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Breaking the Stochasticity Barrier: An Adaptive Variance-Reduced Method for Variational Inequalities
Yungi Jeong and Takumi Otsuka
Stochastic non-convex non-concave optimization, formally characterized as
Stochastic Variational Inequalities (SVIs), presents unique challenges due to rotational dynamics and the absence of a global merit function. While adaptive step-size methods (like Armijo line-search) have revolutionized convex minimization, their application to this setting is hindered by the Stochasticity Barrier: the noise in gradient estimation masks the true operator
curvature, triggering erroneously large steps that destabilize convergence. In this work, we propose VR-SDA-A (Variance-Reduced
Stochastic Descent-Ascent with Armijo), a novel
algorithm that integrates recursive momentum (STORM) with a rigorous Same-Batch
Curvature Verification mechanism. We introduce a theoretical framework based on a Lyapunov potential tracking the Operator Norm, proving that VR- SDA-A achieves an oracle complexity of O(epsilon -3) for finding an epsilon-stationary point in general Lipschitz continuous operators. This matches the optimal rate for non-convex minimization while uniquely enabling automated step-size adaptation in the saddle-point setting. We validate our approach on canonical rotational
benchmarks and non-convex robust regression tasks, demonstrating that our method effectively suppresses limit cycles and accelerates convergence with reduced dependence on manual learning rate scheduling.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Asymptotic Theory of Iterated Empirical Risk Minimization, with Applications to Active Learning
Hugo Cui and Yue M. Lu
We study a class of iterated empirical risk minimization (ERM) procedures in which two successive ERMs are performed on the same
dataset, and the predictions of the first
estimator enter as an argument in the loss function of the second. This setting, which arises naturally in active learning and reweighting schemes, introduces intricate statistical dependencies across samples and fundamentally distinguishes the problem from classical single-stage ERM analyses. For linear models trained with a broad class of convex losses on Gaussian mixture data, we derive a sharp asymptotic characterization of the test error in the high-dimensional regime where the sample size and ambient dimension scale proportionally. Our results provide explicit, fully asymptotic predictions for the performance of the second-stage
estimator despite the reuse of data and the presence of prediction-dependent losses. We apply this theory to revisit a well-studied pool-based active learning problem, removing oracle and sample-splitting assumptions made in prior work. We uncover a fundamental tradeoff in how the labeling budget should be allocated across stages, and demonstrate a double-descent behavior of the test error driven purely by data selection, rather than model size or sample count.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Neural Backward Filtering Forward Guiding
Gefan Yang, Frank van der Meulen and Stefan Sommer
Inference in nonlinear continuous
stochastic processes on trees is challenging, particularly when observations are sparse and the
topology is complex. Exact smoothing via Doob's $h$-transform is intractable for general nonlinear dynamics. We propose Neural Backward Filtering Forward Guiding (NBFFG), a unified framework for both discrete transitions and continuous
diffusions. Our method constructs a variational posterior by leveraging a proxy linear-Gaussian process. This proxy process yields a closed-form backward filter that serves as a guide, steering the generative path toward high-likelihood regions. We then learn a neural residual to capture the non-linear discrepancies. This formulation allows for an unbiased pathwise subsampling scheme, reducing the training complexity from tree-size dependent to path-length dependent. Empirical results show that NBFFG outperforms baselines on synthetic
benchmarks, and we demonstrate the method on a high-dimensional inference task in phylogenetic analysis with reconstruction of ancestral butterfly wing shapes.
Read the full abstract ▾
Machine Learning
Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning
Arvind Mahankali, Kaiyue Wen and Tengyu Ma
Long chain-of-thought
reasoning (Long CoT) is now fundamental to state-of-the-art LLMs, especially in mathematical
reasoning. However, LLM generation is highly sequential, and long CoTs lead to a high latency. We propose to train Divide-and-Conquer CoT (DC-CoT) to reduce the latency. With DC-CoT, the model can act as a director that identifies distinct subtasks that can be performed in parallel in its
reasoning process, and then spawns workers to execute the subtasks. Our goal is to achieve high accuracy, with a low longest path length, which is a theoretical measure of the latency needed for the response. We start with a long CoT base model (DeepScaleR-1.5B-Preview), and first use SFT with a small curated demonstration set to initialize its ability to spawn workers in a certain format. Because SFT degrades the accuracy significantly, we design a multi-stage RL
algorithm, with various data filtering strategies, to recover the accuracy while decreasing the longest path length. Across several
benchmarks including AIME 2024 and HMMT 2025, DC-CoT achieves similar accuracy as DeepScaleR-1.5B-Preview while decreasing longest path length by 35-40%. Our code, SFT
dataset and models are publicly available at https://github.com/amahankali10/DC_CoT_RL_for_Low_Latency_CoT_with_Parallel_
Reasoning.
Read the full abstract ▾