Machine Learning
Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization
Ratun Rahman and Dinh C. Nguyen
Conventional
federated learning (FL) frameworks often suffer from training degradation due to data uncertainty and heterogeneity across local clients. Probabilistic approaches such as
Bayesian neural networks (BNNs) can mitigate this issue by explicitly modeling uncertainty, but they introduce additional runtime, latency, and bandwidth overhead that has rarely been studied in
federated settings. To address these challenges, we propose Meta-BayFL, a personalized probabilistic FL method that combines meta-learning with BNNs to improve training under uncertain and heterogeneous data. The framework is characterized by three main features: (1) BNN-based client models incorporate uncertainty across hidden layers to stabilize training on small and noisy
datasets, (2) meta-learning with adaptive learning rates enables personalized updates that enhance local training under non-IID conditions, and (3) a unified probabilistic and personalized design improves the robustness of global model aggregation. We provide a theoretical convergence analysis and characterize the upper bound of the global model over communication rounds. In addition, we evaluate computational costs (runtime, latency, and communication) and discuss the feasibility of deployment on resource-constrained devices such as edge nodes and IoT systems. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that Meta-BayFL consistently outperforms state-of-the-art methods, including both standard and personalized FL approaches (e.g., pFedMe, Ditto, FedFomo), with up to 7.42% higher test accuracy.
Read the full abstract ▾
Machine Learning
SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training
Prince Zizhuang Wang and Shuli Jiang
Large
Language Model (LLM)
agents have shown strong results on multi-turn tool-use tasks, yet they operate in isolation during training, failing to leverage experiences accumulated across episodes. Existing experience-augmented methods address this by organizing trajectories into retrievable libraries, but they retrieve experiences only once based on the initial task description and hold them constant throughout the episode. In multi-turn settings where observations change at every step, this static
retrieval becomes increasingly mismatched as episodes progress. We propose SLEA-RL (Step-Level Experience-Augmented Reinforcement Learning), a framework that retrieves relevant experiences at each decision step conditioned on the current observation. SLEA-RL operates through three components: (i) step-level observation clustering that groups structurally equivalent environmental states for efficient cluster-indexed
retrieval; (ii) a self-evolving experience library that distills successful strategies and failure patterns through score-based admission and rate-limited extraction; and (iii) policy optimization with step-level credit assignment for fine-grained advantage estimation across multi-turn episodes. The experience library evolves alongside the policy through semantic analysis rather than gradient updates. Experiments on long-horizon multi-turn
agent benchmarks demonstrate that SLEA-RL achieves superior performance compared to various reinforcement learning baselines.
Read the full abstract ▾
Machine Learning
Variational Phasor Circuits for Phase-Native Brain-Computer Interface Classification
Dibakar Sigdel
We present the Variational Phasor
Circuit (VPC), a deterministic classical learning architecture on the continuous $S^1$ unit-circle
manifold. Inspired by variational
quantum circuits, VPC replaces dense weight
matrices with trainable phase shifts, local
unitary mixing, and structured interference in the ambient complex space, giving a unified method for binary and multi-class classification of spatially distributed signals. We evaluate VPC on real motor-imagery electroencephalography (
EEG) from the PhysioNet Motor Movement/Imagery database (10 subjects, Common Spatial Pattern features, subject-wise cross-validation), where it attains a mean decoding accuracy of $0.60$ – the highest among standard
brain–computer-interface baselines (linear discriminant analysis, logistic regression, RBF-SVM, and a multilayer perceptron) – using an order of magnitude fewer parameters and the lowest cross-subject variance. We also characterize capacity honestly: with phase-only shifts and
unitary mixing, VPC realizes a linear decision function in a fixed cosine/sine feature lifting, well matched to the largely separable band-power structure of
EEG but unable to represent parity-type functions, a ceiling that depth does not raise. These results position unit-circle phase interference as a parameter-efficient alternative to dense neural computation for signal classification, and motivate VPC both as a standalone classifier and a front-end for hybrid phasor-
quantum systems.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.BM
Generative Replica-Exchange: A Flow-based Framework for Accelerating Replica Exchange Simulations
Shengjie Huang, Sijie Yang, Jianqiao Yi et al. (+6 more)
Replica exchange (REX) is one of the most widely used enhanced sampling methodologies, yet its efficiency is limited by the requirement for a large number of intermediate temperature replicas. Here we present Generative Replica Exchange (GREX), which integrates deep generative models into the REX framework to eliminate this temperature ladder. Drawing inspiration from reservoir replica exchange (res-REX), GREX utilizes trained normalizing flows to generate high-temperature configurations on demand and map them directly to the target distribution using the potential energy as a constraint, without requiring target-temperature training data. This approach reduces production simulations to a single replica at the target temperature while maintaining thermodynamic rigor through Metropolis exchange acceptance. We validate GREX on three
benchmark systems of increasing complexity, highlighting its superior efficiency and practical applicability for molecular simulations.
Read the full abstract ▾
All authors Shengjie Huang, Sijie Yang, Jianqiao Yi, Rui Zheng, Haocong Liao, Muzammal Hussain, Yaoquan Tu, Xiaoyun Lu, Yang Zhou
Machine Learning
Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling
Junzhuo Ma, Chenghuang Shen, Yi Yu et al. (+15 more)
Technical-service LLM
agents are entering production workflows, where value depends on whether engineers adopt generated replies. Service tickets hide decision logic, contain noisy single-reference responses, and make reward evaluation costly, making standard post-training brittle. Existing post-training and LLM-as-a-Judge approaches improve grounding or feedback, but do not jointly model latent decision logic, response diversity, and reward cost. We address this gap by coupling latent logic augmentation, robust noise reduction, and hybrid reward modeling. The framework augments supervised fine-tuning data with Planning-Aware Trajectory Modeling and
Reasoning Augmentation, builds dual-filtered Multiple Ground Truths, and trains the policy with a hybrid reward that combines a Reranker with an LLM-as-a-Judge. On real Cloud technical-service tasks, the adapted Qwen3-4B achieves the highest Multi-ECS (0.441), lower reward cost, and the highest production adoption rate (46.63%).
Read the full abstract ▾
All authors Junzhuo Ma, Chenghuang Shen, Yi Yu, Xingyan Liu, Jing Gu, Hangyi Sun, Guangquan Hu, Jianfeng Liu, Weiting Liu, Pu Mingyue, Wang Yu, Zhengdong Xiao, Rui Xie, Longjiu Luo, Qianrong Wang, Gurong Cui, Honglin Qiao, Wenlian Lu
Machine Learning · Cross-listed from cs.NE
A Synthesizable RTL Implementation of Predictive Coding Networks
Timothy Oh
Backpropagation has enabled modern deep learning but is difficult to realize as an online, fully distributed hardware learning system due to global error propagation, phase separation, and heavy reliance on centralized
memory. Predictive coding offers an alternative in which inference and learning arise from local prediction-error dynamics between adjacent layers. This paper presents a digital architecture that implements a discrete-time predictive coding update directly in hardware. Each neural core maintains its own activity, prediction error, and synaptic weights, and communicates only with adjacent layers through hardwired connections. Supervised learning and inference are supported via a uniform per-
neuron clamping primitive that enforces boundary conditions while leaving the internal update schedule unchanged. The design is a deterministic, synthesizable RTL substrate built around a sequential MAC datapath and a fixed finite-state schedule. Rather than executing a task-specific instruction sequence inside the learning substrate, the system evolves under fixed local update rules, with task structure imposed through connectivity, parameters, and boundary conditions. The contribution of this work is not a new learning rule, but a complete synthesizable digital substrate that executes predictive-coding learning dynamics directly in hardware.
Read the full abstract ▾
Machine Learning
Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse
Dip Roy, Rajiv Misra and Sanjay Kumar Singh
Extreme neural network sparsification (90% activation reduction) presents a critical challenge for mechanistic interpretability: understanding whether interpretable features survive aggressive compression. This work investigates feature survival under severe capacity constraints in hybrid Variational Autoencoder–Sparse Autoencoder (VAE-SAE) architectures. We introduce an adaptive sparsity scheduling framework that progressively reduces active
neurons from 500 to 50 over 50 training epochs, and provide empirical evidence for fundamental limits of the sparsification-interpretability relationship. Testing across two
benchmark datasets – dSprites and Shapes3D – with both Top-k and L1 sparsification methods, our key finding reveals a pervasive paradox: while global representation quality (measured by Mutual Information Gap) remains stable, local feature interpretability collapses systematically. Under Top-k sparsification, dead
neuron rates reach $34.4\pm0.9\%$ on dSprites and $62.7\pm1.3\%$ on Shapes3D at k=50. L1 regularization – a fundamentally different "soft constraint" paradigm – produces equal or worse collapse: $41.7\pm4.4\%$ on dSprites and $90.6\pm0.5\%$ on Shapes3D. Extended training for 100 additional epochs fails to recover dead
neurons, and the collapse pattern is robust across all tested threshold definitions. Critically, the collapse scales with
dataset complexity: Shapes3D (RGB, 6 factors) shows $1.8\times$ more dead
neurons than dSprites (grayscale, 5 factors) under Top-k and $2.2\times$ under L1. These findings establish that interpretability collapse under sparsification is intrinsic to the compression process rather than an artifact of any particular
algorithm, training duration, or threshold choice.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Unified Spatio-Temporal Token Scoring for Efficient Video VLMs
Jianrui Zhang, Yue Yang, Rohun Tripathi et al. (+5 more)
Token pruning is essential for enhancing the computational efficiency of
vision-
language models (VLMs), particularly for
video-based tasks where temporal redundancy is prevalent. Prior approaches typically prune
tokens either (1) within the
vision transformer (ViT) exclusively for unimodal perception tasks such as action recognition and object
segmentation, without adapting to downstream
vision-
language tasks; or (2) only within the LLM while leaving the ViT output intact, often requiring complex text-conditioned
token selection mechanisms. In this paper, we introduce Spatio-Temporal
Token Scoring (STTS), a simple and lightweight module that prunes
vision tokens across both the ViT and the LLM without text conditioning or
token merging, and is fully compatible with end-to-end training. By learning how to score temporally via an auxiliary loss and spatially via LLM downstream gradients, aided by our efficient packing
algorithm, STTS prunes 50% of
vision tokens throughout the entire architecture, resulting in a 62% improvement in efficiency during both training and inference with only a 0.7% drop in average performance across 13 short and long
video QA tasks. Efficiency gains increase with more sampled frames per
video. Applying test-time scaling for long-
video QA further yields performance gains of 0.5-1% compared to the baseline. Overall, STTS represents a novel, simple yet effective technique for unified, architecture-wide
vision token pruning.
Read the full abstract ▾
All authors Jianrui Zhang, Yue Yang, Rohun Tripathi, Winson Han, Ranjay Krishna, Christopher Clark, Yong Jae Lee, Sangho Lee
Machine Learning · Cross-listed from cs.CV
LoST: Level of Semantics Tokenization for 3D Shapes
Niladri Shekhar Dutt, Zifan Shi, Paul Guerrero et al. (+4 more)
Tokenization is a fundamental technique in the generative modeling of various modalities. In particular, it plays a critical role in autoregressive (AR) models, which have recently emerged as a compelling option for 3D generation. However, optimal tokenization of 3D shapes remains an open question. State-of-the-art (SOTA) methods primarily rely on
geometric level-of-detail (LoD) hierarchies, originally designed for rendering and compression. These spatial hierarchies are often
token-inefficient and lack semantic coherence for AR modeling. We propose Level-of-Semantics Tokenization (LoST), which orders
tokens by semantic salience, such that early prefixes decode into complete, plausible shapes that possess principal semantics, while subsequent
tokens refine instance-specific
geometric and semantic details. To train LoST, we introduce Relational Inter-Distance
Alignment (RIDA), a novel 3D semantic
alignment loss that aligns the relational structure of the 3D shape latent space with that of the semantic DINO feature space. Experiments show that LoST achieves SOTA reconstruction, surpassing previous LoD-based 3D shape tokenizers by large margins on both
geometric and semantic reconstruction metrics. Moreover, LoST achieves efficient, high-quality AR 3D generation and enables downstream tasks like semantic
retrieval, while using only 0.1%-10% of the
tokens needed by prior AR models.
Read the full abstract ▾
All authors Niladri Shekhar Dutt, Zifan Shi, Paul Guerrero, Chun-Hao Paul Huang, Duygu Ceylan, Niloy J. Mitra, Xuelin Chen
Machine Learning
Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training
Ben S. Southworth and Stephen Thomas
Orthogonalized-momentum optimizers such as
Muon improve
transformer training by approximately whitening/orthogonalizing
matrix-valued momentum updates via a short polar-decomposition iteration. However, polar-factor approximations typically require multiple large
matrix multiplications, and the resulting overhead can be substantial and hardware-dependent. We introduce MUD (MomentUm Decorrelation), a complementary whitening approach that replaces
Muon's polar update with a triangular (Cholesky-like) whitening surrogate inspired by classical Gram–Schmidt and Gauss-Seidel ideas. We show that row-orthonormal
matrices are fixed points of the MUD map, relate the inner step to symmetric Gauss-Seidel preconditioning of the Gram
matrix, and prove quadratic local convergence near the fixed point. In terms of time-to-perplexity, MUD yields consistent 10-50% wall-clock improvements over tuned AdamW and
Muon in time-to-perplexity, typically converging slightly slower per step than
Muon but with substantially lower optimizer overhead – relative to
Muon, MUD improves peak
tokens/s by roughly $1.3-2.6\times$ across most settings and up to nearly $3\times$ on GPT-2 large on an A100. We also demonstrate training a ESM-2 150M
protein language model, where MUD matches
Muon-level validation perplexity in significantly less wall-clock time.
Read the full abstract ▾
Machine Learning
Unified Policy Value Decomposition for Rapid Adaptation
Cristiano Capone, Luca Falorsi, Andrea Ciardiello et al. (+1 more)
Rapid adaptation in complex control systems remains a central challenge in reinforcement learning. We introduce a framework in which policy and value functions share a low-dimensional coefficient vector - a goal
embedding - that captures task identity and enables immediate adaptation to novel tasks without retraining representations. During pretraining, we jointly learn structured value bases and compatible policy bases through a bilinear actor-critic decomposition. The critic factorizes as Q = sum_k G_k(g) y_k(s,a), where G_k(g) is a goal-conditioned coefficient vector and y_k(s,a) are learned value basis functions. This multiplicative gating - where a context signal scales a set of state-dependent bases - is reminiscent of gain modulation observed in Layer 5 pyramidal
neurons, where top-down inputs modulate the gain of sensory-driven responses without altering their tuning. Building on Successor Features, we extend the decomposition to the actor, which composes a set of primitive policies weighted by the same coefficients G_k(g). At test time the bases are frozen and G_k(g) is estimated zero-shot via a single forward pass, enabling immediate adaptation to novel tasks without any gradient update. We train a Soft Actor-Critic
agent on the MuJoCo Ant environment under a multi-directional locomotion objective, requiring the
agent to walk in eight directions specified as continuous goal vectors. The bilinear structure allows each policy head to specialize to a subset of directions, while the shared coefficient layer generalizes across them, accommodating novel directions by interpolating in goal
embedding space. Our results suggest that shared low-dimensional goal
embeddings offer a general mechanism for rapid, structured adaptation in high-dimensional control, and highlight a potentially biologically plausible principle for efficient transfer in complex reinforcement learning systems.
Read the full abstract ▾
All authors Cristiano Capone, Luca Falorsi, Andrea Ciardiello, Luca Manneschi
Machine Learning
CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention
Zhongzhu Zhou, Fengxiang Bie, Ziyan Chen et al. (+6 more)
Converting pretrained
attention modules such as grouped-query
attention (GQA) into multi-head latent
attention (MLA) can improve expressivity without increasing KV-cache cost, making it attractive for efficient inference. However, many practical conversion baselines rely on weight-only low-rank approximations (e.g., SVD-style initializations) and uniform rank allocation. They focus on minimizing the difference between weight
matrices rather than on how those weights affect input activations, ignore the covariance structure of activations, and enforce uniform rank across layers, causing activation drift and degraded
attention fidelity. To address these issues, we propose CARE, a Covariance-Aware, Rank-Enhanced MLA conversion pipeline under a fixed KV width. CARE introduces three key steps: (i) activation-preserving factorization, which aligns the approximation with the actual input activations rather than just the weights; (ii) adjusted-rank allocation, which spreads a fixed KV budget across layers by giving more capacity to layers that need it most; and (iii) KV-parity mapping, which reparameterizes the converted K and V to fit the MLA format while keeping the KV-cache size unchanged. Our method outperforms a uniform-rank SVD baseline on Qwen3-4B/30B-A3B-Instruct-2507 and Llama-3.1-8B/70B-Instruct, reducing one-shot perplexity by up to 215x and improving mean accuracy by up to 1.70x at matched KV budgets. With a brief post-SVD healing fine-tune, we fully recover the original model's accuracy.
Read the full abstract ▾
All authors Zhongzhu Zhou, Fengxiang Bie, Ziyan Chen, Zhenyu Zhang, Yibo Yang, Junxiong Wang, Ben Athiwaratkun, Xiaoxia Wu, Shuaiwen Leon Song
Machine Learning · Cross-listed from stat.ME
Multi-Armed Sequential Hypothesis Testing by Betting
Ricardo J. Sandoval, Ian Waudby-Smith and Michael I. Jordan
We consider a variant of sequential testing by betting where, at each time step, the statistician is presented with multiple data sources (arms) and obtains data by choosing one of the arms. We consider the composite global null hypothesis $\mathscr{P}$ that all arms are null in a certain sense (e.g. all dosages of a treatment are ineffective) and we are interested in rejecting $\mathscr{P}$ in favor of a composite alternative $\mathscr{Q}$ where at least one arm is non-null (e.g. there exists an effective treatment dosage). We posit an optimality desideratum that we describe informally as follows: even if several arms are non-null, we seek $e$-processes and sequential tests whose performance are as strong as the ones that have oracle knowledge about which arm generates the most evidence against $\mathscr{P}$. Formally, we generalize notions of log-optimality and expected rejection time optimality to more than one arm, obtaining matching lower and upper bounds for both. A key technical device in this optimality analysis is a modified upper-confidence-bound-like
algorithm for unobservable but sufficiently "estimable" rewards. In the design of this
algorithm, we derive nonasymptotic concentration inequalities for optimal wealth growth rates in the sense of Kelly [1956]. These may be of independent interest.
Read the full abstract ▾
Machine Learning
Weight Clustering Reveals the Importance of Relative Rank in Large Language Models
Borja Aizpurua, Sukhbinder Singh and Román Orús
Large
language models (LLMs) contain billions of parameters, but many exact weight values are not essential. We apply scalar $K$-means clustering to pretrained models, replacing each weight
matrix with only 16–64 shared values. For Llama~3.1-8B-Instruct and SmolLM2-135M, this preserves strong accuracy without retraining, providing a simple way to compress LLMs on disk. Fine-tuning only the cluster centroids further recovers 30–40% of the remaining accuracy loss. Using clustering as an analysis tool, we then keep the cluster assignments fixed and perturb only the shared values, to study which of their properties matter most. We find that preserving their relative rank (which connections are stronger or weaker than others) is more important than preserving their exact magnitudes. Breaking this ordering greatly increases perplexity, while changing the values but keeping the ordering causes little loss in many middle and late layers. However, rank preservation alone is not sufficient. Model performance also depends on maintaining the variance of the reconstructed weights, while early layers remain more sensitive; perturbing the cluster assignments themselves shows that their spatial geometry is also load-bearing. We observe the same behavior across four LLMs from two architecture families. These results suggest that the ordering of weights, together with their overall variance, plays a central role in LLM compression and robustness.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models
Leonardo Defilippis, Florent Krzakala, Bruno Loureiro et al. (+1 more)
Understanding when learning is statistically possible yet computationally hard is a central challenge in high-dimensional statistics. In this work, we investigate this question in the context of single- and multi-index models, classes of functions widely studied as
benchmarks to probe the ability of machine learning methods to discover features in high-dimensional data. Our main contribution is to show that a Noise Sensitivity Exponent (NSE) - a simple quantity determined by the activation function - governs the existence and magnitude of statistical-to-computational gaps within a broad regime of these models. We first establish that, in single-index models with large additive noise, the onset of a computational bottleneck is fully characterized by the NSE. We then demonstrate that the same exponent controls a statistical-computational gap in the specialization transition of large separable multi-index models, where individual components become learnable. Finally, in hierarchical multi-index models, we show that the NSE governs the optimal computational rate in which different directions are sequentially learned. Taken together, our results identify the NSE as a unifying property linking noise robustness, computational hardness, and feature specialization in high-dimensional learning.
Read the full abstract ▾
All authors Leonardo Defilippis, Florent Krzakala, Bruno Loureiro, Antoine Maillard
Machine Learning · Cross-listed from cs.SE
scicode-lint: Detecting Methodology Bugs in Scientific Python Code with LLM-Generated Patterns
Sergey V. Samsonau
Methodology bugs in scientific Python code produce plausible but incorrect results that traditional linters and static analysis tools cannot detect. Several research groups have built ML-specific linters, demonstrating that detection is feasible. Yet these tools share a sustainability problem: dependency on specific pylint or Python versions, limited packaging, and reliance on manual engineering for every new pattern. As AI-generated code increases the volume of scientific software, the need for automated methodology checking (such as detecting data leakage, incorrect cross-validation, and missing random seeds) grows. We present scicode-lint, whose two-tier architecture separates pattern design (frontier models at build time) from execution (small local model at runtime). Patterns are generated, not hand-coded; adapting to new library versions costs
tokens, not engineering hours. On Kaggle notebooks with human-labeled ground truth, preprocessing leakage detection reaches 65% precision at 100% recall; on 38 published scientific papers applying AI/ML, precision is 62% (LLM-judged) with substantial variation across pattern categories; on a held-out paper set, precision is 54%. On controlled tests, scicode-lint achieves 97.7% accuracy across 66 patterns.
Read the full abstract ▾
Machine Learning
RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference
Arpit Singh Gautam and Saurabh Jha
Post training quantization is essential for deploying large
language models (LLMs) on resource constrained hardware, yet state of the art methods enforce uniform bit widths across layers, yielding suboptimal accuracy efficiency trade offs. We present RAMP (Reinforcement Adaptive Mixed Precision), an off policy Soft Actor Critic framework that learns per layer bit width assignments to minimize perplexity under a global bit budget. The policy conditions on an 11 dimensional
embedding of activation statistics, weight properties, and structural descriptors, enabling zero shot transfer across model families and scales. To enable stable sub 4 bit quantization, we introduce Scale Folding, a preconditioning technique that migrates activation outliers into weights via per channel scaling and normalization layer compensation. A quality prioritized reward with asymmetric penalties and budget cliffs drives rapid convergence. On Llama 2 7B, RAMP achieves 5.54 perplexity at 3.68GB (3.65 effective bits), outperforming uniform 4 bit AWQ (5.60 at 3.90 GB) and GPTQ by 6% in size and 1% to3% in quality. Critically, a policy trained only on Llama 2 7B generalizes zero shot to Llama 2 13B and Mistral 7B, often surpassing target specific training, supporting the hypothesis that quantization sensitivity is primarily architectural. The
HALO pipeline exports allocations to GGUF format for kernel free inference on CPUs, GPUs, and edge devices, retaining 99.5% of FP16 commonsense
reasoning performance.
Read the full abstract ▾
Machine Learning
Operator-Theoretic Foundations and Policy Gradient Methods for General MDPs with Unbounded Costs
Abhishek Gupta and Aditya Mahajan
Markov decision processes (MDPs) is viewed as an optimization of an objective function over certain linear operators over general function spaces. A new existence result is established for the existence of optimal policies in general MDPs, which differs from the existence result derived previously in the literature. Using the well-established perturbation theory of linear operators, policy difference lemma is established for general MDPs and the Gauteaux derivative of the objective function as a function of the policy operator is derived. By upper bounding the policy difference via the theory of integral probability metric, a new majorization-minimization type policy gradient
algorithm for general MDPs is derived. This leads to generalization of many well-known
algorithms in reinforcement learning to cases with general state and action spaces. Further, by taking the integral probability metric as maximum mean discrepancy, a low-complexity policy gradient
algorithm is derived for finite MDPs. The new
algorithm, called MM-RKHS, appears to be superior to PPO
algorithm due to low computational complexity, low sample complexity, and faster convergence.
Read the full abstract ▾
Machine Learning
RHYME-XT: A Neural Operator for Spatiotemporal Control Systems
Marijn Ruiter, Miguel Aguiar, Jake Rap et al. (+2 more)
We propose RHYME-XT, an operator-learning framework for surrogate modeling of spatiotemporal control systems governed by input-affine nonlinear partial integro-differential equations (PIDEs) with localized rhythmic behavior. RHYME-XT uses a Galerkin projection to approximate the infinite-dimensional PIDE on a learned finite-dimensional subspace with spatial basis functions parameterized by a neural network. This yields a projected system of ODEs driven by projected inputs. Instead of integrating this non-autonomous system, we directly learn its flow map using an architecture for learning flow functions, avoiding costly computations while obtaining a continuous-time and discretization-invariant representation. Experiments on a neural field PIDE show that RHYME-XT outperforms a state-of-the-art neural operator and is able to transfer knowledge effectively across models trained on different
datasets, through a fine-tuning process.
Read the full abstract ▾
All authors Marijn Ruiter, Miguel Aguiar, Jake Rap, Karl H. Johansson, Amritam Das
Machine Learning
DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
Alexander D. Goldie, Zilin Wang, Adrian Hayler et al. (+17 more)
Automating the development of machine learning
algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate
algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of
algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans billions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise
algorithm discovery
agents (ADAs). We present DiscoBench, a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, and demonstrate its use for ADA optimisation through scaling experiments for automated
prompt tuning. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen.
Read the full abstract ▾
All authors Alexander D. Goldie, Zilin Wang, Adrian Hayler, Deepak Nathani, Edan Toledo, Ken Thampiratwong, Aleksandra Kalisz, Michael Beukman, Hannah Erlebach, Alistair Letcher, Shashank Reddy, Clarisse Wibault, Theo Wolf, Charles O'Neill, Uljad Berdica, Nicholas Roberts, Saeed Rahmani, Roberta Raileanu, Shimon Whiteson, Jakob N. Foerster
Machine Learning
Physics-Aware Machine Learning for Seismic and Volcanic Signal Interpretation
William Thorossian
Modern seismic and volcanic monitoring is increasingly shaped by continuous, multi-sensor observations and by the need to extract actionable information from nonstationary, noisy wavefields. In this context, machine learning has moved from a research curiosity to a practical ingredient of processing chains for detection, phase picking, classification, denoising, and anomaly tracking. However, improved accuracy on a fixed
dataset is not sufficient for operational use. Models must remain reliable under domain shift (new stations, changing noise, evolving volcanic activity), provide uncertainty that supports decision-making, and connect their outputs to physically meaningful constraints. This paper surveys and organizes recent ML approaches for seismic and volcanic signal analysis, highlighting where classical signal processing provides indispensable inductive bias, how self-supervision and generative modeling can reduce dependence on labels, and which evaluation protocols best reflect transfer across regions. We conclude with open challenges for robust, interpretable, and maintainable AI-assisted monitoring.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
How do LLMs Compute Verbal Confidence
Dharshan Kumaran, Arthur Conmy, Federico Barbero et al. (+3 more)
Verbal confidence – prompting LLMs to state their confidence as a number or category – is widely used to extract uncertainty estimates from black-box models. However, how LLMs internally generate such scores remains unknown. We address two questions: first, when confidence is computed – just-in-time when requested, or automatically during answer generation and cached for later
retrieval; and second, what verbal confidence represents –
token log-probabilities, or a richer evaluation of answer quality? Focusing on Gemma 3 27B (across TriviaQA, BigMath, and MMLU), Qwen 2.5 7B, and the
reasoning model Magistral Small 24B, we provide convergent evidence for cached
retrieval. Activation steering, patching, noising, and swap experiments reveal that confidence representations emerge at answer-adjacent positions before appearing at the verbalization site.
Attention blocking pinpoints the information flow: confidence is gathered from answer
tokens, cached at the first post-answer position, then retrieved for output. Critically, linear probing and variance partitioning reveal that these cached representations explain substantial variance in verbal confidence beyond
token log-probabilities, suggesting a richer answer-quality evaluation rather than a simple fluency readout. These findings demonstrate that verbal confidence reflects automatic, sophisticated self-evaluation – not post-hoc reconstruction – with implications for understanding metacognition in LLMs and improving calibration.
Read the full abstract ▾
All authors Dharshan Kumaran, Arthur Conmy, Federico Barbero, Simon Osindero, Viorica Patraucean, Petar Veličković
Machine Learning · Cross-listed from eess.SY
Verification and Validation of Physics-Informed Surrogate Component Models for Dynamic Power-System Simulation
Petros Ellinas, Indrajit Chaudhuri, Johanna Vorwerk et al. (+1 more)
Physics-informed machine learning surrogates are increasingly explored to accelerate dynamic simulation of generators, converters, and other power grid components. The key question, however, is not only whether a surrogate matches a stand-alone component model on average, but whether it remains accurate after insertion into a differential-algebraic simulator, where the surrogate outputs enter the algebraic equations coupling the component to the rest of the system. This paper formulates that in-simulator use as a verification and validation (V&V) problem. A finite-horizon bound is derived that links allowable component-output error to algebraic-coupling sensitivity, dynamic error amplification, and the simulation horizon. Two complementary settings are then studied: model-based verification against a reference component solver, and data-based validation through conformal calibration of the component-output variables exchanged with the simulator. The framework is general, but the case study focuses on physics-informed neural-network surrogates of second-, fourth-, and sixth-order synchronous-machine models. Results show that good stand-alone surrogate accuracy does not by itself guarantee accurate in-simulator behavior, that the largest discrepancies concentrate in stressed operating regions, and that small equation residuals do not necessarily imply small state-trajectory errors.
Read the full abstract ▾
All authors Petros Ellinas, Indrajit Chaudhuri, Johanna Vorwerk, Spyros Chatzivasileiadis
Machine Learning · Cross-listed from cs.CL
Text-to-Stage: Spatial Layouts from Long-form Narratives
Jefferson Hernandez, Swarnadeep Saha, Chenxi Whitehouse et al. (+6 more)
In this work, we probe the ability of a
language model to demonstrate spatial
reasoning from unstructured text, mimicking human capabilities and automating a process that benefits many downstream media applications. Concretely, we study the narrative-to-play task: inferring stage-play layouts (scenes, speaker positions, movements, and room types) from text that lacks explicit spatial, positional, or relational cues. We then introduce a dramaturgy-inspired deterministic evaluation suite and, finally, a training and inference recipe that combines rejection SFT using Best-of-N sampling with RL from verifiable rewards via GRPO. Experiments on a text-only corpus of classical English literature demonstrate improvements over vanilla models across multiple metrics (character attribution, spatial plausibility, and movement economy), as well as
alignment with an LLM-as-a-judge and subjective human preferences.
Read the full abstract ▾
All authors Jefferson Hernandez, Swarnadeep Saha, Chenxi Whitehouse, Sanjeel Parekh, Calvin Murdock, Yuliang Li, W. Owen Brimijoin, Vamsi Krishna Ithapu, Ishwarya Ananthabhotla
Machine Learning
Symmetry-Reduced Physics-Informed Learning of Tensegrity Dynamics
Jing Qin and Muhao Chen
Tensegrity structures possess intrinsic
geometric symmetries that govern their dynamic behavior. However, most existing physics-informed neural network (PINN) approaches for tensegrity dynamics do not explicitly exploit these
symmetries, leading to high computational complexity and unstable optimization. In this work, we propose a
symmetry-reduced physics-informed neural network (SymPINN) framework that embeds group-theory-based
symmetry directly into both the solution expression and the neural network architecture to predict tensegrity dynamics. By decomposing nodes into
symmetry orbits and representing free nodal coordinates using a
symmetry basis, the proposed method constructs a reduced coordinate representation that preserves
geometric symmetry of the structure. The full coordinates are then recovered via
symmetry transformations of the reduced solution learned by the network, ensuring that the predicted configurations automatically satisfy the
symmetry constraints. In this framework, equivariance is enforced through orbit-based coordinate generation,
symmetry-consistent message passing, and physics residual constraints. In addition, SymPINN improves training effectiveness by encoding initial conditions as hard constraints, incorporating
Fourier feature encoding to enhance the representation of dynamic motions, and employing a two-stage optimization strategy. Extensive numerical experiments on symmetric T-bars and lander structures demonstrate significantly improved prediction accuracy and computational efficiency compared to standard physics-informed models, indicating the great potential of
symmetry-aware learning for structure-preserving modeling of tensegrity dynamics.
Read the full abstract ▾
Machine Learning
Discovering Decoupled Functional Modules in Large Language Models
Yanke Yu, Jin Li, Ying Sun et al. (+3 more)
Understanding the internal functional organization of Large
Language Models (LLMs) is crucial for improving their trustworthiness and performance. However, how LLMs organize different functions into modules remains highly unexplored. To bridge this gap, we formulate a functional module discovery problem and propose an Unsupervised LLM Cross-layer MOdule Discovery (ULCMOD) framework that simultaneously disentangles the large set of
neurons in the entire LLM into modules while discovering the topics of input samples related to these modules. Our framework introduces a novel objective function and an efficient Iterative Decoupling (IterD)
algorithm. Extensive experiments show that our method discovers high-quality, disentangled modules that capture more meaningful semantic information and achieve superior performance in various downstream tasks. Moreover, our qualitative analysis reveals that the discovered modules show semantic coherence, correspond to interpretable specializations, and a clear spatial and hierarchical organization within the LLM. Our work provides a novel tool for interpreting the functional modules of LLMs, filling a critical blank in LLM's interpretability research.
Read the full abstract ▾
All authors Yanke Yu, Jin Li, Ying Sun, Ping Li, Zhefeng Wang, Yi Zheng
Machine Learning
Federated Distributional Reinforcement Learning with Distributional Critic Regularization
David Millard, Cecilia Alm, Rashid Ali et al. (+2 more)
Federated reinforcement learning typically aggregates value functions or policies by parameter averaging, which emphasizes expected return and can obscure statistical multimodality and tail behavior that matter in safety-critical settings. We formalize
federated distributional reinforcement learning (FedDistRL), where clients parametrize
quantile value function critics and federate these networks only. We also propose TR-FedDistRL, which builds a per client, risk-aware Wasserstein barycenter over a temporal buffer. This local barycenter provides a reference region to constrain the parameter averaged critic, ensuring necessary distributional information is not averaged out during the federation process. The distributional trust region is implemented as a shrink-squash step around this reference. Under fixed-policy evaluation, the feasibility map is nonexpansive and the update is contractive in a probe-set Wasserstein metric under evaluation. Experiments on a bandit, multi-
agent gridworld, and continuous highway environment show reduced mean-smearing, improved safety proxies (catastrophe/accident rate), and lower critic/policy drift versus mean-oriented and non-
federated baselines.
Read the full abstract ▾
All authors David Millard, Cecilia Alm, Rashid Ali, Pengcheng Shi, Ali Baheri
Machine Learning · Cross-listed from cs.CV
ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
Dmitriy Rivkin, Parker Ewen, Lili Gao et al. (+5 more)
Recent
video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models in the pixel domain incurs prohibitive
memory costs, as activations accumulate across the entire
video sequence. This fundamental limitation also makes fine-tuning these models with pixel-wise losses computationally intractable for long or high-resolution
videos. This paper introduces ChopGrad, a truncated backpropagation scheme for
video decoding, limiting gradient computation to local frame windows while maintaining global consistency. We provide a theoretical analysis of this approximation and show that it enables efficient fine-tuning with frame-wise losses. ChopGrad reduces training
memory from scaling linearly with the number of
video frames (full backpropagation) to constant
memory, and compares favorably to existing state-of-the-art
video diffusion models across a suite of conditional
video generation tasks with pixel-wise losses, including
video super-resolution,
video inpainting,
video enhancement of neural-rendered scenes, and controlled driving
video generation.
Read the full abstract ▾
All authors Dmitriy Rivkin, Parker Ewen, Lili Gao, Julian Ost, Stefanie Walz, Rasika Kangutkar, Mario Bijelic, Felix Heide
Machine Learning
Dropout Robustness and Cognitive Profiling of Transformer Models via Stochastic Inference
Antônio Junior Alves Caiado and Michael Hahsler
Transformer-based
language models are widely deployed for
reasoning, yet their behavior under inference-time stochasticity remains underexplored. While dropout is common during training, its inference-time effects via Monte Carlo sampling lack systematic evaluation across architectures, limiting understanding of model reliability in uncertainty-aware applications. This work analyzes dropout-induced variability across 19
transformer models using MC Dropout with 100
stochastic forward passes per sample. Dropout robustness is defined as maintaining high accuracy and stable predictions under
stochastic inference, measured by standard deviation of per-run accuracies. A cognitive decomposition framework disentangles performance into
memory and
reasoning components. Experiments span five dropout configurations yielding 95 unique evaluations on 1,000 samples. Results reveal substantial architectural variation. Smaller models demonstrate perfect prediction stability while medium-sized models exhibit notable
volatility. Mid-sized models achieve the best overall performance; larger models excel at
memory tasks. Critically, 53% of models suffer severe accuracy degradation under baseline MC Dropout, with task-specialized models losing up to 24 percentage points, indicating unsuitability for uncertainty quantification in these architectures. Asymmetric effects emerge: high dropout reduces
memory accuracy by 27 percentage points while
reasoning degrades only 1 point, suggesting
memory tasks rely on stable representations that dropout disrupts. 84% of models demonstrate
memory-biased performance. This provides the first comprehensive MC Dropout
benchmark for
transformers, revealing dropout robustness is architecture-dependent and uncorrelated with scale. The cognitive profiling framework offers actionable guidance for model selection in uncertainty-aware applications.
Read the full abstract ▾
Machine Learning
RangeAD: Fast On-Model Anomaly Detection
Luca Hinkamp, Simon Klüttermann and Emmanuel Müller
In practice, machine learning methods commonly require anomaly detection (AD) to filter inputs or detect distributional shifts. Typically, this is implemented by running a separate AD model alongside the primary model. However, this separation ignores the fact that the primary model already encodes substantial information about the target distribution. In this paper, we introduce On-Model AD, a setting for anomaly detection that explicitly leverages access to a related machine learning model. Within this setting, we propose RangeAD, an
algorithm that utilizes
neuron-wise output ranges derived from the primary model. RangeAD achieves superior performance even on high-dimensional tasks while incurring substantially lower inference costs. Our results demonstrate the potential of the On-Model AD setting as a practical framework for efficient anomaly detection.
Read the full abstract ▾