Machine Learning
The Stationarity Bias: Stratified Stress-Testing for Time-Series Imputation in Regulated Dynamical Systems
Amirreza Dolatpour Fathkouhi, Alireza Namazi and Heman Shakeri
Time-series imputation
benchmarks employ uniform random masking and shape-agnostic metrics (MSE, RMSE), implicitly weighting evaluation by regime prevalence. In systems with a dominant attractor – homeostatic physiology, nominal industrial operation, stable network traffic – this creates a systematic Stationarity Bias: simple methods appear superior because the
benchmark predominantly samples the easy, low-
entropy regime where they trivially succeed. We formalize this bias and propose a Stratified Stress-Test that partitions evaluation into Stationary and Transient regimes. Using Continuous Glucose Monitoring (CGM) as a testbed – chosen for its rigorous ground-truth forcing functions (meals, insulin) that enable precise regime identification – we establish three findings with broad implications:(i)~Stationary Efficiency: Linear interpolation achieves state-of-the-art reconstruction during stable intervals, confirming that complex architectures are computationally wasteful in low-
entropy regimes.(ii)~Transient Fidelity: During critical transients (post-prandial peaks, hypoglycemic events), linear methods exhibit drastically degraded morphological fidelity (DTW), disproportionate to their RMSE – a phenomenon we term the RMSE Mirage, where low pointwise error masks the destruction of signal shape.(iii)~Regime-Conditional Model Selection: Deep learning models preserve both pointwise accuracy and morphological integrity during transients, making them essential for safety-critical downstream tasks. We further derive empirical missingness distributions from clinical trials and impose them on complete training data, preventing models from exploiting unrealistically clean observations and encouraging robustness under real-world missingness. This framework generalizes to any regulated system where routine stationarity dominates critical transients.
Read the full abstract ▾
Machine Learning
Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
Erkan Turan, Gaspard Abel, Maysam Behmanesh et al. (+2 more)
Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a bifurcation theory perspective, characterizing oversmoothing as convergence to a stable “homogeneous fixed point.” Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous patterns that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-aware initialization and showing its utility in real
benchmark experiments.
Read the full abstract ▾
All authors Erkan Turan, Gaspard Abel, Maysam Behmanesh, Emery Pierson, Maks Ovsjanikov
Machine Learning · Cross-listed from physics.comp-ph
Neural-POD: A Plug-and-Play Neural Operator Framework for Infinite-Dimensional Functional Nonlinear Proper Orthogonal Decomposition
Changhong Mou, Binghang Lu and Guang Lin
AI for science (AI4Science) models often suffer from discretization: learned representations remain tied to the training grid, limiting transfer across resolutions, solvers and applications. We introduce Neural Proper Orthogonal Decomposition (Neural-POD), a plug-and-play neural operator that learns nonlinear, orthogonal basis functions directly in function space and can be integrated in both projection-based reduced order models and operator-learning frameworks such as DeepONet. Neural-POD replaces SVD-derived, resolution-dependent linear modes with continuous, resolution-invariant bases learned via sequential residual minimization, analogous to Gram-Schmidt orthogonalization. The framework supports training under task-specific norms (e.g., $L^2$, $L^1$), improves out-of-distribution generalization to unseen parameter regimes, and captures nonlinear structure in complex systems. Because the learned bases are interpretable and reusable, Neural-POD serves as a general representation module for AI4Science workflows. We demonstrate Neural-POD on Burgers' and Navier-Stokes equations.
Read the full abstract ▾
Machine Learning
DNN-Enabled Multi-User Beamforming for Throughput Maximization under Adjustable Fairness
Kaifeng Lu, Markus Rupp and Stefan Schwarz
Ensuring user
fairness in wireless communications is a fundamental challenge, as balancing the trade-off between
fairness and sum rate leads to a non-convex, multi-objective optimization whose complexity grows with network scale. To alleviate this conflict, we propose an optimization-based unsupervised learning approach based on the wireless
transformer (WiT) architecture that learns from channel state information (CSI) features. We reformulate the trade-off by combining the sum rate and
fairness objectives through a Lagrangian multiplier, which is updated automatically via a dual-ascent
algorithm. This mechanism allows for a controllable
fairness constraint while simultaneously maximizing the sum rate, effectively realizing a trace on the Pareto front between two conflicting objectives. Our findings show that the proposed approach offers a flexible solution for managing the trade-off optimization under prescribed
fairness.
Read the full abstract ▾
Machine Learning
Symbolic recovery of PDEs from measurement data
Erion Morina, Philipp Scholl and Martin Holler
Models based on partial differential equations (PDEs) are powerful for describing a wide range of complex phenomena in the natural sciences. Accurately identifying the PDE model, which represents the underlying physical law, is essential for a proper understanding of the problem. This reconstruction typically relies on indirect and noisy measurements of the system's state and, without specifically tailored methods, rarely yields symbolic expressions, thereby limiting interpretability. In this work, we address this limitation by considering neural network architectures based on rational functions for the symbolic representation of physical laws. These networks combine the approximation power of rational functions with the flexibility to represent arithmetic operations, and generalize ParFam and EQL-type architectures used in symbolic regression for physical law learning. We further establish regularity results for these symbolic networks. Our main contribution is a reconstruction result showing that, if there exists an admissible physical law that is expressible within the symbolic network architecture, then in the limit of noiseless and complete measurements, symbolic networks recover a physical law within the PDE model that is representable by the architecture. Moreover, the recovered law corresponds to a regularization-minimizing parameterization, promoting interpretability and sparsity in case of $L^1$-regularization. Under an additional identifiability condition, the unique true physical law is recovered. These reconstruction and regularity results are derived at the continuous level prior to discretization due to a formulation in function space. Empirical results using the ParFam architecture are consistent with the theoretical findings and suggest the feasibility of reconstructing interpretable physical laws in practice.
Read the full abstract ▾
Machine Learning
Certified Per-Instance Unlearning Using Individual Sensitivity Bounds
Hanna Benarroch, Jamal Atif and Olivier Cappé
Certified machine unlearning can be achieved through noise injection, with differential
privacy guarantees obtained by calibrating the noise to worst-case sensitivity. Yet, the influence of individual training points is highly heterogeneous, making such uniform calibration unnecessarily conservative for most deletion requests. We consider certified per-instance unlearning, a data-dependent notion of certified deletion in which the unlearning mechanism and its certificate can adapt to the point being removed. Our framework turns point-specific sensitivity information into formal unlearning guarantees, allowing the amount of noise to reflect the difficulty of the deletion request rather than the worst case. When training is deterministic, we derive explicit per-instance unlearning certificates. For noisy gradient descent, in the case of ridge regression, similar certificates are obtained through high-probability sensitivity bounds. Experiments on NYU-Depth with frozen DINOv2 features show substantial variation in deletion difficulty across points, and demonstrate that per-instance calibration can substantially improve utility over uniform worst-case calibration.
Read the full abstract ▾
Machine Learning
Multi-Objective Coverage via Constraint Active Search
Zakaria Shams Siam, Xuefeng Liu and Chong Liu
In this paper, we formulate the new multi-objective coverage (MOC) problem where our goal is to identify a small set of representative samples whose predicted outcomes broadly cover the feasible multi-objective space. This problem is of great importance in many critical real-world applications, e.g., drug discovery and materials design, as this representative set can be evaluated much faster than the whole feasible set, thus significantly accelerating the scientific discovery process. Existing works cannot be directly applied as they either focus on sample space coverage or multi-objective optimization that targets the Pareto front. However, chemically diverse samples often yield identical objective profiles, and safety constraints are usually defined on the objectives. To solve this MOC problem, we propose a novel search
algorithm, MOC-CAS, which employs an upper confidence bound-based acquisition function to select optimistic samples guided by Gaussian process posterior predictions. For enabling efficient optimization, we develop a smoothed relaxation of the hard feasibility test and derive an approximate optimizer. Compared to the competitive baselines, we show that our MOC-CAS empirically achieves superior performances across large-scale
protein-target
datasets for SARS-CoV-2 and cancer, each assessed on five objectives derived from SMILES-based features.
Read the full abstract ▾
Machine Learning
A unified theory of feature learning in RNNs and DNNs
Jan P. Bauer, Kirsten Fischer, Moritz Helias et al. (+1 more)
Recurrent and deep neural networks (RNNs/DNNs) are cornerstone architectures in machine learning. Remarkably, RNNs differ from DNNs only by weight sharing, as can be shown through unrolling in time. How does this structural similarity fit with the distinct functional properties these networks exhibit? To address this question, we here develop a unified mean-field theory for RNNs and DNNs in terms of representational kernels, describing fully trained networks in the feature learning ($\mu$P) regime. This theory casts training as
Bayesian inference over sequences and patterns, directly revealing the functional implications induced by the RNNs' weight sharing. In DNN-typical tasks, we identify a phase transition when the learning signal overcomes the noise due to randomness in the weights: below this threshold, RNNs and DNNs behave identically; above it, only RNNs develop correlated representations across timesteps. For sequential tasks, the RNNs' weight sharing furthermore induces an inductive bias that aids generalization by interpolating unsupervised time steps. Overall, our theory offers a way to connect architectural structure to functional biases.
Read the full abstract ▾
All authors Jan P. Bauer, Kirsten Fischer, Moritz Helias, Agostina Palmigiano
Machine Learning · Cross-listed from physics.flu-dyn
Uni-Flow: a unified autoregressive-diffusion model for complex multiscale flows
Xiao Xue, Tianyue Yang, Mingyang Gao et al. (+7 more)
Spatiotemporal flows govern diverse phenomena across physics, biology, and engineering, yet modelling their multiscale dynamics remains a central challenge. Despite major advances in physics-informed machine learning, existing approaches struggle to simultaneously maintain long-term temporal
evolution and resolve fine-scale structure across chaotic, turbulent, and physiological regimes. Here, we introduce Uni-Flow, a unified autoregressive-
diffusion framework that explicitly separates temporal
evolution from spatial refinement for modelling complex dynamical systems. The autoregressive component learns low-resolution latent dynamics that preserve large-scale structure and ensure stable long-horizon rollouts, while the
diffusion component reconstructs high-resolution physical fields, recovering fine-scale features in a small number of denoising steps. We validate Uni-Flow across canonical
benchmarks, including two-dimensional Kolmogorov flow, three-dimensional turbulent channel inflow generation with a
quantum-informed autoregressive prior, and patient-specific simulations of aortic coarctation derived from high-fidelity
lattice Boltzmann hemodynamic solvers. In the cardiovascular setting, Uni-Flow enables task-level faster than real-time inference of pulsatile hemodynamics, reconstructing high-resolution pressure fields over physiologically relevant time horizons in seconds rather than hours. By transforming high-fidelity hemodynamic simulation from an offline, HPC-bound process into a deployable surrogate, Uni-Flow establishes a pathway to faster-than-real-time modelling of complex multiscale flows, with broad implications for scientific machine learning in flow physics.
Read the full abstract ▾
All authors Xiao Xue, Tianyue Yang, Mingyang Gao, Leyu Pan, Maida Wang, Kewei Zhu, Shuo Wang, Jiuling Li, Marco F.P. ten Eikelder, Peter V. Coveney
Machine Learning
Uniform error bounds for quantized dynamical models
Abdelkader Metakalard, Fabien Lauer, Kevin Colin et al. (+1 more)
This paper provides statistical guarantees on the accuracy of dynamical models learned from dependent data sequences. Specifically, we develop uniform error bounds that apply to quantized models and imperfect optimization
algorithms commonly used in practical contexts for system identification, and in particular hybrid system identification. Two families of bounds are obtained: slow-rate bounds via a block decomposition and fast-rate, variance-adaptive, bounds via a novel spaced-point strategy. The bounds scale with the number of bits required to encode the model and thus translate hardware constraints into interpretable statistical complexities.
Read the full abstract ▾
All authors Abdelkader Metakalard, Fabien Lauer, Kevin Colin, Marion Gilson
Machine Learning
Neural Network-Based Parameter Estimation of a Labour Market Agent-Based Model
M Lopes Alves, Joel Dyer, Doyne Farmer et al. (+2 more)
Agent-based modelling (ABM) is a widespread approach to simulate complex systems. Advancements in computational processing and storage have facilitated the adoption of ABMs across many fields; however, ABMs face challenges that limit their use as decision-support tools. A significant issue is parameter estimation in large-scale ABMs, particularly due to computational constraints on exploring the parameter space. This study evaluates a state-of-the-art simulation-based inference (SBI) framework that uses neural networks (NN) for parameter estimation. This framework is applied to an established labour
market ABM based on job transition networks. The ABM is initiated with synthetic
datasets and the real U.S. labour
market. Next, we compare the effectiveness of summary statistics derived from a list of statistical measures with that learned by an embedded NN. The results demonstrate that the NN-based approach recovers the original parameters when evaluating posterior distributions across various
dataset scales and improves efficiency compared to traditional
Bayesian methods.
Read the full abstract ▾
All authors M Lopes Alves, Joel Dyer, Doyne Farmer, Michael Wooldridge, Anisoara Calinescu
Machine Learning
Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment
Davide Casnici, Martin Lefebvre, Justin Dauwels et al. (+1 more)
Predictive coding (PC) is a biologically inspired
algorithm for training neural networks that relies only on local updates, allowing parallel learning across layers. However, practical implementations face two key limitations: error signals must still propagate from the output to early layers through multiple inference-phase steps, and feedback decays exponentially during this process, leading to vanishing updates in early layers. We propose direct Kolen-Pollack predictive coding (DKP-PC), which simultaneously addresses both feedback delay and exponential decay, yielding a more efficient and scalable variant of PC while preserving update locality. Leveraging direct feedback
alignment and direct Kolen-Pollack
algorithms, DKP-PC introduces learnable feedback connections from the output layer to all hidden layers, establishing a direct pathway for error transmission. This yields an
algorithm that reduces the theoretical error propagation time complexity from O(L), with L being the network depth, to O(1), removing depth-dependent delay in error signals. Moreover, empirical results demonstrate that DKP-PC achieves performance at least comparable to, and often exceeding, that of standard PC, while offering improved latency and computational performance, supporting its potential for custom hardware-efficient implementations.
Read the full abstract ▾
All authors Davide Casnici, Martin Lefebvre, Justin Dauwels, Charlotte Frenkel
Machine Learning · Cross-listed from stat.ME
Scenario Approach with Post-Design Certification of User-Specified Properties
Algo Carè, Marco C. Campi and Simone Garatti
The scenario approach is an established data-driven design framework that comes equipped with a powerful theory linking design complexity to generalization properties. In this approach, data are simultaneously used both for design and for certifying the design's reliability, without resorting to a separate test
dataset. This paper takes a step further by guaranteeing additional properties, useful in post-design usage but not considered during the design phase. To this end, we introduce a two-level framework of appropriateness: baseline appropriateness, which guides the design process, and post-design appropriateness, which serves as a criterion for a posteriori evaluation. We provide distribution-free upper bounds on the risk of failing to meet the post-design appropriateness; these bounds are computable without using any additional test data. Under additional assumptions, lower bounds are also derived. As part of an effort to demonstrate the usefulness of the proposed methodology, the paper presents two practical examples in H2 and pole-placement problems. Moreover, a method is provided to infer comprehensive distributional knowledge of relevant performance indexes from the available
dataset.
Read the full abstract ▾
Machine Learning
1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization
Sohir Maskey, Constantin Eichenberg, Johannes Messner et al. (+1 more)
Quantization-aware training (QAT) is an effective method to drastically reduce the
memory footprint of LLMs while keeping performance degradation at an acceptable level. However, the optimal choice of quantization format and bit-width presents a challenge in practice. The full design space of quantization is not fully explored in the context of QAT, and the precise trade-off between quantization and downstream performance is poorly understood, as comparisons often rely solely on perplexity-based evaluations. In this work, we address these shortcomings with an empirical study of QAT in the low-bit regime. We show that k-means based weight quantization outperforms integer formats and can be implemented efficiently on standard hardware. Furthermore, we find that, under a fixed inference
memory budget, the best performance on generative downstream tasks is achieved with $1$-bit quantized weights.
Read the full abstract ▾
All authors Sohir Maskey, Constantin Eichenberg, Johannes Messner, Douglas Orr
Machine Learning · Cross-listed from cs.SE
Latent Regularization in Generative Test Input Generation
Giorgi Merabishvili, Oliver Weißl and Andrea Stocco
This study investigates the impact of regularization of latent spaces through truncation on the quality of generated test inputs for deep learning classifiers. We evaluate this effect using style-based GANs, a state-of-the-art generative approach, and assess quality along three dimensions: validity, diversity, and fault detection. We evaluate our approach on the boundary testing of deep learning image classifiers across three
datasets, MNIST, Fashion MNIST, and CIFAR-10. We compare two truncation strategies: latent code mixing with binary search optimization and random latent truncation for generative exploration. Our experiments show that the latent code-mixing approach yields a higher fault detection rate than random truncation, while also improving both diversity and validity.
Read the full abstract ▾
Machine Learning
CEPAE: Conditional Entropy-Penalized Autoencoders for Time Series Counterfactuals
Tomàs Garriga, Gerard Sanz, Eduard Serrahima de Cambra et al. (+1 more)
The ability to accurately perform counterfactual inference on time series is crucial for decision-making in fields like finance, healthcare, and marketing, as it allows us to understand the impact of events or treatments on outcomes over time. In this paper, we introduce a new counterfactual inference approach tailored to time series data impacted by
market events, which is motivated by an industrial application. Utilizing the abduction-action-prediction procedure and the Structural Causal Model framework, we first adapt methods based on variational autoencoders and
adversarial autoencoders, both previously used in counterfactual literature although not in time series settings. Then, we present the Conditional
Entropy-Penalized Autoencoder (CEPAE), a novel autoencoder-based approach for counterfactual inference, which employs an
entropy penalization loss over the latent space to encourage disentangled data representations. We validate our approach both theoretically and experimentally on synthetic, semi-synthetic, and real-world
datasets, showing that CEPAE generally outperforms the other approaches in the evaluated metrics.
Read the full abstract ▾
All authors Tomàs Garriga, Gerard Sanz, Eduard Serrahima de Cambra, Axel Brando
Machine Learning · Cross-listed from stat.ML
Functional Central Limit Theorem for Stochastic Gradient Descent
Kessang Flamand and Victor-Emmanuel Brunel
We study the asymptotic shape of the trajectory of the
stochastic gradient descent
algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit
theorem for the properly rescaled trajectory. Our result characterizes the long-term fluctuations of the
algorithm around the minimizer by providing a
diffusion limit for the trajectory. In contrast with classical central limit
theorems for the last iterate or Polyak-Ruppert averages, this functional result captures the temporal structure of the fluctuations and applies to non-smooth settings such as robust location estimation, including the
geometric median.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Quantifying construct validity in large language model evaluations
Ryan Othniel Kearns
The LLM community often reports
benchmark results as if they are synonymous with general model capabilities. However,
benchmarks can have problems that distort performance, like test set contamination and annotator error. How can we know that a
benchmark is a reliable indicator of some capability that we want to measure? This question concerns the construct validity of LLM
benchmarks, and it requires separating
benchmark results from capabilities when we model and predict LLM performance. Both social scientists and computer scientists propose formal models - latent factor models and scaling laws - for identifying the capabilities underlying
benchmark scores. However, neither technique is satisfactory for construct validity. Latent factor models ignore scaling laws, and as a result, the capabilities they extract often proxy model size. Scaling laws ignore measurement error, and as a result, the capabilities they extract are both uninterpretable and overfit to the observed
benchmarks. This thesis presents the structured capabilities model, the first model to extract interpretable and generalisable capabilities from a large collection of LLM
benchmark results. I fit this model and its two alternatives on a large sample of results from the OpenLLM Leaderboard. Structured capabilities outperform latent factor models on parsimonious fit indices, and exhibit better out-of-distribution
benchmark prediction than scaling laws. These improvements are possible because neither existing approach separates model scale from capabilities in the appropriate way. Model scale should inform capabilities, as in scaling laws, and these capabilities should inform observed results up to measurement error, as in latent factor models. In combining these two insights, structured capabilities demonstrate better explanatory and predictive power for quantifying construct validity in LLM evaluations.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
Ziyu Zhao, Tong Zhu, Zhi Zhang et al. (+6 more)
Mixture-of-Experts (MoE) effectively scales model capacity while preserving computational efficiency through sparse expert activation. However, training high-quality MoEs from scratch is prohibitively expensive. A promising alternative is to convert pretrained dense models into sparse MoEs. Existing dense-to-MoE methods fall into two categories: dynamic structural
pruning that converts dense models into MoE architectures with moderate sparsity to balance performance and inference efficiency, and downcycling approaches that use pretrained dense models to initialize highly sparse MoE architectures. However, existing methods break the intrinsic activation patterns within dense models, leading to suboptimal expert construction. In this work, we argue that the Gated Linear Unit (GLU) mechanism provides a natural blueprint for dense-to-MoE conversion. We show that the fine-grained neural-wise activation patterns of GLU reveal a coarse-grained structure, uncovering an inherent MoE architecture composed of consistently activated universal
neurons and dynamically activated specialized
neurons. Leveraging this discovery, we introduce ExpertWeaver, a training-free framework that partitions
neurons according to their activation patterns and constructs shared experts and specialized routed experts with layer-adaptive configurations. Our experiments demonstrate that ExpertWeaver significantly outperforms existing methods, both as a training-free dynamic structural
pruning technique and as a downcycling strategy for superior MoE initialization.
Read the full abstract ▾
All authors Ziyu Zhao, Tong Zhu, Zhi Zhang, Tiantian Fan, Jinluan Yang, Kun Kuang, Zhongyu Wei, Fei Wu, Yu Cheng
Machine Learning
The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
Mohammad Taufeeque, Stefan Heimersheim, Adam Gleave et al. (+1 more)
Training against white-box deception detectors has been proposed as a way to make AI systems honest. However, such training risks models learning to obfuscate their deception to evade the detector. Prior work has studied obfuscation only in artificial settings where models were directly rewarded for harmful output. We construct a realistic coding environment where reward hacking via hardcoding test cases naturally occurs, and show that obfuscation emerges in this setting. We introduce a taxonomy of possible outcomes when training against a deception detector. The model either remains honest, or becomes deceptive via two possible obfuscation strategies. (i) Obfuscated activations: the model outputs deceptive text while modifying its internal representations to no longer trigger the detector. (ii) Obfuscated policy: the model outputs deceptive text that evades the detector, typically by including a justification for the reward hack. Empirically, obfuscated activations arise from representation drift during RL, with or without a detector penalty. The detector penalty only incentivizes obfuscated policies; we theoretically show this is expected for policy gradient methods. Sufficiently high KL regularization and detector penalty can yield honest policies, establishing white-box deception detectors as viable training signals for tasks prone to reward hacking.
Read the full abstract ▾
All authors Mohammad Taufeeque, Stefan Heimersheim, Adam Gleave, Chris Cundy
Machine Learning
On the Geometric Coherence of Global Aggregation in Federated Graph Neural Networks
Chethana Prasad Kabgere and Shylaja SS
Federated learning over graph-structured data exposes a fundamental mismatch between standard aggregation mechanisms and the operator nature of graph neural networks (GNNs). While
federated optimization treats model parameters as elements of a shared Euclidean space, GNN parameters induce graph-dependent message-passing operators whose semantics depend on underlying
topology. Under structurally and distributionally heterogeneous client graph distributions, local updates correspond to perturbations of distinct operator
manifolds. Linear aggregation of such updates mixes geometrically incompatible directions, producing global models that converge numerically yet exhibit degraded relational behavior. We formalize this phenomenon as a
geometric failure of global aggregation in cross-domain
federated GNNs, characterized by destructive interference between operator perturbations and loss of coherence in message-passing dynamics. This degradation is not captured by conventional metrics such as loss or accuracy, as models may retain predictive performance while losing structural sensitivity. To address this, we propose GGRS (Global
Geometric Reference Structure), a server-side aggregation framework operating on a data-free proxy of operator perturbations. GGRS enforces
geometric admissibility via directional
alignment, subspace compatibility, and sensitivity control, preserving the structure of the induced message-passing operator.
Read the full abstract ▾
Machine Learning
Approximation Theory for Lipschitz Continuous Transformers
Takashi Furuya, Davide Murari and Carola-Bibiane Schönlieb
Stability and robustness are critical for deploying
Transformers in safety-sensitive settings. A principled way to enforce such behavior is to constrain the model's Lipschitz constant. However, approximation-theoretic guarantees for architectures that explicitly preserve Lipschitz continuity have yet to be established. In this work, we bridge this gap by introducing a class of gradient-descent-type in-context
Transformers that are Lipschitz-continuous by construction. We realize both MLP and
attention blocks as explicit Euler steps of negative gradient flows, ensuring inherent stability without sacrificing expressivity. We prove a universal approximation
theorem for this class within a Lipschitz-constrained function space. Crucially, our analysis adopts a measure-theoretic formalism, interpreting
Transformers as operators on probability measures, to yield approximation guarantees independent of
token count. These results provide a rigorous theoretical foundation for the design of robust, Lipschitz continuous
Transformer architectures.
Read the full abstract ▾
Machine Learning
ExLipBaB: Exact Lipschitz Constant Computation for Piecewise Linear Neural Networks
Tom A. Splittgerber
It has been shown that a neural network's Lipschitz constant can be leveraged to derive robustness guarantees, to improve generalizability via regularization or even to construct invertible networks. Therefore, a number of methods varying in the tightness of their bounds and their computational cost have been developed to approximate the Lipschitz constant for different classes of networks. However, comparatively little research exists on methods for exact computation, which has been shown to be NP-hard. Nonetheless, there are applications where one might readily accept the computational cost of an exact method. These applications could include the benchmarking of new methods or the computation of robustness guarantees for small models on sensitive data. Unfortunately, existing exact
algorithms restrict themselves to only ReLU-activated networks, which are known to come with severe downsides in the context of Lipschitz-constrained networks. We therefore propose a generalization of the LipBaB
algorithm to compute exact Lipschitz constants for arbitrary piecewise linear neural networks and $p$-norms. With our method, networks may contain traditional activations like ReLU or LeakyReLU, activations like GroupSort or the related MinMax and FullSort, which have been of increasing interest in the context of Lipschitz constrained networks, or even other piecewise linear functions like MaxPool.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction
Amartyaveer, Murali Kadambi, Chandra Mohan Sharma et al. (+2 more)
In this study, we have presented a novel approach to predict the Short-Time Objective Intelligibility (STOI) metric using a bottleneck
transformer architecture. Traditional methods for calculating STOI typically requires clean reference
speech, which limits their applicability in the real world. To address this, numerous deep learning-based nonintrusive
speech assessment models have garnered significant interest. Many studies have achieved commendable performance, but there is room for further improvement. We propose the use of bottleneck
transformer, incorporating convolution blocks for learning frame-level features and a multi-head self-
attention (MHSA) layer to aggregate the information. These components enable the
transformer to focus on the key aspects of the input data. Our model has shown higher correlation and lower mean squared error for both seen and unseen scenarios compared to the state-of-the-art model using self-supervised learning (SSL) and spectral features as inputs.
Read the full abstract ▾
All authors Amartyaveer, Murali Kadambi, Chandra Mohan Sharma, Anupam Mondal, Prasanta Kumar Ghosh
Machine Learning
LLM-as-Judge on a Budget
Aadirupa Saha, Aniket Wagde and Branislav Kveton
LLM-as-a-judge has emerged as a cornerstone technique for evaluating large
language models by leveraging LLM
reasoning to score
prompt-response pairs. Since LLM judgments are
stochastic, practitioners commonly query each pair multiple times to estimate mean scores accurately. This raises a critical challenge: given a fixed computational budget $B$, how to optimally allocate queries across $K$
prompt-response pairs to minimize estimation error? We present a principled variance-adaptive approach leveraging multi-armed bandit theory and concentration inequalities. Our method dynamically allocates queries based on estimated score variances, concentrating resources where uncertainty is highest. Further, our
algorithm is shown to achieve a worst-case score-estimation error of $\tilde{O}\left(\sqrt{\frac{\sum_{i=1}^K \sigma_i^2}{B}}\right)$, $\sigma_i^2$ being the unknown score variance for pair $i \in [K]$ with near-optimal budget allocation. Experiments on Summarize-From-Feedback and HelpSteer2 demonstrate that our method significantly outperforms uniform allocation, reducing worst-case estimation error while maintaining identical budgets. Our work establishes a theoretical foundation for efficient LLM evaluation with practical implications for AI safety, model
alignment, and automated assessment at scale.
Read the full abstract ▾
Machine Learning
Evaluating Federated Learning for Cross-Country Mood Inference from Smartphone Sensing Data
Sharmad Kalpande, Saurabh Shirke and Haroon R. Lone
Mood instability is a key behavioral indicator of mental health, yet traditional assessments rely on infrequent and retrospective reports that fail to capture its continuous nature. Smartphone-based mobile sensing enables passive, in-the-wild mood inference from everyday behaviors; however, deploying such systems at scale remains challenging due to
privacy constraints, uneven sensing availability, and substantial variability in behavioral patterns. In this work, we study mood inference using smartphone sensing data in a cross-country
federated learning setting, where each country participates as an independent client while retaining local data. We introduce FedFAP, a feature-aware personalized
federated framework designed to accommodate heterogeneous sensing modalities across regions. Evaluations across geographically and culturally diverse populations show that FedFAP achieves an AUROC of 0.744, outperforming both centralized approaches and existing personalized
federated baselines. Beyond inference, our results offer design insights for mood-aware systems, demonstrating how population-aware personalization and
privacy-preserving learning can enable scalable and mood-aware mobile sensing technologies.
Read the full abstract ▾
Machine Learning
POP: Prior-Fitted First-Order Optimization Policies
Jan Kobiolka, Christian Frey, Gresa Shala et al. (+3 more)
Gradient-based optimizers are highly sensitive to design choices in their adaptive learning rate mechanisms. To address this limitation, we introduce POP, a meta-learned Reinforcement Learning (RL) policy that predicts adaptive learning rates for gradient descent, conditioned on the contextual information provided in the optimization trajectory. Our method introduces a novel RL reward formulation, a new function-scaling strategy for in-distribution generalization, and a novel prior that is used to sample millions of synthetic optimization problems. We evaluate POP on an established
benchmark including 43 optimization functions of various complexity, where it significantly outperforms gradient-based methods. Our evaluation demonstrates strong generalization capabilities without task-specific tuning.
Read the full abstract ▾
All authors Jan Kobiolka, Christian Frey, Gresa Shala, Arlind Kadra, Erind Bedalli, Josif Grabocka
Machine Learning · Cross-listed from physics.flu-dyn
Fluids You Can Trust: Property-Preserving Operator Learning for Incompressible Flows
Ramansh Sharma, Matthew Lowery, Houman Owhadi et al. (+1 more)
We present a novel property-preserving kernel-based operator learning method for incompressible flows governed by the incompressible Navier–Stokes equations. Traditional numerical solvers incur significant computational costs to respect incompressibility. Operator learning offers efficient surrogate models, but current neural operators fail to exactly enforce physical properties such as incompressibility, periodicity, and
turbulence. Our kernel method maps input functions to expansion coefficients of output functions in a property-preserving kernel basis, ensuring that predicted velocity fields analytically and simultaneously preserve the aforementioned physical properties. We present universal approximation results and worst-case a priori convergence rates for our framework; empirically, the observed convergence rates exceed the pessimistic predictions across most
benchmarks, motivating a formulation of more optimistic convergence rates. Another central contribution of this work is a novel computational framework revolving around streaming construction of kernel Gramians and recursive Schur-complement
algorithms to solve the large block linear systems arising from property-preserving kernel methods. This kernel-based framework makes operator learning practical at scales up to $10{,}000$ training functions sampled at up to $10{,}000$ spatial locations. We evaluate the method on challenging 2D and 3D, laminar and turbulent, incompressible flow problems. Our method achieves up to six orders of magnitude lower relative $\ell_2$ errors upon generalization and trains up to five orders of magnitude faster compared to neural operators. Moreover, while our method enforces incompressibility analytically, neural operators exhibit large divergence errors. Our results show that our method provides an accurate and efficient surrogate for incompressible flows.
Read the full abstract ▾
All authors Ramansh Sharma, Matthew Lowery, Houman Owhadi, Varun Shankar
Machine Learning · Cross-listed from physics.soc-ph
The Skeletal Trap: Mapping Spatial Inequality and Ghost Stops in Ankara's Transit Network
Elifnaz Kancan
Ankara's public transport crisis is commonly framed as a shortage of buses or operational inefficiency. This study argues that the problem is fundamentally morphological and structural. The city's leapfrog urban expansion has produced fragmented peripheral clusters disconnected from a rigid, center-oriented bus network. As a result, demand remains intensely concentrated along the Kizilay-Ulus axis and western corridors, while peripheral districts experience either chronic under-service or enforced transfer dependency. The deficiency is therefore not merely quantitative but rooted in the misalignment between urban macroform and network architecture. The empirical analysis draws on a 173-day operational
dataset derived from route-level passenger and trip reports published by EGO under the former "Transparent Ankara" initiative. To overcome the absence of stop-level geospatial data, a Connectivity-Based Weighted Distribution Model reallocates passenger volumes to 1 km x 1 km grid
cells using network centrality. The findings reveal persistent center-periphery asymmetries, structural bottlenecks, and spatially embedded accessibility inequalities.
Read the full abstract ▾
Machine Learning
On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks
Yannic Neuhaus, Nicolas Flammarion, Matthias Hein et al. (+1 more)
Integrating
reasoning in large
language models and large
vision-
language models has recently led to significant improvement of their capabilities. However, the generalization of
reasoning models is still vaguely defined and poorly understood. In this work, we present an evaluation framework to rigorously examine how well chain-of-thought (CoT) approaches generalize on a simple planning task. Specifically, we consider a grid-based
navigation task in which a model is provided with a map and must output a sequence of moves that guides a player from a start position to a goal while avoiding obstacles. The versatility of the task and its data allows us to fine-tune model variants using different input representations (visual and textual) and CoT
reasoning strategies, and systematically evaluate them under both in-distribution (ID) and out-of-distribution (OOD) test conditions. Our experiments show that, while CoT
reasoning improves in-distribution generalization across all representations, out-of-distribution generalization (e.g., to larger maps) remains very limited in most cases when controlling for trivial matches with the ID data. Surprisingly, we find that
reasoning traces which combine multiple text formats yield the best (and non-trivial) OOD generalization. Finally, purely text-based models consistently outperform those utilizing image-based inputs, including a recently proposed approach relying on latent space
reasoning.
Read the full abstract ▾
All authors Yannic Neuhaus, Nicolas Flammarion, Matthias Hein, Francesco Croce