Machine Learning (Statistics)
Learning to Emulate Chaos: Adversarial Optimal Transport Regularization
Gabriel Melo, Leonardo Santiago and Peter Y. Lu
Chaos arises in many complex dynamical systems, from weather to power grids, but is difficult to accurately model with data-driven methods such as machine learning emulators. While emulators are promising tools for accelerating simulations and solving inverse problems, they still struggle to learn chaotic dynamics, where sensitivity to initial conditions renders exact long-term forecasts infeasible, especially given noisy data. Recent work instead trains emulators to match the statistical properties of chaotic attractors, but these approaches often rely on handcrafted summary statistics or large, diverse multi-environment
datasets. In this work, we propose a family of
adversarial optimal transport objectives that can jointly learn high-quality summary statistics and a physically consistent emulator from a single noisy trajectory. We theoretically analyze and experimentally validate a Sinkhorn divergence formulation (2-Wasserstein) and a WGAN-style dual formulation (1-Wasserstein) of our approach. Numerical experiments across a variety of chaotic systems, including ones with high-dimensional spatiotemporal chaos, show that emulators trained using our proposed objectives have significantly improved long-term statistical fidelity.
Read the full abstract ▾
Statistics Applications
Model quality in football: Quantifying the quality of an Expected Threat model
Koen van Arem, Jakob Söhl, Mirjam Bruinsma et al. (+1 more)
The recent growth in data availability in football has increased the risk of incorrect use of data-driven models, making guidelines on their validation and application necessary. The Expected Threat (xT) model is an accessible option for football organisations that start building in-house methods, yet little is known about how to assess its quality. The aim of this study is twofold: to examine how the model quality depends on the number of game states and the number of training points, and to translate these results into guidelines for constructing and applying the model. Using the Markov chain underlying the model, we perform theoretical analyses and simulations to study the estimation error. These show that the estimation error is approximately lognormal for a given number of training points and game states. Additionally, we combine the simulations with expert consultation to establish the estimation error beyond which player evaluations based on the Expected Threat model become unreliable for scouting applications. From this, we derive rules of thumb to ensure the quality of an Expected Threat model before application, and we illustrate through an example how a validated model can be applied in practice. Because the approach generalises to Expected Possession Value models, this paper illustrates a framework to systematically quantify model quality, despite the ground truth being unobservable in football analytics.
Read the full abstract ▾
All authors Koen van Arem, Jakob Söhl, Mirjam Bruinsma, Geurt Jongbloed
Statistics Applications
The geometry of conflict : 3D Spatio-temporal patterns in fatalities prediction
Thomas Schincariol
Understanding how conflict events spread over time and space is crucial for predicting and mitigating future violence. However, progress in this area has been limited by the lack of methods capable of capturing the intricate, dynamic patterns of conflict
diffusion. The complex nature of those trends needs flexibility in the models to untangle them. This study addresses this gap by analyzing spatio-temporal conflict fatality data using an innovative approach that transforms the data into three-dimensional patterns at the Prio-Grid level. In this paper, a shape-based model called ShapeFinder is adapted. By applying the Earth Movers Distance (EMD)
algorithm, we detect and classify these patterns, allowing us to compare and match patterns with high adaptive capacity in all dimensions. Using historical similar patterns, we generate predictions of conflict fatalities and compare these with forecasts from the Views ensemble model, a leading
benchmark. Our findings demonstrate that recognizing and analyzing conflict
diffusion patterns significantly improves predictive accuracy, outperforming the
benchmark model. This research contributes to the study of conflict dynamics by introducing a novel pattern recognition framework that enhances the analysis of spatio-temporal data and offers practical applications for early warning systems.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.CV
Optimizing Diffusion Priors in Image Reconstruction from a Single Observation
Frederic Wang and Katherine L. Bouman
While
diffusion priors generate high-quality posterior samples across many inverse problems, they are often trained on limited training sets or purely simulated data, thus inheriting the errors and biases of these underlying sources. Current approaches to finetuning
diffusion models rely on a large number of observations with varying forward operators, which can be difficult to collect for many applications, and thus lead to overfitting when the measurement set is small. We propose a method for tuning a prior from only a single observation by combining existing
diffusion priors into a single product-of-experts prior and identifying the exponents that maximize the
Bayesian evidence. We validate our method on real-world inverse problems, including black hole imaging, where the true prior is unknown a priori, and image deblurring with text-conditioned priors. We find that the evidence is often maximized by priors that extend beyond those trained on a single
dataset. By generalizing the prior through exponent weighting, our approach enables posterior sampling from both tempered and combined
diffusion models, yielding more flexible priors that improve the trustworthiness of the resulting posterior image distribution.
Read the full abstract ▾
Statistical Methodology
A Functional-Class Meta-Analytic Framework for Quantifying Surrogate Resilience
Emily Hsiao and Layla Parast
A surrogate marker is a biomarker or other physical measurement used to replace a primary outcome in clinical trials to evaluate a treatment effect when the primary outcome of interest is costly, invasive, or takes a long time to observe. However, replacing a primary outcome with a surrogate can lead to the "surrogate paradox," in which a treatment appears beneficial based on the surrogate but is actually harmful with respect to the primary outcome. In this paper, we propose a functional class-based method to assess resilience to the surrogate paradox in a meta-analytic setting. Our method leverages data from K completed studies in which the surrogate marker and primary outcome have been measured to make inference on a new study in which only the surrogate is measured. We do not assume direct transportability of the conditional mean function from the completed studies to the new study; instead, we consider deviations of functions from those observed in the completed studies to estimate the "resilience probability" i.e., the probability of the surrogate paradox in the new study. We investigate the performance of our proposed method through a simulation study and apply our method to data from clinical trials in schizophrenia.
Read the full abstract ▾
Statistical Methodology
Revisiting Bayesian Variable Selection via Optimization
Leo L Duan
Variable selection in linear regression has been a central topic in statistical research for decades.
Bayesian variable selection methods, which account for uncertainty in both the regression coefficients and the noise variance, have achieved broad success through the use of discrete or continuous shrinkage priors and efficient collapsed Gibbs samplers. Despite their popularity and strong empirical performance, an enigma remains: the marginal likelihood, obtained by integrating out the regression coefficients and noise variance, is not log-concave; therefore, there is no guarantee of reliably finding its global optimum. In this article, we study this problem from an optimization perspective. Taking the negative log-marginal likelihood as a loss function of the latent precision parameters, we can rewrite it as a difference of convex functions (DC), and then optimize it via a simple iterative
algorithm. Under mild compact set conditions, the DC
algorithm converges to the global optimum at a linear rate. The positive finding applies to type-II maximum likelihood and extends to maximum marginal posterior under suitable priors, indicating that the problem of mode finding in
Bayesian variable selection is much more benign than the lack of log-concavity might suggest. Besides the theoretical insight, the proposed
algorithm is easy to implement, free of tuning, and extensible to structured sparsity, and thus can serve as an efficient alternative or warm-start for traditional Markov chain Monte Carlo solutions. The method is illustrated through numerical studies and a spatial data application for quantifying the aftershock risk following the 2019 Ridgecrest earthquakes. The source code for the
algorithm is publicly available at https://github.com/leoduan/dca_optimization_variable_selection.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Differentially Private Model Merging
Qichuan Yin, Manzil Zaheer and Tian Li
In machine learning,
privacy requirements at inference or deployment time often evolve due to changing policies, regulations, or user preferences. In this work, we aim to construct a magnitude of models to satisfy any target differential
privacy (DP) requirement without additional training, given a set of existing models trained on the same
dataset with different
privacy/utility tradeoffs. We propose two post-processing techniques, namely random selection and linear combination, to generate final private models satisfying any target
privacy parameter. We provide
privacy accounting of these approaches from the lens of R'enyi DP and
privacy loss distributions on general problems, as well as on private mean estimation, where we precisely characterize the
privacy/utility tradeoffs and compare the two mechanisms. Empirically, we demonstrate the effectiveness of our approaches and validate our analyses on several models and both synthetic and real-world
datasets.
Read the full abstract ▾
Statistical Methodology
ML, PL, QL in Markov chain models
Nils Lid Hjort and Cristiano Varin
In many spatial and spatial-temporal models, and more generally in models with complex dependencies, it may be too difficult to carry out full maximum likelihood (ML) analysis. Remedies include the use of pseudo-likelihood (PL) and quasi-likelihood (QL) (also called the composite likelihood). The present article studies the ML, the PL and the QL methods for general Markov chain models, partly motivated by the desire to understand the precise behaviour of PL and QL methods in settings where this can be analysed. We present limiting normality results and compare performances in different settings. The PL and QL methods can be seen as maximum penalised likelihood methods. We find that the QL strategy is typically preferable to the PL, and that it loses very little to the ML, while earning in model robustness. It has also appeal and potential as a modelling tool. Our methods are illustrated for analysis of DNA sequence
evolution type models.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Early Detection of Latent Microstructure Regimes in Limit Order Books
Prakul Sunil Hiremath and Vruksha Arun Hiremath
Limit order books can transition rapidly from stable to stressed conditions, yet standard early-warning signals such as order flow imbalance and short-term
volatility are inherently reactive. We formalise this limitation via a three-regime causal data-generating process (stable $\to$ latent build-up $\to$ stress) in which a latent deterioration phase creates a prediction window prior to observable stress. Under mild assumptions on temporal drift and regime persistence, we establish identifiability of the latent build-up regime and derive guarantees for strictly positive expected lead-time and non-trivial probability of early detection. We propose a trigger-based detector combining MAX aggregation of complementary signal channels, a rising-edge condition, and adaptive thresholding. Across 200 simulations, the method achieves mean lead-time $+18.6 \pm 3.2$ timesteps with perfect precision and moderate coverage, outperforming classical change-point and microstructure baselines. A preliminary application to one week of BTC/USDT order book data shows consistent positive lead-times while baselines remain reactive. Results degrade in low signal-to-noise and short build-up regimes, consistent with theory.
Read the full abstract ▾
Statistical Methodology · Cross-listed from math.OC
Solving Minimax Problems with Bilinear Objectives with ADMM
Bob Wilson
We consider minimax (saddle-point) problems of the form max_{c \in C} min_{\beta \in S} g(c; \beta), where C and S are compact convex sets, and g is concave-convex. Applying the Alternating Direction Method of Multipliers (ADMM) requires evaluating a proximal operator that is, in general, as hard as the original problem. We show that when the outcome function g is bilinear, i.e. g(c; \beta) = c^T A \beta, the proximal operator reduces to a generalized projection onto the confidence region S. This reduction is exact – it involves no approximation or linearization. The resulting ADMM
algorithm alternates between (i) a generalized projection onto S and (ii) a Euclidean projection onto C. We describe the derivation, state the
algorithm, and discuss convergence.
Read the full abstract ▾
Statistics Theory
The E-measure
Nick W. Koning
We introduce the E-measure: a measure-like generalization of the E-value to a class of hypotheses. Unlike classical measures, E-measures are closed under infimums instead of addition. They arise from a compatibility axiom with logical implications, that there should be at least as much evidence against more specific hypotheses. We show that E-measures are the only non-dominated such objects, if the hypothesis class is closed under intersections. We propose to use the E-measure to present all the relevant evidence for a problem, where the relevance is captured by the choice of hypothesis class. We showcase this by applying the E-measure to decision making, inducing a hypothesis class from the uncertain consequences of decisions. This results in uniform E-consequence bounds on decisions, which nest high-probability loss bounds. Correcting for multiplicity, we consider 'familywise evidence' and 'false evidence rate' control, generalizing from errors and discoveries to continuous evidence. Remarkably, E-measures control these without multiplicity correction if the hypothesis class is intersection-closed. Moreover, we obtain a 'frequentist' notion of updating from E-prior to E-posterior. Abstracting the notion of a 'hypothesis', we advocate for using E-measures for any unknown quantity, leading to predictive E-measures.
Read the full abstract ▾
Machine Learning (Statistics)
Xiaotian Chang, Yangdi Jiang, Cyrus Mostajeran et al. (+1 more)
In this paper, we develop a novel
privacy mechanism for Riemannian
manifold-valued data. Our key contribution lies in uncovering unexpected connections among
geometric analysis, heat
diffusion models, and differential
privacy (DP). We characterize the Renyi divergence via dimension-free Harnack inequalities on Riemannian
manifolds and establish Renyi differential
privacy guarantees governed by
Ricci curvature. For
manifolds with nonnegative
Ricci curvature, we propose a mechanism based on heat
diffusion. In contrast, for general
manifolds we introduce a Langevin-process-based approach that yields intrinsic mechanisms supporting normalization-free sampling and continuous
privacy-utility trade-offs. We derive detailed utility analyses for both mechanisms. As a statistical application, we develop
privacy-preserving estimation of the generalized Frechet mean, including nontrivial sensitivity analysis and phase transition characterizations. Numerical experiments further demonstrate the advantages of the proposed DP mechanisms over existing approaches.
Read the full abstract ▾
All authors Xiaotian Chang, Yangdi Jiang, Cyrus Mostajeran, Qirui Hu
Statistical Methodology
ProfileGLMM: a R Package Extending Bayesian Profile Regression using Generalised Linear Mixed Models
Matteo Amestoy, Mark A. van de Wiel and Wessel N. van Wieringen
ProfileGLMM is an R package integrating Generalised Linear Mixed Models (GLMMs) as the outcome model for
Bayesian profile regression. This statistical framework simultaneously i) explains the variation in the outcome and ii) clusters the observations based on a specified set of interdependent clustering
covariates. The derived cluster memberships are then incorporated, alongside others, as explanatory variables in the regression to model the outcome. This framework efficiently handles complex, highly correlated
covariate structures whose direct inclusion in a standard regression model would be statistically sub-optimal. ProfileGLMM significantly extends
Bayesian profile regression's scope by resolving two key constraints of previous implementations: 1) it allows the analysis of hierarchical and longitudinal data structures through the inclusion of random effects, and 2) it enables the study of interactions between latent clusters and other observable
covariates. ProfileGLMM accommodates various data types, supporting both continuous or binary outcomes and both categorical and continuous clustering
covariates. Built on fast Rcpp code with minimal mandatory parameters, ProfileGLMM offers a flexible analytical tool. It significantly enhances the utility of profile regression for researchers in fields such as epidemiology, social sciences, and clinical studies dealing with complex data.
Read the full abstract ▾
Statistical Methodology
Data Integration for Estimating Subgroup-Specific Conditional Average Treatment Effects (CATEs) Using Coarsened External Information in Randomized Trials
Youqi Yang, Walter Dempsey and Bhramar Mukherjee
Randomized controlled trials (RCTs) are often underpowered to detect treatment heterogeneity in subgroups defined by cross-classifications of multiple
covariates, due to sparse sample sizes in some strata. External RCT data can help, but typically provide treatment effect estimates at a coarser level (e.g., by sex or race) rather than for the finer subgroups of interest (e.g., race-by-sex). We propose a novel James-Stein (JS)-type
estimator that borrows strength from such coarsened external estimates to improve estimation of finer subgroup-specific conditional average treatment effects (CATEs) in an internal study, while accommodating potential incompatibility in marginal CATEs across populations. Based on asymptotic theory, we derive a practical analytic variance
estimator for the JS
estimator that exhibits acceptable empirical performance. Under mild conditions, we show that the proposed
estimator uniformly dominates the ordinary least squares (OLS)
estimator based on internal data regarding a weighted quadratic loss. Simulation studies demonstrate favorable performance compared with existing shrinkage methods, including empirical Bayes and generalized ridge
estimators. We illustrate our method by estimating race-by-sex subgroup CATEs in a tirzepatide weight-loss trial (SURMOUNT-1), borrowing sex-specific and race-specific estimates from two previous semaglutide trials (STEP 1 and STEP 2). The proposed method detects a significantly larger treatment effect on percentage weight loss in the female-White subgroup than in the female-Asian subgroup, a difference not detected using internal data alone.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
Distributional Statistical Models: Weak Moments, Cumulants, and a Central Limit Theorem
R. Labouriau
Many important statistical models fall outside classical moment-based methods due to the non-existence of moments or moment generating functions. We propose a generalised probabilistic framework in which a probability law is represented by a tempered distribution $T \in \mathcal{S}'$, on the same footing as a density, a distribution function, or a characteristic function. Information about the law is extracted by evaluating $T$ on test functions regularised by a given positive Schwartz kernel $\varphi \in \mathcal{S}$ – the kernel serving as a probe, not as part of the law. Expectations are defined via the action of distributions on regularised test functions, yielding well-defined weak moments, weak characteristic functions, and weak cumulants of all orders. These extend classical quantities and retain key algebraic properties such as additivity under independence and natural affine transformation rules. The main results are: (i) a systematic
algebra of weak cumulants; (ii) a weak moment problem where existence of all moments holds unconditionally and uniqueness depends on the kernel, with uniqueness results under Gaussian kernels (via Hermite completeness), positive Schwartz kernels with an exponential tail bound and square-integrable densities (via a Carleman-type criterion), and kernels with exponential decay (via Denjoy-Carleman quasi-analyticity); and (iii) a weak central limit
theorem formulated as convergence of weak characteristic functions to a Gaussian limit, covering cases where the classical
theorem fails. The framework is illustrated with Student's $t$, stable, and
hyperbolic distributions. As a statistical consequence, the weak first moment yields a consistent
estimator of the location parameter in the Cauchy model, where no classical moment-based
estimator exists. A full statistical treatment is given in a companion paper.
Read the full abstract ▾
Statistical Methodology · Cross-listed from astro-ph.IM
Review: A new method for estimation and use of systematic errors in Poisson regression
M. Bonamente
A new method for including systematic errors in the regression with Poisson data is reviewed in this contribution, with emphasis on applications to astronomical spectra. The method consists of generalizing the usual Poisson log-likelihood, known as the Cash statistic $C_{min}$, and its associated likelihood-ratio statistic $\Delta C$, to include the presence of systematic sources of uncertainty. Advantages of this new method include its modeling simplicity and its ability to assess both the level of systematics and the goodness of fit at the same time, including for a nested model component.
Read the full abstract ▾
Statistical Methodology
Doubly Robust Weighted Regression for Causal Inference under Confounder Missingness
Md. Shaddam Hossain Bagmar and Hua Shen
Missingness in confounders is common in observational studies and presents substantial challenges for causal effect estimation by weakening identification and increasing sensitivity to model misspecification. Within the missing-indicator framework, existing approaches typically rely on a single working model and achieve consistency only when that model is correctly specified and are therefore singly robust. In this article, we develop a doubly robust missing indicator weighted ordinary least squares (MI-WOLS)
estimator with partially observed confounders. The proposed method integrates propensity score-based weighting into outcome regression under missing-indicator data representation, yielding a class of balancing weights that account for both confounders and their missingness indicators. Under the missingness-strongly-ignorable treatment allocation assumption and assuming either a Conditionally Independent Treatment or Conditionally Independent Outcome structure, the MI-WOLS
estimator is consistent when at least the treatment or the outcome model is correctly specified. Simulation studies support the theoretical robustness of the MI-WOLS
estimator, demonstrating negligible bias, accurate sandwich-based variance estimation, and near-nominal coverage probability across a wide range of data-generating scenarios. An illustrative application using simulated data designed to reflect a real-world kidney function study further demonstrates the interpretability and practical feasibility of the method, offering a flexible, doubly robust alternative to existing singly robust
estimators.
Read the full abstract ▾
Statistical Methodology
Dynamic Prediction of the Target Survival Time in Metastatic Solid Tumor Cancer Clinical Trials
Sidi Wang, Kelley Kidwell, Bo Huang et al. (+1 more)
Overall survival (OS) is the gold standard for assessing patient benefit and cost-effectiveness of new cancer drugs. However, it is often difficult to use OS as the primary endpoint in randomized clinical trials (RCTs) for patients with metastatic cancer due to multiple reasons. In recent years, progression-free survival (PFS) has increasingly been used as the primary endpoint in metastatic cancer RCTs to accelerate development. However, regulatory authorities often seek mature OS data for approval. Therefore, it is critical to determine the target time when OS data are expected to be mature for reliable statistical inference. Motivated by an advanced renal
cell carcinoma (RCC) clinical trial, we develop and investigate different prediction models leveraging information from disease progression to improve target OS prediction times. We propose a multivariate joint modeling approach considering components of progression and OS and extend three models commonly used for association to be used for OS prediction. To the best of our knowledge, this is the first comprehensive statistical study exploring the prediction of OS using different levels of information on disease progression and illustrating these models using a real, complex
dataset. Our findings have significant implications for OS prediction.
Read the full abstract ▾
All authors Sidi Wang, Kelley Kidwell, Bo Huang, Satrajit Roychoudhury
Machine Learning (Statistics) · Cross-listed from cs.LG
Too Sharp, Too Sure: When Calibration Follows Curvature
Alessandro Morosini, Matea Gjika, Tomaso Poggio et al. (+1 more)
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small
vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration,
curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks
curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss–Newton
curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.
Read the full abstract ▾
All authors Alessandro Morosini, Matea Gjika, Tomaso Poggio, Pierfrancesco Beneventano
Statistics Theory
E-values and sequential power-one tests for monotonicity and unimodality
Hongjian Wang and Aaditya Ramdas
We develop e-values and e-processes testing the null hypothesis that a distribution over nonnegative integers is monotone, and that a distribution over integers is unimodal given a certain mode. Our e-processes lead to tests of power one under any non-null distribution with a sequence of i.i.d. observations, and consistent set-valued mode
estimators that eventually equal the true set of modes. Additionally, we characterize the set of all e-values, and therefore the set of all valid tests, with one monotone and unimodal observation, as well as the most powerful e-value for a fixed alternative. We then show that many of our results can be generalized to continuous random variables, relating them to the existing results in the shape-constrained inference literature.
Read the full abstract ▾
Statistics Applications
Bayesian Inference for Incomplete 2x2 Diagnostic Tables
Sara Antonijevic, Danielle Sitalo and Brani Vidakovic
Incomplete reporting of diagnostic accuracy data remains a persistent problem in medical research. In many studies, only part of the 2x2 diagnostic table is reported, leaving denominators for diseased and non-diseased groups unknown and preventing direct calculation of sensitivity, specificity, predictive values, and related operating characteristics. To address this limitation, we develop hierarchical
Bayesian models for reconstructing incomplete 2x2 diagnostic tables from such partial information. Two motivating scenarios are considered: one in which only a single test-outcome row is observed, and another in which true positives, false positives, and the total sample size are reported but the remaining
cells are missing. The proposed models are illustrated on a
benchmark breast MRI study with complete counts, treated as partially observed in order to assess reconstruction performance under controlled missingness. The framework yields posterior inference for the missing
cell counts and associated diagnostic measures, together with uncertainty quantification in weakly identified settings.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Amortized Vine Copulas for High-Dimensional Density and Information Estimation
Houman Safaai
Modeling high-dimensional dependencies while keeping likelihoods tractable remains challenging. Classical vine-copula pipelines are interpretable but can be expensive, while many neural
estimators are flexible but less structured. In this work, we propose Vine Denoising Copula (VDC), an amortized vine-copula pipeline for continuous-data, simplified-vine dependence modeling. VDC trains a single bivariate denoising model and reuses it across all vine edges. For each edge, given pseudo-observations, the model predicts a piecewise-constant density grid. We then apply an IPFP/Sinkhorn projection that normalizes mass and drives the marginals to uniformity. This preserves the tractable vine-likelihood structure and the usual copula interpretation while replacing repeated per-edge optimization with GPU inference. Across synthetic and real-data
benchmarks, VDC delivers strong bivariate density accuracy, competitive MI/TC estimation, and faster high-dimensional vine fitting. These gains make explicit information estimation and dependence decomposition feasible when repeated vine fitting would otherwise be costly, while conditional downstream tasks remain a limitation.
Read the full abstract ▾
Machine Learning (Statistics)
On Bayesian Softmax-Gated Mixture-of-Experts Models
Nicola Bariletto, Huy Nguyen, Nhat Ho et al. (+1 more)
Mixture-of-experts models provide a flexible framework for learning complex probabilistic input-output relationships by combining multiple expert models through an input-dependent gating mechanism. These models have become increasingly prominent in modern machine learning, yet their theoretical properties in the
Bayesian framework remain largely unexplored. In this paper, we study
Bayesian mixture-of-experts models, focusing on the ubiquitous softmax-based gating mechanism. Specifically, we investigate the asymptotic behavior of the posterior distribution for three fundamental statistical tasks: density estimation, parameter estimation, and model selection. First, we establish posterior contraction rates for density estimation, both in the regimes with a fixed, known number of experts and with a random learnable number of experts. We then analyze parameter estimation and derive convergence guarantees based on tailored Voronoi-type losses, which account for the complex identifiability structure of mixture-of-experts models. Finally, we propose and analyze two complementary strategies for selecting the number of experts. Taken together, these results provide one of the first systematic theoretical analyses of
Bayesian mixture-of-experts models with softmax gating, and yield several theory-grounded insights for practical model design.
Read the full abstract ▾
All authors Nicola Bariletto, Huy Nguyen, Nhat Ho, Alessandro Rinaldo
Statistical Computation · Cross-listed from math.DS
Transient Stability of Nonlinear Stochastic Dynamics: A Finite-Amplitude Logarithmic Measure Framework
Surya Ratna Prakash D and Soumyendu Raha
Spacecraft guidance,
navigation, and control systems operating during mission-critical phases such as atmospheric entry, powered descent, and planetary landing require reliable assessment of finite-time transient behavior under
stochastic uncertainty. Existing
stochastic stability frameworks primarily characterize asymptotic or infinitesimal behavior and therefore provide limited insight into finite-amplitude transient amplification over operationally relevant time horizons. This paper develops a finite-amplitude logarithmic measure for nonlinear Itô
stochastic differential equations, extending classical
matrix measures from infinitesimal to finite-amplitude perturbation
evolution. The proposed framework establishes finite-time mean and variance bounds for logarithmic perturbation growth, derives Chernoff-type probabilistic bounds on transient amplification, and shows that mean transient stability does not necessarily guarantee finite-time pathwise safety. The theory is further extended to projected
stochastic dynamics, leading to a transient-risk index that quantitatively balances deterministic contraction and
diffusion-induced variability for transient-risk-aware system design. The proposed framework is validated through Monte Carlo simulations and flight-like lunar descent telemetry. The results demonstrate substantially improved prediction of nonlinear transient growth compared with classical Jacobian-based approaches and successfully distinguish trajectories exhibiting similar nominal behavior but significantly different transient-risk characteristics. The proposed framework provides a unified methodology for finite-time transient stability analysis, probabilistic transient-risk assessment, and transient-risk-aware guidance,
navigation, and control of nonlinear
stochastic aerospace systems.
Read the full abstract ▾
Machine Learning (Statistics)
Efficient Symbolic Computations for Identifying Causal Effects
Benjamin Hollering, Pratik Misra and Nils Sturma
Determining identifiability of causal effects from observational data under latent confounding is a central challenge in causal inference. For linear structural causal models, identifiability of causal effects is decidable through symbolic computation. However, standard approaches based on Gröbner bases become computationally infeasible beyond small settings due to their doubly exponential complexity. In this work, we study how to practically use symbolic computation for deciding rational identifiability. In particular, we present an efficient
algorithm that provably finds the lowest degree identifying formulas. For a causal effect of interest, if there exists an identification formula of a prespecified maximal degree, our
algorithm returns such a formula in quasi-
polynomial time.
Read the full abstract ▾
Machine Learning (Statistics)
Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms
Yaiza Bermudez, Samir M. Perlaza and Iñaki Esnaola
In this paper, it is shown, for the first time, that centralized performance is achievable in decentralized learning without sharing the local
datasets. Specifically, when clients adopt an empirical risk minimization with relative-
entropy regularization (ERM-RER) learning framework and a forward-backward communication between clients is established, it suffices to share the locally obtained Gibbs measures to achieve the same performance as that of a centralized ERM-RER with access to all the
datasets. The core idea is that the Gibbs measure produced by client~$k$ is used, as reference measure, by client~$k+1$. This effectively establishes a principled way to encode prior information through a reference measure. In particular, achieving centralized performance in the decentralized setting requires a specific scaling of the regularization factors with the local sample sizes. Overall, this result opens the door to novel decentralized learning paradigms that shift the collaboration strategy from sharing data to sharing the local inductive bias via the reference measures over the set of models.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
The Origin of Edge of Stability
Elon Litman
Full-batch gradient descent on neural networks drives the largest Hessian
eigenvalue to the threshold $2/\eta$, where $\eta$ is the learning rate. This phenomenon, the Edge of Stability, has resisted a unified explanation: existing accounts establish self-regulation near the edge but do not explain why the trajectory is forced toward $2/\eta$ from arbitrary initialization. We introduce the edge coupling, a functional on consecutive iterate pairs whose coefficient is uniquely fixed by the gradient-descent update. Differencing its criticality condition yields a step recurrence with stability boundary $2/\eta$, and a second-order expansion yields a loss-change formula whose telescoping sum forces
curvature toward $2/\eta$. The two formulas involve different Hessian averages, but the mean value
theorem localizes each to the true Hessian at an interior point of the step segment, yielding exact forcing of the Hessian
eigenvalue with no gap. Setting both gradients of the edge coupling to zero classifies fixed points and period-two orbits; near a fixed point, the problem reduces to a function of the half-amplitude alone, which determines which directions support period-two orbits and on which side of the critical learning rate they appear.
Read the full abstract ▾
Statistics Applications
Assessing the Shortfall Risk of GB Electricity Grid using Shifts in Winter Weather Conditions
Aninda Bhattacharya, Chris J. Dent, Amy L. Wilson et al. (+1 more)
Extreme weather events during peak winter periods drive resource adequacy risk in Great Britain (GB), with weather sensitivity of the supply-demand balance increasing through additional electric heating and wind generation. This work develops an approach of time-shifting weather within the peak season, through adjustment of the relevant terms in a statistical model for demand. This allows more complete consideration of the security of supply consequences of a weather series, as there will be relevant conditions where demand is suppressed due to weather occurring at a weekend or during the Christmas holiday. Results on a GB example show that consideration of this counterfactual is indeed important, and specifically that winter 2010-11 can either be the most severe in the
dataset, or insignificant within the resource adequacy model, depending on the
alignment of day-of-week with the weather series. Statistical interpretation of the shift model is discussed, which is straightforward for
alignment of day-of-week with weather assuming that all seven
alignments are equiprobable; but is more subtle for shifting weather in and out of Christmas, as there is no natural maximum on the realistic length of shift, but too large a shift may be physically unrealistic. It is likely that in all systems, assessment of a weather year's severity is incomplete without such consideration of the day-of-week effect; however, whether longer shifts of weather with respect to date need to be considered will depend on the presence of a major holiday (such as Christmas in GB) in the peak season.
Read the full abstract ▾
All authors Aninda Bhattacharya, Chris J. Dent, Amy L. Wilson, Gabriele C. Hegerl
Statistics Theory
Likelihood-based inference for birth-death processes with composite birth mechanisms
Marko Lalovic, Nicos Georgiou and Istvan Z. Kiss
We develop a likelihood-based inference for finite-state birth-death processes with composite birth rates, in which multiple distinct mechanisms contribute additively to the total birth intensity. Our main motivating example is an SIS epidemic model with pairwise and higher-order transmission. The process is observed through a single aggregate trajectory, and in the main setting of interest, birth events are unmarked. This creates a deconvolution problem in event space: the state is one-dimensional, but the mechanism underlying each birth is latent. We formulate the inference under a Doob $h$-transformed $Q$-process, which is time-homogeneous and ergodic and which provides a time-homogeneous asymptotic surrogate for the law of the original process conditioned on long survival. We derive the corresponding conditional likelihood and study both the conditional maximum likelihood
estimator and a quasi-maximum likelihood
estimator which is based on a simplified working score. Under the Doob-transform law, we prove consistency and asymptotic normality for both
estimators, with asymptotic covariance determined by the inverse Fisher and inverse Godambe information
matrices, respectively. We also showcase a practical one-dimensional test for the presence of a specific higher-order birth mechanism.
Read the full abstract ▾
Statistics Applications
SHARELIFE Imputations
Giuseppe De Luca and Paolo Li Donni
This report describes the SHARELIFE-MI project, which aims to generate multiple imputations for missing values in the life-course data collected in SHARELIFE Waves 3 and 7. The SHARELIFE study reconstructs individual life histories through retrospective questions covering key biographical domains such as partnerships, fertility, employment, and residence. As in the regular SHARE waves, item nonresponse represents an important source of nonsampling error - particularly for monetary variables, which require conversions across multiple currencies and long time periods. We document the preliminary data recoding and harmonization steps, as well as the design, specification, and implementation of an imputation model based on the fully conditional specification approach. Finally, we assess the internal and external validity of the resulting imputations through comparisons with the observed data, alternative nonresponse adjustments based on inverse propensity weighting, and external
benchmarks from the regular SHARE waves.
Read the full abstract ▾