Statistical Computation
GPU-Accelerated Sequential Monte Carlo for Bayesian Spectral Analysis
Tomohiro Nabika, Yui Hayashi and Masato Okada
Bayesian spectral deconvolution provides a data-driven framework for mathematical model selection and parameter estimation from spectral data. Although highly versatile, it becomes computationally expensive as the number of model parameters, data points, and candidate models increases, often rendering practical applications infeasible. We propose a GPU-accelerated approach in which a sequential Monte Carlo sampler (SMCS) is run in parallel on a GPU to perform
Bayesian model selection of the number of spectral peaks and
Bayesian estimation of peak-function parameters. Numerical experiments demonstrate that the GPU-parallelized SMCS achieves speedups exceeding 500x over CPU-parallelized replica exchange Monte Carlo (REMC). The method is validated on artificial data designed to emulate X-ray photoelectron spectroscopy (XPS) and X-ray diffraction (XRD) measurements, as well as on real experimental spectra. As measurement techniques such as microscopic spectroscopy and in-situ methods continue to drive rapid growth in the volume of spectral data, the proposed approach offers a practical computational foundation for advanced analysis of individual
datasets.
Read the full abstract ▾
Machine Learning (Statistics)
Overfitting and Generalizing with (PAC) Bayesian Prediction in Noisy Binary Classification
Xiaohan Zhu, Mesrob I. Ohannessian and Nathan Srebro
We consider a PAC-Bayes type learning rule for binary classification, balancing the training error of a randomized ”posterior” predictor with its KL divergence to a pre-specified ”prior”. This can be seen as an extension of a modified two-part-code Minimum Description Length (MDL) learning rule, to continuous priors and randomized predictions. With a balancing parameter of $\lambda=1$ this learning rule recovers an (empirical) Bayes posterior and a modified variant recovers the profile posterior, linking with standard
Bayesian prediction (up to the treatment of the single-parameter noise level). However, from a risk-minimization prediction perspective, this
Bayesian predictor overfits and can lead to non-vanishing excess loss in the agnostic case. Instead a choice of $\lambda \gg 1$, which can be seen as using a sample-size-dependent-prior, ensures uniformly vanishing excess loss even in the agnostic case. We precisely characterize the effect of under-regularizing (and over-regularizing) as a function of the balance parameter $\lambda$, understanding the regimes in which this under-regularization is tempered or catastrophic. This work extends previous work by Zhu and Srebro [2025] that considered only discrete priors to PAC Bayes type learning rules and, through their rigorous
Bayesian interpretation, to
Bayesian prediction more generally.
Read the full abstract ▾
Statistical Methodology
When lookout sees crackle: Anomaly detection via kernel density estimation
Rob J Hyndman, Sevvandi Kandanaarachchi and Katharine Turner
We present an updated version of lookout – an
algorithm for detecting anomalies using kernel density estimates with bandwidth based on Rips death diameters – with theoretical guarantees. The kernel density
estimator for updated lookout is shown to be consistent, and the proposed multivariate scaling is robust and efficient. We show our updated
algorithm performs better than the previous version on diverse examples.
Read the full abstract ▾
Statistics Theory
A Martingale Approach To Fluctuations of Rank Estimators in Sensitivity Analysis
Reda Chhaibi, Fabrice Gamboa and Clément Pellegrini
Given a bivariate random pair $(X,Y)$, a natural problem is to estimate, from a single sample $(X_i,Y_i)_{1\le i\le n}$, quantities such as $\mathbb{E}\left[ \mathbb{E}[ Y\mid X ]^2 \right]$. More broadly, sensitivity indices are designed to quantify the possibly nonlinear influence of an input variable $X$ on an output variable $Y$. A classical example is the Sobol' index $$ \frac{\mathrm{Var}(\mathbb{E}[Y\mid X])}{\mathrm{Var}(Y)} \in [0,1] \ . $$ Another important example is the Cramér–von Mises (CvM) index. Following the pioneering work of Chatterjee \cite{chatterjee2021new}, consistent rank-based
estimators are now available for such quantities. In this paper, we prove sharp fluctuation results using martingale methods. Our framework yields a unified treatment of the univariate Sobol' index, a multivariate extension involving several functions of the same scalar input, and the CvM index. As a consequence, we recover, unify, and simplify results from Gamboa et al. \cite{gamboa2022global, gamboa2023erratum}, Lin–Han \cite{lin2022limit}, and Kroll \cite{kroll2024asymptotic}. In particular, we work under minimal regularity assumptions. Furthermore, while the Gaussian fluctuation phenomenon itself was already known, the novelty lies in the structure of the asymptotic variance: for the CvM index, we obtain, to the best of our knowledge, the first explicit formula, while for the Sobol' index, we derive a new expression with a more structured form.
Read the full abstract ▾
Statistical Methodology
Claims, Designs, and Guarantees: A Critical Narrative Review of Statistical Inference Reform and Practice, 2016–2026
Aswini Kumar Patra and Shashi Bhushan Jha
Statistics helps scientists decide what their data show, but its most familiar tool, the $p$-value, is widely misunderstood. Between 2016 and 2026, professional bodies issued new guidance, and researchers developed or popularised alternatives and complements: compatibility intervals, equivalence tests, Bayes factors, and e-values that remain valid when data are checked repeatedly. Reporting rules for trials and reviews were also rewritten. Yet practice has changed little: in the biomedical literature, about 96% of papers that report $p$-values still include at least one below 0.05. This review argues that much of the confusion comes from treating these numbers as interchangeable. Each inferential procedure carries an intended interpretation and, where applicable, a formal guarantee, such as a 5% false-alarm rate, that holds only if the study was run and analysed in a particular way. We call this the Claim–Design–Guarantee framework: a result licenses a claim only when the claim, the design (assignment, sampling, monitoring, selection, and analysis), and the guarantee of the method are aligned. We trace the main developments of the decade, explain each tool in plain terms with formal details in boxes, and show with reproducible examples when the choice of tool changes the conclusion. For example, in a simulated two-group study with 100 interim looks, repeatedly applying an unadjusted $p<0.05$ rule produced false positives in 37.6% of studies, whereas the particular anytime-valid e-value procedure we used produced 2.2%, below its theoretical 5% upper bound. We end with a decision tree, a method-selection table, and a step-by-step workflow linked to current reporting guidelines.
Read the full abstract ▾
Statistical Methodology
Multiple Jump MCMC: A Scalable Algorithm for Bayesian Inference on Binary Model Spaces
Lucas Vogels, Reza Mohammadi, Marit Schoonhoven et al. (+2 more)
This article considers
Bayesian model inference on binary model spaces. Binary model spaces are used by a large class of models, including graphical models, variable selection, mixture distributions, and decision trees. Traditional strategies in this field, such as reversible jump or birth-death MCMC
algorithms, are still popular, despite suffering from a slow exploration of the model space. In this article, we propose an alternative: the Multiple Jump MCMC
algorithm. The
algorithm is simple, rejection-free, and remarkably fast. When applied to undirected Gaussian graphical models, it is 100 to 200 times faster than the state-of-the-art, solving models with $500,000$ parameters in less than a minute. We provide
theorems showing how accurately our
algorithm targets the posterior, and we show how to apply our framework to Gaussian graphical models, Ising models, and variable selection, but note that it applies to most
Bayesian posterior inference on binary model spaces.
Read the full abstract ▾
All authors Lucas Vogels, Reza Mohammadi, Marit Schoonhoven, Sinan Yildirim, Ilker Birbil
Statistical Methodology · Cross-listed from q-fin.RM
Proxy-Reliance Control in Conformal Recalibration of One-Sided Value-at-Risk
Tenghan Zhong
We introduce a proxy-reliance-controlled conformal recalibration framework for one-sided Value-at-Risk (VaR), and study a question that existing state-aware methods do not usually isolate: how strongly should the recalibration adjustment depend on an imperfect
volatility proxy? We formalize this through a proxy-reliance parameter that continuously interpolates between an approximately constant-shift correction and a fully proxy-scaled correction. This makes proxy reliance a distinct and practically interpretable design choice in one-sided VaR recalibration. We show theoretically that larger proxy reliance increases the responsiveness of the tail adjustment to proxy scale, but also increases stressed-state fragility when the proxy underreacts. Empirically, in rolling out-of-sample tests on a six-ETF panel with VIX-linked state variables, and with supporting evidence from SPY, we find that the empirical value of proxy-reliance control lies in improved stressed-state robustness rather than uniform overall dominance. In particular, when the baseline forecast remains exposed to proxy imperfection in stressed states, lower or intermediate proxy reliance can outperform fully proxy-scaled recalibration in stressed left-tail VaR control.
Read the full abstract ▾
Machine Learning (Statistics)
Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling
Young Hyun Cho and Will Wei Sun
Preference-based fine-tuning has become an important component in training large
language models, and the data used at this stage may contain sensitive user information. A central question is how to design a differentially private pipeline that is well suited to the distinct structure of reinforcement learning from human feedback. We propose a
privacy-preserving framework that imposes differential
privacy only on reward learning and derives the final policy from the resulting private reward model. Theoretically, we study the suboptimality gap and show that
privacy contributes an additional additive term beyond the usual non-private statistical error. We also establish a minimax lower bound and show that the dominant term changes with sample size and
privacy level, which in turn characterizes regimes in which the upper bound is rate-optimal up to logarithmic factors. Empirically, synthetic experiments confirm the scaling predicted by the theory, and experiments on the Anthropic HH-RLHF
dataset using the Gemma-2B-IT model show stronger private
alignment performance than existing differentially private baseline methods across
privacy budgets.
Read the full abstract ▾
Statistical Methodology
Variable Selection in Functional Linear Quantile Regression for Identifying Associations between Daily Patterns of Physical Activity and Cognitive Function
Yuanzhen Yue, Stella Self, Yichao Wu et al. (+2 more)
Quantile regression is useful for characterizing the conditional distribution of a response variable and understanding heterogeneity in the
covariate effects at different
quantiles. The rise of high-dimensional physiological data in biomedical research through wearable and sensor devices underscores the need for effective variable selection methods for interpretable and accurate
quantile regression, which can offer robust insights into heterogeneous and dynamic
covariate effects. We develop a flexible variable selection approach for functional linear
quantile regression with multiple functional and scalar predictors. We use a smooth approximation of the
quantile loss function and integrate functional principal component analysis (FPCA) with a group minimax concave penalty (MCP) to impose sparsity on the functional coefficients. A computationally efficient group descent
algorithm is employed for optimization. Through numerical simulations, we demonstrate a satisfactory selection, estimation, and prediction accuracy of the proposed method across different
quantiles for both dense and sparsely observed functional data. The proposed method is applied to accelerometer data from the 2011-2014 National Health and Nutrition Examination Survey (NHANES) to identify key time-varying distributional patterns of physical activity and demographic predictors associated with cognitive function across different
quantiles. Our analysis provides new insights into the complex relationship between the daily distributional patterns of physical activity and cognitive function among older adults, capturing heterogeneous associations across different
quantiles.
Read the full abstract ▾
All authors Yuanzhen Yue, Stella Self, Yichao Wu, Jiajia Zhang, Rahul Ghosal
Machine Learning (Statistics)
SPDE Methods for Nonparametric Bayesian Posterior Contraction and Laplace Approximation
Enric Alberola-Boloix and Ioar Casado-Telletxea
We derive posterior contraction rates (PCRs) and finite-sample Bernstein von Mises (BvM) results for non-parametric
Bayesian models by extending the
diffusion-based framework of Mou et al. (2024) to the infinite-dimensional setting. The posterior is represented as the invariant measure of a Langevin
stochastic partial differential equation (SPDE) on a separable
Hilbert space, which allows us to control posterior moments and obtain non-asymptotic concentration rates in
Hilbert norms under various likelihood
curvature and regularity conditions. We also establish a quantitative Laplace approximation for the posterior. The theory is illustrated in a nonparametric linear Gaussian inverse problem.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
A Theoretical Framework for Energy-Aware Gradient Pruning in Federated Learning
Emmanouil M. Athanasakos
Federated Learning (FL) is constrained by the communication and energy limitations of decentralized edge devices. While gradient sparsification via Top-K magnitude
pruning effectively reduces the communication payload, it remains inherently energy-agnostic. It assumes all parameter updates incur identical downstream transmission and
memory-update costs, ignoring hardware realities. We formalize the
pruning process as an energy-constrained projection problem that accounts for the hardware-level disparities between
memory-intensive and compute-efficient operations during the post-backpropagation phase. We propose Cost-Weighted Magnitude
Pruning (CWMP), a selection rule that prioritizes parameter updates based on their magnitude relative to their physical cost. We demonstrate that CWMP is the optimal greedy solution to this constrained projection and provide a probabilistic analysis of its global energy efficiency. Numerical results on a non-IID CIFAR-10
benchmark show that CWMP consistently establishes a superior performance-energy Pareto frontier compared to the Top-K baseline.
Read the full abstract ▾
Statistical Methodology
Spline Quantile Regression with Cubic and Linear Smoothing Splines
Ta-Hsin Li
Spline
quantile regression (SQR) is a method introduced recently by Li and Megiddo (2026) for linear
quantile regression where the regression coefficients are treated as smooth functions of the
quantile level. With the coefficients represented by cubic splines with fixed
knots on a given set of
quantiles, the SQR method produces an estimate for the functional coefficients by solving a penalized
quantile regression problem. The $\ell_1$-norm of the second derivatives of the coefficients is employed as the penalty for regulating the roughness of the functional coefficients. This extends the SQR method by introducing additional pairings of the functional representation for the regression coefficients and the penalty for their roughness. The resulting cubic and linear SQR solutions are shown to be smoothing splines which are optimal in a functional space larger than the respective spline space with fixed
knots. It is shown that the cubic SQR can be reformulated and solved as a quadratic program and the linear SQR as a linear program. A simulation study demonstrates that the SQR solutions not only offer a concise functional representation of the regression coefficients with distinct smoothness characteristics, but also provide a capability of producing more accurate estimates of the regression coefficients when the underlying functions are suitably smooth. Application of the SQR solutions is demonstrated by real-data examples, including a Granger causality analysis of stock
market indices.
Read the full abstract ▾
Statistical Methodology
On inference in parametric survival data models
Nils Lid Hjort
The usual parametric models for survival data are of the following form. Some parametrically specified hazard rate $\alpha(s,\theta)$ is assumed for possibly censored random life times $X_1^0,\ldots,X_n^0$; one observes only $X_i=\min\{X_i^0,c_i\}$ and $\delta_i=I\{X_i^0\le c_i\}$ for certain censoring times $c_i$ that either are given or come from some censoring distribution. We study the following problems: What do the maximum likelihood
estimator and other
estimators really estimate when the true hazard rate $\alpha(s)$ is different from the parametric hazard rates? What is the limit distribution of an
estimator under such outside-the-model circumstances? How can traditional model-based analyses be made model-robust? Does the model-agnostic viewpoint invite alternative estimation approaches? What are the consequences of carrying out model-based and model-robust bootstrapping? How do theoretical and empirical influence functions generalise to situations with censored data? How do methods and results carry over to more complex models for life history data like regression models and Markov chains?
Read the full abstract ▾
Statistical Methodology
Normalised Local Hazard Plots
Nils Lid Hjort and Thomas Lumley
The purpose of this paper is to develop and illustrate certain classes of graphical plots that can be used for model verification in quite general survival data and life history data models. By suitably comparing nonparametric and parametric estimates of hazard rate functions over time a hazard comparison function can be constructed which under parametric model assumptions is approximately a zero-mean normal process. The test curves we propose are locally normalised versions of such hazard comparison functions. Under model conditions the test function is approximately a standard normal for each time point. This makes the normalised local hazard curves easy to interpret.We give explicit constructions for the most commonly used models of survival analysis, including the exponential, the Weibull, the Gompertz, the gamma, and for parametric Cox regression.
Algorithms carrying this out have been developed in Splus. Various theoretical and practical issues are discussed, including detection power and extensions to time-discrete models. Illustrations are given on simulated and real data.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels
Alexandra Zelenin and Alexandra Zhuravlyova
Weight-Decomposed Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction, but its forward pass requires the row-wise norm of W + sBA, a computation that every major framework we surveyed implements by materializing the dense [d_out, d_in] product BA. At d_in = 8192 and rank r = 384, a single module's norm requires about 512 MB of transient working
memory in bf16, making high-rank DoRA costly and often infeasible on common single-GPU setups once hundreds of adapted modules and checkpointing are involved. We present two systems contributions. A factored norm decomposes the squared norm into base, cross, and Gram terms computable through O(d_out r + r^2) intermediates, eliminating the dense product. Fused Triton kernels collapse the four-kernel DoRA composition into a single pass, reducing
memory traffic by about 4x and using a numerically stable form that avoids catastrophic cancellation in the near-unity rescaling regime where magnitude scales concentrate in practice. Across six 8-32B
vision-
language models (VLMs) on three NVIDIA GPUs (RTX 6000 PRO, H200, B200) at r = 384 in bf16, the fused implementation is 1.5-2.0x faster than Hugging Face PEFT's DoRA implementation for inference and 1.5-1.9x faster for gradient computation (optimizer step excluded), with up to 7 GB lower peak VRAM. Microbenchmarks on six GPUs spanning four architecture generations (L40S, A100, RTX 6000 PRO, H200, B200, B300) confirm 1.5-2.7x compose-kernel speedup. Final-logit cosine similarity exceeds 0.9999 across all model/GPU pairs, and multi-seed training curves match within 7.1 x 10^-4 mean per-step loss delta over 2000 steps.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Confidence-Based Decoding is Provably Efficient for Diffusion Language Models
Changxiao Cai and Gen Li
Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models for
language modeling, allowing flexible generation order and parallel generation of multiple
tokens. However, this flexibility introduces a challenge absent in AR models: the decoding strategy – which determines the order and number of
tokens generated at each iteration – critically affects sampling efficiency. Among decoding strategies explored in practice, confidence-based methods, which adaptively select which and how many
tokens to unmask based on prediction confidence, have shown strong empirical performance. Despite this success, our theoretical understanding of confidence-based decoding remains limited. In this work, we develop the first theoretical analysis framework for confidence-based decoding in DLMs. We focus on an
entropy sum-based strategy that continues unmasking
tokens within each iteration until the cumulative
entropy exceeds a threshold, and show that it achieves $\varepsilon$-accurate sampling in KL divergence with an expected number of iterations $\widetilde O(H(X_0)/\varepsilon)$, where $H(X_0)$ denotes the
entropy of the target data distribution. Notably, this strategy yields substantial sampling acceleration when the data distribution has low
entropy relative to the sequence length, while automatically adapting to the intrinsic complexity of data without requiring prior knowledge or hyperparameter tuning. Overall, our results provide a theoretical foundation for confidence-based decoding and may inform the design of more efficient decoding strategies for DLMs.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting
Qilin Wang
Modern time series
forecasting is evaluated almost entirely through passive observation of single historical trajectories, rendering claims about a model's robustness to non-stationarity fundamentally unfalsifiable. We propose a paradigm shift toward interventionist, exact-statistical benchmarking. By systematically titrating calibrated Gaussian observation noise into known chaotic and
stochastic dynamical systems, we transform
forecasting from a black-box sequence matching game into an exact distributional inference task. Because the underlying data-generating process and noise variance are mathematically explicit, evaluation can rely on exact negative log-likelihoods and calibrated distributional tests rather than heuristic approximations. To fully leverage this framework, we extend the Fern architecture into a probabilistic generative model that natively parameterizes the Symmetric Positive Definite (SPD) cone, outputting calibrated joint covariance structures without the computational bottleneck of generic Jacobian modeling. Under this rigorous evaluation, we find that state-of-the-art zero-shot foundation models behave consistently with the context-parroting mechanism, failing systematically under non-stationary regime shifts and elevated noise. In contrast, Fern explicitly captures the invariant measure and multivariate geometry of the underlying dynamics, maintaining structural fidelity and statistically sharp calibration precisely where massive sequence-matching models collapse.
Read the full abstract ▾
Statistical Methodology
Multiview Graph Fusion with Covariates
Sharmistha Guha, Jose Rodriguez-Acosta and Ivo Dinov
Joint modeling of multiview graphs with a common set of nodes between views and auxiliary predictors is an essential, yet less explored, area in statistical methodology. Traditional approaches often treat graphs in different views as independent or fail to adequately incorporate predictors, potentially missing complex dependencies within and across graph views and leading to reduced inferential accuracy. Motivated by such methodological shortcomings, we introduce an integrative
Bayesian approach for joint learning of a multiview graph with vector-valued predictors. Our modeling framework assumes a common set of nodes for each graph view while allowing for diverse interconnections or edge weights between nodes across graph views, accommodating both binary and continuous valued edge weights. By adopting a hierarchical
Bayesian modeling approach, our framework seamlessly integrates information from diverse graphs through carefully designed prior distributions on model parameters. This approach enables the estimation of crucial model parameters defining the relationship between these graph views and predictors, as well as offers predictive inference of the graph views. Crucially, the approach provides uncertainty quantification in all such inferences. Theoretical analysis establishes that the posterior predictive density for our model asymptotically converges to the true data-generating density, under mild assumptions on the true data-generating density and the growth of the number of graph nodes relative to the sample size. Simulation studies validate the inferential advantages of our approach over predictor-dependent tensor learning and independent learning of different graph views with predictors. We further illustrate model utility by analyzing functional connectivity graphs in neuroscience under cognitive control tasks, relating task-related
brain connectivity with phenotypic measures.
Read the full abstract ▾
Statistics Applications
Identification of physiological shock in intensive care units via Bayesian regime switching models
Emmett B. Kendall, Jonathan P. Williams, Curtis B. Storlie et al. (+3 more)
Detection of occult hemorrhage (i.e., internal bleeding) in patients in intensive care units (ICUs) can pose significant challenges for critical care workers. Because blood loss may not always be clinically apparent, clinicians rely on monitoring vital signs for specific trends indicative of a hemorrhage event. The inherent difficulties of diagnosing such an event can lead to late intervention by clinicians which has catastrophic consequences. Therefore, a methodology for early detection of hemorrhage has wide utility. We develop a
Bayesian regime switching model (RSM) that analyzes trends in patients' vitals and labs to provide a probabilistic assessment of the underlying physiological state that a patient is in at any given time. This article is motivated by a comprehensive
dataset we curated from Mayo Clinic of 33,924 real ICU patient encounters. Longitudinal response measurements are modeled as a vector autoregressive process conditional on all latent states up to the current time point, and the latent states follow a Markov process. We present a novel
Bayesian sampling routine to learn the posterior probability distribution of the latent physiological states, as well as develop an approach to account for pre-ICU-admission physiological changes. A simulation and real case study illustrate the effectiveness of our approach.
Read the full abstract ▾
All authors Emmett B. Kendall, Jonathan P. Williams, Curtis B. Storlie, Misty A. Radosevich, Erica D. Wittwer, Matthew A. Warner
Statistics Theory
Stable Algorithms Lower Bounds for Estimation
Xifan Yu and Ilias Zadik
In this work, we show that for all statistical estimation problems, a natural MMSE instability (discontinuity) condition implies the failure of stable
algorithms, serving as a version of OGP for estimation tasks. Using this criterion, we establish separations between stable and
polynomial-time
algorithms for the following MMSE-unstable tasks (i) Planted Shortest Path, where Dijkstra's
algorithm succeeds, (ii) random Parity Codes, where Gaussian elimination succeeds, and (iii) Gaussian Subset Sum, where
lattice-based methods succeed. For all three, we further show that all low-degree
polynomials are stable, yielding separations against low-degree methods and a new method to bound the low-degree MMSE. In particular, our technique highlights that MMSE instability is a common feature for Shortest Path and the noiseless Parity Codes and Gaussian subset sum. Last, we highlight that our work places rigorous algorithmic footing on the long-standing physics belief that first-order phase transitions–which in this setting translates to MMSE-instability impose fundamental limits on classes of efficient
algorithms.
Read the full abstract ▾
Statistics Applications
Generalized Sequential Monte Carlo Sampling for Redistricting Simulation
Philip O'Sullivan, Kosuke Imai and Cory McCartan
Simulation methods have become important tools for quantifying partisan and racial bias in redistricting plans. We generalize the Sequential Monte Carlo (SMC)
algorithm of McCartan and Imai (2023), one of the commonly used approaches. First, our generalized SMC (gSMC)
algorithm can split off regions of arbitrary size, rather than a single district as in the original SMC framework, enabling the sampling of multi-member districts with a varying number of representatives. Second, the gSMC
algorithm can operate over various sampling spaces, providing additional computational flexibility. Third, we derive optimal-variance incremental weights and show how to compute them efficiently for each sampling space, leading to more efficient sampling. Finally, we propose a hybrid gSMC-MCMC
algorithm by incorporating Markov chain Monte Carlo (MCMC) steps to handle large-scale redistricting applications without changing the target distribution. We demonstrate the effectiveness of the proposed methodology through analyses of the Irish Parliament, which uses multi-member districts of varying sizes, and the Pennsylvania House of Representatives, which has more than 200 single-member districts.
Read the full abstract ▾
Statistics Applications
Data Curation for Machine Learning Interatomic Potentials by Determinantal Point Processes
Joanna Zou and Youssef Marzouk
The development of machine learning interatomic potentials faces a critical computational bottleneck with the generation and labeling of useful training
datasets. We present a novel application of determinantal point processes (DPPs) to the task of selecting informative subsets of atomic configurations to label with reference energies and forces from costly
quantum mechanical methods. Through experiments with hafnium
oxide data, we show that DPPs are competitive with existing approaches to constructing compact but diverse training sets by utilizing kernels of molecular descriptors, leading to improved accuracy and robustness in machine learning representations of molecular systems. Our work identifies promising directions to employ DPPs for unsupervised training data curation with heterogeneous or
multimodal data, or in online active learning schemes for iterative data augmentation during molecular dynamics simulation.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Computationally lightweight classifiers with frequentist bounds on predictions
Shreeram Murali, Cristian R. Rojas and Dominik Baumann
While both classical and neural network classifiers can achieve high accuracy, they fall short on offering uncertainty bounds on their predictions, making them unfit for safety-critical applications. Existing kernel-based classifiers that provide such bounds scale with $\mathcal O (n^{\sim3})$ in time, making them computationally intractable for large
datasets. To address this, we propose a novel, computationally efficient classification
algorithm based on the Nadaraya-Watson
estimator, for whose estimates we derive frequentist uncertainty intervals. We evaluate our classifier on synthetically generated data and on electrocardiographic heartbeat signals from the MIT-BIH Arrhythmia database. We show that the method achieves competitive accuracy $>$\SI{96}{\percent} at $\mathcal O(n)$ and $\mathcal O(\log n)$ operations, while providing actionable uncertainty bounds. These bounds can, e.g., aid in flagging low-confidence predictions, making them suitable for real-time settings with resource constraints, such as diagnostic monitoring or implantable devices.
Read the full abstract ▾
Statistical Methodology
Detecting change regions on spheres
Di Su, Yining Chen and Tengyao Wang
While change point detection in time series data has been extensively studied, little
attention has been given to its generalisation to data observed on spheres or other
manifolds, where changes may occur within spatially complex regions with irregular boundaries, posing significant challenges. We propose a new class of
estimators, namely, Change Region Identification and SeParation (CRISP), to locate changes in the mean function of a signal-plus-noise model defined on $d$-dimensional spheres. The CRISP
estimator applies to scenarios with a single change region, and is extended to multiple change regions via a newly developed generic scheme. The convergence rate of the CRISP
estimator is shown to depend on the VC dimension of the hypothesis class that characterises the change regions in general. We also carefully study the case where change regions have the geometry of spherical caps. Simulations confirm the promising finite-sample performance of this approach. The CRISP
estimator's practical applicability is further demonstrated through two real data sets on global temperature and ozone hole.
Read the full abstract ▾
Machine Learning (Statistics)
Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data
Atticus Rex, Elizabeth Qian and David Peterson
Supervised machine learning describes the practice of fitting a parameterized model to labeled input-output data. Supervised machine learning methods have demonstrated promise in learning efficient surrogate models that can (partially) replace expensive high-fidelity models, making many-query analyses, such as optimization, uncertainty quantification, and inference, tractable. However, when training data must be obtained through the evaluation of an expensive model or experiment, the amount of training data that can be obtained is often limited, which can make learned surrogate models unreliable. In many engineering and scientific settings, cheaper low-fidelity models may be available, for example arising from simplified physics modeling or coarse grids. These models may be used to generate additional low-fidelity training data. The goal of multifidelity machine learning is to use both high- and low-fidelity training data to learn a surrogate model which is cheaper to evaluate than the high-fidelity model, but more accurate than any available low-fidelity model. This work proposes a new multifidelity training approach for Gaussian process regression which uses low-fidelity data to define additional features that augment the input space of the learned model. Similarly to cokriging
estimators, the proposed approach conditions the high-fidelity surrogate model on the predictions of all available low-fidelity surrogate models, while benefiting from the computational efficiency of autoregressive
estimators. Numerical experiments on several test problems demonstrate both increased predictive accuracy and reduced computational cost relative to the state of the art.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors
Julius Kobialka, Emanuel Sommer, Chris Kolb et al. (+3 more)
Bayesian neural network (BNN) posteriors are often considered impractical for inference, as
symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: balancedness, weight reallocation on equal-probability
manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions.
Read the full abstract ▾
All authors Julius Kobialka, Emanuel Sommer, Chris Kolb, Juntae Kwon, Daniel Dold, David Rügamer
Statistical Methodology
Cost-Aware Optimized Front-Door Experimental Design
Leopold Mareis and Mathias Drton
Causal effect estimation often succeeds cost-constrained sequential data collection. This work considers multivariate linear front-door models with arbitrary unobserved confounding on treatment and response. We optimize the experimental design by balancing the statistical efficiency and measurement costs through partial data. The full-data efficient influence function for the causal effect is derived, together with the geometry of all observed-data influence functions. This characterization yields a closed-form optimal sampling policy and an
estimator to minimize the asymptotic variance of regular asymptotically linear (RAL)
estimators within a class of augmented full-data influence functions. The resulting design also covers back-door estimation. In simulations and applications to biological, medical, and industrial
datasets, the optimized designs achieve substantial efficiency gains ($5.3\%$ to $31.9\%$) over naive full-sampling strategies.
Read the full abstract ▾
Statistical Methodology · Cross-listed from astro-ph.CO
A plug-and-play approach with fast uncertainty quantification for weak lensing mass mapping
Hubert Leterme, Andreas Tersenov, Jalal Fadili et al. (+1 more)
Upcoming stage-IV surveys such as Euclid and Rubin will deliver vast amounts of high-precision data, opening new opportunities to constrain cosmological models with unprecedented accuracy. A key step in this process is the reconstruction of the dark matter distribution from noisy weak-
lensing shear measurements. Current deep-learning-based mass-mapping methods achieve high reconstruction accuracy, but either require retraining a model for each new observed sky region (limiting practicality) or rely on slow Markov chain Monte Carlo sampling. Efficient exploitation of future survey data therefore calls for a new method that is accurate, flexible, and fast at inference. In addition, an uncertainty quantification with coverage guarantees is essential for a reliable cosmological parameter estimation. We introduce PnPMass, a plug-and-play approach for weak-
lensing mass mapping. The
algorithm produces point estimates by alternating between a gradient descent step with a carefully chosen data fidelity term and a denoising step implemented with a single deep-learning model trained on simulated data corrupted by Gaussian white noise. We also propose a fast sampling-free uncertainty quantification scheme based on moment networks, with calibrated error bars obtained through conformal prediction to ensure coverage guarantees. Finally, we
benchmark PnPMass against model-driven and data-driven mass-mapping techniques. PnPMass achieves a performance close to that of the currently best deep-learning methods while offering fast inference. It converges in just a few iterations, and it requires only a single training phase, regardless of the noise covariance of the observations. It therefore combines flexibility, efficiency, and reconstruction accuracy while delivering tighter error bars than existing approaches, making it well suited for upcoming weak-
lensing surveys.
Read the full abstract ▾
All authors Hubert Leterme, Andreas Tersenov, Jalal Fadili, Jean-Luc Starck