Peer Revenue Equity in science

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Tuesday 24 March

Statistics Theory

Efficient partially replicated block designs with each replication number one or two

We investigate block designs, under the A- and MV-criteria, when each treatment can have only one or two replications due to resource constraints, as can happen, for example, in early generation varietal trials. While these are commonly known as partially replicated designs, a key new feature of the present work is that no restriction about a constant block size is imposed on the subdesign consisting of the twice replicated treatments. This makes the derivation more challenging but allows us to entertain a wider class of competing designs and hence increases the flexibility of the results. Considering all treatments as equally important, design-independent, sharp lower bounds on the A- and MV-criteria are derived, so as to find highly efficient designs over this wider class. The roles of (a) linked block designs, (b) designs in an online catalog designtheory.org, and (c) partially balanced incomplete block designs, or duals thereof, as adapted to our setup, are explored at length. Illustrative examples are presented.
Read the full abstract

Statistics Theory

Fixed-level calibration of the Cauchy combination test

The Cauchy combination test (CCT) is widely used because it yields a closed-form combined $p$-value and is known to be asymptotically valid as the nominal level $\alpha\downarrow0$ under broad dependence structures. We study a different asymptotic question: whether the usual Cauchy cutoff remains accurate at an ordinary fixed level when the number $K$ of combined $p$-values grows under dependence. Under a canonical one-factor equicorrelated Gaussian copula model, we show that the raw CCT is generally not asymptotically exact at fixed $\alpha$. With fixed positive correlation, the statistic converges to a random latent-factor limit, so there is no universal fixed-level reference law. When the common correlation $\rho_K$ weakens with $K$, fixed-level behaviour is governed by the boundary-layer scale $s_K=\sqrt{\rho_K}(\log K)^{3/2}$, and the raw CCT is asymptotically exact if and only if $\rho_K(\log K)^3\to0$. Because the size distortion arises entirely from the reference law and not from the statistic, it can be corrected without modifying the test statistic itself. We propose the boundary-layer calibrated CCT (BL-CCT), which replaces the standard Cauchy reference by a one-parameter Gaussian-smoothed Cauchy family. Unlike recent variants that modify the test statistic, BL-CCT leaves the statistic unchanged and corrects only the reference law. BL-CCT is asymptotically exact under the weaker condition $\rho_K\log K\to0$ and provides a useful finite-$K$ approximation on bounded boundary layers. We also conduct several power analyses: although BL-CCT only raises the cutoff, it incurs no first-order power loss relative to the raw CCT on the exactness scale, under local dense, sparse, and dense Gaussian alternatives. Numerical experiments support the calibration theory.
Read the full abstract

Monday 23 March

Statistical Computation

GPU-Accelerated Sequential Monte Carlo for Bayesian Spectral Analysis

Bayesian spectral deconvolution provides a data-driven framework for mathematical model selection and parameter estimation from spectral data. Although highly versatile, it becomes computationally expensive as the number of model parameters, data points, and candidate models increases, often rendering practical applications infeasible. We propose a GPU-accelerated approach in which a sequential Monte Carlo sampler (SMCS) is run in parallel on a GPU to perform Bayesian model selection of the number of spectral peaks and Bayesian estimation of peak-function parameters. Numerical experiments demonstrate that the GPU-parallelized SMCS achieves speedups exceeding 500x over CPU-parallelized replica exchange Monte Carlo (REMC). The method is validated on artificial data designed to emulate X-ray photoelectron spectroscopy (XPS) and X-ray diffraction (XRD) measurements, as well as on real experimental spectra. As measurement techniques such as microscopic spectroscopy and in-situ methods continue to drive rapid growth in the volume of spectral data, the proposed approach offers a practical computational foundation for advanced analysis of individual datasets.
Read the full abstract

Machine Learning (Statistics)

Overfitting and Generalizing with (PAC) Bayesian Prediction in Noisy Binary Classification

We consider a PAC-Bayes type learning rule for binary classification, balancing the training error of a randomized ”posterior” predictor with its KL divergence to a pre-specified ”prior”. This can be seen as an extension of a modified two-part-code Minimum Description Length (MDL) learning rule, to continuous priors and randomized predictions. With a balancing parameter of $\lambda=1$ this learning rule recovers an (empirical) Bayes posterior and a modified variant recovers the profile posterior, linking with standard Bayesian prediction (up to the treatment of the single-parameter noise level). However, from a risk-minimization prediction perspective, this Bayesian predictor overfits and can lead to non-vanishing excess loss in the agnostic case. Instead a choice of $\lambda \gg 1$, which can be seen as using a sample-size-dependent-prior, ensures uniformly vanishing excess loss even in the agnostic case. We precisely characterize the effect of under-regularizing (and over-regularizing) as a function of the balance parameter $\lambda$, understanding the regimes in which this under-regularization is tempered or catastrophic. This work extends previous work by Zhu and Srebro [2025] that considered only discrete priors to PAC Bayes type learning rules and, through their rigorous Bayesian interpretation, to Bayesian prediction more generally.
Read the full abstract

Statistical Methodology

When lookout sees crackle: Anomaly detection via kernel density estimation

We present an updated version of lookout – an algorithm for detecting anomalies using kernel density estimates with bandwidth based on Rips death diameters – with theoretical guarantees. The kernel density estimator for updated lookout is shown to be consistent, and the proposed multivariate scaling is robust and efficient. We show our updated algorithm performs better than the previous version on diverse examples.
Read the full abstract

Statistics Theory

A Martingale Approach To Fluctuations of Rank Estimators in Sensitivity Analysis

Given a bivariate random pair $(X,Y)$, a natural problem is to estimate, from a single sample $(X_i,Y_i)_{1\le i\le n}$, quantities such as $\mathbb{E}\left[ \mathbb{E}[ Y\mid X ]^2 \right]$. More broadly, sensitivity indices are designed to quantify the possibly nonlinear influence of an input variable $X$ on an output variable $Y$. A classical example is the Sobol' index $$ \frac{\mathrm{Var}(\mathbb{E}[Y\mid X])}{\mathrm{Var}(Y)} \in [0,1] \ . $$ Another important example is the Cramér–von Mises (CvM) index. Following the pioneering work of Chatterjee \cite{chatterjee2021new}, consistent rank-based estimators are now available for such quantities. In this paper, we prove sharp fluctuation results using martingale methods. Our framework yields a unified treatment of the univariate Sobol' index, a multivariate extension involving several functions of the same scalar input, and the CvM index. As a consequence, we recover, unify, and simplify results from Gamboa et al. \cite{gamboa2022global, gamboa2023erratum}, Lin–Han \cite{lin2022limit}, and Kroll \cite{kroll2024asymptotic}. In particular, we work under minimal regularity assumptions. Furthermore, while the Gaussian fluctuation phenomenon itself was already known, the novelty lies in the structure of the asymptotic variance: for the CvM index, we obtain, to the best of our knowledge, the first explicit formula, while for the Sobol' index, we derive a new expression with a more structured form.
Read the full abstract

Statistical Methodology

Claims, Designs, and Guarantees: A Critical Narrative Review of Statistical Inference Reform and Practice, 2016–2026

Statistics helps scientists decide what their data show, but its most familiar tool, the $p$-value, is widely misunderstood. Between 2016 and 2026, professional bodies issued new guidance, and researchers developed or popularised alternatives and complements: compatibility intervals, equivalence tests, Bayes factors, and e-values that remain valid when data are checked repeatedly. Reporting rules for trials and reviews were also rewritten. Yet practice has changed little: in the biomedical literature, about 96% of papers that report $p$-values still include at least one below 0.05. This review argues that much of the confusion comes from treating these numbers as interchangeable. Each inferential procedure carries an intended interpretation and, where applicable, a formal guarantee, such as a 5% false-alarm rate, that holds only if the study was run and analysed in a particular way. We call this the Claim–Design–Guarantee framework: a result licenses a claim only when the claim, the design (assignment, sampling, monitoring, selection, and analysis), and the guarantee of the method are aligned. We trace the main developments of the decade, explain each tool in plain terms with formal details in boxes, and show with reproducible examples when the choice of tool changes the conclusion. For example, in a simulated two-group study with 100 interim looks, repeatedly applying an unadjusted $p<0.05$ rule produced false positives in 37.6% of studies, whereas the particular anytime-valid e-value procedure we used produced 2.2%, below its theoretical 5% upper bound. We end with a decision tree, a method-selection table, and a step-by-step workflow linked to current reporting guidelines.
Read the full abstract

Statistical Methodology

Multiple Jump MCMC: A Scalable Algorithm for Bayesian Inference on Binary Model Spaces

This article considers Bayesian model inference on binary model spaces. Binary model spaces are used by a large class of models, including graphical models, variable selection, mixture distributions, and decision trees. Traditional strategies in this field, such as reversible jump or birth-death MCMC algorithms, are still popular, despite suffering from a slow exploration of the model space. In this article, we propose an alternative: the Multiple Jump MCMC algorithm. The algorithm is simple, rejection-free, and remarkably fast. When applied to undirected Gaussian graphical models, it is 100 to 200 times faster than the state-of-the-art, solving models with $500,000$ parameters in less than a minute. We provide theorems showing how accurately our algorithm targets the posterior, and we show how to apply our framework to Gaussian graphical models, Ising models, and variable selection, but note that it applies to most Bayesian posterior inference on binary model spaces.
Read the full abstract

All authors Lucas Vogels, Reza Mohammadi, Marit Schoonhoven, Sinan Yildirim, Ilker Birbil

Statistical Methodology · Cross-listed from q-fin.RM

Proxy-Reliance Control in Conformal Recalibration of One-Sided Value-at-Risk

We introduce a proxy-reliance-controlled conformal recalibration framework for one-sided Value-at-Risk (VaR), and study a question that existing state-aware methods do not usually isolate: how strongly should the recalibration adjustment depend on an imperfect volatility proxy? We formalize this through a proxy-reliance parameter that continuously interpolates between an approximately constant-shift correction and a fully proxy-scaled correction. This makes proxy reliance a distinct and practically interpretable design choice in one-sided VaR recalibration. We show theoretically that larger proxy reliance increases the responsiveness of the tail adjustment to proxy scale, but also increases stressed-state fragility when the proxy underreacts. Empirically, in rolling out-of-sample tests on a six-ETF panel with VIX-linked state variables, and with supporting evidence from SPY, we find that the empirical value of proxy-reliance control lies in improved stressed-state robustness rather than uniform overall dominance. In particular, when the baseline forecast remains exposed to proxy imperfection in stressed states, lower or intermediate proxy reliance can outperform fully proxy-scaled recalibration in stressed left-tail VaR control.
Read the full abstract

Machine Learning (Statistics)

Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling

Preference-based fine-tuning has become an important component in training large language models, and the data used at this stage may contain sensitive user information. A central question is how to design a differentially private pipeline that is well suited to the distinct structure of reinforcement learning from human feedback. We propose a privacy-preserving framework that imposes differential privacy only on reward learning and derives the final policy from the resulting private reward model. Theoretically, we study the suboptimality gap and show that privacy contributes an additional additive term beyond the usual non-private statistical error. We also establish a minimax lower bound and show that the dominant term changes with sample size and privacy level, which in turn characterizes regimes in which the upper bound is rate-optimal up to logarithmic factors. Empirically, synthetic experiments confirm the scaling predicted by the theory, and experiments on the Anthropic HH-RLHF dataset using the Gemma-2B-IT model show stronger private alignment performance than existing differentially private baseline methods across privacy budgets.
Read the full abstract

Statistical Methodology

Variable Selection in Functional Linear Quantile Regression for Identifying Associations between Daily Patterns of Physical Activity and Cognitive Function

Quantile regression is useful for characterizing the conditional distribution of a response variable and understanding heterogeneity in the covariate effects at different quantiles. The rise of high-dimensional physiological data in biomedical research through wearable and sensor devices underscores the need for effective variable selection methods for interpretable and accurate quantile regression, which can offer robust insights into heterogeneous and dynamic covariate effects. We develop a flexible variable selection approach for functional linear quantile regression with multiple functional and scalar predictors. We use a smooth approximation of the quantile loss function and integrate functional principal component analysis (FPCA) with a group minimax concave penalty (MCP) to impose sparsity on the functional coefficients. A computationally efficient group descent algorithm is employed for optimization. Through numerical simulations, we demonstrate a satisfactory selection, estimation, and prediction accuracy of the proposed method across different quantiles for both dense and sparsely observed functional data. The proposed method is applied to accelerometer data from the 2011-2014 National Health and Nutrition Examination Survey (NHANES) to identify key time-varying distributional patterns of physical activity and demographic predictors associated with cognitive function across different quantiles. Our analysis provides new insights into the complex relationship between the daily distributional patterns of physical activity and cognitive function among older adults, capturing heterogeneous associations across different quantiles.
Read the full abstract

All authors Yuanzhen Yue, Stella Self, Yichao Wu, Jiajia Zhang, Rahul Ghosal

Machine Learning (Statistics)

SPDE Methods for Nonparametric Bayesian Posterior Contraction and Laplace Approximation

We derive posterior contraction rates (PCRs) and finite-sample Bernstein von Mises (BvM) results for non-parametric Bayesian models by extending the diffusion-based framework of Mou et al. (2024) to the infinite-dimensional setting. The posterior is represented as the invariant measure of a Langevin stochastic partial differential equation (SPDE) on a separable Hilbert space, which allows us to control posterior moments and obtain non-asymptotic concentration rates in Hilbert norms under various likelihood curvature and regularity conditions. We also establish a quantitative Laplace approximation for the posterior. The theory is illustrated in a nonparametric linear Gaussian inverse problem.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

A Theoretical Framework for Energy-Aware Gradient Pruning in Federated Learning

Federated Learning (FL) is constrained by the communication and energy limitations of decentralized edge devices. While gradient sparsification via Top-K magnitude pruning effectively reduces the communication payload, it remains inherently energy-agnostic. It assumes all parameter updates incur identical downstream transmission and memory-update costs, ignoring hardware realities. We formalize the pruning process as an energy-constrained projection problem that accounts for the hardware-level disparities between memory-intensive and compute-efficient operations during the post-backpropagation phase. We propose Cost-Weighted Magnitude Pruning (CWMP), a selection rule that prioritizes parameter updates based on their magnitude relative to their physical cost. We demonstrate that CWMP is the optimal greedy solution to this constrained projection and provide a probabilistic analysis of its global energy efficiency. Numerical results on a non-IID CIFAR-10 benchmark show that CWMP consistently establishes a superior performance-energy Pareto frontier compared to the Top-K baseline.
Read the full abstract

Statistical Methodology

Spline Quantile Regression with Cubic and Linear Smoothing Splines

Spline quantile regression (SQR) is a method introduced recently by Li and Megiddo (2026) for linear quantile regression where the regression coefficients are treated as smooth functions of the quantile level. With the coefficients represented by cubic splines with fixed knots on a given set of quantiles, the SQR method produces an estimate for the functional coefficients by solving a penalized quantile regression problem. The $\ell_1$-norm of the second derivatives of the coefficients is employed as the penalty for regulating the roughness of the functional coefficients. This extends the SQR method by introducing additional pairings of the functional representation for the regression coefficients and the penalty for their roughness. The resulting cubic and linear SQR solutions are shown to be smoothing splines which are optimal in a functional space larger than the respective spline space with fixed knots. It is shown that the cubic SQR can be reformulated and solved as a quadratic program and the linear SQR as a linear program. A simulation study demonstrates that the SQR solutions not only offer a concise functional representation of the regression coefficients with distinct smoothness characteristics, but also provide a capability of producing more accurate estimates of the regression coefficients when the underlying functions are suitably smooth. Application of the SQR solutions is demonstrated by real-data examples, including a Granger causality analysis of stock market indices.
Read the full abstract

Statistical Methodology

On inference in parametric survival data models

The usual parametric models for survival data are of the following form. Some parametrically specified hazard rate $\alpha(s,\theta)$ is assumed for possibly censored random life times $X_1^0,\ldots,X_n^0$; one observes only $X_i=\min\{X_i^0,c_i\}$ and $\delta_i=I\{X_i^0\le c_i\}$ for certain censoring times $c_i$ that either are given or come from some censoring distribution. We study the following problems: What do the maximum likelihood estimator and other estimators really estimate when the true hazard rate $\alpha(s)$ is different from the parametric hazard rates? What is the limit distribution of an estimator under such outside-the-model circumstances? How can traditional model-based analyses be made model-robust? Does the model-agnostic viewpoint invite alternative estimation approaches? What are the consequences of carrying out model-based and model-robust bootstrapping? How do theoretical and empirical influence functions generalise to situations with censored data? How do methods and results carry over to more complex models for life history data like regression models and Markov chains?
Read the full abstract

2603.22374 · 23 Mar · 34 pages, no figures. Statistical Research Report, Department of Mathematics, University of Oslo, October 1991. The paper is published in essentially this form in International Statistical Review, 1992, vol. 60, pages 355-387, at this url: https://www.jstor.org/stable/1403683?seq=1

Statistical Methodology

Normalised Local Hazard Plots

The purpose of this paper is to develop and illustrate certain classes of graphical plots that can be used for model verification in quite general survival data and life history data models. By suitably comparing nonparametric and parametric estimates of hazard rate functions over time a hazard comparison function can be constructed which under parametric model assumptions is approximately a zero-mean normal process. The test curves we propose are locally normalised versions of such hazard comparison functions. Under model conditions the test function is approximately a standard normal for each time point. This makes the normalised local hazard curves easy to interpret.We give explicit constructions for the most commonly used models of survival analysis, including the exponential, the Weibull, the Gompertz, the gamma, and for parametric Cox regression. Algorithms carrying this out have been developed in Splus. Various theoretical and practical issues are discussed, including detection power and extensions to time-discrete models. Illustrations are given on simulated and real data.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels

Weight-Decomposed Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction, but its forward pass requires the row-wise norm of W + sBA, a computation that every major framework we surveyed implements by materializing the dense [d_out, d_in] product BA. At d_in = 8192 and rank r = 384, a single module's norm requires about 512 MB of transient working memory in bf16, making high-rank DoRA costly and often infeasible on common single-GPU setups once hundreds of adapted modules and checkpointing are involved. We present two systems contributions. A factored norm decomposes the squared norm into base, cross, and Gram terms computable through O(d_out r + r^2) intermediates, eliminating the dense product. Fused Triton kernels collapse the four-kernel DoRA composition into a single pass, reducing memory traffic by about 4x and using a numerically stable form that avoids catastrophic cancellation in the near-unity rescaling regime where magnitude scales concentrate in practice. Across six 8-32B vision-language models (VLMs) on three NVIDIA GPUs (RTX 6000 PRO, H200, B200) at r = 384 in bf16, the fused implementation is 1.5-2.0x faster than Hugging Face PEFT's DoRA implementation for inference and 1.5-1.9x faster for gradient computation (optimizer step excluded), with up to 7 GB lower peak VRAM. Microbenchmarks on six GPUs spanning four architecture generations (L40S, A100, RTX 6000 PRO, H200, B200, B300) confirm 1.5-2.7x compose-kernel speedup. Final-logit cosine similarity exceeds 0.9999 across all model/GPU pairs, and multi-seed training curves match within 7.1 x 10^-4 mean per-step loss delta over 2000 steps.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Confidence-Based Decoding is Provably Efficient for Diffusion Language Models

Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models for language modeling, allowing flexible generation order and parallel generation of multiple tokens. However, this flexibility introduces a challenge absent in AR models: the decoding strategy – which determines the order and number of tokens generated at each iteration – critically affects sampling efficiency. Among decoding strategies explored in practice, confidence-based methods, which adaptively select which and how many tokens to unmask based on prediction confidence, have shown strong empirical performance. Despite this success, our theoretical understanding of confidence-based decoding remains limited. In this work, we develop the first theoretical analysis framework for confidence-based decoding in DLMs. We focus on an entropy sum-based strategy that continues unmasking tokens within each iteration until the cumulative entropy exceeds a threshold, and show that it achieves $\varepsilon$-accurate sampling in KL divergence with an expected number of iterations $\widetilde O(H(X_0)/\varepsilon)$, where $H(X_0)$ denotes the entropy of the target data distribution. Notably, this strategy yields substantial sampling acceleration when the data distribution has low entropy relative to the sequence length, while automatically adapting to the intrinsic complexity of data without requiring prior knowledge or hyperparameter tuning. Overall, our results provide a theoretical foundation for confidence-based decoding and may inform the design of more efficient decoding strategies for DLMs.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting

Modern time series forecasting is evaluated almost entirely through passive observation of single historical trajectories, rendering claims about a model's robustness to non-stationarity fundamentally unfalsifiable. We propose a paradigm shift toward interventionist, exact-statistical benchmarking. By systematically titrating calibrated Gaussian observation noise into known chaotic and stochastic dynamical systems, we transform forecasting from a black-box sequence matching game into an exact distributional inference task. Because the underlying data-generating process and noise variance are mathematically explicit, evaluation can rely on exact negative log-likelihoods and calibrated distributional tests rather than heuristic approximations. To fully leverage this framework, we extend the Fern architecture into a probabilistic generative model that natively parameterizes the Symmetric Positive Definite (SPD) cone, outputting calibrated joint covariance structures without the computational bottleneck of generic Jacobian modeling. Under this rigorous evaluation, we find that state-of-the-art zero-shot foundation models behave consistently with the context-parroting mechanism, failing systematically under non-stationary regime shifts and elevated noise. In contrast, Fern explicitly captures the invariant measure and multivariate geometry of the underlying dynamics, maintaining structural fidelity and statistically sharp calibration precisely where massive sequence-matching models collapse.
Read the full abstract

Statistical Methodology

Multiview Graph Fusion with Covariates

Joint modeling of multiview graphs with a common set of nodes between views and auxiliary predictors is an essential, yet less explored, area in statistical methodology. Traditional approaches often treat graphs in different views as independent or fail to adequately incorporate predictors, potentially missing complex dependencies within and across graph views and leading to reduced inferential accuracy. Motivated by such methodological shortcomings, we introduce an integrative Bayesian approach for joint learning of a multiview graph with vector-valued predictors. Our modeling framework assumes a common set of nodes for each graph view while allowing for diverse interconnections or edge weights between nodes across graph views, accommodating both binary and continuous valued edge weights. By adopting a hierarchical Bayesian modeling approach, our framework seamlessly integrates information from diverse graphs through carefully designed prior distributions on model parameters. This approach enables the estimation of crucial model parameters defining the relationship between these graph views and predictors, as well as offers predictive inference of the graph views. Crucially, the approach provides uncertainty quantification in all such inferences. Theoretical analysis establishes that the posterior predictive density for our model asymptotically converges to the true data-generating density, under mild assumptions on the true data-generating density and the growth of the number of graph nodes relative to the sample size. Simulation studies validate the inferential advantages of our approach over predictor-dependent tensor learning and independent learning of different graph views with predictors. We further illustrate model utility by analyzing functional connectivity graphs in neuroscience under cognitive control tasks, relating task-related brain connectivity with phenotypic measures.
Read the full abstract

Statistics Applications

Identification of physiological shock in intensive care units via Bayesian regime switching models

Detection of occult hemorrhage (i.e., internal bleeding) in patients in intensive care units (ICUs) can pose significant challenges for critical care workers. Because blood loss may not always be clinically apparent, clinicians rely on monitoring vital signs for specific trends indicative of a hemorrhage event. The inherent difficulties of diagnosing such an event can lead to late intervention by clinicians which has catastrophic consequences. Therefore, a methodology for early detection of hemorrhage has wide utility. We develop a Bayesian regime switching model (RSM) that analyzes trends in patients' vitals and labs to provide a probabilistic assessment of the underlying physiological state that a patient is in at any given time. This article is motivated by a comprehensive dataset we curated from Mayo Clinic of 33,924 real ICU patient encounters. Longitudinal response measurements are modeled as a vector autoregressive process conditional on all latent states up to the current time point, and the latent states follow a Markov process. We present a novel Bayesian sampling routine to learn the posterior probability distribution of the latent physiological states, as well as develop an approach to account for pre-ICU-admission physiological changes. A simulation and real case study illustrate the effectiveness of our approach.
Read the full abstract

All authors Emmett B. Kendall, Jonathan P. Williams, Curtis B. Storlie, Misty A. Radosevich, Erica D. Wittwer, Matthew A. Warner

Statistics Theory

Stable Algorithms Lower Bounds for Estimation

In this work, we show that for all statistical estimation problems, a natural MMSE instability (discontinuity) condition implies the failure of stable algorithms, serving as a version of OGP for estimation tasks. Using this criterion, we establish separations between stable and polynomial-time algorithms for the following MMSE-unstable tasks (i) Planted Shortest Path, where Dijkstra's algorithm succeeds, (ii) random Parity Codes, where Gaussian elimination succeeds, and (iii) Gaussian Subset Sum, where lattice-based methods succeed. For all three, we further show that all low-degree polynomials are stable, yielding separations against low-degree methods and a new method to bound the low-degree MMSE. In particular, our technique highlights that MMSE instability is a common feature for Shortest Path and the noiseless Parity Codes and Gaussian subset sum. Last, we highlight that our work places rigorous algorithmic footing on the long-standing physics belief that first-order phase transitions–which in this setting translates to MMSE-instability impose fundamental limits on classes of efficient algorithms.
Read the full abstract

Statistics Applications

Generalized Sequential Monte Carlo Sampling for Redistricting Simulation

Simulation methods have become important tools for quantifying partisan and racial bias in redistricting plans. We generalize the Sequential Monte Carlo (SMC) algorithm of McCartan and Imai (2023), one of the commonly used approaches. First, our generalized SMC (gSMC) algorithm can split off regions of arbitrary size, rather than a single district as in the original SMC framework, enabling the sampling of multi-member districts with a varying number of representatives. Second, the gSMC algorithm can operate over various sampling spaces, providing additional computational flexibility. Third, we derive optimal-variance incremental weights and show how to compute them efficiently for each sampling space, leading to more efficient sampling. Finally, we propose a hybrid gSMC-MCMC algorithm by incorporating Markov chain Monte Carlo (MCMC) steps to handle large-scale redistricting applications without changing the target distribution. We demonstrate the effectiveness of the proposed methodology through analyses of the Irish Parliament, which uses multi-member districts of varying sizes, and the Pennsylvania House of Representatives, which has more than 200 single-member districts.
Read the full abstract

Statistics Applications

Data Curation for Machine Learning Interatomic Potentials by Determinantal Point Processes

The development of machine learning interatomic potentials faces a critical computational bottleneck with the generation and labeling of useful training datasets. We present a novel application of determinantal point processes (DPPs) to the task of selecting informative subsets of atomic configurations to label with reference energies and forces from costly quantum mechanical methods. Through experiments with hafnium oxide data, we show that DPPs are competitive with existing approaches to constructing compact but diverse training sets by utilizing kernels of molecular descriptors, leading to improved accuracy and robustness in machine learning representations of molecular systems. Our work identifies promising directions to employ DPPs for unsupervised training data curation with heterogeneous or multimodal data, or in online active learning schemes for iterative data augmentation during molecular dynamics simulation.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Computationally lightweight classifiers with frequentist bounds on predictions

While both classical and neural network classifiers can achieve high accuracy, they fall short on offering uncertainty bounds on their predictions, making them unfit for safety-critical applications. Existing kernel-based classifiers that provide such bounds scale with $\mathcal O (n^{\sim3})$ in time, making them computationally intractable for large datasets. To address this, we propose a novel, computationally efficient classification algorithm based on the Nadaraya-Watson estimator, for whose estimates we derive frequentist uncertainty intervals. We evaluate our classifier on synthetically generated data and on electrocardiographic heartbeat signals from the MIT-BIH Arrhythmia database. We show that the method achieves competitive accuracy $>$\SI{96}{\percent} at $\mathcal O(n)$ and $\mathcal O(\log n)$ operations, while providing actionable uncertainty bounds. These bounds can, e.g., aid in flagging low-confidence predictions, making them suitable for real-time settings with resource constraints, such as diagnostic monitoring or implantable devices.
Read the full abstract

Statistical Methodology

Detecting change regions on spheres

While change point detection in time series data has been extensively studied, little attention has been given to its generalisation to data observed on spheres or other manifolds, where changes may occur within spatially complex regions with irregular boundaries, posing significant challenges. We propose a new class of estimators, namely, Change Region Identification and SeParation (CRISP), to locate changes in the mean function of a signal-plus-noise model defined on $d$-dimensional spheres. The CRISP estimator applies to scenarios with a single change region, and is extended to multiple change regions via a newly developed generic scheme. The convergence rate of the CRISP estimator is shown to depend on the VC dimension of the hypothesis class that characterises the change regions in general. We also carefully study the case where change regions have the geometry of spherical caps. Simulations confirm the promising finite-sample performance of this approach. The CRISP estimator's practical applicability is further demonstrated through two real data sets on global temperature and ozone hole.
Read the full abstract

Machine Learning (Statistics)

Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data

Supervised machine learning describes the practice of fitting a parameterized model to labeled input-output data. Supervised machine learning methods have demonstrated promise in learning efficient surrogate models that can (partially) replace expensive high-fidelity models, making many-query analyses, such as optimization, uncertainty quantification, and inference, tractable. However, when training data must be obtained through the evaluation of an expensive model or experiment, the amount of training data that can be obtained is often limited, which can make learned surrogate models unreliable. In many engineering and scientific settings, cheaper low-fidelity models may be available, for example arising from simplified physics modeling or coarse grids. These models may be used to generate additional low-fidelity training data. The goal of multifidelity machine learning is to use both high- and low-fidelity training data to learn a surrogate model which is cheaper to evaluate than the high-fidelity model, but more accurate than any available low-fidelity model. This work proposes a new multifidelity training approach for Gaussian process regression which uses low-fidelity data to define additional features that augment the input space of the learned model. Similarly to cokriging estimators, the proposed approach conditions the high-fidelity surrogate model on the predictions of all available low-fidelity surrogate models, while benefiting from the computational efficiency of autoregressive estimators. Numerical experiments on several test problems demonstrate both increased predictive accuracy and reduced computational cost relative to the state of the art.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors

Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: balancedness, weight reallocation on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions.
Read the full abstract

All authors Julius Kobialka, Emanuel Sommer, Chris Kolb, Juntae Kwon, Daniel Dold, David Rügamer

Statistical Methodology

Cost-Aware Optimized Front-Door Experimental Design

Causal effect estimation often succeeds cost-constrained sequential data collection. This work considers multivariate linear front-door models with arbitrary unobserved confounding on treatment and response. We optimize the experimental design by balancing the statistical efficiency and measurement costs through partial data. The full-data efficient influence function for the causal effect is derived, together with the geometry of all observed-data influence functions. This characterization yields a closed-form optimal sampling policy and an estimator to minimize the asymptotic variance of regular asymptotically linear (RAL) estimators within a class of augmented full-data influence functions. The resulting design also covers back-door estimation. In simulations and applications to biological, medical, and industrial datasets, the optimized designs achieve substantial efficiency gains ($5.3\%$ to $31.9\%$) over naive full-sampling strategies.
Read the full abstract

Statistical Methodology · Cross-listed from astro-ph.CO

A plug-and-play approach with fast uncertainty quantification for weak lensing mass mapping

Upcoming stage-IV surveys such as Euclid and Rubin will deliver vast amounts of high-precision data, opening new opportunities to constrain cosmological models with unprecedented accuracy. A key step in this process is the reconstruction of the dark matter distribution from noisy weak-lensing shear measurements. Current deep-learning-based mass-mapping methods achieve high reconstruction accuracy, but either require retraining a model for each new observed sky region (limiting practicality) or rely on slow Markov chain Monte Carlo sampling. Efficient exploitation of future survey data therefore calls for a new method that is accurate, flexible, and fast at inference. In addition, an uncertainty quantification with coverage guarantees is essential for a reliable cosmological parameter estimation. We introduce PnPMass, a plug-and-play approach for weak-lensing mass mapping. The algorithm produces point estimates by alternating between a gradient descent step with a carefully chosen data fidelity term and a denoising step implemented with a single deep-learning model trained on simulated data corrupted by Gaussian white noise. We also propose a fast sampling-free uncertainty quantification scheme based on moment networks, with calibrated error bars obtained through conformal prediction to ensure coverage guarantees. Finally, we benchmark PnPMass against model-driven and data-driven mass-mapping techniques. PnPMass achieves a performance close to that of the currently best deep-learning methods while offering fast inference. It converges in just a few iterations, and it requires only a single training phase, regardless of the noise covariance of the observations. It therefore combines flexibility, efficiency, and reconstruction accuracy while delivering tighter error bars than existing approaches, making it well suited for upcoming weak-lensing surveys.
Read the full abstract

All authors Hubert Leterme, Andreas Tersenov, Jalal Fadili, Jean-Luc Starck