Machine Learning (Statistics)
Beyond Normality: Reliable A/B Testing with Non-Gaussian Data
Junpeng Gong, Chunkai Wang, Hao Li et al. (+3 more)
A/B testing has become the cornerstone of decision-making in online
markets, guiding how platforms launch new features, optimize pricing strategies, and improve user experience. In practice, we typically employ the pairwise $t$-test to compare outcomes between the treatment and control groups, thereby assessing the effectiveness of a given strategy. To be trustworthy, these experiments must keep Type I error (i.e., false positive rate) under control; otherwise, we may launch harmful strategies. However, in real-world applications, we find that A/B testing often fails to deliver reliable results. When the data distribution departs from normality or when the treatment and control groups differ in sample size, the commonly used pairwise $t$-test is no longer trustworthy. In this paper, we quantify how skewed, long tailed data and unequal allocation distort error rates and derive explicit formulas for the minimum sample size required for the $t$-test to remain valid. We find that many online feedback metrics require hundreds of millions samples to ensure reliable A/B testing. Thus we introduce an Edgeworth-based correction that provides more accurate $p$-values when the available sample size is limited. Offline experiments on a leading A/B testing platform corroborate the practical value of our theoretical minimum sample size thresholds and demonstrate that the corrected method substantially improves the reliability of A/B testing in real-world conditions.
Read the full abstract ▾
All authors Junpeng Gong, Chunkai Wang, Hao Li, Jinyong Ma, Haoxuan Li, Xu He
Statistical Methodology
Unifying regression-based and design-based causal inference in time-series experiments
Zhexiao Lin and Peng Ding
Time-series experiments, also called switchback experiments or N-of-1 trials, play increasingly important roles in modern applications in medical and industrial areas. Under the potential outcomes framework, recent research has studied time-series experiments from the design-based perspective, relying solely on the randomness in the design to drive the statistical inference. Focusing on simpler statistical methods, we examine the design-based properties of regression-based methods for estimating treatment effects in time-series experiments. We demonstrate that the treatment effects of interest can be consistently estimated using ordinary least squares with an appropriately specified working model and transformed regressors. Our analysis allows for estimating a diverging number of treatment effects simultaneously, and establishes the consistency and asymptotic normality of the regression-based
estimators. Additionally, we show that asymptotically, the heteroskedasticity and autocorrelation consistent variance
estimators provide conservative estimates of the true, design-based variances. Importantly, although our approach relies on regression, our design-based framework allows for misspecification of the regression model.
Read the full abstract ▾
Statistical Methodology
A powerful goodness-of-fit test using the probability integral transform of order statistics
Christian T. Covington and Jeffrey W. Miller
Goodness-of-fit (GoF) tests are a fundamental component of statistical practice, essential for checking model assumptions and testing scientific hypotheses. Despite their widespread use, popular GoF tests exhibit surprisingly low statistical power against substantial departures from the null hypothesis. To address this, we introduce PITOS, a novel GoF test based on applying the probability integral transform (PIT) to the $j$th order statistic (OS) given the $i$th order statistic for selected pairs $i,j$. Under the null, for any pair $i,j$, this yields a $\mathrm{Uniform}(0,1)$ random variable, which we map to a p-value via $u\mapsto 2\min(u, 1-u)$. We compute these p-values for a structured collection of pairs $i,j$ generated via a discretized transformed Halton sequence, and aggregate them using the Cauchy combination technique to obtain the PITOS p-value. Our method maintains approximately valid Type I error control, has an efficient $O(n \log n)$ runtime, and can be used with any null distribution via the Rosenblatt transform. In empirical demonstrations, we find that PITOS has much higher power than popular GoF tests on distributions characterized by local departures from the null, while maintaining competitive power across all distributions tested.
Read the full abstract ▾
Statistical Methodology
Testing for a common subspace in compositional datasets with structural zeros
Francesco Porro, Fabio Rapallo and Sara Sommariva
In this paper, we consider the problem of testing for a common principal subspace in compositional
datasets with structural zeros. In particular, we address the problem in the general setting in which two groups are compared, each characterized by its own pattern of structural zeros. This situation prevents the direct use of standard logratio-based principal subspace comparison methods, since the two groups cannot be represented in a common logratio coordinate system in the usual way. We thus define a test for the presence of a common subspace that is fully compatible with the Aitchison geometry and logratio analysis. The proposed construction allows the principal subspaces associated with the two groups to be compared despite the different patterns of structural zeros. Under logratio-normality, we derive an analytical approximation to the null distribution of the test statistic. Alongside this parametric version, we also introduce a nonparametric bootstrap procedure that does not rely on distributional assumptions. The finite-sample behaviour of the test is investigated through simulations. The method is finally illustrated on a reproducible microbiome
dataset available through Bioconductor.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Self-induced stochastic resonance: A physics-informed machine learning approach
Divyesh Savaliya and Marius E. Yamakou
Self-induced
stochastic resonance (SISR) is the emergence of coherent oscillations in slow-fast excitable systems driven solely by noise, without external periodic forcing or proximity to a bifurcation. This work presents a physics-informed machine learning framework for modeling and predicting SISR in the
stochastic FitzHugh-Nagumo
neuron. We embed the governing
stochastic differential equations and SISR-asymptotic timescale-matching constraints directly into a Physics-Informed Neural Network (PINN) based on a Noise-Augmented State Predictor architecture. The composite loss integrates data fidelity, dynamical residuals, and barrier-based physical constraints derived from Kramers' escape theory. The trained PINN accurately predicts the dependence of spike-train coherence on noise intensity, excitability, and timescale separation, matching results from direct
stochastic simulations with substantial improvements in accuracy and generalization compared with purely data-driven methods, while requiring significantly less computation. The framework provides a data-efficient and interpretable surrogate model for simulating and analyzing noise-induced coherence in multiscale
stochastic systems.
Read the full abstract ▾
Statistical Computation · Cross-listed from cs.LG
Clustering by Denoising: Latent plug-and-play diffusion for single-cell data
Dominik Meier, Shixing Yu, Sagnik Nandy et al. (+2 more)
Single-
cell RNA sequencing (scRNA-seq) enables the study of cellular heterogeneity. Yet, clustering accuracy, and with it downstream analyses based on
cell labels, remain challenging due to measurement noise and biological variability. In standard latent spaces (e.g., obtained through PCA), data from different
cell types can be projected close together, making accurate clustering difficult. We introduce a latent plug-and-play
diffusion framework that separates the observation and denoising space. This separation is operationalized through a novel Gibbs sampling procedure: the learned
diffusion prior is applied in a low-dimensional latent space to perform denoising, while to steer this process, noise is reintroduced into the original high-dimensional observation space. This unique "input-space steering" ensures the denoising trajectory remains faithful to the original data structure. Our approach offers three key advantages: (1) adaptive noise handling via a tunable balance between prior and observed data; (2) uncertainty quantification through principled uncertainty estimates for downstream analysis; and (3) generalizable denoising by leveraging clean reference data to denoise noisier
datasets, and via averaging, improve quality beyond the training set. We evaluate robustness on both synthetic and real single-
cell genomics data. Our method improves clustering accuracy on synthetic data across varied noise levels and
dataset shifts. On real-world single-
cell data, our method demonstrates improved biological coherence in the resulting
cell clusters, with cluster boundaries that better align with known
cell type markers and developmental trajectories.
Read the full abstract ▾
All authors Dominik Meier, Shixing Yu, Sagnik Nandy, Promit Ghosal, Kyra Gan
Machine Learning (Statistics) · Cross-listed from cs.AI
HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning
Long H Dang and David Rawlinson
The Hierarchical
Reasoning Model (HRM) has impressive
reasoning abilities given its small size, but has only been applied to supervised, static, fully-observable problems. One of HRM's strengths is its ability to adapt its computational effort to the difficulty of the problem. However, in its current form it cannot integrate and reuse computation from previous time-steps if the problem is dynamic, uncertain or partially observable, or be applied where the correct action is undefined, characteristics of many real-world problems. This paper presents HRM-
Agent, a variant of HRM trained using only reinforcement learning. We show that HRM can learn to navigate to goals in dynamic and uncertain maze environments. Recent work suggests that HRM's
reasoning abilities stem from its recurrent inference process. We explore the dynamics of the recurrent inference process and find evidence that it is successfully reusing computation from earlier environment time-steps.
Read the full abstract ▾
Statistical Methodology
Efficiently Learning Synthetic Control Models for High-dimensional Disaggregated Data
Ye Shen, Rui Song and Alberto Abadie
The Synthetic Control method (SC) has become a valuable tool for estimating causal effects. Originally designed for single-treated unit scenarios, it has recently found applications in high-dimensional disaggregated settings with multiple treated units. However, challenges in practical implementation and computational efficiency arise in such scenarios. To tackle these challenges, we propose a novel approach that integrates the Multivariate Square-root Lasso method into the synthetic control framework. We rigorously establish the estimation error bounds for fitting the Synthetic Control weights using Multivariate Square-root Lasso, accommodating high-dimensionality and time series dependencies. Additionally, we quantify the estimation error for the Average Treatment Effect on the Treated (ATT). Through simulation studies, we demonstrate that our method offers superior computational efficiency without compromising estimation accuracy. We apply our method to assess the causal impact of COVID-19 Stay-at-Home Orders on the monthly unemployment rate in the United States at the county level.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
A Theory of the Mechanics of Information: Generalization Through Measurement of Uncertainty (Learning is Measuring)
Christopher J. Hazard, Michael Resnick, Jacob Beel et al. (+7 more)
Traditional machine learning relies on explicit models and domain assumptions, often limiting flexibility and interpretability. We introduce a "model-free" framework of instance-based learning using surprisal (information theoretic uncertainty) to directly analyze and perform inferences from raw data, eliminating distribution modeling, reducing bias, and enabling efficient updates including direct edits and deletion of training data. By quantifying relevance through uncertainty, the approach enables generalizable inference across tasks including generative inference, causal discovery, anomaly detection, and time series
forecasting. Our approach emphasizes traceability, interpretability, and data-driven decision making, offering a unified, human-understandable framework for machine learning, can be adapted to virtually all machine learning tasks, achieving competency across all and state-of-the-art performance on anomaly detection and some supervised tasks. The mathematical foundations create a "physics" of information, which enable these techniques to apply effectively to a wide variety of complex data types, including missing data. Our results suggest that this may be a viable foundation upon which to build scalable machine learning and artificial intelligence that can maintain human understandability of the underlying mechanics.
Read the full abstract ▾
All authors Christopher J. Hazard, Michael Resnick, Jacob Beel, Jack Xia, Cade Mack, Dominic Glennie, Matthew Fulp, David Maze, Andrew Bassett, Martin Koistinen
Statistics Theory
Composite goodness-of-fit test with the Kernel Stein Discrepancy and a bootstrap for degenerate U-statistics with estimated parameters
Florian Brück, Veronika Reimoser and Fabian Baier
This paper formally derives the asymptotic distribution of a goodness-of-fit test based on the Kernel Stein Discrepancy introduced in (Oscar Key et al., "Composite Goodness-of-fit Tests with Kernels", Journal of Machine Learning Research 26.51 (2025), pp. 1-60). The test enables the simultaneous estimation of the optimal parameter within a parametric family of candidate models. Its asymptotic distribution is shown to be a weighted sum of infinitely many $\chi^2$-distributed random variables plus an additional disturbance term, which is due to the parameter estimation. Further, we provide a general framework to bootstrap degenerate parameter-dependent $U$-statistics and use it to derive a new Kernel Stein Discrepancy composite goodness-of-fit test.
Read the full abstract ▾
Statistics Applications
On the simultaneous inference of susceptibility distributions and intervention effects from epidemic curves
Ibrahim Mohammed, Chris Robertson and M. Gabriela M. Gomes
Susceptible-Exposed-Infectious-Recovered (SEIR) models with inter-individual variation in susceptibility or exposure to infection were proposed early in the COVID-19 pandemic as a potential element of the mathematical/statistical toolset available to policy development. In comparison with other models employed at the time, those designed to fully estimate the effects of such variation tended to predict small epidemic waves and hence require less containment to achieve the same outcomes. However, these models never made it to mainstream COVID-19 policy making due to lack of prior validation of their inference capabilities. Here we report the results of the first systematic investigation of this matter. We simulate
datasets using the model with strategically chosen parameter values, and then conduct maximum likelihood estimation to assess how well we can retrieve the assumed parameter values. We identify some identifiability issues which can be overcome by creatively fitting multiple epidemics with shared parameters.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.PR
Beyond the Semicircle: Free Diffusion Models with Prescribed Equilibria
Swagatam Das
A growing class of machine-learning objects – covariance and Gram
matrices, kernel and
attention matrices,
MIMO channel
matrices, density operators – are naturally spectra rather than coordinate vectors. Building a denoising
diffusion model for such data by corrupting
eigenvalues coordinatewise is not merely elegant: it converges to the wrong limit, because
eigenvalues repel rather than move independently. Free probability theory supplies the correct forward process, with free convolution replacing classical convolution and Voiculescu's conjugate variable replacing the score, but constant-coefficient free
diffusions have a hidden limitation of their own: their only possible
equilibrium is the semicircular law, whatever the target distribution looks like. We show that state-dependent free
volatility removes this restriction. For any sufficiently regular compactly supported target law, we explicitly construct, through a closed-form
Hilbert-transform drift, a free Fokker–Planck flow having that law as a stationary spectral distribution. Under an additional convex-potential condition, the construction admits a globally relaxing
diffusion interpretation, reverse-time dynamics, and a trainable score. The generalization is genuine rather than cosmetic: no change of spectral variable reduces it to the constant-coefficient case, and it is a Wasserstein gradient flow only for an explicitly characterized family of coefficients that contains no bounded non-constant member. We instantiate the construction on the Marchenko–Pastur law, the limiting spectrum of isotropic sample covariance
matrices, and validate it numerically: the free-versus-coordinatewise gap, an exact dimension-independent score driving reverse-time
matrix dynamics at several
matrix sizes, the designed Marchenko–Pastur
equilibrium, and end-to-end generation with a learned score.
Read the full abstract ▾
Statistics Theory
Estimation of periodically correlated random fields that are isotropic on a sphere
Iryna Golichenko, Oleksandr Masyutka and Mykhailo Moklyachuk
The problem of optimal linear estimation of functionals depending on the unknown values of a spatial temporal isotropic random field $\zeta(j,x)$, which is periodically correlated with respect to discrete time argument $j\in\mathrm Z$ and mean-square continuous isotropic on the unit sphere ${S_n}$ with respect to spatial argument $x\in{S_n}$. Estimates are based on observations of the field $\zeta(j,x)+\theta(j,x)$ at points $(j,x):$ $j\in Z\backslash\{0, 1, .... , N\}$, $x\in S_{n}$, where $\theta(j,x)$ is an uncorrelated with $\zeta(t,x)$ spatial temporal isotropic random field, which is periodically correlated with respect to discrete time argument $j\in\mathrm Z$ and mean-square continuous isotropic on the sphere ${S_n}$ with respect to spatial argument $x\in{S_n}$. Formulas for calculating the mean square errors and the spectral characteristics of the optimal linear estimate of the functional are derived in the case where the spectral density
matrices are exactly known. Formulas that determine the least favourable spectral density
matrices and the minimax (robust) spectral characteristics are proposed in the case where the spectral density
matrices are not exactly known but a class of admissible spectral density
matrices is given.
Read the full abstract ▾
Statistics Theory
Interpolation of functionals of stochastic sequences with stationary increments from observations with noise
Maksym Luz and Mykhailo Moklyachuk
The problem of optimal estimation of linear functional ${{A}_{N}}\xi =\sum\limits_{k=0}^{N}{a(k)\xi (k)}\,$ depending on the unknown values of a
stochastic sequence $\xi (m)$ with stationary $n$-th increments from observations of the sequence $\xi (k)$ at points $k=-1,-2,\ldots $ and of the sequence $\xi (k)+\eta (k)$ at points of time $k=N+1,N+2,\ldots $ is considered. Formulas for calculating the mean square error and the spectral characteristic of the optimal linear estimate of the functional are proposed under condition of spectral certainty, where spectral densities of the sequences $\xi (m)$ and $\eta (m)$ are exactly known. Minimax (robust) method of estimation is applied in the case where the spectral densities are not known exactly while some sets of admissible spectral densities are given. Formulas that determine the least favorable spectral densities and the minimax spectral characteristics are proposed for some specific sets of admissible densities.
Read the full abstract ▾
Machine Learning (Statistics)
OEUVRE: OnlinE Unbiased Variance-Reduced loss Estimation
Kanad Pardeshi, Bryan Wilder and Aarti Singh
Online learning
algorithms continually update their models as data arrive, making it essential to accurately estimate the expected loss at the current time step. The prequential method is an effective estimation approach which can be practically deployed in various ways. However, theoretical guarantees have previously been established under strong conditions on the
algorithm, and practical
algorithms have hyperparameters which require careful tuning. We introduce OEUVRE, an
estimator that evaluates each incoming sample on the function learned at the current and previous time steps, recursively updating the loss estimate in constant time and
memory. We use algorithmic stability, a property satisfied by many popular online learners, for optimal updates and prove consistency, convergence rates, and concentration bounds for our
estimator. We design a method to adaptively tune OEUVRE's hyperparameters and test it across diverse online and
stochastic tasks. We observe that OEUVRE matches or outperforms other
estimators even when their hyperparameters are tuned with oracle access to ground truth.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Centrum: Model-based Database Auto-tuning with Minimal Distributional Assumptions
Yuanhao Lai, Pengfei Zheng, Chenpeng Ji et al. (+5 more)
Gaussian-Process-based
Bayesian optimization (GP-BO), is a prevailing model-based framework for DBMS auto-tuning. However, recent work shows GP-BO-based DBMS auto-tuners significantly outperformed auto-tuners based on SMAC, which features random forest surrogate models; such results motivate us to rethink and investigate the limitations of GP-BO in auto-tuner design. We find the fundamental assumptions of GP-BO are widely violated when modeling and optimizing DBMS performance, while tree-ensemble-BOs (e.g., SMAC) can avoid the assumption pitfalls and deliver improved tuning efficiency and effectiveness. Moreover, we argue that existing tree-ensemble-BOs restrict further advancement in DBMS auto-tuning. First, existing tree-ensemble-BOs can only achieve distribution-free point estimates, but still impose unrealistic distributional assumptions on uncertainty estimates, compromising surrogate modeling and distort the acquisition function. Second, recent advances in gradient boosting, which can further enhance surrogate modeling against vanilla GP and random forest counterparts, have rarely been applied in optimizing DBMS auto-tuners. To address these issues, we propose a novel model-based DBMS auto-tuner, Centrum. Centrum improves distribution-free point and interval estimation in surrogate modeling with a two-phase learning procedure of
stochastic gradient boosting ensembles. Moreover, Centrum adopts a generalized SGBE-estimated locally-adaptive conformal prediction to facilitate a distribution-free uncertainty estimation and acquisition function. To our knowledge, Centrum is the first auto-tuner to realize distribution-freeness, enhancing BO's practicality in DBMS auto-tuning, and the first to seamlessly fuse gradient boosting ensembles and conformal inference in BO. Extensive physical and simulation experiments on two DBMSs and three workloads show Centrum outperforms 21 SOTA methods.
Read the full abstract ▾
All authors Yuanhao Lai, Pengfei Zheng, Chenpeng Ji, Yan Li, Songhan Zhang, Rutao Zhang, Zhengang Wang, Yunfei Du
Statistics Applications
Imaging Genetics Analysis of Alzheimer's Disease
Riddhik Basu and Arkaprava Roy
Alzheimer's disease (AD) is a progressive neurodegenerative disorder characterized by cognitive decline, structural
brain changes, and genetic predispositions. This study leverages machine-learning and statistical techniques to investigate the mechanistic relationships between cognitive function, genetic markers, and neuroimaging biomarkers in AD progression. Using data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), we perform both low-dimensional and high-dimensional analyses to identify key predictors of disease states, including cognitively normal (CN), mild cognitive impairment (MCI), and AD. Our low-dimensional approach utilizes multiple linear and ordinal logistic regression to examine the influence of cognitive scores, cerebrospinal fluid (CSF) biomarkers, and demographic factors on disease classification. The results highlight significant associations between Mini-Mental State Examination (MMSE), Clinical Dementia Rating Sum of Boxes (CDRSB), and phosphorylated tau levels in predicting cognitive decline. The high-dimensional analysis employs Sure Independence Screening (SIS) and LASSO regression to reduce dimensionality and identify genetic markers correlated with cognitive impairment and white matter integrity. Genes such as CLIC1, NAB2, and TGFBR1 emerge as significant predictors across multiple analyses, linking genetic expression to neurodegeneration. Additionally, imaging genetic analysis reveals shared genetic influences across
brain hemispheres and the corpus callosum, suggesting distinct genetic contributions to white matter degradation. These findings enhance our understanding of AD pathology by integrating cognitive, genetic, and imaging data. Future research should explore longitudinal analyses and potential gene-environment interactions to further elucidate the biological mechanisms underlying AD progression.
Read the full abstract ▾
Statistical Methodology
Testing Copula Hypothesis with Copula Entropy
Jian Ma
Testing copula hypothesis is of fundamental importance in the applications of copula theory. In this paper we proposed a copula hypothesis testing with copula
entropy. Since copula
entropy is a unified theory in probability and therefore testing copula hypothesis based on it can be applied to any types of copula function. The test statistic is defined as the difference of copula
entropy of copula hypothesis and true copula
entropy. We propose the estimation method of the proposed statistic and two special cases for Gaussian copula hypothesis and Gumbel copula hypothesis. We test the effectiveness of the proposed method with simulation experiments.
Read the full abstract ▾
Statistical Methodology
Gini representations of skewness, kurtosis and higher-order moments
Jean-Marie Dufour, Abderrahim Taamouti and Meilin Tong
We provide pairwise-difference (Gini-type) representations of higher-order central moments for both general random variables and empirical moments. Such representations do not require a measure of location. For third and fourth moments, this yields pairwise-difference representations of skewness and kurtosis coefficients. We show that all central moments possess such representations, so no reference to the mean is needed for moments of any order. This is done by considering i.i.d. replications of the random variables considered, by observing that central moments can be interpreted as covariances between a random variable and powers of the same variable, and by giving recursions which link the pairwise-difference representation of any moment to lower order ones.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Identification of Causal Direction under an Arbitrary Number of Latent Confounders
Wei Chen, Linjun Peng, Zhiyi Huang et al. (+4 more)
Recovering causal structure in the presence of latent variables is an important but challenging task. While many methods have been proposed to handle it, most of them require strict and/or untestable assumptions on the causal structure. In real-world scenarios, observed variables may be affected by multiple latent variables simultaneously, which, generally speaking, cannot be handled by these methods. In this paper, we consider the linear, non-Gaussian case, and make use of the joint higher-order cumulant
matrix of the observed variables constructed in a specific way. We show that, surprisingly, causal asymmetry between two observed variables can be directly seen from the rank deficiency properties of such higher-order cumulant
matrices, even in the presence of an arbitrary number of latent confounders. Identifiability results are established, and the corresponding identification methods do not even involve iterative procedures. Experimental results demonstrate the effectiveness and asymptotic correctness of our proposed method.
Read the full abstract ▾
All authors Wei Chen, Linjun Peng, Zhiyi Huang, Haoyue Dai, Zhifeng Hao, Ruichu Cai, Kun Zhang
Statistical Methodology
Sample size determination for win statistics in cluster-randomized trials
Xi Fang, Zhiqiang Cao and Fan Li
Composite endpoints are increasingly used in clinical trials to capture treatment effects across multiple or hierarchically ordered outcomes. Although inference procedures based on win statistics, such as the win ratio, win odds, and win difference (net benefit) have gained traction in individually randomized trials, their methodological development for cluster-randomized trials remains limited. In particular, there is no formal framework for power and sample size determination when using win statistics with composite time-to-event outcomes. We develop a unified framework for power and sample size calculation for win statistics under cluster randomization. Analytical variance expressions are derived for a class of win statistics, yielding closed-form variance expressions and power procedures that avoid computationally intensive simulations. The variance expressions explicitly characterize the roles of the rank intracluster correlation coefficient, cluster size, tie probability, and outcome prioritization for study planning purposes. Importantly, our variances nest existing formulas for univariate outcomes as special cases while extending them to complex, hierarchically ordered composite endpoints. Simulation studies confirm accurate finite-sample performance, and we supply a case study to illustrate the use of our proposed method.
Read the full abstract ▾
Statistics Theory
Stopping Rules for Monte Carlo Methods of Martingale Difference Type
Jiezhong Wu and Reiichiro Kawai
We establish a practical and easy-to-implement sequential stopping rule for the martingale central limit
theorem, focusing on Monte Carlo methods for estimating the mean of a non-iid sequence of martingale difference type. Starting with an impractical scheme based on the standard martingale central limit
theorem, we progressively address its limitations from implementation perspectives in the non-asymptotic regime. Along the way, we compare the proposed schemes with their counterparts in the asymptotic regime. The developed framework has potential applications in various domains. Numerical results are provided to demonstrate the effectiveness of the developed stopping rules in terms of reliability and complexity.
Read the full abstract ▾
Statistical Methodology
Stopping Rules for Monte Carlo Methods: A Review
Jiezhong Wu and Reiichiro Kawai
Sequential analysis encompasses simulation theories and methods where the sample size is determined dynamically based on accumulating data. Since the conceptual inception, numerous sequential stopping rules have been introduced, and many more are currently being refined and developed. This article aims to deliver an up-to-date review of recent developments in sequential stopping rules, with a deliberate emphasis on Monte Carlo methods for estimating an unknown expectation, including binomial proportions, primarily under standard iid sampling and also under certain lightly generalized settings. These methodologies have long served and likely will continue to serve, as fundamental bases for both theoretical and practical developments in stopping rules for general statistical inference, advanced Monte Carlo techniques, and their modern applications. Building upon over a hundred references and empirical studies, we explore the essential aspects of these methods, such as core assumptions, numerical
algorithms, convergence properties, and practical trade-offs to guide further developments, particularly at the intersection of sequential stopping rules and related areas of research.
Read the full abstract ▾
Machine Learning (Statistics)
Block Coordinate Descent for Neural Networks Provably Finds Global Minima
Shunta Akiyama
In this paper, we consider a block coordinate descent (BCD)
algorithm for training deep neural networks and provide a new global convergence guarantee under strictly monotonically increasing activation functions. While existing works demonstrate convergence to stationary points for BCD in neural networks, our contribution is the first to prove convergence to global minima, ensuring arbitrarily small loss. We show that the loss with respect to the output layer decreases exponentially while the loss with respect to the hidden layers remains well-controlled. Additionally, we derive generalization bounds using the Rademacher complexity framework, demonstrating that BCD not only achieves strong optimization guarantees but also provides favorable generalization performance. Moreover, we propose a modified BCD
algorithm with skip connections and non-negative projection, extending our convergence guarantees to ReLU activation, which are not strictly monotonic. Empirical experiments confirm our theoretical findings, showing that the BCD
algorithm achieves a small loss for strictly monotonic and ReLU activations.
Read the full abstract ▾
Statistics Theory · Cross-listed from cond-mat.stat-mech
The Gravitational Aspect of Information: The Physical Reality of Asymmetric "Distance"
Tomoi Koide and Armin van de Venn
We show that when a Brownian bridge is physically constrained to satisfy a canonical condition, its time
evolution exactly coincides with an m-geodesic on the statistical
manifold of Gaussian distributions. This identification provides a direct physical realization of a
geometric concept in information geometry. It implies that purely random processes evolve along informationally straight trajectories, analogous to geodesics in general relativity. Our findings suggest that the asymmetry of informational “distance" (divergence) plays a fundamental physical role, offering a concrete step toward an equivalence principle for information.
Read the full abstract ▾
Machine Learning (Statistics)
Semi-Supervised Learning under General Causal Models
Archer Moore, Heejung Shim, Jingge Zhu et al. (+1 more)
Semi-supervised learning (SSL) aims to train a machine learning model using both labelled and unlabelled data. While the unlabelled data have been used in various ways to improve the prediction accuracy, the reason why unlabelled data could help is not fully understood. One interesting and promising direction is to understand SSL from a causal perspective. In light of the independent causal mechanisms principle, the unlabelled data can be helpful when the label causes the features but not vice versa. However, the causal relations between the features and labels can be complex in real world applications. In this paper, we propose a SSL framework that works with general causal models in which the variables have flexible causal relations. More specifically, we explore the causal graph structures and design corresponding causal generative models which can be learned with the help of unlabelled data. The learned causal generative model can generate synthetic labelled data for training a more accurate predictive model. We verify the effectiveness of our proposed method by empirical studies on both simulated and real data.
Read the full abstract ▾
All authors Archer Moore, Heejung Shim, Jingge Zhu, Mingming Gong
Machine Learning (Statistics)
Statistical Analysis of the Sinkhorn Iterations for Two-Sample Schrödinger Bridge Estimation
Ibuki Maeda, Rentian Yao and Atsushi Nitanda
The Schrödinger bridge problem seeks the optimal
stochastic process that connects two given probability distributions with minimal energy modification. While the Sinkhorn
algorithm is widely used to solve the static optimal transport problem, a recent work (Pooladian and Niles-Weed, 2024) proposed the Sinkhorn bridge, which estimates Schrödinger bridges by plugging optimal transport into the time-dependent drifts of SDEs, with statistical guarantees in the one-sample estimation setting where the true source distribution is fully accessible. In this work, to further justify this method, we study the statistical performance of intermediate Sinkhorn iterations in the two-sample estimation setting, where only finite samples from both source and target distributions are available. Specifically, we establish a statistical bound on the squared total variation error of Sinkhorn bridge iterations: $O(1/m+1/n + r^{4k})~(r \in (0,1))$, where $m$ and $n$ are the sample sizes from the source and target distributions, respectively, and $k$ is the number of Sinkhorn iterations. This result provides a theoretical guarantee for the finite-sample performance of the Schrödinger bridge
estimator and offers practical guidance for selecting sample sizes and the number of Sinkhorn iterations. Notably, our theoretical results apply to several representative methods such as [SF]$^2$M, DSBM-IMF, BM2, and LightSB(-M) under specific settings, through the previously unnoticed connection between these
estimators.
Read the full abstract ▾
Statistical Methodology
Surface decomposition method for sensitivity analysis of first-passage dynamic reliability of linear systems
Jianhua Xian, Sai Hung Cheung and Cheng Su
This work presents a novel surface decomposition method for the sensitivity analysis of first-passage dynamic reliability of linear systems subjected to Gaussian random excitations. The method decomposes the sensitivity of first-passage failure probability into a sum of surface integrals over the constrained component limit-state hypersurfaces. The evaluation of these surface integrals can be accomplished, owing to the availability of closed-form linear expressions of both the component limit-state functions and their sensitivities for linear systems. An importance sampling strategy is introduced to further enhance the efficiency for estimating the sum of these surface integrals. The number of function evaluations required for the reliability sensitivity analysis is typically on the order of 10^2 to 10^3. The approach is particularly advantageous when a large number of design parameters are considered, as the results of function evaluations can be reused across different parameters. Three numerical examples are investigated to demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
Statistics Applications
Regularization method in the variable selection for logistic regression on BRFSS data
Jinbo Niu
Stroke remains a leading cause of death and disability worldwide, yet effective prediction of stroke risk using large-scale population data remains challenging due to data imbalance and high-dimensional features. In this study, we develop and evaluate regularized logistic regression models for stroke prediction using data from the 2022 Behavioral Risk Factor Surveillance System (BRFSS), comprising 445132 U.S. adult respondents and 328 health-related variables. To address data imbalance, we apply several resampling techniques including oversampling, undersampling, class weighting, and the Synthetic Minority Oversampling Technique (SMOTE). We further employ Lasso, Elastic Net, and Group Lasso regularization methods to perform feature selection and dimensionality reduction. Model performance is assessed using ROC-AUC, sensitivity, and specificity metrics. Among all methods, the Lasso-based model achieved the highest predictive performance (AUC = 0.761), while the Group Lasso method identified a compact set of key predictors: Age, Heart Disease, Physical Health, and Dental Health. These findings demonstrate the potential of regularized regression techniques for interpretable and efficient prediction of stroke risk from large-scale behavioral health data.
Read the full abstract ▾
Statistical Methodology
Semi-supervised Vertex Hunting, with Applications in Network and Text Analysis
Yicong Jiang and Zheng Tracy Ke
Vertex hunting (VH) is the task of estimating a simplex from noisy data points and has many applications in areas such as network and text analysis. We introduce a new variant, semi-supervised vertex hunting (SSVH), in which partial information is available in the form of barycentric coordinates for some data points, known only up to an unknown transformation. To address this problem, we develop a method that leverages properties of orthogonal projection
matrices, drawing on novel insights from linear
algebra. We establish theoretical error bounds for our method and demonstrate that it achieves a faster convergence rate than existing unsupervised VH
algorithms. Finally, we apply SSVH to two practical settings, semi-supervised network mixed membership estimation and semi-supervised topic modeling, resulting in efficient and scalable
algorithms.
Read the full abstract ▾