Machine Learning (Statistics)
Learning Asymptotics with Convergence-Rate Guarantees using Linear Least Squares
Christos N. Efrem
We introduce a new research area that is called Asymptotics Learning Theory (ALT) and combines optimization with asymptotic analysis. In particular, ALT provides a unified approach for computing unknown constants/parameters in proven asymptotic expansions using optimization theory. In this paper, we focus on a general asymptotic form which includes a broad class of asymptotics. Furthermore, we study two powerful numerical methods, namely, sliding Linear Least Squares (sLLSQ) and sliding Tikhonov Linear Least Squares (sT-LLSQ). For these techniques we rigorously prove asymptotic estimates that lead to sufficient conditions for convergence (to the correct values of unknown parameters) and convergence-rate guarantees. Despite their strengths, both methods have also limitations, e.g., slow convergence—or even, counterintuitively, divergence—in some cases. Moreover, we present fundamental applications in analytic combinatorics, a beautiful field of mathematics that deals with asymptotic enumeration of discrete structures using complex analysis. The proposed techniques complement existing approaches, such as the ratio method and its variants. Numerical examples also verify the theoretical results. Finally, we discuss interesting research directions in ALT.
Read the full abstract ▾
Statistical Computation
Uniqueness in multivariate curve resolution, re-tuned
Robert Rajko and Rabeea Razaq
There are many misunderstandings about the term and interpretation of uniqueness in multivariate curve resolution tasks. In CAC2026 Tarragona, it turned out that even mathematicians do not properly construe Manne's
theorems. So we decided to provide this re-tuned summary, using several original quotes, their explanations, and newly created illustrative figures to get the points across. This contribution then explicitly offers the following: (i) a unified interpretation of Manne's conditions, DBU, GRU, minimal constrained duality, and particular-solution-based criteria; (ii) newly constructed visual illustrations, including a counterexample; and (iii) a practical workflow for diagnosing uniqueness in nonnegative bilinear MCR.
Read the full abstract ▾
Statistics Theory · Cross-listed from eess.SP
GLRT for Reconfigurable Intelligent Surface aided Spectrum Sensing
Nikhilsingh Parihar, Praful Mankar and Sachin Chaudhari
Spectrum sensing (SS) is crucial for realising cognitive
radio networks, where the secondary user (SU) needs to detect the presence of a primary user (PU) in order to utilise the spectrum. However, the ability of detection is influenced by unknown propagation environment factors such as multipath fading, correlated noise, transmission power of PU, etc. This paper investigates reconfigurable intelligent surfaces (RIS)-aided SS under correlated noise conditions using a generalised likelihood ratio test (GLRT) and energy detector (ED) frameworks. We first derive maximum likelihood estimates of the unknown channel state and transmit power, and employ these estimates to construct the GLRT-based test statistic using the signal received with an optimally configured RIS. The RIS phase shift
matrix is optimally determined to maximise the gain of the estimated channel. Besides, the detection and false alarm probabilities of ED with optimally configured RIS are also derived. The numerical receiver output characteristics (ROC) demonstrate that the proposed GLRT achieves superior detection probability compared to ED, particularly under correlated noise and limited number of observations.
Read the full abstract ▾
Statistics Applications
Towards Optimal Estimators for Randomized Control Trials
Harsh Parikh, Gabriel Levin-Konigsberg, Nilesh Tripuraneni et al. (+5 more)
Randomized controlled trials (RCTs) are fundamental tools for causal inference across technology companies, pharmaceutical research, and federal agencies. While the standard difference-in-means
estimator provides unbiased treatment effect estimates, it often lacks precision, particularly when treatment effects are heterogeneous or outcomes exhibit heavy-tailed distributions. Although numerous precision-enhancing methods exist—from
covariate adjustment techniques to variance reduction strategies—recent research demonstrates that no single
estimator performs optimally across all
datasets. Rather than seeking the best
estimator for individual RCTs, which risks compromising scientific validity through convenient selection, we propose a principled framework for identifying optimal
estimators within families of RCTs based on specific analytical goals. Our approach uses sample splitting to estimate the distribution of evaluation metrics (e.g., mean squared error, regret) across RCT families, enabling systematic comparisons between
estimators while maintaining asymptotic guarantees. We demonstrate this framework using a sample of Amazon's Supply Chain Optimization Technology trials and the Strengthening Democracy Challenge
dataset (25 interventions). Results reveal that optimal
estimators vary significantly by analytical objective: weighted least squares performs best for inference goals, while difference-in-means minimizes regret for decision-making contexts. This work provides actionable guidance for
estimator selection while preserving methodological rigor across diverse research applications.
Read the full abstract ▾
All authors Harsh Parikh, Gabriel Levin-Konigsberg, Nilesh Tripuraneni, Dhruv Madeka, Michael I. Jordan, Dean Foster, Dominique Perrault-Joncas, Alexander Volfovsky
Machine Learning (Statistics)
FedSLIM: Privacy-Preserving Federated MDL-Based Descriptive Pattern Mining Across Data Silos
Samar Samir Khalil, Noha S. Tawfik and Marco Spruit
Federated learning has achieved considerable success for predictive modelling, yet
federated descriptive analytics remains largely unexplored. Existing
federated pattern mining approaches are predominantly support-based and do not optimise a principled global objective such as Minimum Description Length (MDL). We introduce FedSLIM, the first
federated MDL-based framework for descriptive pattern mining. Building on the SLIM principle, FedSLIM enables collaborative optimisation of compact pattern models across distributed databases without sharing raw transactions. We propose two complementary variants that balance
privacy, communication, and optimisation fidelity under different deployment assumptions. To evaluate
federated MDL mining, we introduce fidelity and discovery-oriented metrics that quantify agreement with a centralised baseline and assess recovery of globally informative patterns. Experiments on multiple real-world
datasets under IID and non-IID partitioning show that both variants preserve high-quality compression structure while requiring orders of magnitude less search than the centralised baseline. We further reveal a local-global discovery gap in distributed MDL mining, where globally compressive patterns may be undiscoverable through isolated local optimisation. Both variants recover globally informative patterns absent from all standalone local models, demonstrating the benefits of
federated optimisation beyond independent local mining. These results establish
federated MDL mining as a practical foundation for
privacy-preserving descriptive analytics across distributed data silos.
Read the full abstract ▾
Statistics Applications
CoLaDAG: Compositional Latent Log-ratio DAG Analysis of the Gut Microbiome under Silver Nanoparticle Exposure
Shuyan Chen, Ziliang Shen and Xinlei Wang
Directed network analysis of microbiome counts is complicated by compositional sampling, high dimensionality, and limited biological replication. We present CoLaDAG, a fixed-reference latent additive log-ratio (ALR)
estimator for generating sparse directed conditional-dependence hypotheses from compositional counts. The method combines a multinomial observation model, a working linear Gaussian structural equation model, nonconvex DC-ADMM optimization, and post-estimation thresholding with greedy acyclic projection. Under simulations aligned with this observation model, CoLaDAG obtained the largest mean exact-direction Matthews correlation and the smallest mean false discovery rate among the evaluated implementations; performance deteriorated under continuous-data and dropout misspecification. In the 12-mouse silver-nanoparticle (AgNP) case study, the 58-node fitted graph was sensitive to block resampling and ALR reference choice: 60 of 284 primary edges attained a mouse-block selection frequency of at least 0.60. The reported orientations and dose-stratified slopes are exploratory, coordinate-specific hypotheses rather than identified causal or exposure effects. The leading stable relations prioritize anaerobic gut taxa for targeted abundance, metabolite, and perturbation studies, but do not establish cross-feeding or toxicological mechanisms.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
Jinshu Huang, Yiming Jiang and Chunlin Wu
Despite the empirical success of score-based
diffusion models, a complete theoretical understanding of how finite-sample learning, network parameterization, and numerical discretization jointly dictate generative quality remains underdeveloped. Existing sampling analyses often evaluate the generative performance conditional on an oracle score or a pre-specified error threshold. In this work, we establish a unified convergence and generalization framework for score-based
diffusion models parameterized by practical ResNet-type architectures. We analyze the generalization and convergence properties from the practical finite-sample, discrete-time learning problem of the score function to the ideal continuous-time, population-level objective. Based on the generalization result of the learning problem of score function, we analyze the sampling process induced by the learned score function and provide an end-to-end total variation distance estimate for the generated terminal distribution. This estimate explicitly decomposes the overall generative error into four interpretable components: the truncation error of the forward process, the reverse-time discretization error, the generalization error incorporating both finite data and forward-time discretization, and the training optimization gap. Our results quantitatively characterize how the training sample size, temporal discretization grids, and optimization accuracy jointly control the final fidelity of samples generated by
diffusion models.
Read the full abstract ▾
Machine Learning (Statistics)
Beyond ICA: Identifiability by Symmetry Breaking
Pengzhou Wu
We prove the identifiability of deep generative models (DGMs) with piecewise-affine (PWA) decoders and Gaussian mixture model (GMM) priors, in a purely unsupervised setting. We introduce three algebraic contrast principles for
symmetry breaking: domain contrast, which trivializes the mixture
symmetry group; mechanism contrast, which ensures every decoder branch is witnessed by a unique boundary; and interaction contrast, which forbids parameter conspiracies between latent components and decoder branches. Together they exploit the interplay between the discrete combinatorics of the PWA map and the continuous
symmetry structure of the latent GMM. Continuity is replaced by algebraic
symmetry conditions; injectivity is decoupled from structural identification and required only for pointwise inversion. Our results form a hierarchy: from law identifiability (LID; latent distribution up to a global affine map) through map identifiability (MID; decoder up to the same map) to posterior and pointwise identifiability. The ICA-form ambiguity emerges under conditions on diagonal component covariances. Assumptions are only on the data-generating process, not on learning methods, except for the interaction contrast. To our knowledge this is the first to make algebraic
symmetry-breaking the engine of nonlinear identifiability, the first to admit discontinuous decoders, and the first to handle fully non-injective decoders, where every observation admits multiple latent codes.
Read the full abstract ▾
Machine Learning (Statistics)
Adaptive Multi-Scale Forecasting and Gate-Localized Conformal Prediction for Multivariate Nonstationary Time Series
Ziling Ma, Junshu Jiang, Ángel López-Oriona et al. (+2 more)
We propose ABF-T-GLCP, a model-agnostic framework for
forecasting and uncertainty quantification in nonstationary multivariate time series. The central idea is to learn an adaptive predictive state representation for point
forecasting and reuse it for conformal calibration. The
forecasting module combines horizon-specific temporal experts through a learned gate and refines predictions using sparse predictive transfer across related series. The uncertainty module, Gate-Localized Conformal Prediction (GLCP), uses the learned gate state, together with temporal recency, to select locally relevant calibration residuals, thereby coupling uncertainty calibration to the predictive regimes used by the
forecasting model. This shared representation allows point forecasts and prediction intervals to adapt consistently under evolving temporal dynamics while retaining the model-agnostic nature of conformal prediction and yielding approximate local coverage under mild stability conditions. Experiments on a large-scale high-frequency commodity
forecasting benchmark show consistent gains in point
forecasting accuracy and substantially narrower prediction intervals with empirical coverage close to the nominal level. Additional results indicate that the framework extends beyond the motivating financial application.
Read the full abstract ▾
All authors Ziling Ma, Junshu Jiang, Ángel López-Oriona, Ying Sun, Hernando Ombao
Statistical Methodology
Anytime-Valid Confirmation of Covariate Balance for Prespecified Corrections
Seungjin Choi
Many
covariate-shift adaptation methods construct a correction $w(x)$, but users must still determine whether the corrected distribution is sufficiently balanced for the target stream. We study anytime-valid confirmation of prespecified corrections from sequential unlabeled target inputs. Our primary contribution is a procedure for confirming
covariate balance. For a prespecified class of balancing functions and tolerances, time-uniform confidence sequences permit continuous monitoring and data-dependent stopping once all plausible target moments lie within their tolerance bands. If the correction is out of tolerance for at least one function, the probability of ever incorrectly confirming balance is at most the prescribed level. Upon stopping, the procedure yields a certificate local to the chosen functions and tolerances, yet providing an absolute downstream-adequacy statement that ordinary shift diagnostics generally do not. With finite source data, contracted bands preserve this guarantee while accounting for uncertainty in weighted source moments, whereas expanded bands support only compatibility diagnostics. As complementary information, we study a source-calibrated likelihood-ratio e-process whose KL-drift identity characterizes correction directions relative to the source. Under the source-reference distribution, the probability of ever crossing its evidence threshold is controlled, but crossing does not confirm balance. We also give an exponential-tilt test for departures beyond an acceptable correction region and deploy balance-confirmed corrections in weighted conformal prediction. Experiments illustrate false-confirmation control, locality to the balancing-function class, KL-drift diagnostics, acceptable-region monitoring, finite-source effects, and downstream conformal coverage under
covariate shift.
Read the full abstract ▾
Statistical Methodology
Decorated graphons for temporal network estimation
Charles Dufour and Sofia C. Olhede
We propose a unified nonparametric framework for modeling time-evolving networks using decorated graphons (also known as probability-graphons): symmetric functions that assign to each node pair a probability distribution over binary edge time series. This generalizes the static decorated-graphon construction to dynamic graphs while preserving node exchangeability and allowing temporal dynamics such as
memory and periodicity. Models in which edges evolve independently given the latent variables, such as autoregressive and Markov edge processes, arise as special cases. We develop a two-stage estimation procedure that separates temporal modeling from network structure. Because the network stage requires only mild regularity conditions on the edge-process
estimator, a broad class of temporal edge models can be used in the first stage. We establish nonparametric convergence rates in both block-model and Hölder-smooth regimes, and make explicit how the rate depends on the number of observed time steps and on the quality of the edge-level estimation. We illustrate the method on simulated data and a hospital contact network, recovering latent community structure and time-varying interaction patterns. The framework gives a nonparametric baseline for dynamic network analysis with explicit convergence guarantees.
Read the full abstract ▾
Statistical Methodology
Identification and Honest Recovery from Semantic Observation Kernels: Operator Error, Coarsening, and Stability
Matthew Francis Dixon
Probabilistic text generators, such as large
language models, assign probabilities to phrases, but consequential decisions require posterior uncertainty over meaningful states. These are not interchangeable:
language probabilities depend on the
prompt, may be incomplete and need not reliably identify state uncertainty. Without a statistical bridge, fluent responses and numerical confidence are insufficient for inference or governance. We formulate recovery of the target posterior as a semiparametric inverse problem and develop honest recovery guarantees that account jointly for calibration error, measurement noise, incomplete probabilities and weak identification. Simulations demonstrate the predicted coverage and stability behaviour, while two frozen
language-model studies demonstrate held-out recovery. The resulting method determines when a semantic measurement can be trusted for inference and when use, review, recalibration or abstention is warranted, providing a statistical foundation for runtime AI governance.
Read the full abstract ▾
Machine Learning (Statistics)
Operator Neural Jump ODEs: $L^2$-optimal prediction in function spaces
Florian Krach, Oliver Löthgren and Josef Teichmann
In this paper, we study the extension of Neural Jump ODEs to infinite-dimensional function spaces. In particular, the underlying process $X$ now takes values in $L^2(\Xi, \mathbb{R}^{d_X})$ instead of $\mathbb{R}^{d_X}$ and the Operator NJ-ODE approximates the optimal predictor of this process by producing a representative of the conditional expectation. The NJ-ODE model is a framework for online learning the optimal prediction of continuous-time
stochastic processes, given discrete, possibly irregular and incomplete past observations. In a series of works, this model has been extended to deal with generic path-dependent processes, with observation noise and dependent observations, with long-term predictions, and with input-output systems. However, throughout all of these works, the underlying processes were restricted to be finite-dimensional. In particular, function-valued problems, like yield curve or
volatility surface predictions, could only be handled through discretization, which inherently leads to a loss of information. In this work, we build on ideas from Neural Operator methods that allow us to extend the NJ-ODE framework to an infinite-dimensional output process. To prove convergence of the NJ-ODE to the optimal prediction process, we develop a new approximation strategy that also generalizes previous works in the finite-dimensional setting by considerably weakening the assumptions.
Read the full abstract ▾
Statistical Methodology
Convergence of Estimative Density to Information Projection for Misspecified Normal Distribution Model
Yo Sheena
This paper investigates the convergence of an estimative multivariate normal density when the true distribution is a misspecified multivariate t-distribution. The statistical model is the family of k-dimensional normal distributions (N_k(\mu,\Sigma)), whereas the observations are assumed to follow (t_k(0,I_k,\nu)), with (\nu>6). The information projection of the true distribution onto the normal model is first identified as the normal distribution with mean zero and covariance
matrix (\nu/(\nu-2)I_k). The main objective is to evaluate the expected Kullback-Leibler divergence between this information projection and the normal density obtained by substituting the maximum likelihood
estimator into the model. Using a general asymptotic expansion for estimative densities, the paper derives explicit first- and second-order terms of the risk as functions of the sample size (n), the dimension (k), and the degrees of freedom (\nu). To obtain the second-order term, the paper calculates the required moments, information
matrices, and higher-order cumulants under both the multivariate normal and multivariate t-distributions. In particular, the complicated third- and fourth-order cumulants involving quadratic sufficient statistics are classified according to their index patterns, and their values and multiplicities are systematically derived.
Read the full abstract ▾
Statistical Methodology
The normalized expectation-maximization (N-EM) algorithm
Guo-Liang Tian, Xuanyu Liu and Yuanfan Zhao
Although the $\textit{expectation-maximization}$ (EM)
algorithm is a powerful optimization tool in statistics, it can only be applied to missing/incomplete data problems or to problems with a latent-variable structure. It is well known that the introduction of latent variables (or the data augmentation) is an art; i.e., it could only be done case by case. In this paper, we propose a new
algorithm, a so-called $\textit{normalized EM}$ (N-EM)
algorithm, for a class of log-likelihood functions with integrals. As an extension of the original EM
algorithm, the N-EM
algorithm inherits all advantages of EM-type
algorithms and consists of three steps: normalization step (N-step), expectation step (E-step) and maximization step (M-step), where the N-step is to construct a $\textit{normalized density function}$ (ndf), the E-step is to compute a well-established surrogate $Q$-function and the M-step is to maximize the $Q$-function as in the original EM
algorithm. The ascent property, the best choice of the ndf, and those N-EM
algorithms with a difficult M-step are also explored. By multiple real applications, we have shown that the N-EM
algorithm can solve some problems which cannot be addressed by the EM
algorithm. Next, for problems to which the EM can be applied (often case by case), the N-EM
algorithm can be employed in a unified framework. Numerical experiments are performed and convergence properties are also established.
Read the full abstract ▾
Statistics Theory
Identifiability and Information-Based Inference for Epidemic Transmission Models Under Partial Observation
Md Asaduzzaman
Inference for epidemic transmission on dynamic networks is fundamentally limited by latent infection times, incomplete contact histories, imperfect observation, and external sources of infection. Although coherent likelihood formulations are available for partially observed epidemic processes, considerably less is known about the theoretical limits of statistical inference under such observation mechanisms. This paper develops a unified framework for studying identifiability and Fisher information in epidemic transmission models observed on dynamic contact networks. We establish conditions for structural and local identifiability, derive observed and complete-data information
matrices, and quantify information loss arising from unobserved transmission events and missing network information through a missing-information decomposition. We further investigate how observation frequency, network coverage, and measurement accuracy influence parameter estimability and statistical efficiency, providing a principled basis for evaluating surveillance strategies. Simulation studies demonstrate that the proposed framework accurately characterises the relationship between observation design, statistical information, and parameter estimation, with theoretical predictions closely matching finite-sample performance. The proposed framework clarifies the relationship between observation design, identifiability, and inferential precision, and provides a theoretical foundation for statistical inference in partially observed epidemic transmission models.
Read the full abstract ▾
Statistical Methodology
Copulas for Geostatistical Data: Foundations, Modeling Principles and Statistical Inference
Axel Bücher and Nadja Klein
Spatial statistics commonly describes spatial dependence through second-order quantities such as covariance functions and variograms, often within Gaussian random-field models and under structural assumptions such as stationarity, isotropy, or distance-based decay. Copulas offer a complementary framework that separates marginal distributions from dependence and permits a broad range of non-Gaussian dependence structures. Because the finite-dimensional distributions of a spatial random field can always be decomposed into margins and copulas through Sklar's
theorem, copulas provide a natural
language for studying spatial dependence beyond second-order summaries. Yet the relevant literature has developed along several largely separate strands across spatial statistics, copula modeling,
stochastic processes, and application domains, often with different terminology and modeling objectives. This review brings these strands together: We revisit classical concepts from spatial statistics through a copula lens, discuss copula-based tools for describing spatial dependence, and systematically review constructions of spatial copula models. Particular emphasis is placed on Kolmogorov consistency and on the distinction between models defined for a fixed set of locations and genuinely process-level constructions. We also discuss statistical inference, extensions to spatio-temporal settings, and emerging directions involving flexible marginal and dependence models. By clarifying the relationships among existing approaches and their respective strengths and limitations, the review provides a unified perspective on the interface between copula modeling and spatial statistics.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CV
When Less Is More: A Controlled Benchmark of Lightweight CNNs for Satellite Land-Cover Segmentation on DeepGlobe
Atiq Ur Rehman and Joseph Michael Donovan
High-resolution satellite imagery is the backbone of good land-cover classification, and without that, environmental monitoring, urban planning, and sustainable resource management all fall short. Deep learning architectures perform well in semantic
segmentation, but the efficiency-accuracy trade-off across classical convolutional encoders is not well quantified under controlled, reproducible conditions. This study compares five architectures VGG16, MobileNetV2, InceptionV3, AlexNet, and CNN on the DeepGlobe Land Cover Classification
dataset using three progressively optimized iterations to isolate regularisation, transfer learning, and architectural depth. To ensure performance differentials reflect architectural properties, all experiments used identical preprocessing, hyperparameter, and training protocols without data augmentation or class-imbalance correction. At 24.98 MB, MobileNetV2_v1 had the highest overall accuracy (0.7906) and mean Intersection over Union (0.4625), outperforming deeper alternatives like InceptionV3_v2 (125.17 MB, accuracy 0.7610) and VGG16_v2 (71.13 MB, accuracy 0.7653). Class-wise analysis showed strength in urban, agricultural, and water categories, but rangeland-barren confusion showed that architectural optimization alone cannot optimize spectrally similar minority classes. Strong spatial generalization and crisp boundary delineation were confirmed on held-out test imagery, validating operational applicability. These results show that lightweight, transfer-learned models can match or outperform deeper models in resource-constrained remote-sensing environments, enabling scalable land-cover mapping.
Read the full abstract ▾
Machine Learning (Statistics)
Covariance-Boosted Gaussian Processes for Spatiotemporal Irregularities
Jeremy Ovadia
Nonstationary Gaussian process (GP) models are powerful tools for capturing input-dependent variability by adapting to observed data. However, with limited sampling and highly parameterized covariance structure, they are often prone to overfitting and overconfident uncertainty estimates, potentially leading to misleading predictions in safety-critical applications. Motivated by ionospheric modeling for satellite-based augmentation systems (SBAS), this paper proposes a Covariance-Boosted Gaussian Process (CBGP) framework centered upon boosting covariance priors to discover nonstationary latent functions for signal and observation variation that capture irregularities in the input domain. An additional layer of GP modeling of "partially-whitened" observations guides latent function relative error estimation that is used to iteratively update weak priors in a gradient descent-like procedure. Following boosting, restrictions are imposed upon prior covariances to prevent overfitting while posterior uncertainties are inflated to prevent model overconfidence. CBGP model efficacy and robustness are demonstrated through out-of-sample testing of both simulated and real-world applications that meet a three-nines integrity standard. The modeling of an extensive ionospheric storm
dataset over South America suggests accurate and reliable means to compute SBAS ionospheric corrections in the most challenging space weather environment using regional models that are more informed and responsive than local fitting performed by currently-operating SBAS.
Read the full abstract ▾
Statistical Methodology
Monte Carlo testing: non-asymptotic guarantees without joint exchangeability
Rina Foygel Barber and Aaditya Ramdas
In hypothesis testing, Monte Carlo tests are usually justified either by exact null simulation or by joint exchangeability of the observed data and its simulated copies. This leaves a gap for common computational procedures, such as parallel MCMC sampling initialized at the observed data, where each copy may be marginally null and even pairwise exchangeable with the observation, but the full collection is not jointly exchangeable. In such cases the usual empirical p-value can be invalid when the chain has not mixed, while exactly exchangeable constructions such as the Besag–Clifford hub-and-spoke sampler may suffer from high conditional Monte Carlo variability. We give finite-sample guarantees for this intermediate regime. If, under the null, the observed data $X$ and a copy $X'\sim P(\cdot\mid X)$ are conditionally i.i.d.\ given a latent variable, then for any prespecified statistic and any finite number $m$ of conditionally independent Monte Carlo copies, the resulting empirical p-value obeys $\mathbb P\{p_m\le \alpha\}\le 2\alpha$. This guarantee requires no mixing conditions and holds for any number of copies $m$, and it explains finite-sample oscillatory behavior in inference via MCMC sampling. In addition, we further show that the guarantee provides insights into inference problems arising in other settings, including inference on
Bayesian models (recovering a classical result showing validity up to a factor of $2$ for posterior predictive p-values), and inference via balanced permutation tests.
Read the full abstract ▾
Statistical Computation
SuSIE: Subset Simulation with Intrepid Exploration
Adwait Sharma, Promit Chakroborty and Michael D. Shields
This work explores the challenges associated with the subset simulation framework - a well-established
algorithm for estimating structural reliability - in settings involving multiple, possibly disconnected regions of failure, or involving discontinuous or sharply changing performance functions. We demonstrate that these drawbacks of subset simulation stem from the limitations of the Markov chain Monte Carlo (MCMC) sampler employed within the method, not from the subset simulation framework itself. Traditional random-walk Metropolis
algorithms, which are known to struggle severely with sampling from
multimodal distributions, are conventionally applied within subset simulation, which leads to inaccurate failure probability estimates in such cases. In this work, we instead utilize a modified version of the Intrepid MCMC sampler, which has recently been shown to be more effective than vanilla random-walk Metropolis
algorithms in sampling from
multimodal probability distributions. The subset simulation method with the proposed Intrepid sampler is demonstrated to address these complicating features. Several illustrative examples are considered, ranging from 2 to 1003 dimensions and exhibiting multiple failure regions or highly nonlinear performance functions, including both analytical problems and structural engineering applications.
Read the full abstract ▾
Machine Learning (Statistics)
Robust Conformalized Selection with Noisy Responses
Chengyao Yu, Hongxin Wei and Bingyi Jing
Conformalized selection has been widely applied to select high-quality candidates from large
datasets with rigorous uncertainty quantification, such as reliable labeling, drug discovery, and the
alignment of large
language models. Nevertheless, existing methods assume clean responses on calibration data, an assumption that rarely holds in practice. In this paper, we formulate the above tasks as selecting candidates with true predicted labels or with responses exceeding certain values. We demonstrate that existing conformal selection methods fail to control the false discovery rate (FDR) or suffer from severe power loss under contaminated calibration data. To that end, we propose Robust Conformalized Selection (RCS), a unified framework for selective classification with valid FDR control under general label contamination. The key insight of RCS lies in a novel statistical reduction: by separately conditioning on different classes, we translate the intractable label noise into a localized
covariate shift problem, which then enables a
covariate-adjusted empirical-Bayes-type estimate of the number of false selections. Statistical properties such as the asymptotic FDR control, power optimality, and robustness of RCS are established. We further develop an instantiation of RCS under randomized response model, and also apply RCS to the task of selecting candidates with large response values. Extensive experiments on both simulated and real-world
datasets demonstrate the effectiveness of RCS.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
Asha Barua and Sajad Khodadadian
Natural Policy Gradient (NPG) is a well-established Reinforcement Learning
algorithm that underlies widely used methods such as Trust Region Policy Optimization and Proximal Policy Optimization, both of which have demonstrated strong empirical success. In this paper, we study exact NPG in finite-horizon Markov Decision Processes with known dynamics and horizon-dependent transition kernels. We provide the first finite-time convergence guarantees for this
algorithm in this setting, for which we consider both constant and increasing step size regimes. With a constant step size $\eta_t=\eta$, we prove that NPG converges sublinearly with a rate of $\mathcal{O}(H^{2}/t)$ after $t$ iterations, where $H$ is the horizon length. We also extend this constant step size analysis to linear MDPs in an exact population-projection oracle under a full support projection distribution, recovering the same sublinear rate as in the tabular setting. Furthermore, with increasing step sizes, we prove that this
algorithm achieves a linear convergence rate of $\mathcal{O}\left(\left(1-\frac{1}{\vartheta_\rho}\right)^t\right)$ for a problem-dependent constant $\vartheta_\rho > 1$, and the horizon-only robust schedule of the form $\eta_t=\eta_0(H/(H-1))^t$ where $\eta_0>0$ and $H \geq 2$, attains this same
geometric rate.
Read the full abstract ▾
Machine Learning (Statistics)
Variable Importance Identification Through Lazy Training for Binary Classification
Anand Singh, Luke Pennella, Eshan Kabir et al. (+1 more)
Deep neural networks have been widely used in many applications (e.g., computer
vision and natural
language processing); however, understanding their explainability remains a challenging task. Recently, substantial research has been devoted to improving the explainability of deep neural networks, with most of this work focusing on the regression framework. In this paper, we instead focus on the binary classification framework and adopt a variable-importance framework combined with the idea of lazy training to propose an efficient
algorithm for identifying important features. From a theoretical perspective, our method relies on only a minimal set of assumptions and achieves well-controlled error rates. The validity of the proposed method and
algorithm is examined through extensive simulation studies and real-data applications.
Read the full abstract ▾
All authors Anand Singh, Luke Pennella, Eshan Kabir, Xiaoxi Shen
Statistics Theory
On the Order-Conditional Optimality of Gaffke's Bound
George Bissias and Erik Learned-Miller
Let $X = (X_1, \ldots, X_n)$ be a random vector from any Borel probability law on $\mathbb{R}_+^n$. We revisit the problem of deriving a lower confidence bound (LCB) on a scalar parameter of that law. We recast classical work, beginning with Buehler, in purely probabilistic terms to form a more accessible and extensible framework. We then specialize the framework to the case where the components of $X$ are independent. In this context, we prove that Gaffke's bound is Buehler optimal for the order that it induces with respect to the maximum marginal mean parameter: $max_{i \in [n]} E_Q[X_i]$, which reduces to the common mean when the $X_i$ are independent and identically distributed. That is to say, no other valid LCB that orders samples in the same way as Gaffke's bound can improve on it with respect to this parameter.
Read the full abstract ▾