Statistical Methodology
Scenario Approach with Post-Design Certification of User-Specified Properties
Algo Carè, Marco C. Campi and Simone Garatti
The scenario approach is an established data-driven design framework that comes equipped with a powerful theory linking design complexity to generalization properties. In this approach, data are simultaneously used both for design and for certifying the design's reliability, without resorting to a separate test
dataset. This paper takes a step further by guaranteeing additional properties, useful in post-design usage but not considered during the design phase. To this end, we introduce a two-level framework of appropriateness: baseline appropriateness, which guides the design process, and post-design appropriateness, which serves as a criterion for a posteriori evaluation. We provide distribution-free upper bounds on the risk of failing to meet the post-design appropriateness; these bounds are computable without using any additional test data. Under additional assumptions, lower bounds are also derived. As part of an effort to demonstrate the usefulness of the proposed methodology, the paper presents two practical examples in H2 and pole-placement problems. Moreover, a method is provided to infer comprehensive distributional knowledge of relevant performance indexes from the available
dataset.
Read the full abstract ▾
Other Statistics
Either a Confidence Interval Covers, or It Doesn't (Or Does It?): A Model-Based View of Ex-Post Coverage Probability
Scott Lee
In Neyman's original formulation, a 1-alpha confidence interval procedure is justified by its long-run coverage properties, and a single realized interval is to be described only by the slogan that it either covers the parameter or it does not. On this view, post-data probability statements about the coverage of an individual interval are taken to be conceptually out of bounds. In this paper, I present two kinds of arguments against treating that "either-or" reading as the only legitimate interpretation of confidence. The first is informal, via a set of thought experiments in which the same joint probability model is used to compute both forward-looking and backward-looking probabilities for occurred-but-unobserved events. The second is more formal, recasting the standard confidence-interval construction in terms of infinite sequences of trials and their associated 0/1 coverage indicators. In that representation, the design-level coverage probability 1-alpha and the degenerate conditional probabilities given the full data appear simply as different conditioning levels of the same model. I argue that a strict behavioristic reading that privileges only the latter is in tension with the very mathematical machinery used to define long-run error rates. I then sketch an alternative view of confidence as a predictive probability (or forecast) about the coverage indicator, together with a simple normative rule for when intermediate probabilities for single coverage events should be allowed. Keywords: confidence intervals; coverage probability; frequentist inference; single-case probability; predictive probability; Neyman. Disclaimer: The findings and conclusions in this report are those of the author and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Read the full abstract ▾
Statistical Methodology
Fixed-Horizon Self-Normalized Inference for Adaptive Experiments via Martingale AIPW/DML with Logged Propensities
Gabriel Saco
Adaptive randomized experiments update treatment probabilities as data accrue, but still require an end-of-study interval for the average treatment effect (ATE) at a prespecified horizon. Under adaptive assignment, propensities can keep changing, so the predictable quadratic variation of AIPW/DML score increments may remain random. When no deterministic variance limit exists, Wald statistics normalized by a single long-run variance target can be conditionally miscalibrated given the realized variance regime. We assume no interference, sequential randomization, i.i.d. arrivals, and executed overlap on a prespecified scored set, and we require two auditable pipeline conditions: the platform logs the executed randomization probability for each unit, and the nuisance regressions used to score unit $t$ are constructed predictably from past data only. These conditions make the centered AIPW/DML scores an exact martingale difference sequence. Using self-normalized martingale limit theory, we show that the Studentized statistic, with variance estimated by realized quadratic variation, is asymptotically N(0,1) at the prespecified horizon, even without variance stabilization. Simulations validate the theory and highlight when standard fixed-variance Wald reporting fails.
Read the full abstract ▾
Machine Learning (Statistics)
Functional Central Limit Theorem for Stochastic Gradient Descent
Kessang Flamand and Victor-Emmanuel Brunel
We study the asymptotic shape of the trajectory of the
stochastic gradient descent
algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit
theorem for the properly rescaled trajectory. Our result characterizes the long-term fluctuations of the
algorithm around the minimizer by providing a
diffusion limit for the trajectory. In contrast with classical central limit
theorems for the last iterate or Polyak-Ruppert averages, this functional result captures the temporal structure of the fluctuations and applies to non-smooth settings such as robust location estimation, including the
geometric median.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Approximation Theory for Lipschitz Continuous Transformers
Takashi Furuya, Davide Murari and Carola-Bibiane Schönlieb
Stability and robustness are critical for deploying
Transformers in safety-sensitive settings. A principled way to enforce such behavior is to constrain the model's Lipschitz constant. However, approximation-theoretic guarantees for architectures that explicitly preserve Lipschitz continuity have yet to be established. In this work, we bridge this gap by introducing a class of gradient-descent-type in-context
Transformers that are Lipschitz-continuous by construction. We realize both MLP and
attention blocks as explicit Euler steps of negative gradient flows, ensuring inherent stability without sacrificing expressivity. We prove a universal approximation
theorem for this class within a Lipschitz-constrained function space. Crucially, our analysis adopts a measure-theoretic formalism, interpreting
Transformers as operators on probability measures, to yield approximation guarantees independent of
token count. These results provide a rigorous theoretical foundation for the design of robust, Lipschitz continuous
Transformer architectures.
Read the full abstract ▾
Statistical Methodology
Confidence Distributions for FIC scores
Céline Cunen and Nils Lid Hjort
When using the Focused Information Criterion (FIC) for assessing and ranking candidate models with respect to how well they do for a given estimation task, it is customary to produce a so-called FIC plot. This plot has the different point estimates along the y-axis and the root-FIC scores on the x-axis, these being the estimated root-mean-square scores. In this paper we address the estimation uncertainty involved in each of the points of such a FIC plot. This needs careful assessment of each of the
estimators from the candidate models, taking also modelling bias into account, along with the relative precision of the associated estimated mean squared error quantities. We use confidence distributions for these endeavours. This leads to fruitful CD-FIC plots, helping the statistician to judge to what extent the seemingly best models really are better than other models, etc. These efforts also lead to two further developments. The first is a new tool for model selection, which we call the
quantile FIC, which helps overcome certain difficulties associated with the usual FIC procedures, related to somewhat arbitrary schemes for handling estimated squared biases. A particular case is the median-FIC. The second development is to form model averaged
estimators with fruitful weights determined by the relative sizes of the median- and
quantile-FIC scores. And Mrs. Jones is pregnant.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Logit Distance Bounds Representational Similarity
Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele et al. (+2 more)
For a broad family of discriminative models that includes autoregressive
language models, identifiability results imply that if two models induce the same conditional distributions, then their internal representations are equal up to an invertible linear transformation. We ask whether an analogous conclusion holds approximately when the distributions are close instead of equal. Building on the observation of Nielsen et al. (2025) that closeness in KL divergence need not imply high linear representational similarity, we study a distributional distance based on logit differences and show that closeness in this distance does yield linear similarity guarantees. Specifically, we define a representational dissimilarity measure based on the models' identifiability class and prove that it is bounded by the logit distance. We further show that, when model probabilities are bounded away from zero, KL divergence upper-bounds logit distance; yet the resulting bound fails to provide nontrivial control in practice. As a consequence, KL-based
distillation can match a teacher's predictions while failing to preserve linear representational properties, such as linear-probe recoverability of human-interpretable concepts. In
distillation experiments on synthetic and image
datasets, logit-distance
distillation yields students with higher linear representational similarity and better preservation of the teacher's linearly recoverable concepts.
Read the full abstract ▾
All authors Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele, Andrea Dittadi, Simon Buchholz
Statistics Applications
Deep description of static and dynamic network ties in Honduran villages
Marios Papamichalis, Nikolaos Nakis and Nicholas A. Christakis
We examine static and dynamic social network structure in 176 villages within the Copan Department of Honduras across two data waves (2016, 2019), using detailed data on multiplex networks for 20,232 individuals enrolled in a longitudinal survey. These networks capture friendship, health advice, financial help, and
adversarial relationships, allowing us to show how cooperation and conflict jointly shape social structure. Using node-level network measures derived from near-census sociocentric village networks, we leverage mixed-effects zero-inflated negative binomial models to assess the influence of individual attributes, such as gender, marital status, education, religion, and indigenous status, and of village characteristics, on the dynamics of social networks over time. We complement these node-level models with dyadic assortativity (odds-ratio-based homophily) and community-level measures to describe how sorting by key attributes differs across network types and between waves. Our results demonstrate significant assortativity based on gender and religion, particularly within health and financial networks. Across networks, gender and religion exhibit the most consistent assortative mixing. Additionally, community-level assortativity metrics indicate that educational and financial factors increasingly influence social ties over time. Our findings provide insights into how personal attributes and community dynamics interact to shape network formation and socio-economic relationships in rural settings over time.
Read the full abstract ▾
Statistical Methodology
Space-filling lattice designs for computer experiments
Naoki Sakai and Takashi Goda
This paper investigates the construction of space-filling designs for computer experiments. The space-filling property is characterized by the covering and separation radii of a design, which are integrated through the unified criterion of quasi-uniformity. We focus on a special class of designs, known as quasi-Monte Carlo (QMC)
lattice point sets, and propose two construction
algorithms. The first
algorithm generates rank-1
lattice point sets as an approximation of quasi-uniform Kronecker sequences, where the generating vector is determined explicitly. As a byproduct of our analysis, we prove that this explicit point set achieves an isotropic discrepancy of $O(N^{-1/d})$. The second
algorithm utilizes Korobov
lattice point sets, employing the Lenstra–Lenstra–Lovász (LLL) basis reduction
algorithm to identify the generating vector that ensures quasi-uniformity. Numerical experiments are provided to validate our theoretical claims regarding quasi-uniformity. Furthermore, we conduct empirical comparisons between various QMC point sets in the context of Gaussian process regression, showcasing the efficacy of the proposed designs for computer experiments.
Read the full abstract ▾
Statistical Methodology
Bayesian Nonparametrics for Gene-Gene and Gene-Environment Interactions in Case-Control Studies: A Synthesis and Extension
Durba Bhattacharya and Sourabh Bhattacharya
Gene-gene and gene-environment interactions are widely believed to play significant roles in explaining the variability of complex traits. While substantial research exists in this area, a comprehensive statistical framework that addresses multiple sources of uncertainty simultaneously remains lacking. In this article, we synthesize and propose extension of a novel class of
Bayesian nonparametric approaches that account for interactions among genes, loci, and environmental factors while accommodating uncertainty about population substructure. Our contribution is threefold: (1) We provide a unified exposition of hierarchical
Bayesian models driven by Dirichlet processes for genetic interactions, clarifying their conceptual advantages over traditional regression approaches; (2) We shed light on new computational strategies that combine transformation-based MCMC with parallel processing for scalable inference; and (3) We present enhanced hypothesis testing procedures for identifying disease-predisposing loci.Through applications to myocardial infarction data, we demonstrate how these methods offer biological insights not readily obtainable from standard approaches. Our synthesis highlights the advantages of
Bayesian nonparametric thinking in genetic epidemiology while providing practical guidance for implementation.
Read the full abstract ▾
Statistics Applications
From Chain-Ladder to Individual Claims Reserving
Ronald Richman and Mario V. Wüthrich
The chain-ladder (CL) method is the most widely used claims reserving technique in non-life insurance. This manuscript introduces a novel approach to computing the CL reserves based on a fundamental restructuring of the data utilization for the CL prediction procedure. Instead of rolling forward the cumulative claims with estimated CL factors, we estimate multi-period factors that project the latest observations directly to the ultimate claims. This alternative perspective on CL reserving creates a natural pathway for the application of machine learning techniques to individual claims reserving. As a proof of concept, we present a small-scale real data application employing neural networks for individual claims reserving.
Read the full abstract ▾
Statistical Methodology
Joint Modeling of Longitudinal EHR Data with Shared Random Effects for Informative Visiting and Observation Processes
Cheng-Han Yang, Xu Shi and Bhramar Mukherjee
Longitudinal electronic health record (EHR) data offer opportunities to study biomarker trajectories; however, association estimates-the primary inferential target-from standard models designed for regular observation times may be biased by a two-stage hierarchical missingness mechanism. The first stage is the visiting process (informative presence), where encounters occur at irregular times driven by patient health status; the second is the observation process (informative observation), where biomarkers are selectively measured during visits. To address these mechanisms, we propose a unified semiparametric joint modeling framework that simultaneously characterizes the visiting, biomarker observation, and longitudinal outcome processes. Central to this framework is a shared subject-specific Gaussian latent variable that captures unmeasured frailty and induces dependence across all components. We develop a three-stage estimation procedure and establish the consistency and asymptotic normality of our
estimators. We also introduce a sequential procedure that imputes missing biomarkers prior to adjusting for irregular visiting and examine its performance. Simulation results demonstrate that our method yields unbiased estimates under this mechanism, whereas existing approaches can be substantially biased; notably, methods adjusting only for irregular visiting may exhibit even greater bias than those ignoring both mechanisms. We apply our framework to data from the All of Us Research Program to investigate associations between neighborhood-level socioeconomic status indicators and six blood-based biomarker trajectories, providing a robust tool for outpatient settings where irregular monitoring and selective measurement are prevalent.
Read the full abstract ▾
Statistics Theory
Non-Stationary Covariance Functions for Spatial Data on Linear Networks
Alfredo Alegría
We introduce a novel class of non-stationary covariance functions for random fields on linear networks that allows both the variance and the correlation range of the random field to vary spatially. The proposed covariance functions are useful to model random fields with a spatial dependence that is locally isotropic with respect to the resistance metric, a distance that reflects the
topology of the network. The framework admits explicit
stochastic representations of the associated random fields and can be naturally extended to
matrix-valued covariance functions for vector-valued random fields. We assess the statistical and computational performance of a weighted local likelihood
estimator for the proposed models using synthetic data generated on the street network of the University of Chicago neighborhood.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis et al. (+1 more)
Machine learning model performance improvements tend to arise from competition and application. For deployment, we consider prescriptive scaling laws: given a pre-training compute budget, what downstream accuracy is attainable with contemporary post-training practice, and how stable is that mapping as the field evolves? Using large-scale observational evaluations with 5k existing and 2k newly evaluated model checkpoints spanning 2022-2026 across six
benchmarks, we estimate capability boundaries, high conditional
quantiles of
benchmark scores as a function of log pre-training FLOPs, via smoothed
quantile regression with a monotone, saturating sigmoid parameterization. We validate temporal reliability by fitting on earlier model generations and evaluating on later releases: across four of six tasks, the out-of-distribution coverage error remains below 2%, while math
reasoning exhibits a consistently advancing boundary over time. For instance, at a budget of 10^24 FLOPs, the estimated attainable accuracies are 0.83 on IFEval and 0.54 on MATH Lvl 5. We then extend our approach to analyze task-dependent saturation and to probe contamination-related shifts on math
reasoning tasks. Finally, we introduce a balanced I-optimal sampling
algorithm that recovers near-full-data frontiers using roughly 20% of the parameter-count-weighted evaluation budget, as low as 5% on some tasks, while maintaining comparable calibration. Together, our work releases Proteus-2k, the latest model performance evaluation
dataset, and introduces a practical methodology for translating compute budgets into reliable performance expectations and for monitoring when capability boundaries shift across time.
Read the full abstract ▾
All authors Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis, Sham Kakade
Statistical Methodology
Bayesian Inference for Joint Tail Risk in Paired Biomarkers via Archimedean Copulas with Restricted Jeffreys Priors
Agnideep Aich, Md. Monzur Murshed, Sameera Hewage et al. (+1 more)
We propose a
Bayesian copula-based framework to quantify clinically interpretable joint tail risks from paired continuous biomarkers. After converting each biomarker margin to rank-based pseudo-observations, we model dependence using one-parameter Archimedean copulas and focus on three probability-scale summaries at tail level $\alpha$: the lower-tail joint risk $R_L(\theta)=C_\theta(\alpha,\alpha)$, the upper-tail joint risk $R_U(\theta)=2\alpha-1+C_\theta(1-\alpha,1-\alpha)$, and the conditional lower-tail risk $R_C(\theta)=R_L(\theta)/\alpha$. Uncertainty is quantified via a restricted Jeffreys prior on the copula parameter and grid-based posterior approximation, which induces an exact posterior for each tail-risk functional. In simulations from Clayton and Gumbel copulas across multiple dependence strengths, posterior credible intervals achieve near-nominal coverage for $R_L$, $R_U$, and $R_C$. We then analyze NHANES 2017–2018 fasting glucose (GLU) and HbA1c (GHB) ($n=2887$) at $\alpha=0.05$, obtaining tight posterior credible intervals for both the dependence parameter and induced tail risks. The results reveal markedly elevated extremal co-movement relative to independence; under the Gumbel model, the posterior mean joint upper-tail risk is $R_U(\alpha)=0.0286$, approximately $11.46\times$ the independence
benchmark $\alpha^2=0.0025$. Overall, the proposed approach provides a principled, dependence-aware method for reporting joint and conditional extremal-risk summaries with
Bayesian uncertainty quantification in biomedical applications.
Read the full abstract ▾
All authors Agnideep Aich, Md. Monzur Murshed, Sameera Hewage, Ashit Baran Aich
Machine Learning (Statistics) · Cross-listed from cs.CV
Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models
Tai Le-Gia and Jaehyun Ahn
Zero-shot anomaly detection (ZSAD) has gained increasing
attention in medical imaging as a way to identify abnormalities without task-specific supervision, but most advances remain limited to 2D
datasets. Extending ZSAD to 3D medical images has proven challenging, with existing methods relying on slice-wise features and
vision-
language models, which fail to capture volumetric structure. In this paper, we introduce a fully training-free framework for ZSAD in 3D
brain MRI that constructs localized volumetric
tokens by aggregating multi-axis slices processed by 2D foundation models. These 3D patch
tokens restore cubic spatial context and integrate directly with distance-based, batch-level anomaly detection pipelines. The framework provides compact 3D representations that are practical to compute on standard GPUs and require no fine-tuning,
prompts, or supervision. Our results show that training-free, batch-based ZSAD can be effectively extended from 2D encoders to full 3D MRI volumes, offering a simple and robust approach for volumetric anomaly detection.
Read the full abstract ▾
Machine Learning (Statistics)
Sparse Additive Model Pruning for Order-Based Causal Structure Learning
Kentaro Kanamori, Hirofumi Suzuki and Takuya Takagi
Causal structure learning, also known as causal discovery, aims to estimate causal relationships between variables as a form of a causal directed acyclic graph (DAG) from observational data. One of the major frameworks is the order-based approach that first estimates a topological order of the underlying DAG and then prunes spurious edges from the fully-connected DAG induced by the estimated topological order. Previous studies often focus on the former ordering step because it can dramatically reduce the search space of DAGs. In practice, the latter
pruning step is equally crucial for ensuring both computational efficiency and estimation accuracy. Most existing methods employ a
pruning technique based on generalized additive models and hypothesis testing, commonly known as CAM-
pruning. However, this approach can be a computational bottleneck as it requires repeatedly fitting additive models for all variables. Furthermore, it may harm estimation quality due to multiple testing. To address these issues, we introduce a new
pruning method based on sparse additive models, which enables direct
pruning of redundant edges without relying on hypothesis testing. We propose an efficient
algorithm for learning sparse additive models by combining the randomized tree
embedding technique with group-wise sparse regression. Experimental results on both synthetic and real
datasets demonstrated that our method is significantly faster than existing
pruning methods while maintaining comparable or superior accuracy.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.IT
On the Entropy of General Mixture Distributions
Namyoon Lee
Mixture distributions are a workhorse model for
multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential
entropy of the mixture is notoriously hard to compute because the mixture couples a logarithm with a sum. This paper develops a deterministic, closed-form toolkit for bounding and accurately approximating mixture
entropy directly from component parameters. Our starting point is an information-theoretic channel viewpoint: the latent mixture label plays the role of an input, and the observation is the output. This viewpoint separates mixture
entropy into an average within-component uncertainty plus an overlap term that quantifies how much the observation reveals about the hidden label. We then bound and approximate this overlap term using pairwise overlap integrals between component densities, yielding explicit expressions whenever these overlaps admit a closed form. A simple, family-dependent offset corrects the systematic bias of the Jensen overlap bound and is calibrated to be exact in the two limiting regimes of complete overlap and near-perfect separation. A final clipping step guarantees that the estimate always respects universal information-theoretic bounds. Closed-form specializations are provided for Gaussian, factorized
Laplacian, uniform, and hybrid mixtures, and numerical experiments validate the resulting bounds and approximations across separation, dimension, number of components, and correlated covariances.
Read the full abstract ▾
Statistics Theory
Bayes Risk for Goodness of Fit Tests
Nicholas G. Polson, Vadim Sokolov and Daniel Zantedeschi
We develop a unified framework for goodness-of-fit (GOF) testing through the lens of Bayes risk. Classical GOF procedures are commonly calibrated either at fixed significance level (CLT scale) or through exponential error exponents (LDP scale). We establish that Bayes-risk optimal calibration operates on the moderate-deviation (MDP) scale, producing canonical $\sqrt{\log n}$ inflation of rejection thresholds and polynomially decaying Type I error. Our main contributions are: (i) we formalise the Rubin–Sethuraman program for KS-type statistics as a risk-calibration
theorem with explicit regularity conditions on priors and empirical-process functionals; (ii) we develop the precise connection between Bayes-risk expansions and Sanov information asymptotics, showing how $\log n$-order truncations arise naturally when risk, rather than pure exponents, is the evaluation criterion; (iii) we provide detailed applications to location testing under Laplace families, shape testing via Bayes factors, and connections to Fisher information geometry. The organizing principle throughout is that sample size enters Bayes-optimal GOF cutoffs through the MDP scale, unifying KS-based and Sanov-based perspectives under a single risk criterion.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
The Information Geometry of Softmax: Probing and Steering
Kiho Park, Todd Nief, Yo Joong Choe et al. (+1 more)
This paper concerns the question of how AI systems encode semantic structure into the
geometric structure of their representation spaces. The motivating observation is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. In this case, we argue that the natural geometry is information geometry. Our focus is on the role of information geometry on semantic encoding and the linear representation hypothesis. As an illustrative application, we develop "dual steering", a method for robustly steering representations to exhibit a particular concept using linear probes. We prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. Empirically, we find that dual steering enhances the controllability and stability of concept manipulation.
Read the full abstract ▾
All authors Kiho Park, Todd Nief, Yo Joong Choe, Victor Veitch
Statistical Methodology
Structural grouping of extreme value models via graph fused lasso
Takuma Yoshida, Koki Momoki and Shuichi Kawano
The generalized Pareto distribution (GPD) is a fundamental model for analyzing the tail behavior of a distribution. In particular, the shape parameter of the GPD characterizes the extremal properties of the distribution. As described in this paper, we propose a method for grouping shape parameters in the GPD for clustered data via graph fused lasso. The proposed method simultaneously estimates the model parameters and identifies which clusters can be grouped together. We establish the asymptotic theory of the proposed
estimator and demonstrate that its variance is lower than that of the cluster-wise
estimator. This variance reduction not only enhances estimation stability but also provides a principled basis for identifying homogeneity and heterogeneity among clusters in terms of their tail behavior. We assess the performance of the proposed
estimator through Monte Carlo simulations. As an illustrative example, our method is applied to rainfall data from 996 clustered sites across Japan.
Read the full abstract ▾