Statistical Methodology
Measures of classification bias derived from sample size analysis
Ioannis Ivrissimtzis, Shauna Concannon, Matthew Houliston et al. (+1 more)
We propose the use of a simple intuitive principle for measuring algorithmic classification bias: the significance of the differences in a classifier's error rates across the various demographics is inversely commensurate with the sample size required to statistically detect them. That is, if large sample sizes are required to statistically establish biased behavior, the
algorithm is less biased, and vice versa. In a simple setting, we assume two distinct demographics, and non-parametric estimates of the error rates on them, e1 and e2, respectively. We use a well-known approximate formula for the sample size of the chi-squared test, and verify some basic desirable properties of the proposed measure. Next, we compare the proposed measure with two other commonly used statistics, the difference e2-e1 and the ratio e2/e1 of the error rates. We establish that the proposed measure is essentially different in that it can rank
algorithms for bias differently, and we discuss some of its advantages over the other two measures. Finally, we briefly discuss how some of the desirable properties of the proposed measure emanate from fundamental characteristics of the method, rather than the approximate sample size formula we used, and thus, are expected to hold in more complex settings with more than two demographics.
Read the full abstract ▾
All authors Ioannis Ivrissimtzis, Shauna Concannon, Matthew Houliston, Graham Roberts
Machine Learning (Statistics)
Microeconomic Foundations of Multi-Agent Learning
Nassim Helou
Modern AI systems increasingly operate inside
markets and institutions where data, behavior, and incentives are endogenous. This paper develops an economic foundation for multi-
agent learning by studying a principal-
agent interaction in a Markov decision process with strategic externalities, where both the principal and the
agent learn over time. We propose a two-phase incentive mechanism that first estimates implementable transfers and then uses them to steer long-run dynamics; under mild regret-based rationality and exploration conditions, the mechanism achieves sublinear social-welfare regret and thus asymptotically optimal welfare. Simulations illustrate how even coarse incentives can correct inefficient learning under stateful externalities, highlighting the necessity of incentive-aware design for safe and welfare-aligned AI in
markets and insurance.
Read the full abstract ▾
Statistical Methodology
On estimands in target trial emulation
Edoardo Efrem Gervasoni, Liesbet De Bus, Stijn Vansteelandt et al. (+1 more)
The target trial framework enables causal inference from longitudinal observational data by emulating randomized trials initiated at multiple time points. Precision is often improved by pooling information across trials, with standard models typically assuming - among other things - a time-constant treatment effect. However, this obscures interpretation when the true treatment effect varies, which we argue to be likely as a result of relying on noncollapsible estimands. To address these challenges, this paper introduces a model-free strategy for target trial analysis, centered around the choice of the estimand, rather than model specification. This ensures that treatment effects remain clearly interpretable for well-defined populations even under model misspecification. We propose estimands suitable for different study designs, and develop accompanying G-computation and inverse probability weighted
estimators. Applications on simulations and real data on antimicrobial de-escalation in an intensive care unit setting demonstrate the greater clarity and reliability of the proposed methodology over traditional techniques.
Read the full abstract ▾
All authors Edoardo Efrem Gervasoni, Liesbet De Bus, Stijn Vansteelandt, Oliver Dukes
Machine Learning (Statistics) · Cross-listed from cs.CL
A path to natural language through tokenisation and transformers
David S. Berman and Alexander G. Stapleton
Natural
languages exhibit striking regularities in their statistical structure, including notably the emergence of Zipf's and Heaps' laws. Despite this, it remains broadly unclear how these properties relate to the modern tokenisation schemes used in contemporary
transformer models. In this note, we analyse the information content (as measured by the Shannon
entropy) of various corpora under the assumption of a Zipfian frequency distribution, and derive a closed-form expression for the slot
entropy expectation value. We then empirically investigate how byte–pair encoding (BPE) transforms corpus statistics, showing that recursive applications of BPE drive
token frequencies toward a Zipfian power law while inducing a characteristic growth pattern in empirical
entropy. Utilizing the ability of
transformers to learn context dependent
token probability distributions, we train
language models on corpora tokenised at varying BPE depths, revealing that the model predictive
entropies increasingly agree with Zipf-derived predictions as the BPE depth increases.
Attention-based diagnostics further indicate that deeper tokenisation reduces local
token dependencies, bringing the empirical distribution closer to the weakly dependent (near IID) regime. Together, these results clarify how BPE acts not only as a compression mechanism but also as a statistical transform that reconstructs key informational properties of natural
language.
Read the full abstract ▾
Machine Learning (Statistics)
On the Identifiability of Regime-Switching Models with Multi-Lag Dependencies
Carles Balsells-Rodas, Toshiko Matsui, Pedro A.M. Mediano et al. (+2 more)
Identifiability is central to the interpretability of deep latent variable models, ensuring parameterisations are uniquely determined by the data-generating distribution. However, it remains underexplored for deep regime-switching time series. We develop a general theoretical framework for multi-lag Regime-Switching Models (RSMs), encompassing Markov Switching Models (MSMs) and Switching Dynamical Systems (SDSs). For MSMs, we formulate the model as a temporally structured finite mixture and prove identifiability of both the number of regimes and the multi-lag transitions in a nonlinear-Gaussian setting. For SDSs, we establish identifiability of the latent variables up to permutation and scaling via temporal structure, which in turn yields conditions for identifiability of regime-dependent latent causal graphs (up to regime/node permutations). Our results hold in a fully unsupervised setting through architectural and noise assumptions that are directly enforceable via neural network design. We complement the theory with a flexible variational
estimator that satisfies the assumptions and validate the results on synthetic
benchmarks. Across real-world
datasets from neuroscience, finance, and
climate, identifiability leads to more trustworthy interpretability analysis, which is crucial for scientific discovery.
Read the full abstract ▾
All authors Carles Balsells-Rodas, Toshiko Matsui, Pedro A.M. Mediano, Yixin Wang, Yingzhen Li
Statistical Methodology
Progressive Bayesian Confidence Architectures for Cold-Start Personal Health Analytics: Formalizing Early Insight Through Posterior Contraction and Risk-Aware Interpretation
Richik Chakraborty
Personal health analytics systems face a persistent cold-start dilemma: users expect meaningful insights early in data collection, while conventional statistical inference requires data volumes that often exceed engagement horizons. Existing approaches either delay inference until fixed statistical thresholds are met – leading to user disengagement – or surface heuristic insights without formal uncertainty quantification, risking false confidence. We propose a progressive
Bayesian confidence architecture that formalizes early-stage inference through phased interpretation of posterior uncertainty. Drawing on
Bayesian updating and epistemic strategies from financial risk modeling under sparse observations, we map posterior contraction to interpretable tiers of insight, ranging from exploratory directional evidence to robust associative inference. We demonstrate the framework's performance through controlled experimentation with synthetic N-of-1 health data, showing that calibrated early insights can be generated within 5–7 days while maintaining explicit epistemic humility. Compared to fixed-threshold baselines requiring 30+ days of data, the proposed approach yields earlier directional signals (mean: 5.3 vs 31.7 days, p<0.001) while controlling false discovery rates below 6% (5.9% at day 30) despite 26-day earlier detection, compared to 0% FDR for fixed-threshold baselines that delay insights by 30 days. In addition, we show strong uncertainty calibration (76% credible interval coverage for ground-truth correlations at day 90). This work contributes a methodological framework for uncertainty-aware early inference in personalized health analytics that bridges the gap between user engagement requirements and statistical rigor.
Read the full abstract ▾
Machine Learning (Statistics)
Self-Supervised Learning from Noisy and Incomplete Data
Julián Tachella and Mike Davies
Many important problems in science and engineering involve inferring a signal from noisy and/or incomplete observations, where the observation process is known. Historically, this problem has been tackled using hand-crafted regularization (e.g., sparsity, total-variation) to obtain meaningful estimates. Recent data-driven methods often offer better solutions by directly learning a solver from examples of ground-truth signals and associated observations. However, in many real-world applications, obtaining ground-truth references for training is expensive or impossible. Self-supervised learning methods offer a promising alternative by learning a solver from measurement data alone, bypassing the need for ground-truth references. This manuscript provides a comprehensive summary of different self-supervised methods for inverse problems, with a special emphasis on their theoretical underpinnings, and presents practical applications in imaging inverse problems.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
PET-TURTLE: Deep Unsupervised Support Vector Machines for Imbalanced Data Clusters
Javier Salazar Cavazos
Foundation
vision, audio, and
language models enable zero-shot performance on downstream tasks via their latent representations. Recently, unsupervised learning of data group structure with deep learning methods has gained popularity. TURTLE, a state of the art deep clustering
algorithm, uncovers data labeling without supervision by alternating label and hyperplane updates, maximizing the hyperplane margin, in a similar fashion to support vector machines (SVMs). However, TURTLE assumes clusters are balanced; when data is imbalanced, it yields non-ideal hyperplanes that cause higher clustering error. We propose PET-TURTLE, which generalizes the cost function to handle imbalanced data distributions by a power law prior. Additionally, by introducing sparse logits in the labeling process, PET-TURTLE optimizes a simpler search space that in turn improves accuracy for balanced
datasets. Experiments on synthetic and real data show that PET-TURTLE improves accuracy for imbalanced sources, prevents over-prediction of minority clusters, and enhances overall clustering.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from quant-ph
Shallow-circuit Supervised Learning on a Quantum Processor
Luca Candelori, Swarnadeep Majumder, Antonio Mezzacapo et al. (+6 more)
Quantum computing has long promised transformative advances in data analysis, yet practical
quantum machine learning has remained elusive due to fundamental obstacles such as a steep
quantum cost for the loading of classical data and poor trainability of many
quantum machine learning
algorithms designed for near-term
quantum hardware. In this work, we show that one can overcome these obstacles by using a linear
Hamiltonian-based machine learning method which provides a compact
quantum representation of classical data via ground state problems for k-local
Hamiltonians. We use the recent sample-based Krylov
quantum diagonalization method to compute low-energy states of the data
Hamiltonians, whose parameters are trained to express classical
datasets through local gradients. We demonstrate the efficacy and scalability of the methods by performing experiments on
benchmark datasets using up to 50
qubits of an IBM Heron
quantum processor.
Read the full abstract ▾
All authors Luca Candelori, Swarnadeep Majumder, Antonio Mezzacapo, Javier Robledo Moreno, Kharen Musaelian, Santhanam Nagarajan, Sunil Pinnamaneni, Kunal Sharma, Dario Villani
Machine Learning (Statistics) · Cross-listed from cs.LG
From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
Marc Finzi, Shikai Qiu, Yiding Jiang et al. (+3 more)
Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated without considering a downstream task? On these questions, Shannon information and Kolmogorov complexity come up nearly empty-handed, in part because they assume observers with unlimited computational capacity and do not target the useful information content. In this work, we identify and exemplify three seeming paradoxes in information theory: (1) information cannot be increased by deterministic transformations; (2) information is independent of the order of data; (3) likelihood modeling is merely distribution matching. To shed light on the tension between these results and modern practice, and to quantify the value of data, we introduce epiplexity, a formalization of information capturing what computationally bounded observers can learn from data. Epiplexity captures the structural content in data while excluding time-bounded
entropy, the random unpredictable content exemplified by pseudorandom number generators and chaotic dynamical systems. With these concepts, we demonstrate how information can be created with computation, how it depends on the ordering of the data, and how likelihood modeling can produce more complex programs than present in the data generating process itself. We also present practical procedures to estimate epiplexity which we show capture differences across data sources, track with downstream performance, and highlight
dataset interventions that improve out-of-distribution generalization. In contrast to principles of model selection, epiplexity provides a theoretical foundation for data selection, guiding how to select, generate, or transform data for learning systems.
Read the full abstract ▾
All authors Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, Andrew Gordon Wilson
Statistics Applications
Computationally Efficient Estimation of Localized Treatment Effects for Multi-Level, Multi-Component Interventions to Address the Opioid Crisis
Abdulrahman A. Ahmed, M. Amin Rahimian, Qiushi Chen et al. (+1 more)
The opioid epidemic remains a major public health challenge in the United States, requiring a multi-pronged intervention approach to mitigate harms to communities. Given the heterogeneity of the epidemic, it is crucial for policymakers to understand localized treatment effects of different intervention components and utilize limited resources efficiently. While locally calibrated simulation models can project epidemic outcomes for any given intervention policy, collecting simulation results for all intervention combinations to estimate localized treatment effects for each community is impractical because the number of combinations grows exponentially with the number of interventions and the levels at which they are applied. To tackle this, we develop a two-stage metamodel framework with a two-step sequential design for efficient sampling. The metamodel consists of a response function linking health outcomes to each intervention component's treatment effect, and a Gaussian process regression (GPR) to learn spatial and socio-economic structures of the treatment effects based on locally-contextualized
covariates. With two-step sequential sampling, we leverage spatial correlations and posterior uncertainty to sequentially sample the most informative counties and treatment conditions. We apply this framework to estimate the treatment effects of buprenorphine dispensing and naloxone distribution on overdose mortality rates using a calibrated
agent-based opioid epidemic model in Pennsylvania counties. Our approach achieves less than 5% average relative error using fewer than 2% of the runs required for an exhaustive simulation. Our two-stage framework provides a computationally efficient approach to support policymakers, enabling an efficient evaluation of alternative resource-allocation strategies to mitigate the opioid epidemic in local communities.
Read the full abstract ▾
All authors Abdulrahman A. Ahmed, M. Amin Rahimian, Qiushi Chen, Praveen Kumar
Machine Learning (Statistics) · Cross-listed from cs.LG
Time-Aware Synthetic Control
Saeyoung Rho, Cyrus Illick, Samhitha Narasipura et al. (+3 more)
The synthetic control (SC) framework is widely used for observational causal inference with time-series panel data. SC has been successful in diverse applications, but existing methods typically treat the ordering of pre-intervention time indices interchangeable. This invariance means they may not fully take advantage of temporal structure when strong trends are present. We propose Time-Aware Synthetic Control (TASC), which employs a state-space model with a constant trend while preserving a low-rank structure of the signal. TASC uses the Kalman filter and Rauch-Tung-Striebel smoother: it first fits a generative time-series model with expectation-maximization and then performs counterfactual inference. We evaluate TASC on both simulated and real-world
datasets, including policy evaluation and sports prediction. Our results suggest that TASC offers advantages in settings with strong temporal trends and high levels of observation noise.
Read the full abstract ▾
All authors Saeyoung Rho, Cyrus Illick, Samhitha Narasipura, Alberto Abadie, Daniel Hsu, Vishal Misra
Statistical Methodology
A non-parametric approach for estimating the correlation between log-rank test statistics with applications to a conjunctive power calculation
Anne Lyngholm Soerensen, Paul Blanche, Henrik Ravn et al. (+1 more)
We present a method for estimating the correlation between log-rank test statistics evaluating separate null hypotheses for two time-to-event endpoints. The correlation is estimated using subject-level data by a non-parametric approach based on the independent and identically distributed (iid) decomposition of the log-rank test statistic under any alternative. Using the iid decomposition, we are able to make an assumption-lean estimation of the correlation. A motivating example using the developed approach is provided. Here, we illustrate how the suggested approach can be used to give a realistic quantification of expected conjunctive power that can guide the design of a new randomized clinical trial using historical data. Finally, we investigate the method's finite sample properties via a simulation study that confirms unbiased and consistent behavior of the proposed approach. In addition, the simulation study gives insight into the effects of censoring on the correlation between the log-rank test statistics.
Read the full abstract ▾
All authors Anne Lyngholm Soerensen, Paul Blanche, Henrik Ravn, Christian Pipper
Statistical Methodology
On the bias of the Hoover index estimator: Results for the gamma distribution
Roberto Vila and Helton Saulo
The Hoover index is a widely used measure of inequality with an intuitive interpretation, yet little is known about the finite-sample properties of its empirical
estimator. In this paper, we derive a simple expression for the expected value of the Hoover index
estimator for general non-negative populations, based on Laplace transform techniques and exponential tilting. This unified framework applies to both continuous and discrete distributions. Explicit bias expressions are obtained for gamma population, showing that the
estimator is generally biased in finite samples. Numerical and simulation results illustrate the magnitude of the bias and its dependence on the underlying distribution and sample size.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Multi-Distribution Robust Conformal Prediction
Yuqi Yang and Ying Jin
In many
fairness and distribution robustness problems, one has access to labeled data from multiple source distributions yet the test data may come from an arbitrary member or a mixture of them. We study the problem of constructing a conformal prediction set that is uniformly valid across multiple, heterogeneous distributions, in the sense that no matter which distribution the test point is from, the coverage of the prediction set is guaranteed to exceed a pre-specified level. We first propose a max-p aggregation scheme that delivers finite-sample, multi-distribution coverage given any conformity scores associated with each distribution. Upon studying several efficiency optimization programs subject to uniform coverage, we prove the optimality and tightness of our aggregation scheme, and propose a general
algorithm to learn conformity scores that lead to efficient prediction sets after the aggregation under standard conditions. We discuss how our framework relates to group-wise distributionally robust optimization, sub-population shift,
fairness, and multi-source learning. In synthetic and real-data experiments, our method delivers valid worst-case coverage across multiple distributions while greatly reducing the set size compared with naively applying max-p aggregation to single-source conformity scores, and can be comparable in size to single-source prediction sets with popular, standard conformity scores.
Read the full abstract ▾
Statistical Methodology
Bayes Factor Group Sequential Designs
Samuel Pawel and Leonhard Held
The Bayes factor, the data-based updating factor from prior to posterior odds, is a principled measure of relative evidence for two competing hypotheses. It is naturally suited to sequential data analysis in settings such as clinical trials and animal experiments, where early stopping for efficacy or futility is desirable. However, designing such studies is challenging because computing design characteristics, such as the probability of obtaining conclusive evidence or the expected sample size, typically requires computationally intensive Monte Carlo simulations, as no closed-form or efficient numerical methods exist. To address this issue, we extend results from classical group sequential design theory to sequential Bayes factor designs. The key idea is to derive Bayes factor stopping regions in terms of the z-statistic and use the known distribution of the cumulative z-statistics to compute stopping probabilities through multivariate normal integration. The resulting method is fast, accurate, and simulation-free. We illustrate it with examples from clinical trials, animal experiments, and psychological studies. We also provide an open-source implementation in the bfpwr R package. Our method makes exploring sequential Bayes factor designs as straightforward as classical group sequential designs, enabling experiments to rapidly design informative and efficient experiments.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.SI
Modeling ICD-10 Morbidity and Multidimensional Poverty as a Spatial Network: Evidence from Thailand
Pratana Kukieattikool, Kittiya Ku-kiattikun, Anukool Noymai et al. (+5 more)
Health and poverty in Thailand exhibit pronounced geographic structuring, yet the extent to which they operate as interconnected regional systems remains insufficiently understood. This study analyzes ICD-10 chapter-level morbidity and multidimensional poverty as outcomes embedded in a spatial interaction network. Interpreting Thailand's 76 provinces as nodes within a fixed-degree regional graph, we apply tools from spatial econometrics and social network analysis, including Moran's I, Local Indicators of Spatial Association (LISA), and Spatial Durbin Models (SDM), to assess spatial dependence and cross-provincial spillovers. Our findings reveal strong spatial clustering across multiple ICD-10 chapters, with persistent high-high morbidity zones, particularly for digestive, respiratory, musculoskeletal, and symptom-based diseases, emerging in well-defined regional belts. SDM estimates demonstrate that spillover effects from neighboring provinces frequently exceed the influence of local deprivation, especially for living-condition, health-access, accessibility, and poor-household indicators. These patterns are consistent with contagion and contextual influence processes well established in social network theory. By framing morbidity and poverty as interdependent attributes on a spatial network, this study contributes to the growing literature on structural
diffusion, health inequality, and regional vulnerability. The results highlight the importance of coordinated policy interventions across provincial boundaries and demonstrate how network-based modeling can uncover the spatial dynamics of health and deprivation.
Read the full abstract ▾
All authors Pratana Kukieattikool, Kittiya Ku-kiattikun, Anukool Noymai, Navaporn Surasvadi, Jantakarn Makma, Pubodin Pornratchpum, Watcharakon Noothong, Chainarong Amornbunchornvej
Statistics Theory · Cross-listed from math.PR
Yueqi Cao
Motivated by Varadhan's
theorem, we study Varadhan functions and variances on compact Riemannian
manifolds together with their minimizers, the
diffusion means introduced in Eltzner et al. (2023). Given independent and identically distributed samples, we prove uniform laws of large numbers and uniform central limit
theorems for their empirical versions. Through the small time asymptotics of gradients and Hessians of Varadhan functions, we build a strong connection to the central limit
theorem for Fréchet means on compact Riemannian
manifolds by recovering the “non-standard” correction term from Hotz et al. (2024).
Read the full abstract ▾
Statistics Theory
Evidence thresholds of collapsed structured block models
Marios Papamichalis and Regina Ruane
A
stochastic block model explains a network by splitting its nodes into communities and a
matrix of tie probabilities between them. Collapsing those probabilities out scores each split by one number, its marginal likelihood, and every decision the model takes, how many communities, whether a weak one survives, what generates the ties inside it, is a comparison of two such numbers. The comparisons fail in known ways: weak structure is dropped, small communities are merged, a group of hubs is returned in place of the real groups. Each failure has a threshold, and the thresholds have not been computed. We compute them. We prove that the best single partition keeps a planted structure only above the first-moment bound, which sits above the Kesten-Stigum threshold for up to ten communities, while the evidence summed over all partitions switches at Kesten-Stigum itself: between the two, the posterior favours structure that every partition a sampler can report rejects. We prove three further thresholds, one for merging small communities, one for the density at which the representation alone decides, one for the degree prior that cuts a community along its degrees. The empirical finding confirms the theory: on planted partitions and on thirteen labelled networks, from the karate club to citation and co-purchase networks of up to 19,717 nodes, each threshold falls where it is predicted, against likelihood, spectral, penalised and nonparametric alternatives.
Read the full abstract ▾