Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Wednesday 7 January

Statistical Methodology

Multi-transport Distributional Regression

We study distribution-on-distribution regression problems in which a response distribution depends on multiple distributional predictors. Such settings arise naturally in applications where the outcome distribution is driven by several heterogeneous distributional sources, yet remain challenging due to the nonlinear geometry of the Wasserstein space. We propose an intrinsic regression framework that aggregates predictor-specific transported distributions through a weighted Fréchet mean in the Wasserstein space. The resulting model admits multiple distributional predictors, assigns interpretable weights quantifying their relative contributions, and defines a flexible regression operator that is invariant to auxiliary construction choices, such as the selection of a reference distribution. From a theoretical perspective, we establish identifiability of the induced regression operator and derive asymptotic guarantees for its estimation under a predictive Wasserstein semi-norm, which directly characterizes convergence of the composite prediction map. Extensive simulation studies and a real data application demonstrate the improved predictive performance and interpretability of the proposed approach compared with existing Wasserstein regression methods.
Read the full abstract

All authors Yuanying Chen, Tongyu Li, Yang Bai, Zhenhua Lin

Statistical Methodology

Small area estimation of dependent extreme value indices

In extreme value analysis, tail behavior of a heavy-tailed data distribution is modeled by a Pareto-type distribution in which the so-called extreme value index (EVI) controls the tail behavior. For heavy-tailed data obtained from multiple population subgroups, or areas, this study efficiently predicts the EVIs of all areas using information among areas. For this purpose, we propose a mixed effects model, which is a useful approach in small area estimation. In this model, we represent differences among areas in the EVIs by latent variables called random effects. Using correlated random effects across areas, we incorporate the relations among areas into the model. The obtained model achieves simultaneous prediction of EVIs of all areas. Herein, we describe parameter estimation and random effect prediction in the model, and clarify theoretical properties of the estimator. Additionally, numerical experiments are presented to demonstrate the effectiveness of the proposed method. As an application of our model, we provide a risk assessment of heavy rainfall in Japan.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Local Intrinsic Dimensionality of Ground Motion Data for Early Detection of Catastrophic Slope Failure

Local Intrinsic Dimensionality (LID) has shown strong potential for anomaly detection in high-dimensional data, including landslide failure detection in granular media, where early and accurate identification of failure zones is crucial for effective geohazard mitigation. However, this task is still challenging due to the spatial correlations and temporal dynamics that are inherently present in surface displacement data. To address this gap, we propose a novel unsupervised framework called spatiotemporal LID (st-LID) that generalizes the LID for robust failure detection in landslide monitoring networks. Our approach introduces three key innovations: (1) Kinematic enhancement, incorporating velocity into the LID computation to capture instantaneous deformation rates and short-term temporal dynamics; (2) Bayesian spatial fusion, which aggregates LID values across spatial neighborhoods via Bayesian estimation, to embed spatial correlations and account for localized noise; and (3) Temporal modeling (t-LID), a new variant that characterizes long-term dynamics of displacement data, providing a robust temporal representation of displacement behavior. By unifying these components, st-LID identifies complex, multi-stage failure zones often overlooked by existing methods. Extensive experiments show that st-LID consistently outperforms state-of-the-art unsupervised baselines in detection precision and lead-time, providing a robust foundation for landslide early warning systems and targeted risk intervention to enhance community resilience and preparedness strategies.
Read the full abstract

Statistical Methodology · Cross-listed from econ.GN

Governance of Technological Transition: A Predator-Prey Analysis of AI Capital in China's Economy and Its Policy Implications

The rapid integration of Artificial Intelligence (AI) into China's economy presents a classic governance challenge: how to harness its growth potential while managing its disruptive effects on traditional capital and labor markets. This study addresses this policy dilemma by modeling the dynamic interactions between AI capital, physical capital, and labor within a Lotka-Volterra predator-prey framework. Using annual Chinese data (2016-2023), we quantify the interaction strengths, identify stable equilibria, and perform a global sensitivity analysis. Our results reveal a consistent pattern where AI capital acts as the 'prey', stimulating both physical capital accumulation and labor compensation (wage bill), while facing only weak constraining feedback. The equilibrium points are stable nodes, indicating a policy-mediated convergence path rather than volatile cycles. Critically, the sensitivity analysis shows that the labor market equilibrium is overwhelmingly driven by AI-related parameters, whereas the physical capital equilibrium is also influenced by its own saturation dynamics. These findings provide a systemic, quantitative basis for policymakers: (1) to calibrate AI promotion policies by recognizing the asymmetric leverage points in capital vs. labor markets; (2) to anticipate and mitigate structural rigidities that may arise from current regulatory settings; and (3) to prioritize interventions that foster complementary growth between AI and traditional economic structures while ensuring broad-base distribution of technological gains.
Read the full abstract

Machine Learning (Statistics)

Online Learning with Limited Information in the Sliding Window Model

Motivated by recent work on the experts problem in the streaming model, we consider the experts problem in the sliding window model. The sliding window model is a well-studied model that captures applications such as traffic monitoring, epidemic tracking, and automated trading, where recent information is more valuable than older data. Formally, we have $n$ experts, $T$ days, the ability to query the predictions of $q$ experts on each day, a limited amount of memory, and should achieve the (near-)optimal regret $\sqrt{nW}\text{polylog}(nT)$ regret over any window of the last $W$ days. While it is impossible to achieve such regret with $1$ query, we show that with $2$ queries we can achieve such regret and with only $\text{polylog}(nT)$ bits of memory. Not only are our algorithms optimal for sliding windows, but we also show for every interval $\mathcal{I}$ of days that we achieve $\sqrt{n|\mathcal{I}|}\text{polylog}(nT)$ regret with $2$ queries and only $\text{polylog}(nT)$ bits of memory, providing an exponential improvement on the memory of previous interval regret algorithms. Building upon these techniques, we address the bandit problem in data streams, where $q=1$, achieving $n T^{2/3}\text{polylog}(T)$ regret with $\text{polylog}(nT)$ memory, which is the first sublinear regret in the streaming model in the bandit setting with polylogarithmic memory; this can be further improved to the optimal $\mathcal{O}(\sqrt{nT})$ regret if the best expert's losses are in a random order.
Read the full abstract

All authors Vladimir Braverman, Sumegha Garg, Chen Wang, David P. Woodruff, Samson Zhou

Statistical Methodology

Propagating Surrogate Uncertainty in Bayesian Inverse Problems

Standard Bayesian inference schemes are infeasible for inverse problems with computationally expensive forward models. A common solution is to replace the model with a cheaper surrogate. To avoid overconfident conclusions, it is essential to acknowledge the surrogate approximation by propagating its uncertainty. At present, a variety of distinct uncertainty propagation methods have been suggested, with little understanding of how they vary. To fill this gap, we propose a mixture distribution termed the expected posterior (EP) as a general baseline for uncertainty-aware posterior approximation, justified by decision theoretic and modular Bayesian inference arguments. We then investigate the expected unnormalized posterior (EUP), a popular heuristic alternative, analyzing when it may deviate from the EP baseline. Our results show that this heuristic can break down when the surrogate uncertainty is highly non-uniform over the design space, as can be the case when the log-likelihood is emulated by a Gaussian process. Finally, we present the random kernel preconditioned Crank-Nicolson (RKpCN) algorithm, an approximate Markov chain Monte Carlo scheme that provides practical EP approximation in the challenging setting involving infinite-dimensional Gaussian process surrogates.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Universal concentration for sums under arbitrary dependence

We present a universal concentration bound for sums of random variables under arbitrary dependence, and we prove that it is asymptotically optimal for broad families of marginals admitting a uniform integrable tail-quantile envelope. The bound follows directly from the subadditivity of expected shortfall, a property well known in the risk-measure literature. Our sharpness result relies on an explicit construction of asymptotically extremal couplings. We furthermore provide practical sufficient conditions – based on convex transformation order comparisons with exponential and power-law envelopes – under which the bound admits simple, explicit tail profiles.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from q-bio.NC

A Quantifiable Information-Processing Hierarchy Provides a Necessary Condition for Detecting Agency

As intelligent systems are developed across diverse substrates - from machine learning models and neuromorphic hardware to in vitro neural cultures - understanding what gives a system agency has become increasingly important. Existing definitions, however, tend to rely on top-down descriptions that are difficult to quantify. We propose a bottom-up framework grounded in a system's information-processing order: the extent to which its transformation of input evolves over time. We identify three orders of information processing. Class I systems are reactive and memoryless, mapping inputs directly to outputs. Class II systems incorporate internal states that provide memory but follow fixed transformation rules. Class III systems are adaptive; their transformation rules themselves change as a function of prior activity. While not sufficient on their own, these dynamics represent necessary informational conditions for genuine agency. This hierarchy offers a measurable, substrate-independent way to identify the informational precursors of agency. We illustrate the framework with neurophysiological and computational examples, including thermostats and receptor-like memristors, and discuss its implications for the ethical and functional evaluation of systems that may exhibit agency.
Read the full abstract

All authors Brett J. Kagan, Valentina Baccetti, Brian D. Earp, J. Lomax Boyd, Julian Savulescu, Adeel Razi

Statistical Methodology

Differentially Private Bayesian Inference for Gaussian Copula Correlations

Gaussian copulas are widely used to estimate multivariate distributions and relationships. We present algorithms for estimating Gaussian copula correlations that ensure differential privacy. We first convert data values into sets of two-way tables of counts above and below marginal medians. We then add noise to these counts to satisfy differential privacy. We utilize the one-to-one correspondence between the true counts and the copula correlation to estimate a posterior distribution of the copula correlation given the noisy counts, marginalizing over the distribution of the underlying true counts using a composite likelihood. We also present an alternative, maximum likelihood approach for point estimation. Using simulation studies, we compare these methods to extant methods in the literature for computing differentially private copula correlations.
Read the full abstract

Statistics Applications · Cross-listed from cs.AI

Personalization of Large Foundation Models for Health Interventions

Large foundation models (LFMs) transform healthcare AI in prevention, diagnostics, and treatment. However, whether LFMs can provide truly personalized treatment recommendations remains an open question. Recent research has revealed multiple challenges for personalization, including the fundamental generalizability paradox: models achieving high accuracy in one clinical study perform at chance level in others, demonstrating that personalization and external validity exist in tension. This exemplifies broader contradictions in AI-driven healthcare: the privacy-performance paradox, scale-specificity paradox, and the automation-empathy paradox. As another challenge, the degree of causal understanding required for personalized recommendations, as opposed to mere predictive capacities of LFMs, remains an open question. N-of-1 trials – crossover self-experiments and the gold standard for individual causal inference in personalized medicine – resolve these tensions by providing within-person causal evidence while preserving privacy through local experimentation. Despite their impressive capabilities, this paper argues that LFMs cannot replace N-of-1 trials. We argue that LFMs and N-of-1 trials are complementary: LFMs excel at rapid hypothesis generation from population patterns using multimodal data, while N-of-1 trials excel at causal validation for a given individual. We propose a hybrid framework that combines the strengths of both to enable personalization and navigate the identified paradoxes: LFMs generate ranked intervention candidates with uncertainty estimates, which trigger subsequent N-of-1 trials. Clarifying the boundary between prediction and causation and explicitly addressing the paradoxical tensions are essential for responsible AI integration in personalized medicine.
Read the full abstract

All authors Stefan Konigorski, Johannes E. Vedder, Babajide Alamu Owoyele, İbrahim Özkan

Statistical Methodology

Improving operating characteristics of clinical trials by augmenting control arm using propensity score-weighted borrowing-by-parts power prior

Borrowing external data can improve estimation efficiency but may introduce bias when populations differ in covariate distributions or outcome variability. A proper balance needs to be maintained between the two datasets to justify the borrowing. We propose a propensity score weighting borrowing-by-parts power prior (PSW-BPP) that integrates causal covariate adjustment through propensity score weighting with a flexible Bayesian borrowing approach to address these challenges in a unified framework. The proposed approach first applies propensity score weighting to align the covariate distribution of the external data with that of the current study, thereby targeting a common estimand and reducing confounding due to population heterogeneity. The weighted external likelihood is then incorporated into a Bayesian model through a borrowing-by-parts power prior, which allows distinct power parameters for the mean and variance components of the likelihood, enabling differential and calibrated information borrowing. Additionally, we adopt the idea of the minimal plausibility index (mPI) to calculate the power parameters. This separate borrowing provides greater robustness to prior-data conflict compared with traditional power prior methods that impose a single borrowing parameter. We study the operating characteristics of PSW-BPP through extensive simulation and a real data example. Simulation studies demonstrate that PSW-BPP yields more efficient and stable estimation than no borrowing and fixed borrowing, particularly under moderate covariate imbalance and outcome heterogeneity. The proposed framework offers a principled and extensible methodological contribution for Bayesian inference with external data in observational and hybrid study designs.
Read the full abstract

All authors Apu Chandra Das, Sakib Salam, Aninda Roy, Rakhi Chowdhury, Antar Chandra Das, Ashim Chandra Das

Tuesday 6 January

Statistical Methodology

Measures of classification bias derived from sample size analysis

We propose the use of a simple intuitive principle for measuring algorithmic classification bias: the significance of the differences in a classifier's error rates across the various demographics is inversely commensurate with the sample size required to statistically detect them. That is, if large sample sizes are required to statistically establish biased behavior, the algorithm is less biased, and vice versa. In a simple setting, we assume two distinct demographics, and non-parametric estimates of the error rates on them, e1 and e2, respectively. We use a well-known approximate formula for the sample size of the chi-squared test, and verify some basic desirable properties of the proposed measure. Next, we compare the proposed measure with two other commonly used statistics, the difference e2-e1 and the ratio e2/e1 of the error rates. We establish that the proposed measure is essentially different in that it can rank algorithms for bias differently, and we discuss some of its advantages over the other two measures. Finally, we briefly discuss how some of the desirable properties of the proposed measure emanate from fundamental characteristics of the method, rather than the approximate sample size formula we used, and thus, are expected to hold in more complex settings with more than two demographics.
Read the full abstract

All authors Ioannis Ivrissimtzis, Shauna Concannon, Matthew Houliston, Graham Roberts

Machine Learning (Statistics)

Microeconomic Foundations of Multi-Agent Learning

Modern AI systems increasingly operate inside markets and institutions where data, behavior, and incentives are endogenous. This paper develops an economic foundation for multi-agent learning by studying a principal-agent interaction in a Markov decision process with strategic externalities, where both the principal and the agent learn over time. We propose a two-phase incentive mechanism that first estimates implementable transfers and then uses them to steer long-run dynamics; under mild regret-based rationality and exploration conditions, the mechanism achieves sublinear social-welfare regret and thus asymptotically optimal welfare. Simulations illustrate how even coarse incentives can correct inefficient learning under stateful externalities, highlighting the necessity of incentive-aware design for safe and welfare-aligned AI in markets and insurance.
Read the full abstract

Statistical Methodology

On estimands in target trial emulation

The target trial framework enables causal inference from longitudinal observational data by emulating randomized trials initiated at multiple time points. Precision is often improved by pooling information across trials, with standard models typically assuming - among other things - a time-constant treatment effect. However, this obscures interpretation when the true treatment effect varies, which we argue to be likely as a result of relying on noncollapsible estimands. To address these challenges, this paper introduces a model-free strategy for target trial analysis, centered around the choice of the estimand, rather than model specification. This ensures that treatment effects remain clearly interpretable for well-defined populations even under model misspecification. We propose estimands suitable for different study designs, and develop accompanying G-computation and inverse probability weighted estimators. Applications on simulations and real data on antimicrobial de-escalation in an intensive care unit setting demonstrate the greater clarity and reliability of the proposed methodology over traditional techniques.
Read the full abstract

All authors Edoardo Efrem Gervasoni, Liesbet De Bus, Stijn Vansteelandt, Oliver Dukes

Machine Learning (Statistics) · Cross-listed from cs.CL

A path to natural language through tokenisation and transformers

Natural languages exhibit striking regularities in their statistical structure, including notably the emergence of Zipf's and Heaps' laws. Despite this, it remains broadly unclear how these properties relate to the modern tokenisation schemes used in contemporary transformer models. In this note, we analyse the information content (as measured by the Shannon entropy) of various corpora under the assumption of a Zipfian frequency distribution, and derive a closed-form expression for the slot entropy expectation value. We then empirically investigate how byte–pair encoding (BPE) transforms corpus statistics, showing that recursive applications of BPE drive token frequencies toward a Zipfian power law while inducing a characteristic growth pattern in empirical entropy. Utilizing the ability of transformers to learn context dependent token probability distributions, we train language models on corpora tokenised at varying BPE depths, revealing that the model predictive entropies increasingly agree with Zipf-derived predictions as the BPE depth increases. Attention-based diagnostics further indicate that deeper tokenisation reduces local token dependencies, bringing the empirical distribution closer to the weakly dependent (near IID) regime. Together, these results clarify how BPE acts not only as a compression mechanism but also as a statistical transform that reconstructs key informational properties of natural language.
Read the full abstract

Machine Learning (Statistics)

On the Identifiability of Regime-Switching Models with Multi-Lag Dependencies

Identifiability is central to the interpretability of deep latent variable models, ensuring parameterisations are uniquely determined by the data-generating distribution. However, it remains underexplored for deep regime-switching time series. We develop a general theoretical framework for multi-lag Regime-Switching Models (RSMs), encompassing Markov Switching Models (MSMs) and Switching Dynamical Systems (SDSs). For MSMs, we formulate the model as a temporally structured finite mixture and prove identifiability of both the number of regimes and the multi-lag transitions in a nonlinear-Gaussian setting. For SDSs, we establish identifiability of the latent variables up to permutation and scaling via temporal structure, which in turn yields conditions for identifiability of regime-dependent latent causal graphs (up to regime/node permutations). Our results hold in a fully unsupervised setting through architectural and noise assumptions that are directly enforceable via neural network design. We complement the theory with a flexible variational estimator that satisfies the assumptions and validate the results on synthetic benchmarks. Across real-world datasets from neuroscience, finance, and climate, identifiability leads to more trustworthy interpretability analysis, which is crucial for scientific discovery.
Read the full abstract

All authors Carles Balsells-Rodas, Toshiko Matsui, Pedro A.M. Mediano, Yixin Wang, Yingzhen Li

Statistical Methodology

Progressive Bayesian Confidence Architectures for Cold-Start Personal Health Analytics: Formalizing Early Insight Through Posterior Contraction and Risk-Aware Interpretation

Personal health analytics systems face a persistent cold-start dilemma: users expect meaningful insights early in data collection, while conventional statistical inference requires data volumes that often exceed engagement horizons. Existing approaches either delay inference until fixed statistical thresholds are met – leading to user disengagement – or surface heuristic insights without formal uncertainty quantification, risking false confidence. We propose a progressive Bayesian confidence architecture that formalizes early-stage inference through phased interpretation of posterior uncertainty. Drawing on Bayesian updating and epistemic strategies from financial risk modeling under sparse observations, we map posterior contraction to interpretable tiers of insight, ranging from exploratory directional evidence to robust associative inference. We demonstrate the framework's performance through controlled experimentation with synthetic N-of-1 health data, showing that calibrated early insights can be generated within 5–7 days while maintaining explicit epistemic humility. Compared to fixed-threshold baselines requiring 30+ days of data, the proposed approach yields earlier directional signals (mean: 5.3 vs 31.7 days, p<0.001) while controlling false discovery rates below 6% (5.9% at day 30) despite 26-day earlier detection, compared to 0% FDR for fixed-threshold baselines that delay insights by 30 days. In addition, we show strong uncertainty calibration (76% credible interval coverage for ground-truth correlations at day 90). This work contributes a methodological framework for uncertainty-aware early inference in personalized health analytics that bridges the gap between user engagement requirements and statistical rigor.
Read the full abstract

Machine Learning (Statistics)

Self-Supervised Learning from Noisy and Incomplete Data

Many important problems in science and engineering involve inferring a signal from noisy and/or incomplete observations, where the observation process is known. Historically, this problem has been tackled using hand-crafted regularization (e.g., sparsity, total-variation) to obtain meaningful estimates. Recent data-driven methods often offer better solutions by directly learning a solver from examples of ground-truth signals and associated observations. However, in many real-world applications, obtaining ground-truth references for training is expensive or impossible. Self-supervised learning methods offer a promising alternative by learning a solver from measurement data alone, bypassing the need for ground-truth references. This manuscript provides a comprehensive summary of different self-supervised methods for inverse problems, with a special emphasis on their theoretical underpinnings, and presents practical applications in imaging inverse problems.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

PET-TURTLE: Deep Unsupervised Support Vector Machines for Imbalanced Data Clusters

Foundation vision, audio, and language models enable zero-shot performance on downstream tasks via their latent representations. Recently, unsupervised learning of data group structure with deep learning methods has gained popularity. TURTLE, a state of the art deep clustering algorithm, uncovers data labeling without supervision by alternating label and hyperplane updates, maximizing the hyperplane margin, in a similar fashion to support vector machines (SVMs). However, TURTLE assumes clusters are balanced; when data is imbalanced, it yields non-ideal hyperplanes that cause higher clustering error. We propose PET-TURTLE, which generalizes the cost function to handle imbalanced data distributions by a power law prior. Additionally, by introducing sparse logits in the labeling process, PET-TURTLE optimizes a simpler search space that in turn improves accuracy for balanced datasets. Experiments on synthetic and real data show that PET-TURTLE improves accuracy for imbalanced sources, prevents over-prediction of minority clusters, and enhances overall clustering.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from quant-ph

Shallow-circuit Supervised Learning on a Quantum Processor

Quantum computing has long promised transformative advances in data analysis, yet practical quantum machine learning has remained elusive due to fundamental obstacles such as a steep quantum cost for the loading of classical data and poor trainability of many quantum machine learning algorithms designed for near-term quantum hardware. In this work, we show that one can overcome these obstacles by using a linear Hamiltonian-based machine learning method which provides a compact quantum representation of classical data via ground state problems for k-local Hamiltonians. We use the recent sample-based Krylov quantum diagonalization method to compute low-energy states of the data Hamiltonians, whose parameters are trained to express classical datasets through local gradients. We demonstrate the efficacy and scalability of the methods by performing experiments on benchmark datasets using up to 50 qubits of an IBM Heron quantum processor.
Read the full abstract

All authors Luca Candelori, Swarnadeep Majumder, Antonio Mezzacapo, Javier Robledo Moreno, Kharen Musaelian, Santhanam Nagarajan, Sunil Pinnamaneni, Kunal Sharma, Dario Villani

Machine Learning (Statistics) · Cross-listed from cs.LG

From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence

Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated without considering a downstream task? On these questions, Shannon information and Kolmogorov complexity come up nearly empty-handed, in part because they assume observers with unlimited computational capacity and do not target the useful information content. In this work, we identify and exemplify three seeming paradoxes in information theory: (1) information cannot be increased by deterministic transformations; (2) information is independent of the order of data; (3) likelihood modeling is merely distribution matching. To shed light on the tension between these results and modern practice, and to quantify the value of data, we introduce epiplexity, a formalization of information capturing what computationally bounded observers can learn from data. Epiplexity captures the structural content in data while excluding time-bounded entropy, the random unpredictable content exemplified by pseudorandom number generators and chaotic dynamical systems. With these concepts, we demonstrate how information can be created with computation, how it depends on the ordering of the data, and how likelihood modeling can produce more complex programs than present in the data generating process itself. We also present practical procedures to estimate epiplexity which we show capture differences across data sources, track with downstream performance, and highlight dataset interventions that improve out-of-distribution generalization. In contrast to principles of model selection, epiplexity provides a theoretical foundation for data selection, guiding how to select, generate, or transform data for learning systems.
Read the full abstract

All authors Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, Andrew Gordon Wilson

Statistics Applications

Computationally Efficient Estimation of Localized Treatment Effects for Multi-Level, Multi-Component Interventions to Address the Opioid Crisis

The opioid epidemic remains a major public health challenge in the United States, requiring a multi-pronged intervention approach to mitigate harms to communities. Given the heterogeneity of the epidemic, it is crucial for policymakers to understand localized treatment effects of different intervention components and utilize limited resources efficiently. While locally calibrated simulation models can project epidemic outcomes for any given intervention policy, collecting simulation results for all intervention combinations to estimate localized treatment effects for each community is impractical because the number of combinations grows exponentially with the number of interventions and the levels at which they are applied. To tackle this, we develop a two-stage metamodel framework with a two-step sequential design for efficient sampling. The metamodel consists of a response function linking health outcomes to each intervention component's treatment effect, and a Gaussian process regression (GPR) to learn spatial and socio-economic structures of the treatment effects based on locally-contextualized covariates. With two-step sequential sampling, we leverage spatial correlations and posterior uncertainty to sequentially sample the most informative counties and treatment conditions. We apply this framework to estimate the treatment effects of buprenorphine dispensing and naloxone distribution on overdose mortality rates using a calibrated agent-based opioid epidemic model in Pennsylvania counties. Our approach achieves less than 5% average relative error using fewer than 2% of the runs required for an exhaustive simulation. Our two-stage framework provides a computationally efficient approach to support policymakers, enabling an efficient evaluation of alternative resource-allocation strategies to mitigate the opioid epidemic in local communities.
Read the full abstract

All authors Abdulrahman A. Ahmed, M. Amin Rahimian, Qiushi Chen, Praveen Kumar

Machine Learning (Statistics) · Cross-listed from cs.LG

Time-Aware Synthetic Control

The synthetic control (SC) framework is widely used for observational causal inference with time-series panel data. SC has been successful in diverse applications, but existing methods typically treat the ordering of pre-intervention time indices interchangeable. This invariance means they may not fully take advantage of temporal structure when strong trends are present. We propose Time-Aware Synthetic Control (TASC), which employs a state-space model with a constant trend while preserving a low-rank structure of the signal. TASC uses the Kalman filter and Rauch-Tung-Striebel smoother: it first fits a generative time-series model with expectation-maximization and then performs counterfactual inference. We evaluate TASC on both simulated and real-world datasets, including policy evaluation and sports prediction. Our results suggest that TASC offers advantages in settings with strong temporal trends and high levels of observation noise.
Read the full abstract

All authors Saeyoung Rho, Cyrus Illick, Samhitha Narasipura, Alberto Abadie, Daniel Hsu, Vishal Misra

Statistical Methodology

A non-parametric approach for estimating the correlation between log-rank test statistics with applications to a conjunctive power calculation

We present a method for estimating the correlation between log-rank test statistics evaluating separate null hypotheses for two time-to-event endpoints. The correlation is estimated using subject-level data by a non-parametric approach based on the independent and identically distributed (iid) decomposition of the log-rank test statistic under any alternative. Using the iid decomposition, we are able to make an assumption-lean estimation of the correlation. A motivating example using the developed approach is provided. Here, we illustrate how the suggested approach can be used to give a realistic quantification of expected conjunctive power that can guide the design of a new randomized clinical trial using historical data. Finally, we investigate the method's finite sample properties via a simulation study that confirms unbiased and consistent behavior of the proposed approach. In addition, the simulation study gives insight into the effects of censoring on the correlation between the log-rank test statistics.
Read the full abstract

All authors Anne Lyngholm Soerensen, Paul Blanche, Henrik Ravn, Christian Pipper

Statistical Methodology

On the bias of the Hoover index estimator: Results for the gamma distribution

The Hoover index is a widely used measure of inequality with an intuitive interpretation, yet little is known about the finite-sample properties of its empirical estimator. In this paper, we derive a simple expression for the expected value of the Hoover index estimator for general non-negative populations, based on Laplace transform techniques and exponential tilting. This unified framework applies to both continuous and discrete distributions. Explicit bias expressions are obtained for gamma population, showing that the estimator is generally biased in finite samples. Numerical and simulation results illustrate the magnitude of the bias and its dependence on the underlying distribution and sample size.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Multi-Distribution Robust Conformal Prediction

In many fairness and distribution robustness problems, one has access to labeled data from multiple source distributions yet the test data may come from an arbitrary member or a mixture of them. We study the problem of constructing a conformal prediction set that is uniformly valid across multiple, heterogeneous distributions, in the sense that no matter which distribution the test point is from, the coverage of the prediction set is guaranteed to exceed a pre-specified level. We first propose a max-p aggregation scheme that delivers finite-sample, multi-distribution coverage given any conformity scores associated with each distribution. Upon studying several efficiency optimization programs subject to uniform coverage, we prove the optimality and tightness of our aggregation scheme, and propose a general algorithm to learn conformity scores that lead to efficient prediction sets after the aggregation under standard conditions. We discuss how our framework relates to group-wise distributionally robust optimization, sub-population shift, fairness, and multi-source learning. In synthetic and real-data experiments, our method delivers valid worst-case coverage across multiple distributions while greatly reducing the set size compared with naively applying max-p aggregation to single-source conformity scores, and can be comparable in size to single-source prediction sets with popular, standard conformity scores.
Read the full abstract

Statistical Methodology

Bayes Factor Group Sequential Designs

The Bayes factor, the data-based updating factor from prior to posterior odds, is a principled measure of relative evidence for two competing hypotheses. It is naturally suited to sequential data analysis in settings such as clinical trials and animal experiments, where early stopping for efficacy or futility is desirable. However, designing such studies is challenging because computing design characteristics, such as the probability of obtaining conclusive evidence or the expected sample size, typically requires computationally intensive Monte Carlo simulations, as no closed-form or efficient numerical methods exist. To address this issue, we extend results from classical group sequential design theory to sequential Bayes factor designs. The key idea is to derive Bayes factor stopping regions in terms of the z-statistic and use the known distribution of the cumulative z-statistics to compute stopping probabilities through multivariate normal integration. The resulting method is fast, accurate, and simulation-free. We illustrate it with examples from clinical trials, animal experiments, and psychological studies. We also provide an open-source implementation in the bfpwr R package. Our method makes exploring sequential Bayes factor designs as straightforward as classical group sequential designs, enabling experiments to rapidly design informative and efficient experiments.
Read the full abstract

Statistics Applications · Cross-listed from cs.SI

Modeling ICD-10 Morbidity and Multidimensional Poverty as a Spatial Network: Evidence from Thailand

Health and poverty in Thailand exhibit pronounced geographic structuring, yet the extent to which they operate as interconnected regional systems remains insufficiently understood. This study analyzes ICD-10 chapter-level morbidity and multidimensional poverty as outcomes embedded in a spatial interaction network. Interpreting Thailand's 76 provinces as nodes within a fixed-degree regional graph, we apply tools from spatial econometrics and social network analysis, including Moran's I, Local Indicators of Spatial Association (LISA), and Spatial Durbin Models (SDM), to assess spatial dependence and cross-provincial spillovers. Our findings reveal strong spatial clustering across multiple ICD-10 chapters, with persistent high-high morbidity zones, particularly for digestive, respiratory, musculoskeletal, and symptom-based diseases, emerging in well-defined regional belts. SDM estimates demonstrate that spillover effects from neighboring provinces frequently exceed the influence of local deprivation, especially for living-condition, health-access, accessibility, and poor-household indicators. These patterns are consistent with contagion and contextual influence processes well established in social network theory. By framing morbidity and poverty as interdependent attributes on a spatial network, this study contributes to the growing literature on structural diffusion, health inequality, and regional vulnerability. The results highlight the importance of coordinated policy interventions across provincial boundaries and demonstrate how network-based modeling can uncover the spatial dynamics of health and deprivation.
Read the full abstract

All authors Pratana Kukieattikool, Kittiya Ku-kiattikun, Anukool Noymai, Navaporn Surasvadi, Jantakarn Makma, Pubodin Pornratchpum, Watcharakon Noothong, Chainarong Amornbunchornvej

Statistics Theory · Cross-listed from math.PR

Uniform Limit Theorems for Diffusion Means on Compact Riemannian Manifolds

Motivated by Varadhan's theorem, we study Varadhan functions and variances on compact Riemannian manifolds together with their minimizers, the diffusion means introduced in Eltzner et al. (2023). Given independent and identically distributed samples, we prove uniform laws of large numbers and uniform central limit theorems for their empirical versions. Through the small time asymptotics of gradients and Hessians of Varadhan functions, we build a strong connection to the central limit theorem for Fréchet means on compact Riemannian manifolds by recovering the “non-standard” correction term from Hotz et al. (2024).
Read the full abstract

Statistics Theory

Evidence thresholds of collapsed structured block models

A stochastic block model explains a network by splitting its nodes into communities and a matrix of tie probabilities between them. Collapsing those probabilities out scores each split by one number, its marginal likelihood, and every decision the model takes, how many communities, whether a weak one survives, what generates the ties inside it, is a comparison of two such numbers. The comparisons fail in known ways: weak structure is dropped, small communities are merged, a group of hubs is returned in place of the real groups. Each failure has a threshold, and the thresholds have not been computed. We compute them. We prove that the best single partition keeps a planted structure only above the first-moment bound, which sits above the Kesten-Stigum threshold for up to ten communities, while the evidence summed over all partitions switches at Kesten-Stigum itself: between the two, the posterior favours structure that every partition a sampler can report rejects. We prove three further thresholds, one for merging small communities, one for the density at which the representation alone decides, one for the degree prior that cuts a community along its degrees. The empirical finding confirms the theory: on planted partitions and on thirteen labelled networks, from the karate club to citation and co-purchase networks of up to 19,717 nodes, each threshold falls where it is predicted, against likelihood, spectral, penalised and nonparametric alternatives.
Read the full abstract