Peer Revenue Impact factor: volatile

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 10 August

Statistical Methodology

COMPACT: Spectral Adjustment Scores from a Complete and Irreducible Causal Criterion

Observational datasets frequently contain many baseline variables, yet investigators estimating causal effects may not know which variables to include in the adjustment set. Confounding information may also be distributed weakly across many variables. Propensity scores can simplify adjustment by reducing high-dimensional covariates to a scalar with binary treatment. Although the propensity score is the coarsest balancing score, this distributional optimality does not imply maximal specificity over causal graphs. We instead examine all causal graphs among a candidate score, treatment, and outcome while allowing latent variables. Under faithfulness, we identify the largest set of unconditional and conditional dependence relations whose truth is invariant to whether treatment causes the outcome, leaving treatment-effect estimation to the downstream analysis. This criterion defines the maximally specific graph class expressible through these relations. We then develop the proposed algorithm, which operationalizes the criterion through a generalized eigenvalue problem whose score space targets the span of a balancing coordinate and an outcome-guided coordinate. We show that sufficiently informative proxies can recover this span without direct observation of the adjustment variables, characterize the resulting estimation and causal errors, and establish bootstrap validity for the complete procedure. Simulations and a real-data application demonstrate superior performance over several alternatives.
Read the full abstract

Statistical Methodology

Cohen's f or Mean Standardized Differences? Assessing Covariate Balance with Multivalued Treatments

Assessing covariate balance across more than two treatment groups has no established omnibus standard: the prevailing practice averages, or takes the maximum of, pairwise standardized mean differences (SMD), while Cohen's f - the classical generalization of Cohen's d to more than two groups - offers an alternative grounded in an established effect-size framework, but the two have not been formally compared. We extend both to arbitrary weighted and covariate-adjusted models via a community-contributed Stata command, esizereg, and validate them in a simulation study of three, four, and six treatment groups under correctly specified and misspecified generalized-propensity-score weighting, correlating each statistic against downstream treatment-effect bias. Cohen's f tracks estimation bias comparably to mean absolute SMD, both pooled (r = 0.93) and within weighting arm (r approximately 0.79); maximum absolute SMD is generally weakest. This ranking was unchanged under a deliberately adversarial nonlinear/interaction outcome model. Cohen's f and the SMD-based statistics are not numerically comparable: we derive an exact representation of f as a size-weighted quadratic function of the pairwise SMDs and prove the minimum attainable ratio between f and mean absolute SMD under equal group weights, so conventional SMD thresholds should not apply directly to f. We recommend reporting f alongside its per-level and pairwise decomposition.
Read the full abstract

Statistical Methodology

Structured dispersion modelling for mortality data using a Conway–Maxwell–Poisson specification

Mortality models that attempt to capture dispersion typically assume a fixed dispersion structure, an assumption that is rarely satisfied in practice and that can lead to miscalibrated uncertainty and poor predictive performance. In this paper, we introduce a flexible framework for explicitly modelling dispersion in mortality data using the Conway–Maxwell–Poisson (CMP) distribution, which accommodates underdispersion, equidispersion, and overdispersion within a unified specification. Rather than imposing a global dispersion parameter, the framework allows both the type and degree of dispersion to vary by age and over time, thus capturing structural heterogeneity that simpler models may overlook. A Bayesian formulation treats dispersion as unknown, with prior structures that coherently propagate parameter, process, and distributional uncertainty. Estimation is carried out via Markov chain Monte Carlo (MCMC) methods. Using empirical death data for males in England and Wales, we show that variability in mortality counts differs substantially across ages and across time periods. This has meaningful implications for the calibration of longevity risk and the pricing of annuity products.
Read the full abstract

Statistical Methodology

Joint return levels of maximum temperature and minimum relative humidity by combining copulas with an extreme value framework for bimodal data

Climate change has become a growing concern, particularly in regions experiencing increasingly frequent extreme events. In Brasília, the capital of Brazil, significant shifts in climate patterns have drawn attention, including episodes of intense heat, unusually cold weather, and prolonged dry periods, often accompanied by wildfires. These phenomena directly affect the population and local ecosystems, requiring detailed analyses to understand their causes and effects. In this work, the return period for the joint distribution of maximum air temperature and minimum relative humidity in Brasília is determined using a copula-based approach. First, the dependence structure between the variables is modeled through a copula, while the marginal distributions are fitted using the novel methodology for modelling extreme values in complex systems. Different copula families, including rotated versions, were evaluated using information criteria, leading to the selection of a model that adequately captured the dependence structure, including asymmetry and tail behavior. The selected model also reproduced the multimodal pattern observed in the joint density, a feature consistent with the possible presence of multiple climate regimes. In the context of climate data modelling, the bivariate return level was determined using a conditional approach to assess the occurrence of extreme scenarios characterized by high temperatures combined with low relative humidity. This analysis makes it possible to quantify the expected frequency of such events, providing valuable information for environmental risk monitoring, the planning of preventive measures, and the development of climate change adaptation strategies in the region.
Read the full abstract

All authors Beatriz G da Cruz Albernaz, Cira E G Otiniano, Carolyne Soares de Brito, Fidel E C Morales, Enzo Porto Brasil

Statistics Applications

Estimating Media Mix Models with Demand-Marketing Interactions: A Constrained Genetic Algorithm Approach

This paper proposes a novel extension to Media Mix Modeling (MMM) that introduces a multiplicative interaction between marketing activity and underlying consumer demand. Unlike standard MMM frameworks that assume additive and independent effects of media and baseline demand drivers, our specification allows marketing effectiveness to vary with prevailing demand conditions. However, the proposed structure introduces significant statistical challenges, particularly identifiability issues that can lead to unstable and biased parameter estimates. Using comprehensive simulation studies, we analyze the nature and severity of these identification issues through bias assessment and their implications for statistical inference. To address these issues, we develop a constrained genetic algorithm optimization approach which facilitates robust estimation that simultaneously mitigates biases arising from the aforementioned issues. The proposed approach also offers flexibility to incorporate economically meaningful parameter constraints. Finally, the proposed methodology is applied to real-world data to demonstrate its effectiveness in enhancing estimation accuracy while taking into account additional commercial constraints, facilitating budget allocation decisions.
Read the full abstract

All authors J. S. T. Wong, G. Hughes, Y. Bao, H. Dai, V. Giagos, H.M. Fernanda, H.O. Bakan, K. Passmore

Statistics Theory · Cross-listed from math.PR

The sharp SAT/UNSAT phase transition in random ellipsoid fitting

Let $x_1,\ldots,x_n$ be independent standard Gaussian vectors in $\mathbb{R}^d$. An ellipsoid fit is a matrix $S \succeq 0$ such that $x_i^\top S x_i =d$ for every $i$, so that all the points lie on the boundary of the centered ellipsoid $\{ x : x^\top S x = d\}$. Saunderson, Parrilo and Willsky conjectured that, as $n,d \to \infty$, this semidefinite feasibility problem undergoes a sharp transition at $n \sim d^2/4$. We prove this conjecture. If $\lim \sup n/d^2 = \alpha^* <1/4$, then, with probability tending to one, an ellipsoid fit exists; moreover, one can choose $S$ with all eigenvalues in a fixed interval $[\lambda_- , \lambda_+] \subset (0,\infty)$ depending only on $\alpha^*$. Conversely, if $\lim \inf n/d^2 > 1/4$, then, with probability tending to one, no ellipsoid fit exists, without any spectral restriction. Our proof builds on the Gaussian-equivalence framework developed by Bandeira and Maillard (2025) and closes the two gaps left open in their work: establishing exact fitting and removing the operator-norm constraint. On the satisfiable side, the new ingredients are a head-tail decomposition of the dual vector, exact correction of the sparse head constraints, and a Gaussian comparison principle for the low-influence tail. On the unsatisfiable side, we split a candidate into a low-rank spectral head and a Schatten-3 diffuse bulk, Gaussianize the bulk conditionally on the head, and apply a projected Gordon escape argument. The threshold is governed by the statistical dimension $d(d+1)/4$ of the positive semidefinite cone.
Read the full abstract

Statistical Methodology

Bandable Cumulant Tensors: Optimal Estimation and Applications in Non-Gaussian Data Modeling

Higher-order cumulants capture the non-Gaussian dependence that covariance misses, but they are hard to use in high dimensions. An order-$d$ cumulant tensor has $p^d$ entries, and the plug-in sample cumulant is generally not even rate-optimal under the tensor spectral norm. For ordered data, both difficulties admit one remedy: assuming that higher-order interactions decay away from the main tensor diagonal, we introduce a bandable cumulant class and a tapered sample cumulant estimator that computes only $O(pk^{d-1})$ local entries at bandwidth $k$ and never forms the full tensor. Under exponential-type tail conditions, we prove nonasymptotic spectral-norm bounds that separate tapering bias from stochastic error, and a minimax lower bound over the same class that matches the leading bias and stochastic terms of the upper bound; for sub-Gaussian observations, the tapered estimator attains the minimax rate whenever $n\gg(k+\log p)^{d-1}$ at the oracle bandwidth $k$, with the ambient dimension entering only through $\log p$. Localization also suppresses the higher-order fluctuations behind this suboptimality, so tapering plays a stronger role here than in bandable covariance estimation. The spectral-norm guarantee transfers directly to downstream tasks, yielding plug-in error bounds for cumulant Yule–Walker estimation in autoregressive models, minimum-distance estimation in moving-average models, and matched-filter source localization in sensor arrays. Simulations corroborate the theory, and real-data analyses of RR-interval, air-quality, and Neuropixels recordings illustrate the resulting stability gains.
Read the full abstract

All authors Runshi Tang, Anru R. Zhang, Yuefeng Han, Wei Biao Wu

Statistics Theory

Optimal Inference with Black-box Predictions

Powerful black-box predictive models have motivated many proposals for combining observed data with predictions to perform valid statistical inference. Despite this progress, the field lacks a unifying principle that explains how hypothesis tests should integrate data and predictions in a way that is both valid and efficient. In this work, we address this gap in the high-dimensional Gaussian sequence model. We characterize the information-theoretic limits of inference with black-box predictions when their accuracies are known and, for orthogonal predictions, when they are unknown. Building on these characterizations, we develop practical hypothesis tests that adapt to the unknown accuracies of the predictions while benefiting from strong alignment among them.
Read the full abstract

All authors Lucas Kania, Abhinav Chakraborty, Edward Kennedy, Larry Wasserman, Sivaraman Balakrishnan

Statistics Applications

Estimating Epidemic Rate Parameters: Adaptivity, Bias, and Convolution

Metrics like the case-fatality rate and reproduction number are key descriptors of epidemics from the COVID-19 pandemic to the seasonal flu. In retrospect, these quantities enrich our understanding of infectious disease outbreaks; in real-time, they are absolutely critical to informing public health response. Thus, an important question in epidemiology is how best to estimate such metrics, especially in real-time. This question is complicated by practical considerations like data availability, as well as the fact that the metrics themselves may change as the epidemic unfolds.
Read the full abstract

Statistical Methodology

Asymptotics of Nonparametric Estimation under General Non-monotone MAR Missingness: A Nonparametric Maximum Likelihood Approach

Missing data constitute a pervasive challenge in empirical research. Consequently, there is an ever-growing number of methods designed to address this challenge, with multiple imputation and inverse probability weighting the dominant strategies. Despite this, theoretical guarantees remain limited, particularly in the challenging case of non-monotone missing at random (MAR). When guarantees exist, they are often confined to simplified settings such as missing completely at random, monotone or block-wise missingness, or rest on restrictive assumptions about the missingness mechanism. In this paper, we utilize the theory of sieve maximum likelihood to establish a general rate of convergence under MAR that requires no modeling of the missingness mechanism and no restriction on the configuration of missing patterns, beyond MAR itself and a natural positivity condition. Applying this result to density estimation, we show that the complete-data density can be estimated at the minimax rate over a Hölder class, up to a logarithmic factor, for any prescribed smoothness level. The missingness does not affect the rate and enters only through a constant. The estimator is approximated in practice by a simple expectation-maximization (EM) algorithm operating on the incomplete data directly. In simulations, it performs comparably to the kernel density estimator supplied with the complete data across a wide range of missingness levels.
Read the full abstract

Machine Learning (Statistics)

Deciding When to Switch: E-Processes for Adaptive Minimax Training for Generative Adversarial Nets

Modern data science increasingly gives rise to hypothesis-testing problems that are not naturally formulated in terms of parameters within prespecified statistical models. One important example is the dynamic evaluation of optimization algorithms, where decisions must be made during training about whether further updates remain beneficial or the algorithm should switch to a different phase. This issue is particularly relevant in stochastic min-max optimization. Generative adversarial networks (GANs) provide a canonical example, as their training requires repeated decisions about when to switch between discriminator and generator updates, yet existing methods typically rely on fixed update ratios or heuristic criteria. We formulate this switching problem as sequential hypothesis testing and develop an e-process-based adaptive training procedure. During discriminator updates, one e-process tests the null that the discriminator-induced separation between the empirical data distribution and the generator law remains below a target level. During generator updates, with the discriminator fixed, a second e-process tests the reverse null that this separation remains above a refresh level. Conditional on the observed training sample, we prove that fresh empirical indices and latent draws yield conditional e-values that can be accumulated into e-processes, providing anytime-valid Type I error control under adaptive model updates and data-dependent switching. Across multimodal synthetic distributions and image benchmark datasets, the proposed method matches or outperforms the best fixed-ratio baselines under several widely used GAN objectives.
Read the full abstract

All authors Hyunjoo Kim, Sicheng Wu, Agastya Venkatraman, Guang Lin, Sehwan Kim

Statistical Methodology

Covariate Informed Identification of Heterogeneity and Outliers in Longitudinal Data

We often observe heterogeneity in longitudinal data, where the mean and variance for certain profiles meaningfully differ from the rest. Some profiles may also exhibit outliers at a limited number of measurements. Using a standard mixed effects model, which assumes homogeneity, can lead to overestimating the residual variance and inefficient estimation. In this work, we identify and account for three sources of heterogeneity in longitudinal data: incompatible mean trajectories, increased residual variance, and outliers at individual measurements. Our Bayesian mixture model incorporates binary indicators of heterogeneity for each of these features, modeled through logistic regression using covariates. We perform statistical inference using Markov chain Monte Carlo and implement model selection to evaluate the inclusion of various heterogeneous components. Simulations demonstrate that our model can accurately identify heterogeneity and produce efficient estimates of the fixed effects parameters. We further validate our approach using the DHEAS hormone data from the SWAN study.
Read the full abstract

All authors Anish Mukherjee, Jeremy T. Gaskins, Jeffrey M. Albert, Tanujit Dey, Sounak Chakraborty

Statistics Applications

Status Association Does Not Reliably Predict Decision Leakage

Bias evaluations often move too quickly from evidence that a model encodes a social association to claims that the same association will alter consequential decisions. We test whether that inference is warranted using Chilean surnames as controlled socioeconomic probes. We evaluate eight frozen model-provider cells on 1,032 prompts each, yielding 8,256 verified primary responses. The design separates forced latent association from matched consequential decisions across academic selection, professional hiring, research fellowship selection, and legal-aid intake. Elite-coded surnames received higher forced high-status probability mass than common surnames in seven of eight models and higher mass than rare-frequency controls in all eight. Yet elite-minus-common decision effects were close to zero for most systems. Five models were statistically equivalent within a predeclared (Plus-Minus)0.10 standard-deviation margin, while the remaining three were imprecise or borderline, with no consistent elite advantage. Association strength did not reliably predict decision leakage across models (r = 0.201, p = 0.633) or across frozen surname-pair-by-model cells (r = 0.065, p = 0.565). The central result is a measurement dissociation: latent social association and consequential treatment are empirically distinct constructs. Evaluations should measure the transition from association to action directly.
Read the full abstract

Statistics Theory

Dynamic $e$-closure for online hypotheses with any-time-valid evidence: closure principles and projective mergers

Many modern testing problems are sequential along two axes: new hypotheses may arrive over time, while evidence for hypotheses already under consideration continues to evolve and may be inspected at arbitrary stopping times. We develop dynamic $e$-closure for this setting. At a global stopping time the active true-null intersection is random. Future-extension coherence allows its certificate to be compared with that of a fixed terminal intersection, yielding simultaneous stopped-FDR control. If the certificates are also time-monotone, the resulting closure controls simultaneous SupFDR and is setwise persistent. Conversely, every procedure satisfying either criterion is contained in a dynamic closure generated by canonical normalized-loss processes. For pointwise mergers, fixed-dimensional admissibility is equivalent to ordinary arbitrary-dependence $e$-merging. Coherence across horizons then forces a single globally summable weight sequence and exact neutrality under padding by the $e$-value one; on a countably infinite hypothesis universe, this rules out nontrivial symmetric mergers in the admissible pointwise class. The theory extends from FDP to bounded losses that are monotone in the possible true-null configuration and local in the reported action. We also give a coherence counterexample, persistent constructions, and a globally valid shared-control model.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.NE

BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

We introduce BDH-CQ, a reasoning model that combines in-context learning with recurrent latent reasoning. Inputs presented at inference time continuously update the model's recurrent memory; the model then solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning. We evaluate the model on the public ARC-AGI-1 evaluation set and use controlled ARC-like interventions to study what it learns from demonstrations, how consistently it applies an inferred transformation, and which concepts remain difficult. A 150M-parameter configuration reaches 29.5% pass@2 at a computed inference cost of \$0.0007 per task. This operating point breaks through the previously reported ARC-AGI-1 cost-accuracy Pareto frontier, establishing a new state of the art in benchmark cost efficiency.
Read the full abstract

All authors Björn Engdahl, Adrian Kosowski, Jan Chorowski, Zuzanna Stamirowska, Przemysław Uznański, Junlin Jiang, Rohan Phadke, Remigiusz Kinas, Richard Zhong

Statistical Methodology · Cross-listed from math.OC

Safe Start: Configuring Optimization Algorithms for Decision-Making under Extreme Risks

We consider stochastic optimization where the goal is not only to optimize an average-case objective, but also to mitigate the occurrence of rare catastrophic events. This problem is motivated by safety-aware decision-making and AI training. We first argue that, in the presence of a simulation model, natural attempts to integrate variance reduction into optimization, even executed in a reasonable adaptive fashion, encounter fundamental challenges in guaranteeing realistic runtime when using common stochastic gradient descent algorithms. This challenge arises from the extreme sensitivity of tail-based objectives with respect to the decision variables, which renders a dichotomic failure of convergence regardless of what step size we select. We offer remedies based on a new notion of safe start that allows for efficient finite-time error control, and show how the sampling complexity scales favorably under the combination of safe start and variance reduction. We illustrate our methodologies on examples in portfolio optimization and robust classification with neural networks.
Read the full abstract

Machine Learning (Statistics)

Logarithmic-Free Moment and Generalization Bounds for Uniformly Stable Algorithms

Uniform stability is a classical tool for controlling the generalization error of a learning algorithm. Bousquet, Klochkov, and Zhivotovskiy (2020) showed that the problem can be reduced to a moment inequality for a sum of weakly interacting functions of independent random variables. Their bound contains an additional factor $\log n$, and they asked whether this factor can be removed. We answer this upper-bound question affirmatively. More specifically, let $Z=(Z_1,\ldots,Z_n)$ have independent coordinates and let $g_i(Z)$ satisfy $\mathbb E[g_i(Z)\mid Z_{-i}]=0, \ \left| \mathbb E[g_i(Z)\mid Z_i]\right|\le M, \ \text{for every } i = 1, \dots, n, $ where $Z_{-i}$ denotes all coordinates except $Z_i$. Assume additionally that changing any coordinate $Z_j$, $j\neq i$, changes $g_i$ by at most $\beta$, we prove that, for every $p\ge2$, for every $p\ge2$, $$ \left\| \sum_{i=1}^n g_i(Z)\right\|_p \le 16pn\beta+M\sqrt{2pn}. $$ This removes the $\log n$ factor from the previous bound and matches the lower bound of Bousquet, Klochkov, and Zhivotovskiy up to universal constants in the range covered by their construction. Our proof first establishes the required estimate on the Rademacher cube, then transfers it to arbitrary product distributions by a two-copy randomization argument.
Read the full abstract

Statistical Methodology

The Earth Moves, But So Does the Bias: Systematic Upward Bias of the Wasserstein (Earth Mover's) Distance and Permutation-Based Null Calibration

The Earth Mover's Distance (EMD) is gaining increasing interest among political scientists for assessing similarity in preference distributions. However, there remains a risk of finite-sample upward bias induced by sampling variation in empirical probability measures, which is under-recognized by existing studies. This problem is especially severe in high-dimensional or sparse settings, including conjoint distributions that serve as an illustrative example in this paper. As political scientists are broadening their use cases of EMD, this paper cautions against interpreting standard bootstrap uncertainty bounds as a correction for the upward bias of empirical EMD. It proposes a permutation-based null calibration framework for more robust hypothesis testing. As a non-parametric approach, it frees researchers from making directional or distributional shape assumptions. While alternative estimators require these rigid assumptions to correct for upward bias, political science data often fail to meet them in practice. Through four sets of Monte Carlo simulations, this paper demonstrates the utility of this framework. The proposed approach also applies more generally to empirical comparisons of two probability distributions defined on a common metric space, provided that the ground distance between support points is substantively meaningful.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Langevin dynamics along the zero set of real-analytic potentials

We consider the Langevin diffusion $dX_t = - \beta \nabla V(X_t) dt + \sqrt{2} dB_t$ for a general nonnegative real-analytic potential $V$ and a large parameter $\beta$. In the large-$\beta$ limit the process is confined to the zero set of $V$, assuming that it starts there. We derive a candidate limiting evolution on the zero set. To do so, the zero set is partitioned into strata according to a measure of local codimension known as the local learning coefficient and its multiplicity. It is then shown that the Dirichlet form associated with $X$ converges in a certain sense to a hierarchy of Dirichlet forms corresponding to a stochastic evolution on the strata. This evolution is strongly biased toward higher-dimensional, or "more singular", strata. This result is motivated by a question from Watanabe's singular learning theory regarding the learning dynamics of overparameterized statistical models and the generalization puzzle in deep learning. The result suggests a mechanism for the observation that stochastic gradient methods tend to be biased toward singular solutions that generalize well.
Read the full abstract

Statistical Methodology

Clustering Informed Inverse Probability Weighting Strategies for Causal Effect Estimation in Observational Studies

Inverse probability weighting (IPW) is widely used to estimate causal effects in observational studies but depends on adequate propensity-score specification. We compare three strategies for addressing treatment assignment heterogeneity: standard IPW, clustering augmented IPW with cluster specific propensity score models, and a global propensity score model including estimated cluster membership as a covariate. Through simulations with and without latent cluster structure and under correctly specified and omitted covariate propensity score models, we evaluate bias, mean squared error (MSE), and confidence interval coverage across sample sizes of 100 to 500. Both cluster informed strategies reduced bias and MSE from omitted covariate misspecification relative to standard IPW, but neither uniformly dominated: clustering augmented IPW achieved lower MSE when latent cluster structure was present, whereas the global model generally provided lower bias and better coverage at smaller sample sizes. We also apply the methods to 966 breast cancer patients treated with carboplatin, using generalized propensity scores to estimate the dose response relationship between treatment cycles and hypersensitivity reaction risk. Standard and clustered analyses produced similar pooled estimates, while clustering additionally provided subgroup specific estimates and diagnostic profiles. Overall, cluster informed strategies may improve robustness to propensity score misspecification, with relative performance depending on subgroup structure, sample size, and inferential priorities.
Read the full abstract

All authors Ruohui Chen, Scott Zuo, Whitney Stevens, Seth Pollack, Wenna Xi, Lucia Petito, Lihui Zhao, Hui Zhang

Statistical Methodology · Cross-listed from econ.EM

Bias-robust causal inference for panel data

We develop a bias-robust causal inference method for observational panel data settings. Such methods typically impute untreated outcomes, so counterfactual error passes straight into the estimated treatment effect while conventional standard errors ignore it. We adapt bias-aware minimax methods, developed for estimating regression coefficients in factor-model panels, to a causal target: the average effect on the treated, which has to be imputed and may vary across units and periods. The estimator corrects the imputed counterfactual with weighted untreated residuals and reports intervals with an explicit allowance for the error that remains. In simulations the proposed method holds nominal coverage where alternatives such as the generalized synthetic control have almost none, especially when the factor rank is underfitted, at the cost of wider intervals. By applying the developed methodology to real data the estimated effect remains significant for counterfactual errors nearly twice the size that the design's placebos typically exhibit.
Read the full abstract

Statistics Applications

Longitudinal Bayesian networks for assessing team performance in the National Basketball Association

Assessing the performance of a basketball team requires the consideration of multiple sources of information. In recent years, the volume and the quality of data generated in sport has increased considerably, particularly in basketball. In this work, we propose a Bayesian graphical modelling framework for the longitudinal analysis of basketball team performance. The framework is based on Bayesian networks that explicitly represent the nodes, that is, the random variables of interest, as longitudinal stochastic processes. We propose three baseline longitudinal models: a static Bayesian network, a dynamic Bayesian network with an autoregressive structure between successive games, and a dynamic Bayesian network based on a hidden Markov structure. We illustrate the proposed framework through a real-world sports analytics case study involving the Philadelphia 76ers of the National Basketball Association (NBA) during the 2005–06 season. The analysis includes player participation, minutes played, fouls drawn, and one-, two-, and three-point shots attempted and made.
Read the full abstract

All authors Gabriel Calvo, Francisco Palmí-Perales, Carmen Armero, Virgilio Gómez-Rubio

Machine Learning (Statistics) · Cross-listed from cs.LG

SR-OPSD: Self-Referenced On-Policy Self-Distillation

On-policy self-distillation (OPSD) converts feedback into dense token-level supervision on student-generated trajectories, complementing reinforcement learning with sparse outcome rewards. Its self-teacher, derived from the student's current or exponentially averaged parameters and conditioned on additional context, evolves alongside the student and its rollout context distribution. The benefit of modifying this moving target depends on how target–student probability mismatches translate into updates. We propose Self-Referenced On-Policy Self-Distillation (SR-OPSD), which constructs a normalized geometric target from the self-teacher and a frozen initial policy, then minimizes the forward Rényi divergence from this target to the student. The interpolation coefficient controls the self-teacher's contribution, while the Rényi order controls the power weighting of target-to-student probability ratios in the gradient. For fixed contexts and target components, we establish a conditional variational characterization and derive the exact token-logit gradient, revealing how anchoring and projection jointly shape the effective update target. Experiments across scientific reasoning, tool use, mathematical reasoning, and code generation demonstrate strong performance across multiple model families and scales. Ablations further show that reference anchoring can improve or degrade performance depending on the projection objective, supporting the joint design of target construction and projection geometry.
Read the full abstract

All authors Zhuo Sun, Entong Li, Yanlong Zhao, Xiaoyuan Cheng, Wenxuan Yuan, Kaiyu Li, Che Liu, Huihang Liu, Baihua He, Xinyu Zhang, Harrison Bo Hua Zhu, Li Zeng

Statistics Applications

Bayesian Node Edge Modeling of Road Crashes in Central Bogotá

Introduction: Crash counts on road segments and intersections exhibit differ- ent exposure and connectivity patterns that conventional analyses may obscure. Methodology: A Bayesian negative binomial node edge model was fitted to 8,169 road segments and 8,398 intersections in six central districts of Bogotá. Separate predictors represented road hierarchy, pavement, speed, signalization, intersection configuration, and land-use treatment. Model performance was examined through pre- dictive summaries and spatial diagnostics, while computational details are reported in the appendix. Results: Intersections with at least four incident segments and higher maximum incident speeds had higher expected crash counts. Land use treatment and signalized access intensity also showed posterior associations. Road hierarchy and signalization were the clearest segment-level factors; however, this component had weak raw scale predictive performance and numerical uncertainty for some pavement categories. Conclusion: Treating intersections and segments as distinct network elements provides an interpretable baseline for urban crash analysis, but the segment results require cautious interpretation and motivate spatially structured extensions.
Read the full abstract

Statistics Theory

Censored Heteroscedastic Extremes

We study estimation of tail heterogeneity for non-identically distributed extreme observations subject to random right-censoring. In the uncensored setting, such heterogeneity is described by the event scedasis function, which measures the relative contribution of different design points to the upper tail. Under censoring, however, the observed tail heterogeneity is contaminated by the censoring scedasis functions, and applying uncensored techniques targets the wrong object. We propose a Beran-type estimator of the relative event scedasis, which is consistent under mild conditions. To obtain these results, survival analysis representations at an upper order statistics are extended to the non-identically distributed case; specifically, we develop conditional Nelson–Aalen and Beran theory on increasing intervals whose random endpoint is dominated, with probability tending to one, by a deterministic high local quantile. In particular, we derive a martingale array representation of the conditional Nelson–Aalen estimator with explicit error bounds depending only on the sample fraction and the bandwidth. Simulations demonstrate the finite-sample performance of the method, and an application to French property-casualty insurance claims illustrates how heterogeneous censoring can distort naive scedasis estimates.
Read the full abstract

Statistics Applications

Comparing Tobit and Two-Part Hurdle Models for Semi-Continuous Longitudinal Data with an Application to Clonal Hematopoiesis

Zero-inflated nonnegative continuous longitudinal data frequently arise in biomedical studies where outcomes consist of a mixture of excess zeros and positive continuous measurements. Two widely used approaches for analyzing such data are mixed-model versions of Tobit and the two-part hurdle models. The Tobit assumes a latent regression model that is censored below a specified threshold, while the hurdle separately models the continuous positive outcomes and the binary indicator of being positive. The choice between these models has rarely been systematically discussed, and inappropriate model choice may lead to biased estimation and misleading scientific interpretations. In this paper, we derive rigorous mathematical conditions under which the two models are equivalent and show that the Tobit can be viewed as a special case of the hurdle model when the link function for the binary process is probit. Based on simulation studies, we found that the hurdle is more flexible and robust than the Tobit model. On the other hand, if the assumptions of the Tobit are met, this model is easier to interpret since it does not require distinct inferences on both the continuous and binary processes. We therefore recommend that the Tobit model only be used when these assumptions are scientifically plausible and empirically supported; otherwise, the hurdle model is preferable. We applied both models to study the dynamics of somatic mosaicism using longitudinal clonal fraction measurements from the Prostate, Lung, Colorectal, and Ovarian study data while accounting for excess zero values. Estimates obtained from the Tobit and hurdle models were broadly consistent with those from a standard linear model that ignored zero inflation. These findings provide additional support for previously reported associations in studies of clonal hematopoiesis across different types of mosaic chromosomal alterations.
Read the full abstract

All authors Sumaja Bandreddi, Pei Zhang, Rebecca L. Kelly, Mitchell J. Machiela, Paul S. Albert

Statistical Methodology

Why the unrestricted weighted least squares should be routinely reported in medical meta-analyses

The unrestricted weighted least squares (UWLS) meta-analysis estimator of mean effect is an alternative to the conventional random-effects model (RE). It is a weighted least squares regression estimator that can be represented as a multiplicative random-effects model. UWLS has been shown to fit medical research better than RE as measured by AIC/BIC widely across Cochrane Database of Systematic Reviews (CDSR). The independence of UWLS's mean and heterogeneity estimators provide small-sample advantages that RE does not possess. Large small-sample biases and uncertainty in RE's heterogeneity variance estimates explain most of RE's relatively poor fit along with RE's boundary problem, where RE's heterogeneity variance is estimated to be zero. We prove that UWLS almost always has superior fit at RE's boundary with uncommon exceptions. UWLS has also been found to have generally superior statistical properties: bias, MSE, and coverage relative to RE across 1,665 simulation designs compiled from four published studies authored by different teams of researchers. A recent study in this journal replicated UWLS's superior goodness of fit widely across both the CDSR and a new set of simulations. Due to reporting and interpretation errors, this recent study calls for the continued use of RE as the default meta-analysis estimator with limited applications of UWLS. We address this recent study's concerns and show why UWLS should be routinely reported in medical meta-analyses.
Read the full abstract

All authors T.D. Stanley, John P.A. Ioannidis, Maximilian Maier, Hristos Doucouliagos, Willem M. Otte, František Bartoš

Statistics Applications

Amplitude-Phase Analysis of the COVID-19 Point Process and the Early Countermeasures

We investigate how governmental restrictions relate to the spread and temporal dynamics of COVID-19 early in the pandemic. We model daily infection data from each US state as realisations of a point process, taking the random intensity measure to be the latent object of interest and, crucially, allowing these realisations to vary not only in magnitude but also in the temporal dynamics. By non-parametrically separating these amplitude and phase variations, we examine how government restrictions relate to each source of variability, relating the infection curves to the Oxford Stringency Index, which we treat as a measure on the same time window. Employing Wasserstein PCA, we analyse the temporal variability of both the infections and the restrictions. We then use the resulting scores, together with the scalars representing the overall stringency budget and the total infection count, as inputs to a linear vector-on-vector regression model. Our findings suggest that, when considering the separate contributions of amplitude and phase variability, earlier implementation of restrictions is associated with flatter infection curves. By contrast, we do not find significant evidence of an association between stringency and total infection counts, nor between the overall stringency budget and the infection outcomes considered.
Read the full abstract

Statistical Methodology

A Spatial Fay-Herriot Model when the Auxiliary Variables are based on Non-traditional Data

Small area estimation methods combine direct survey estimates with model-based predictions to produce reliable estimates of population quantities. When covariates are measured with error, as often occurs when auxiliary information is coming from big data sources, standard area-level approaches such as the traditional Fay-Herriot model can perform poorly if this measurement error is ignored, potentially yielding biased estimates. Building on the extension proposed by Ybarra and Lohr to account for measurement error in covariates, we further develop this modelling framework by incorporating spatial dependence between areas. We propose a spatial measurement error Fay-Herriot model that jointly accounts for covariate measurement error and spatial correlation, enabling information borrowing across relevant neighbouring areas while properly adjusting for the additional uncertainty introduced by the covariates. We derive the properties of the proposed model and its parameter estimators and outline a parametric bootstrap procedure for estimating the mean squared error of the resulting empirical best linear unbiased predictor. A simulation study examines model performance under a range of measurement error scenarios. The proposed approach is then applied to European Social Survey data, supplemented with big data auxiliary information, to estimate regional attitudes towards climate change in Spain.
Read the full abstract

Statistics Theory · Cross-listed from math.NA

Diffusion Maps Kernel Ridge Regression

In this paper, we study kernel ridge regression using the data-driven diffusion maps (DM) kernel, which is constructed through algebraic manipulations of the diffusion maps algorithm. Under the assumptions that the data lie on a manifold and are sampled uniformly, we prove that the appropriately scaled DM kernel converges uniformly to the heat kernel on the manifold for sufficiently large times as the dataset size increases and the kernel bandwidth is scaled appropriately. Consequently, the limiting Reproducing Kernel Hilbert Space (RKHS) induced by the DM kernel coincides with the RKHS associated with the heat kernel on the manifold. We further show that the RKHS induced by the DM kernel is isometrically isomorphic to the RKHS of the Gaussian kernel, which is continuously embedded in the RKHS of a Matérn kernel whose norm is equivalent to an appropriate Sobolev norm. This result implies that standard risk bounds for kernel ridge regression applicable to Matérn kernels also apply to the DM kernel. Finally, we provide numerical results that (1) validate the convergence of the heat kernel approximation, (2) demonstrate the greater expressiveness of the DM kernel compared to the Gaussian kernel for supervised learning over a larger class of functions on manifolds with boundary, and (3) demonstrate the advantage of the DM kernel over the Gaussian kernel in learning functions with varying frequencies and co-dimensions.
Read the full abstract