Peer Revenue Impact factor: volatile

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Friday 21 August

Statistical Methodology

Public Signals, Concealed Choices: Dynamic Measurement without Behavioral Identification

Members of collective institutions may leave public traces while their individual choices remain concealed. This paper separates a corpus-conditional public position from the behavioral rule linking that position to participation and secret choice. I measure the first with a dynamic ordinal state-space model and establish a likelihood-exclusion result for the second. If behavioral-link parameters enter only the distribution of an entirely unobserved outcome and are a priori independent of measurement parameters, public signals leave their posterior equal to their prior. Declared monotone mappings are therefore reported as sensitivity envelopes, not estimates or identified bounds. Cabinet-sized simulations compare dynamic measurement with a static ordinal model and transparent summaries, exposing gains from temporal pooling and failures under weak measurement or omitted dimensionality. The application reconstructs 41 pre-decision official-source records preceding a concealed Slovenian cabinet decision. Only six of 21 ministers have attributable signals. The model locates a small visible group but leaves the remainder prior-dominated. Event-dependence adjustments, coding perturbations, rolling prediction, and the subsequently disclosed record support the same conclusion. Uncertainty should track the public information environment rather than be converted into unsupported behavioral precision.
Read the full abstract

Statistics Applications

Knowledge-guided Transfer Prediction In Underrepresented Populations: A GRU-D-Static Framework For Maternal And Neonatal Outcomes

Integrating summary-level scientific knowledge into neural network models provides a practical strategy for transferring prediction models trained on adequately sampled source cohorts to underrepresented target populations, where individual-level data in the target domain are often limited or unavailable. In this study, we propose transfer prediction strategies incorporating external summary-level scientific knowledge and illustrate its application on the PRISMA Maternal and Neonatal Health Study, training a neural network model on the source data to predict adverse outcomes in the target cohorts. Besides, we also extend the existing GRU-D framework by incorporating static feature embeddings and attention weights to jointly leverage temporal and static information for improved prediction. Our approach employs soft labels derived from summary-level statistics describing the target population to fine-tune GRU-D-Static models that are initially trained on the source populations. We evaluate six maternal and neonatal outcomes, including stillbirth, preterm birth, low birth weight, small vulnerable newborn, neonatal death, and maternal near miss. Across all tested scenarios, fine-tuning using soft labels from just basic covariates substantially improved predictive performance compared with deep learning models trained on the source sample. Furthermore, the performance slightly improves more when additional covariates were incorporated into the logistic regression model or when partial input features from the target population were available for fine-tuning. These findings demonstrate that integrating existing scientific knowledge in the literature through transfer prediction of source neural network models can enhance prediction performance in underrepresented target populations, reducing reliance on large-scale data collection and supporting risk prediction in global health.
Read the full abstract

All authors Yipeng Wei, Zahra Hoodbhoy, Emily R. Smith, Fang Jin, Muhammad Imran Nisar, Muhammad Farrukh Qazi, Christopher Mores, Victor Akelo, Caleb Sagam, Florence Aweyo, Charlotte Tawiah, Veronica Agyemang, Kwaku Poku Asante, Sam Newton, Santosh Joseph Benjamin, Anne George Cherian, Devakumar Devadhas, James A, Margaret P. Kasaro, Augustine Tunga, Sarmila Mazumder, Neeraj Sharma, Wilbroad Mutale, Mae Bridget Spelke, Qing Pan

Machine Learning (Statistics) · Cross-listed from cs.LG

Free-Probability Kernels for Zero-Rollout Hyperparameter Selection in Reservoir Computing

Reservoir computing (RC) couples a fixed recurrent dynamical system with a trained lightweight readout, but this efficiency is partly lost during hyperparameter selection: the recurrent gain, input scale, and leakage rate determine the reservoir's stability and temporal processing regime and are usually tuned through many rollouts. We introduce a deterministic, pilot-informed selector for leaky linear reservoirs followed by coordinate-wise nonlinear features. Free probability yields cross-lag propagation coefficients that summarize how the reservoir mixes past inputs. In the large-width limit, these coefficients define a deterministic temporal kernel that approximates the finite-reservoir feature geometry. Kernel ridge regression on a short labelled pilot sequence therefore ranks candidate operating regimes without instantiating or rolling out a reservoir, and the selected configuration transfers across widths. Across ten synthetic temporal benchmarks, zero-rollout selection obtains a mean deployment score of $0.772$, compared with $0.774$ for exhaustive simulation-based search, while avoiding $156\,600$ selection rollouts. With a small rollout budget, the proposed ranking provides the strongest mean performance at every tested budget and reaches the exhaustive reference using $4.8\%$ of its rollout cost. On four public electricity-transformer-temperature (ETT) forecasting datasets, five retained candidates recover the exhaustive operating point on three datasets. On multivariate cellular-traffic forecasting, 15 rollouts per cell reach the 462-rollout exhaustive reference and outperform random search and Bayesian optimization at low budgets. These results position free-probability kernels as deterministic surrogates for selecting reservoir operating regimes when validation rollouts are scarce.
Read the full abstract

Statistics Applications

Calibration of clinical trial sample size based on design utility

Clinical trial design relies on both statistical and clinical considerations for pre-specification of potentially practice-changing target treatment effects. As larger trials tend to be associated with high power and modest minimal detectable benefit, trial sample size is typically calibrated with reference to relevant precedents to prevent overpowering. Albeit trial sponsors and regulators are accustomed to this practice, there is scope for simplification to enhance the robustness, transparency and cross-trial consistency of this aspect of the design process. To this end, a design utility index is proposed here as a formal basis for sample size calibration, balancing the increase in power at higher sample sizes against the corresponding reduction in the magnitude of minimum detectable treatment effects, without requiring additional statistical assumptions or bespoke software. Application of utility calibration to a broad range of designs demonstrates consistency with regulatory expectations for minimum power, particularly in confirmatory settings, and effective protection against overpowering and against overly aggressive interim analyses.
Read the full abstract

All authors Charu Gupta, Gabriel Innocenzi, Christina Yap, Daniel Jackson, Fabio Rigat

Machine Learning (Statistics) · Cross-listed from cs.LG

A Critical Audit of Spatiotemporal Forecasting Benchmark Datasets and Models

Graph neural networks (GNNs) are routinely employed for spatiotemporal forecasting, yet their performance across widely used benchmark datasets is inconsistent. Here, we perform an audit of dataset properties and baseline models to assess the quality of the benchmarks, and the robustness of the conclusions drawn from them. Using classical statistical tools, we characterise spatiotemporal lagged dependencies in benchmarks, and examine how temporal differencing changes these relationships and affects model rankings. Motivated by this, we re-evaluate temporal linear baselines, significantly reducing the apparent gains from GNNs on several benchmarks, and surpassing GNNs on others. Suspecting that GNNs struggle to extract linear, node-wise signals, we find that supplying them with autoregressive residuals improves their performance particularly on non-traffic benchmarks. Finally, controlled synthetic experiments reveal that GNNs are sensitive to heterogeneity in temporal dynamics and spatial graph interactions. Together, our findings demonstrate that baseline specification, data pre-processing and system heterogeneity shape the interpretations drawn from benchmark rankings, informing the design and robust evaluation of GNNs.
Read the full abstract

All authors Kenneth Martin, Simon Heilig, Asja Fischer, Michel F. C. Haddad, Adam M. Sykulski, Moshe Eliasof

Statistical Methodology

From Cumulative Weights to Marginal Density Ratios: Per-Protocol Estimation in Sequential Target Trial Emulation

Sequential target trial emulation evaluates eligibility at multiple baseline times to emulate a sequence of randomized trials using observational data. Estimating per-protocol effects in this setting is challenging because treatment deviations and loss to follow-up induce selection among individuals who remain observed and adherent over time. Conventional inverse-probability methods address this selection using cumulative weights constructed from estimated adherence and censoring probabilities, but these weights can be highly variable, leading to unstable and imprecise effect estimates. We propose a different approach based on marginal density ratios (MDRs). The MDR directly compares the state distribution among individuals who would remain event-free under a target treatment strategy with the corresponding distribution among observed-adherent individuals. We use longitudinal g-computation to generate the target risk sets and a probabilistic classifier to estimate density ratios for reweighting the observed outcomes. Building on this approach, we also develop a doubly robust extension. Favorable performance across the simulation study suggests that MDR weighting is a promising alternative to cumulative longitudinal weights when its identification assumptions are plausible.
Read the full abstract

All authors Zern Ke, Mingshi Cui, Feng Dai, Birol Emir, Javier Cabrera, Demissie Alemayehu

Statistical Methodology

Spatial function-on-function quantile regression

This paper introduces a novel penalized spatial function-on-function quantile regression framework for analyzing spatially indexed functional data, bridging a critical gap between spatial functional models and quantile regression. Our work makes three key contributions. First, we propose the first spatial function-on-function quantile regression model that jointly accounts for spatial correlation across curves through a functional spatial autoregressive structure while allowing inference on arbitrary conditional quantiles of the functional response. Unlike traditional mean-based alternatives, this approach successfully captures state-dependent volatility and distributional dynamics beyond the conditional mean. Second, we develop a two-stage instrumental-variable estimation strategy to address endogeneity induced by the functional spatial lag. By utilizing tensor-product B-spline expansions with tensor-product roughness penalties, our method ensures optimal smoothness without the destructive information loss inherent in principal component truncation. Third, for fixed spline dimensions, we establish $\sqrt n$-asymptotic normality of the spline coefficient estimators and the induced finite-rank Gaussian-process limits for the reconstructed coefficient surfaces. Extensive Monte Carlo experiments and a high-resolution analysis of Italian PM$_{2.5}$ air quality data demonstrate that spatial function-on-function quantile regression significantly outperforms non-spatial and mean-based competitors, providing a robust and informative tool for environmental risk management and complex functional data analysis. Our method has been implemented in the SpatialFoFReg R package.
Read the full abstract

Statistics Theory · Cross-listed from cs.IT

Minimax Quantile Bounds via Information Measures

We develop a unified information-theoretic framework for lower bounding minimax quantiles. The starting point is a loss-adapted Neyman–Pearson metaconverse that bounds the minimax success probability at every loss threshold and confidence level. The bound separates the small-ball behaviour of the prior under the loss from the statistical distinguishability of the observation model, and is optimised over an auxiliary output distribution. Different relaxations of this Neyman–Pearson bound yield converses based on \(f\)-informativity, Sibson mutual information \(I_\alpha\), Maximal Leakage, and Amemiya norms. Classical Fano and Le Cam lower bounds are recovered as special cases. The framework also clarifies why different information measures are suited to different recovery criteria. Maximal Leakage is exact for a class of symmetric exact-recovery problems. We use this identity to derive finite-sample bounds on the full minimax exact-recovery risk in the balanced Gaussian weighted stochastic block model, as well as two-sided finite-sample minimax-quantile bounds for low-rank matrix estimation under isotropic bounded-energy noise. For approximate Hamming recovery, we exhibit a heterogeneous binary model in which an optimised finite Sibson order yields a strong converse while the Maximal Leakage specialisation is trivial. Finally, for one-coordinate Poisson localisation, a Bennett-type Young function used through its Amemiya norm recovers the exact success-probability scale, whereas classical Fano and fixed-power relaxations are strictly weaker. These results show that sharp converses for minimax quantiles require adapting the information measure to the recovery resolution, whether exact or approximate, and to the tail behaviour of the likelihood ratio.
Read the full abstract

Statistical Methodology

Heterogeneous Effects of Continuous Treatments via Conditional Modified Treatment Policies

For continuous treatments such as drug dose or ventilator intensity, a key clinically actionable question is whether a modest, patient-specific adjustment to the current dose would help or harm, rather than whether to treat at all. Standard estimands such as average or conditional dose-response functions require positivity across a wide range of doses, an assumption that routinely fails in observational clinical data where protocols tie dosing to patient characteristics. We develop a framework for characterizing heterogeneity in the effects of small shifts (“nudges”) of a continuous treatment. We study two estimands: the conditional nudge effect, the expected outcome change if an individual at dose $a$ with covariates $\bx$ had their dose shifted by $\delta$, and the conditional modified treatment policy (CMTP) effect, which averages the nudge over the observed dose distribution given covariates. Both are identified under weak, shift-specific local positivity and exchangeability conditions. Recasting the $\delta$-shift as a two-arm comparison through a duplicated-data construction, we develop weighting and A-learning estimators based on squared-error and negative log-likelihood losses, introduce augmented versions that reduce variance without changing the target, and establish asymptotic normality of proposed estimators. Simulations support the theory, and an analysis of mechanical ventilation data from MIMIC-III identifies patient profiles predicted to benefit from a modest reduction in mechanical power.
Read the full abstract

Statistical Methodology

A Multiscale Ball Test for Conditional Mean Independence

Tests of conditional mean independence can lose power when departures are confined to a bounded part of a multivariate predictor space and the relevant spatial scale is unknown. We propose a Multiscale Ball Conditional Mean Independence (MBCMI) test that aggregates support-weighted local mean contrasts in an outcome variable across balls centered on each data point in a predictor set. Fixed-grid theory identifies the population target, establishes consistency for grid-visible alternatives, and derives a Pitman local-power limit governed by the ball-smoothed mean departure. For serial data, feasible recursive-sign-bootstrap validity for stable finite-order autoregressions with conditionally sign-symmetric innovations is established. Application-aligned serial null experiments reject 4.25% of the time. MBCI is demonstrated to be strongest for local and radial signals. Predictor-law experiments show that these conclusions are not an artefact of independent Gaussian covariates. In monthly U.S. finance data, cross-fitted residual MBCMI tests remove every full-sample rejection, suggesting contemporaneous conditional-mean dependence rather than evidence of distinctive nonlinear structure.
Read the full abstract

Statistical Methodology

A modified score function for monotone likelihood in promotion time cure rate models

Survival models that incorporate a cure fraction provide a flexible framework for jointly modeling the cure and the survival distributions. However, when the data comprise a high proportion of censored observations or highly unbalanced binary covariates, maximum likelihood estimation may become unstable, leading to parameter estimates that diverge to infinity. This phenomenon, commonly referred to in the literature as monotone likelihood, compromises statistical inference by precluding the existence of finite maximum likelihood estimates. Specifically, the likelihood function increases monotonically along certain directions in the parameter space, so no finite maximizer exists. To the best of our knowledge, the monotone likelihood problem has received little or no attention in the context involving the promotion time model. This paper addresses this gap by proposing a modified score function based on Firth's bias-reduction method, which adjusts the estimation procedure to ensure finite and stable parameter estimates. The performance of the proposed approach is evaluated through extensive Monte Carlo simulation studies. An application to a real dataset further demonstrates its practical advantages, showing that the mitosis factor, an established prognostic marker, becomes statistically significant under the proposed methodology.
Read the full abstract

Machine Learning (Statistics)

Minimax Optimality of Score-Entropy Discrete Diffusion

Discrete diffusion models have demonstrated strong performance across a range of datasets, including natural language data and graph-structured data. Among many variants, score-entropy discrete diffusion (SEDD) has achieved particularly strong empirical results. In SEDD, new samples are generated by iteratively evaluating a sequence of concrete score functions, which are learned by minimizing a score-entropy loss. While much of the prior theoretical literature on discrete diffusion has focused on the sampling efficiency of SEDD under the assumption of small score estimation error, recent work has begun to investigate the finite-sample properties of score estimation itself. In this work, we take a different route by investigating the fundamental statistical limits of concrete score estimation. We focus on uniform and masking discrete diffusions, two of the most widely adopted discrete diffusion models. We establish a minimax lower bound under the score-entropy loss, and propose an MLE-based thresholding estimator that matches this lower bound up to constant and polylogarithmic factors that depend on neighboring density ratios. We further show that, for any target distribution, this density ratio is naturally controlled under both uniform and masking discrete diffusion models, yielding nearly matching minimax lower and upper bounds for the aggregated score estimation error. Our results imply that, with appropriate initialization and discretization, SEDD can achieve nearly optimal minimax sample complexity, as measured by the KL divergence between the target and generated distributions.
Read the full abstract

Statistical Computation · Cross-listed from math.NA

A new analysis of the randomly pivoted Cholesky algorithm

The randomly pivoted Cholesky algorithm is one of the leading methods for computing a low-rank approximation to a large positive-semidefinite matrix. However, while it consistently achieves accuracy comparable to or better than competing methods of its type in experiments, its theoretical analysis lags somewhat behind other methods. This paper closes this gap, proving that randomly pivoted Cholesky produces an approximation with expected error within a $1+\varepsilon$ factor of the optimal rank-$r$ approximation in $\mathcal{O}(r/\varepsilon + r\sqrt{\log r})$ steps. This result nearly matches the optimal complexity $\Theta(r/\varepsilon)$ for any low-rank approximation method based on a partial Cholesky decomposition (also known as a column Nyström approximation). The paper also presents bounds on the randomly pivoted Cholesky trace and spectral-norm errors that hold with high probability. The mathematical argument is largely due to GPT 5.6-Sol (Pro), with some refinements by the author.
Read the full abstract

Thursday 20 August

Statistical Computation

Uniform efficiency of the Ulrich-Wood sampler for the von Mises-Fisher distribution

The standard approach to stochastic simulation from the von Mises-Fisher distribution is a rejection sampler proposed by Ulrich and Wood. This note provides a theoretical justification of its efficiency, lower-bounding its acceptance probability uniformly over the dimension and concentration parameter. A novel interpretation of the proposal is given which provides some intuition for this efficiency.
Read the full abstract

Statistical Methodology

Covariance Kernels on Unordered Pair Spaces: Theory and Applications to Network-Valued Data

Many scientific problems are relational: the quantity of interest is a connection between two objects, while information about similarity is available for the objects themselves. We develop a covariance framework for unordered relationships that transfers object-level geometry to the relations they form while preserving endpoint identity and invariance to ordering. Building on symmetric pairwise-kernel representations, we develop theory for the loop-free domains used in undirected networks. We establish spectral interlacing and trace-loss results after self-pairs are removed, connect the relational spectrum to regularization and risk, derive an exact inferential error for spectral truncation, and quantify how perturbations of the underlying geometry propagate to pair covariance and estimation. Simulations show when structured borrowing improves estimation and how geometric misspecification can erode that benefit. We apply the framework to autism neuroimaging using resting-state functional-connectivity data from the Autism Brain Imaging Data Exchange (ABIDE). With a 116-region parcellation and 6,670 unique connections, the application shows that a large connectome can have a much smaller effective covariance dimension. It also demonstrates that high explained covariance alone is insufficient for choosing a low-rank representation when inferential accuracy is the goal. The framework provides a principled foundation for covariance and regularization when the statistical units are unordered relationships.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Evaluation Choices Decide the Forecasting Leaderboard: Evidence from a Production Marketplace Panel

A forecasting benchmark reports which method won. We show that the answer is set by the evaluator's choices before any model is fitted. We benchmark 24 forecasting methods and one textbook reference, including six 2025-era time series foundation models, on a production marketplace panel of 1,887 business customers over 67 months. We hold the data, the horizon and the period fixed, and vary only the evaluation design. Three choices each reverse or dissolve a headline conclusion. Changing the unit of analysis from the market total to the individual customer moves our production baseline from second of nineteen, beaten by nothing, to twenty-third of twenty-five. Nineteen of its twenty-four challengers beat it there. Changing how much error is pooled decides whether a Diebold-Mariano test finds anything at all. Scoring prediction intervals rather than point forecasts reorders the field almost completely, with a rank correlation of 0.02 on intermittent demand. We then measure what the deployed system gets from this. Its selection rule captures 55% of the distance between doing nothing and choosing with hindsight. The reversal is not a quirk of our data. We ran the released protocol, unchanged, on the public M5 retail panel. The same baseline shape places first at the market total and last per series, beaten by everything, and a replayed selection rule closes 64.7% of the same floor-to-ceiling distance there. Adding five zero-shot foundation models to that roster changes who wins at the total, not the shape. The bands' blind spot travels too: conformal bands under-cover most on the spikiest items. Splitting our own panel into ever smaller groups turns the contrast into a curve: the baseline's rank worsens at every level of disaggregation. We release the evaluation protocol and report an error of our own that inverted a result before we caught it.
Read the full abstract

Statistical Methodology

Let Time Tell: Identification and Gaussian Process Estimation for Interrupted Time Series

We study causal inference in interrupted time series designs where a treatment affects every unit simultaneously, so that the contemporaneous controls used by difference-in-differences and synthetic control are unavailable and the counterfactual must be extrapolated from a unit's own pre-treatment history. We establish identification within the potential outcomes framework and estimate the counterfactual by Gaussian process regression. Rather than committing to a single best-fitting trend, the estimator retains the functions consistent with the pre-treatment series and widens its intervals where extrapolation magnifies their divergence. Connecting it to reproducing kernel Hilbert space theory, we derive a bias decomposition that isolates the component extrapolation inflates and a worst-case bound on that component, justifying the Gaussian process estimator's posterior variance as extrapolation-aware uncertainty quantification. In closed form, the band equals the worst-case divergence the model class permits among functions consistent with the pre-treatment data. The method is illustrated with calibrated simulations and an analysis of handgun purchases after the Supreme Court's Heller decision, a universal treatment whose practical effect concentrates in a single jurisdiction. An R package, gpss, implements the approach.
Read the full abstract

Statistical Methodology

A Comprehensive Bayesian Approach to Entity Resolution for Data with Multiple Truths

In many applications, from government to ecology, integrating data from diverse and noisy sources is critical for downstream inference. However, a unique identifier to link records cleanly from the same entity may not exist. Entity resolution (also referred to as de-duplication or record linkage) merges such databases to identify duplicates, allowing for more complete data for inference. A multitude of methods from statistics and computer science in recent years assume a single, immutable true value for each variable used in linkage. We argue this restrictive assumption is often violated in practice, leading researchers to discard useful data and biasing downstream inference. For example, a respondent's education status may truly change between two different surveys, yet existing methods would treat at least one observation as a distorted version of the truth, even though both are correct. In this paper, we propose a novel entity resolution model that comprehensively accommodates "multiple truths" by introducing a mixture of exponential family distributions that further handles multiple data types. We provide options to fit the model with Markov chain Monte Carlo routines and variational inference for massive datasets. We demonstrate the method's value via simulation and linking a longitudinal survey of Italian household wealth.
Read the full abstract

Statistical Methodology

Targeted Deep Survival Contrasts: Valid Inference for Treatment-Specific Survival Benefit with Neural Networks

Neural survival models are increasingly asked to support counterfactual claims—how much a treatment would change survival in a population—rather than only prognostic risk scores. Answering such questions from observational data requires valid inference for treatment-specific survival contrasts under confounding and covariate-dependent censoring, targets for which standard deep survival estimators are biased and provide no honest uncertainty. We propose Targeted Deep Survival Contrasts (TDSC), which extends Targeted Deep Architectures (TDA)—targeted maximum likelihood estimation embedded in a network's weight space—to the full vector of treatment-specific survival curves over a time grid, and hence to the benefit curve and the restricted mean survival time (RMST) difference. A single universal targeting path, one ridge projection of the stacked efficient influence functions onto closed-form last-layer gradients per iteration, simultaneously solves the projected estimating equations for all coordinates; a one-step residual top-up converts the plug-in into a doubly robust estimator of the unrestricted target; and a multiplier bootstrap yields simultaneous confidence bands for the benefit curve. We prove joint asymptotic linearity, band validity, and double robustness of the top-up for a cross-fitted variant requiring no Donsker conditions. Across seven Monte Carlo banks with confounded treatment, sign-varying effect heterogeneity, and dependent censoring, the TDSC plug-in attains nominal pointwise and simultaneous coverage with 35% lower MSE than a per-timepoint one-step (AIPCW) built from the same nuisance fits. Under a badly wrong outcome model the plug-in tracks its working parameter and its intervals fail (44% coverage), while the top-up restores nominal inference for the unrestricted causal target (94-95%)—and in-sample diagnostics separate the two regimes.
Read the full abstract

Statistical Methodology

Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data

Mixed-type data containing both numerical and categorical variables arise in many scientific and real-world applications. Existing representation learning and generative modeling approaches typically focus either on reconstruction accuracy or unconditional data generation, but often fail to recover the full conditional distribution of the data while preserving interpretable structural relationships between heterogeneous variable types. In this work, we introduce Conditional-Independence-Regularized Distributional Autoencoders, a framework for learning low-dimensional representations of mixed-type data through conditional distribution matching and structural regularization. Our method combines an energy-score-based objective for numerical variables, a likelihood-based objective for categorical variables, and an auxiliary conditional independence regularization term encouraging the learned representation to capture the dependence between numerical and categorical components. We provide theoretical analysis showing that the optimal representation balances unexplained numerical variability, conditional entropy of categorical variables, and residual conditional dependence. Empirically, the proposed method achieves strong performance on both synthetic and real-world datasets, substantially improving categorical distribution recovery, achieving competitive overall conditional distribution recovery, and preserving mixed-type dependence structure. The code has been made available at GitHub.
Read the full abstract

Other Statistics

Statistics in the Age of AI

Artificial intelligence (AI) can automate programming, model fitting, visualization, simulation, literature synthesis, and increasingly sophisticated methodological tasks, but it cannot remove the logical conditions under which data support scientific claims or consequential decisions. We formalize these conditions through statistical warrant, which connects data to a claim through the target, observation regime, assumptions, procedure, uncertainty assessment, validation criterion, loss structure, governance and accountability. No algorithm can consistently recover a target that is not identified by the observation regime without additional information or assumptions. From this principle, we organize the argument around five statements. Questions and targets are integral to statistical methods. Data acquire evidential meaning only through design, provenance, and assumptions. Description, prediction, causal inference, and decision are mathematically distinct tasks. Analytical abundance requires accounting for how analyses are selected, uncertainty across the analytical system, and deployment validation. The statistician's fundamental role is therefore to construct, criticize, and safeguard statistical warrant, including by developing new methodology when existing theory is inadequate. This role requires statistical reasoning and attributable human and institutional responsibility.
Read the full abstract

Statistical Methodology

[EDGE] A grouped calibration test for logistic regression that tolerates a few corrupted records

The Hosmer-Lemeshow calibration test groups patients by predicted risk for a valid reference and loses power that more groups cannot recover. Later tests weight each record's residual alone, which makes them fragile: on an otherwise correct model, one record in a thousand with a corrupted covariate raises the false-alarm rate of Stukel's score test from 5% to 12%, and ten to 63%. Pooling on equal-size groups bounds a wrong record's influence, dividing its residual by its groupmates' variance; without pooling, the same directions are as fragile as Stukel's test after a refit. EDGE, the test proposed here, projects the grouped residuals onto a small basis of calibration shapes. Its degrees of freedom do not depend on the number of groups, so the partition becomes a setting, fine-grained for power and coarse when records may be wrong, and its reference needs no resampling. In a prespecified simulation EDGE was comparable in power to Stukel's test on clean data and 0.064 more powerful on average than the ten-group Hosmer-Lemeshow test. With one and ten exaggerated covariates in a thousand its false-alarm rates were 0.056 and 0.097, and with ten groups below 0.10 up to about ten exaggerated or five reversed. In external validation, at ten groups, its average power was that of Cox's recalibration test and the GiViTI belt, and its false-alarm rate stayed below 0.10 with ten reversed records in a thousand, where theirs did not. In a 70,000-patient cohort its verdict did not change with the partition.
Read the full abstract

2608.20511 · 20 Aug · 25 pages, 6 figures, 6 tables; Supporting Information as an ancillary file. v2: revised and retitled, with new studies of corrupted records and external validation and a clinical cohort. R package ebrahim.gof (CRAN). Archive: https://doi.org/10.5281/zenodo.23079225. Develops a method introduced in one chapter of the first author's M.Sc. thesis (arXiv:2608.11140)

Statistical Methodology

Counterfactual Optimization of Policy Interventions: Lexical Ordering and Leapfrogging

Most data-driven policy learning methods maximize average outcomes, overlooking the possibility that a policy beneficial on average may still harm a substantial fraction of individuals. Motivated by the ethical principle of "first do no harm", we study how to design a change from a baseline policy that improves overall welfare while keeping the worst-case probability or expectation of individual harm below a specified limit. We establish sufficient conditions under which an optimal policy transition has a lexical leapfrogging structure: groups defined by covariates and current treatment are ranked by a priority score, and any treatment change moves them directly to the conditionally optimal treatment. We derive this score under several models for the dependence among potential outcomes. We demonstrate this harm-aware policy optimization approach in a reanalysis of the I-SPY2 breast cancer platform trial and show how the consideration of counterfactual harm may lead to different conclusions about which treatment-subgroup pairs may warrant deprioritization in further clinical evaluation.
Read the full abstract

Statistics Applications

Testing Additivity of Lead and Benzo[a]pyrene-induced Neurotoxicity in Caenorhabditis elegans Assays

Exposure to environmental contaminants is a recognized cause of neurotoxicity, contributing to the onset of a broad range of neurological conditions. In realistic settings, such exposure involves com- plex mixtures, and the combined effect of their components may differ from what their individual effects would predict. Characterizing such interactions and testing them against a principled notion of additivity is central to assessing the neurotoxicological risk. We take up these questions for two widespread and independently neurotoxic pollutants, lead (Pb) and benzo[a]pyrene (BaP), through a novel C. elegans assay in which nematodes were subjected to single and joint exposures across a range of doses. Morphological damage is quantified on an ordinal scale at the level of individual dopaminergic neurons. To analyze these data, we model the full distribution of the ordinal response as a convex mixture between an unexposed and a maximally affected profile. The weight of this mixture varies with chemical doses, modeled flexibly via monotone splines and, for the joint effect, in a radial coordinate system. Additivity is assessed via a likelihood ratio test against established null models, and is calibrated via parametric bootstrap. Applied to the C. elegans assay, our analysis reveals a localized, asymmetric synergy between Pb and BaP, concentrated where moderate BaP meets high Pb exposure.
Read the full abstract

All authors Niccolo Anceschi, Javier Huayta, Joel N. Meyer, David B. Dunson, Amy H. Herring

Statistics Applications

Robust Parameter Estimation for Snow Load Induced by Annual Maximum Snow Accumulation Using Constrained Bayesian Priors

This paper develops a Bayesian framework to estimate the parameters of the Generalized Extreme Value (GEV) distribution for the weight induced by annual maximum accumulations of snow, referred to as the snow load, using a Hamiltonian Monte Carlo (HMC) algorithm as implemented in the \enquote{extremMHMC} R package developed alongside this paper. Key to the approach is the use of strong prior distributions for the shape parameter that are appropriate in the context of snow loads, which helps to ensure robustness in the distribution parameter estimates for annual maximum snow loads despite small sample sizes. This robustness is key to ensuring that structural reliability analyses, which rely on the GEV distribution, produce physically realistic estimates of snow loads. Information on strong prior distributions is derived from existing studies on extreme rainfall and snowfall, with the novel use of hyperbolic tangent functions to transition the prior distribution parameters between low and high snow regimes. This approach enables global applicability of the strong prior approach while maintaining physical realism. Additionally, simulation studies confirm a reduction in Root Mean Square Error (RMSE) of the shape parameter estimate compared to frequentist methods, particularly for small sample sizes. Finally, a real-world application using a data from 9715 stations further demonstrates the feasibility of the proposed Bayesian framework for large scale implementation.
Read the full abstract

Machine Learning (Statistics)

Uncertainty propagation in auto-regressive random neural network models

We develop analytical and particle-based methods for uncertainty propagation in random neural network models, where both the inputs and network parameters are allowed to be random. Building on the piecewise-linear structure of the Leaky ReLU activation function, we derive a local approximation of the neural network output with respect to perturbations in both its inputs and parameters. This approximation is exact for perturbations that preserve the network activation pattern, and it allows us to compute analytical expressions for the probability density function and characteristic function of the network output, together with closed-form approximations for its mean and covariance. We extend this uncertainty propagation framework to autonomous dynamical systems whose one-step evolution map is represented by a random neural network. Repeated application of this map defines an autoregressive model, for which we derive recursive equations to propagate uncertainty in both the state and network parameters over time. These equations explicitly account for the state-parameter cross-covariance that develops under successive iterations of the network. Numerical experiments on the Lorenz-63 system and the Kuramoto-Sivashinsky equation demonstrate accurate uncertainty propagation through the predictability horizon and the applicability of the proposed framework to high-dimensional dynamical systems.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

When Clean Data Hurts: Learning with Monotone Corruptions Beyond Binary Classification

Optimal learners are tailored to exploit the i.i.d.\ data assumption underlying the classic PAC model. What if an i.i.d.\ training sample were corrupted with correctly labeled examples drawn from an otherwise unrelated, even adversarial source? This model of learning with monotone adversarial corruptions was recently introduced by Larsen et al. (2026), who demonstrated that all known optimal binary learners suffer increased error rates in this setting, from $O(d / n)$ in the PAC model to $\Omega (d \log(n / d) / n)$ under monotone corruption. Mehrotra (2026) proved this logarithmic factor to be necessary for binary classification, but left open the consequences of corruption for more general learning settings, such as multiclass classification and partial binary concept classes. As our primary result, we demonstrate that monotone adversaries are frighteningly more powerful in each of these settings. We exhibit a learnable multiclass problem, of DS dimension only 2, that becomes altogether unlearnable under a monotone adversary, and show an analogous result for partial binary concept classes. These results are achieved by an adaptive adversary permitted to view the original i.i.d.\ training set $S$ and to insert $b < \infty$ corrupted datapoints into $S$. In the multiclass example, the adversary need only insert a linear number $b = |S| = n$ of datapoints. We complement these impossibility results by proving that every class remains learnable when the number of adaptive additions is $o(n)$, which our previous multiclass lower bound proves to be tight. We further observe that the classic multiclass error rate of $O(d_{\mathrm{DS}} / n)$ remains achievable against adaptive adversaries restricted to a known constant budget $b = O(1)$, against semi-adaptive adversaries viewing only a $p$-fraction of $S$ for $p \in (0, 1)$, and against oblivious adversaries that cannot view $S$.
Read the full abstract

Statistical Computation

A Complexity Bound for the Kent-Ganeiber-Mardia Sampler for the Bingham Distribution

The Bingham distribution is a family of antipodally symmetric distributions on the unit sphere, characterised by an exponential-of-quadratic change of measure with respect to the uniform distribution. Kent, Ganeiber and Mardia proposed a rejection sampler for generating samples from Bingham distributions using proposals from an angular central Gaussian (ACG) distribution. Their empirical results suggest that the least efficient regime is the high-concentration limit, where the acceptance probability is of order $d^{-1/2}$ in dimension $d$, implying a polynomial complexity guarantee. In this note, we verify this dimension-dependent prediction, establishing the uniform guarantee $\inf\{\alpha_D:D=D^\top\in\mathbb{R}^{d\times d}\}\ge c_\star/\sqrt{d}$, where $c_\star=0.759\ldots$. A one-dimensional high-concentration limit demonstrates that the $d^{-1/2}$ rate is unimprovable and that even the constant $c_\star$ cannot be improved beyond $0.857\ldots$. The proof relies on a novel interpretation of the acceptance probability and a comparison principle for weighted sums of chi-squared random variables, which may be of independent interest.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Amortized Bandwidth Learning for Kernel Density Estimation under Logarithmic Score

Kernel density estimation converts finite samples into probability densities, but its performance depends critically on bandwidth selection. Classical selectors prescribe the sample-to-bandwidth rule analytically or asymptotically, or solve a new optimization for each sample. An amortized framework is proposed that instead learns this mapping across a distribution of density-estimation tasks by optimizing the logarithmic score. A truncated-and-renormalized bounded-support formulation enables stable learning across heterogeneous tasks, while affine standardization allows a selector trained on a single reference interval to transfer across bounded intervals. Experiments under Gaussian sampling, a multi-family benchmark, and randomized Gaussian-mixture training show that the amortized selector consistently and substantially outperforms Silverman's rule, the Sheather–Jones selector, and least-squares cross-validation, with especially large gains in small and heterogeneous samples. Finite Gaussian mixtures provide a generic training mechanism supported by their $L^1$ approximation property. Selectors trained in this way generalize strongly across different density structures, allowing the same trained selector to be applied directly to finite samples from unknown densities without specifying or fitting a distributional family. This combination of broad applicability and strong empirical performance makes the framework attractive for a wide range of applications in which finite samples or ensembles must be converted into continuous probability densities.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Information on trajectories: martingales and random times

Accounting for information flow on the path space of trajectories of a nonnegative martingale yields exact variational identities for it, even at arbitrary random times. This recovers the widely used classical concentration inequalities, from Ville to PAC-Bayes, and measures what each one discards. The tail a bound controls is itself a relative entropy, resolved by the chain rule into per-step conditional divergences. The discarded slack has an exact form in each of three geometries: a Gibbs tilt for the Azuma-Hoeffding and PAC-Bayes bounds, the crossing itself for Ville's and for pooled tests, and a dominating certificate for the $L^p$ maximal bound. That certificate's optional-stopping deficit resolves per step into Bregman divergences of the running maximum. On a path-time space, the same identity gains one factor that prices anticipation: an arbitrary random time carries an e-process “peeking penalty.” The partition function can be read as a coalescent–a prefix-sharing probability of independent copies–and geometric mixtures of test martingales gain a pooling benefit for multi-model safe testing.
Read the full abstract