Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Tuesday 16 June

Statistical Methodology

A closed-form sample size correction for always-valid inference with optional stopping

Sequential tests that allow continuous monitoring are common in A/B experimentation. Power calculations for these tests require simulations that are hard to scale across many metrics on an experimentation platform. Instead, a common sizing heuristic inflates the fixed-sample size until the marginal rejection probability at the planned endpoint reaches $1-\beta$. This last-point rule is conservative because always-valid (AV) power is the probability of a boundary crossing at any time during the run, not at the endpoint alone. We give a closed-form correction factor $k^(\alpha, \beta, t_0)$ expressed in elementary functions and the bivariate normal CDF, where $t_0 = m/n_z$ is the burn-in fraction. The closed-form approximation depends on the boundary only through its value and slope at the planned endpoint and can be evaluated for any smooth concave boundary. We work out three cases: the confidence sequences of Waudby-Smith et al. (2023) and Maharaj et al. (2023), and the mixture sequential probability ratio test of Johari et al. (2022). Setting the total sample size to $k^ \cdot n_z$, where $n_z$ is the fixed-sample size for allocation ratio $r$, hits empirical power within approximately 3 percentage points of target in Gaussian simulations. The correction factor depends on the allocation ratio $r$ only through $t_0 = m/n_z(r)$. We study sensitivity to the burn-in parameter and show that the correction saves 8–20% of the last-point sample budget across the operating range.
Read the full abstract

Statistical Methodology

Logarithmic energy distances and Gini covariance for Hilbert-valued random elements

For $\alpha\in(0,2)$, the generalized energy distance and the Gini covariance statistic are based on kernels of the form $(x,y)\mapsto \|x-y\|^\alpha$, where $\|\cdot\|$ denotes the norm in a real separable Hilbert space. This paper investigates the boundary regime $\alpha\downarrow 0$. After suitable normalization, the corresponding energy distance converges to a logarithmic energy distance involving the kernel $(x,y)\mapsto\log\|x-y\|$. We establish that the resulting logarithmic energy distance retains the fundamental characterization property of ordinary energy distances in separable Hilbert spaces and derive a representation in terms of Gaussian-kernel maximum mean discrepancies. Motivated by this representation, we introduce a logarithmic Gini covariance for the $k$-sample problem and investigate its structural and asymptotic properties. In particular, we derive a representation in terms of pairwise logarithmic energy distances, establish a characterization theorem for equality of distributions, develop asymptotic null and alternative theory for the corresponding empirical statistic, and discuss permutation-based implementation. The logarithmic framework reveals a new boundary phenomenon within the family of energy-type statistics and provides connections with kernel methods, functional data analysis, and high-dimensional inference.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

Gaussian width is a central geometric complexity measure in high-dimensional probability, compressed sensing, convex optimization, and learning theory. It quantifies the average extent of a set along random directions, thereby capturing the effective dimension of constraint sets, hypothesis classes, and descent cones. However, this notion is intrinsically Euclidean. Statistical models instead carry a natural Riemannian geometry induced by the Fisher information metric, where directions are scaled according to statistical distinguishability rather than ambient Euclidean length. We introduce Fisher width, a Fisher-geometric analogue of Gaussian width for statistical manifolds. At a parameter point $\theta$, Fisher width replaces the Euclidean identity by the local metric tensor $G(\theta)^{1/2}$, measuring the Gaussian width of the Fisher-rescaled set. This makes the resulting quantity sensitive to local statistical curvature and invariant under smooth reparameterizations. We develop the basic theory of Fisher width, showing that it retains key structural features of Gaussian width, including concentration, metric perturbation stability, and spectral comparison bounds with the Euclidean baseline, while also capturing anisotropic geometric effects invisible to Euclidean measures. As an application, we prove a generalization bound for Fisher-Lipschitz hypothesis classes and propose computable estimators, which we evaluate empirically on MNIST across three model classes. Fisher width is to statistical manifolds what Gaussian width is to Euclidean convex bodies. This work lays the foundation for studying complexity and learning on curved statistical manifolds.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.PR

Finite-Time Queue Peak Laws in Stochastic Networks: Logarithmic Scaling After Geometric Thresholds

We study finite-horizon queue peaks in generalized switches, a standard stochastic-network model in which many queues share constrained service resources. Arrivals may be dependent, nonstationary, and responsive to the system history; the only load condition is uniform interior slack, meaning the conditional mean arrival vector stays in a fixed contraction of the capacity region. We show that this slack reshapes the finite-time peak law for drift-minimizing scheduling policies such as MaxWeight. The square-root envelope that is sharp without slack persists only up to a geometry-dependent threshold; beyond that threshold, the running maximum grows only logarithmically with the horizon, both with high probability and in expectation. The mechanism is self-normalization: in the current queue direction, the projected fluctuation scale is normalized by the stabilizing drift scale. This removes capacity geometry from the logarithmic coefficient, while geometry remains in the threshold. Matching lower bounds show that both the logarithmic term and a geometric threshold are unavoidable. When finite-time state-space collapse is available, the threshold can be sharpened using local bottleneck geometry. For generalized input-queued switches, we obtain finite-time peak bounds with tight logarithmic coefficients. Simulations illustrate the two-phase envelope, local geometric refinements, and variance-sensitive improvements predicted by the theory.
Read the full abstract

Statistics Theory

Conformal Prediction Intervals with Tail-Specific Guarantees

This paper extends classical conformal frameworks for constructing prediction intervals with global marginal coverage $1-\alpha$ to intervals that provide explicitly calibrated guarantees for the upper and lower tails separately. Focusing on split conformal prediction, we first construct lower and upper one-sided conformal intervals that achieve marginal validity, and then derive the induced two-sided interval by intersection. Theoretical results prove both tail-specific and global marginal coverage of the induced two-sided interval. Results are presented first for the exchangeable setting, where coverage has finite-sample guarantees, and then for non-exchangeable data, where guarantees are asymptotic. Simulation studies show that the proposed approach achieves improved directional calibration relative to classical two-sided intervals, especially relevant in skewed data. Finally, the benefit of the proposed framework is showcased in a financial application, where one aims for return maximization while seeking strict control on the left tail.
Read the full abstract

Statistics Applications

Sensitivity Bounds and Conservative Inference for Contagion under Latent Homophily

Whether connected units are similar because influence spreads across ties or because similar units form ties is a long standing problem. We study a fixed network with two waves of nodal outcomes. Rather than positing a parametric model for network formation, we consider identification of contagion under latent homophily as a selection bias problem. We define a focal component controlled direct effect (CDE) that holds a tie present, changes one alter's lagged outcome, and permits dependence on the remaining fixed network background. We show that the gap between the CDE and the observed connected dyad risk ratio is governed by how strongly a latent variable shifts the composition of connected dyads. Under stated mean exchangeability and sensitivity restrictions, we develop interpretable nonparametric bounds for the primary naturally connected target. For inference conditional on the observed network and baseline outcomes, heterogeneous dyad specific means can invalidate the usual inclusion exclusion variance estimator; we establish asymptotically conservative one sided limits under conditional actor dissociation and stated regularity conditions. A simulation study characterizes the bounds' error control and power. We apply the framework to the 2008 U.S. House votes on the Troubled Asset Relief Program. The connected contrast among legislators suggests vote contagion and survives mild latent homophily and outcome susceptibility.
Read the full abstract

Machine Learning (Statistics)

A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

Temporal difference (TD) learning with linear function approximation is a core method for policy evaluation. Its classical continuous-time description is an ordinary differential equation (ODE), which captures the asymptotic mean dynamics but neglects stochastic fluctuations determining the error floor. We introduce a stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise. The resulting model distinguishes the contraction dynamics governed by the projected Bellman operator from the influence of Markovian sampling. As a consequence, the model explains the constant-stepsize error floor through the interaction between Markovian long-run covariance and the contraction geometry of the projected Bellman operator.
Read the full abstract

All authors M. Forzo, E. Monzio Compagnoni, A. Russo, A. Pacchiano

Statistical Methodology

Spatial Disease Mapping and Disparity Detection Using Generative AI: An Amortized Bayesian Learning Framework

We introduce an amortized Bayesian framework for spatial boundary detection that generalizes posterior inference across areal graphs with varying numbers of regions and diverse adjacency structures. The underlying model couples a Poisson count likelihood with a covariate-driven rule to interrupt smoothing across dissimilar neighboring areas, utilizing a directed acyclic graph autoregressive (DAGAR) prior to capture residual spatial dependence. To approximate the target posterior distribution, a neural engine is trained on simulated maps: a permutation-invariant summary network encodes graph-aware representations of the observed counts, offsets, covariates, and adjacency matrices, while a conditional normalizing flow generates the approximate posterior draws. Simulation studies demonstrate accurate parameter recovery, near-nominal interval coverage, well-calibrated posterior predictive behavior, and informative posterior boundary probabilities. Benchmarking against Markov chain Monte Carlo (MCMC) confirms close agreement regarding primary boundary evidence, and an ablation study validates the inclusion of model-guided graph summaries. Finally, applications to Glasgow respiratory disease and California lung cancer data demonstrate that a single trained neural engine can be seamlessly deployed across real-world maps with distinct graph structures, yielding boundary conclusions consistent with established localized smoothing analyses.
Read the full abstract

Statistical Methodology

Bayesian Threshold-Aligned Joint Disease Progression Modeling for Alzheimer's Disease

Alzheimer's disease is characterized by the progressive accumulation of amyloid-$\beta$ and tau followed years later by cognitive impairment. Despite this established motif, substantial subject-level variability exists in the age of pathological progression and the onset of cognitive symptoms. To understand the source of this variation, subjects must be aligned across heterogeneous disease timelines via frameworks that jointly model disease progression and time to cognitive impairment with reference to landmark positivity thresholds. Existing neurodegenerative disease progression models rely on restrictive parametric forms, fail to anchor disease timelines to positivity thresholds, and decouple biomarker trajectories from cognitive survival endpoints. To address these limitations, we introduce the Bayesian Threshold-Aligned Joint Disease Progression Model (B-TAJ DPM). This generative, semi-parametric framework models multivariate disease progression trajectories over latent disease timelines anchored at landmark positivity thresholds. Crucially, the framework integrates a survival model to link pathological progression to cognitive impairment. Posterior inference and posterior predictions for unseen subjects are carried out in open-source software. Simulation studies demonstrate excellent estimation accuracy and interval coverage. When applied to Alzheimer's Disease Neuroimaging Initiative data, B-TAJ DPM characterizes non-linear progression patterns, quantifies subject-level variation in positivity age, and reveals links between age of tau positivity and acceleration of cognitive impairment.
Read the full abstract

All authors Rong Wu, Duygu Tosun, Isabella Hausle, Margo Heston, Aaron Wolfe Scheffler

Statistical Methodology

Undocumented Behavior in the gsynth R package and its Consequences for Three Published Studies

Prior to the version 1.3.1 update on CRAN in December 2025, gsynth, a popular R package for estimating Interactive Fixed Effects (IFE) models, could drastically and systematically underestimate standard errors. This underestimation would occur when two estimation options (inference = "parametric", and EM = TRUE) were used together, in which case the package would apply a parametric bootstrap procedure to Gobillon and Magnac (2016)'s IFE-EM estimator. The package ceased supporting this combination in December 2025, and the latest documentation now describes the parametric bootstrap as not suitable for use with the IFE-EM estimator due to a theoretical incompatibility. Our focus is an implementation error we identified in the pre-1.3.1 versions of gsynth: the parametric bootstrap used when EM = TRUE did not match the algorithm proposed in Xu (2017), using in-sample residuals instead of out-of-sample errors. We show that this implementation error can cause underestimation by orders of magnitude. We conduct an empirical Monte Carlo study using randomly assigned placebo treatments on a series of state-level panel datasets, and show that gsynth could yield high false positive rates in realistic settings. We identify three papers published in the American Political Science Review that are affected by this behavior. Reanalyzing the relevant sections of these papers, we show that (i) correcting the implementation error renders most findings insignificant, and (ii) using Xu (2017)'s Generalized Synthetic Control method in place of IFE-EM renders every finding insignificant.
Read the full abstract

Statistical Methodology

Spatial prediction of environmental processes using random forests: How best to account for spatial dependence?

Geostatistical spatial prediction for environmental processes is typically undertaken using Gaussian process models via Kriging, while machine learning (ML) algorithms are state-of-the-art for non-spatial prediction. An exciting recent fusion of these ideas imbibes traditional ML algorithms with the capacity to deal with spatial autocorrelation, leading to improved predictive performance. A range of approaches have been proposed, including fusion with Gaussian processes, observation-driven correlation structures, spatial basis functions and local geographical fitting. However, there has been no numerical comparison of their relative predictive performances, which is needed to advise environmental scientists on the optimal approach to use. This paper fills this knowledge gap, and focuses on random forests as the ML algorithm because they are more computationally and conceptually straightforward to implement than deep learning algorithms. The results from two studies are presented, the first being a controlled simulation experiment investigating whether any single approach is consistently superior across different spatial autocorrelation types. The second study focuses on the prediction of air pollution concentrations within a tuberculosis prevalence study in Blantyre, Malawi. The results show that whilst no single approach is universally superior, utilising spatial basis functions appears to perform consistently well across both the simulation and real data studies.
Read the full abstract

All authors Duncan Lee, Vinny Davies, Helen R. Savage, Hussein Twabi, Marriott Nliwasa, Peter MacPherson

Machine Learning (Statistics)

Tensor-based second-order causal discovery

Causal discovery seeks to uncover the causal dependencies among variables. For this purpose, we propose an algorithm called Tensor-based Second-order Causal Discovery (TSCD). Its input is a tensor obtained from the covariance matrices of observational and interventional data. Assuming the causal dependencies follow a linear structural equation model on a directed acyclic graph (DAG), TSCD outputs the DAG and the functions on its edges, requiring only that the noise variables are uncorrelated. We also implement a version of the approach for nonlinear models. Our focus on second-order statistics (via the covariance matrices) is motivated by their statistical and computational efficiency relative to higher-order moments, their identifiability relative to first-order statistics, and that they work regardless of whether the variables are Gaussian. We show that TSCD has identifiable causal order and parameters from a number of interventions that is logarithmic in the number of variables. Experiments show that TSCD is robust to noise, competitive with existing methods, and scales to hundreds of variables.
Read the full abstract

Statistics Applications

Model-based clustering of compositional trajectories for the analysis of mobility data

Understanding urban mobility patterns is crucial for designing efficient and sustainable transportation systems. Motivated by an application to the municipality of Padova and its surroundings, we propose a novel statistical framework for the analysis and clustering of mobility trajectories derived from telephonic data. We introduce a compositional representation of individual movements that integrates the uncertain device location with information on the surrounding road network, encoding at each time point the proportions of different road types compatible with the observed position. This formulation naturally accounts for measurement uncertainty and yields trajectories evolving in the simplex. To model these data, we develop a state-space framework for compositional time series that captures both the telephonic measurement error and the temporal dynamics of the latent mobility process. Building on this representation, we propose a model-based clustering approach based on mixtures of state-space models to identify groups of trajectories with similar evolution. This allows us to aggregate individual movements into interpretable mobility patterns at the population level. The results of the case study demonstrate the ability of the approach to uncover meaningful mobility behaviors, providing insights that are potentially relevant to policy makers.
Read the full abstract

Machine Learning (Statistics)

Fast Nonparametric Conditional Independence Testing via Two-Stage Regression

Constraint-based causal discovery relies on repeated conditional independence tests, but fast nonparametric tests often sacrifice calibration, especially when variables depend on the conditioning set through nonlinear relationships. We introduce BLITZ (Broad-to-Local Independence Testing via residualiZation), a nonparametric conditional independence test designed to run well under a second while maintaining the accuracy needed for the thousands of queries performed by constraint-based causal discovery algorithms. BLITZ first removes broad smooth dependence on the conditioning set using low-order polynomial regression, then applies a small nonlinear feature map and residualizes those features with shallow tree regressions. The resulting statistic tests residual cross-covariance, with a moment-matched chi-square approximation to the null distribution. We show theoretically that the two-stage design reduces the effective complexity faced by the tree residualizers, allowing shallow trees to control residual conditional-mean bias while avoiding excessive overfitting. In simulations, BLITZ provides better null calibration than fast kernel, random-feature, and regression-based competitors while remaining among the fastest methods tested. In causal discovery experiments on synthetic graphs and flow-cytometry data, BLITZ yields more reliable endpoint orientations among retained adjacencies and competitive structural recovery. These results suggest that broad-to-local residualization is a practical route to calibrated, scalable nonparametric conditional independence testing for causal discovery.
Read the full abstract

Machine Learning (Statistics)

Differential Privacy of Gaussian Process Posterior Sampling

We study the privacy of releasing functional posterior sample paths from a Gaussian process (GP) when the entire training set including covariates and responses is private. Unlike standard differential-privacy (DP) mechanisms that inject external noise, posterior sampling is intrinsically random and we show that this randomness provides useful privacy guarantees. We derive Rényi-DP guarantees separating privacy leakage through the posterior mean from a distinct channel induced by the data-dependent posterior covariance. The analysis identifies effective ridge regularisation and covariance scale as the principal privacy-controlling quantities and yields sharper guarantees in several regimes of practical interest as well as extensions to repeated and adaptive releases. Membership inference attacks confirm the predicted dependence on regularisation, covariance scale and the number of released paths. Utility experiments on downstream posterior sampling tasks identify noisy observation regimes where privacy-compatible regularisation preserves useful samples. Finally we identify large-data asymptotic regime in which the privacy parameter and posterior mean-square risk vanish simultaneously, yielding privacy for free. Together, these results provide a comprehensive characterisation of privacy and utility of GP posterior sampling.
Read the full abstract

Statistics Applications

Understanding Long-Term Dynamics of Individual Metro Usage: A Hidden Semi-Markov State Framework with Survival Analysis

Understanding how individual metro usage evolves over multi-year horizons is essential for transit planning and passenger retention. However, existing approaches typically characterize mobility patterns as static clusters or short-term variability, leaving the lifecycle dynamics of transit participation underexplored. This study proposes a state-based lifecycle modeling framework that integrates Hidden Semi-Markov Models (HSMM) with discrete-time survival analysis to characterize the evolution of individual metro mobility. The HSMM infers latent mobility states with explicit duration distributions and a transition matrix governing regime changes, while the survival component models exit and re-entry events via state-dependent hazard functions conditioned on mobility-state trajectories and behavioral history. Applied to four years of smart card data from the Shanghai metro system (2021-2024), the framework enables the identification of interpretable mobility states, the characterization of transition dynamics, and the quantification of state-dependent exit and re-entry processes. The analysis reveals five robust mobility states with a directional transition hierarchy centered on an occasional-usage gateway state, and fundamentally different temporal mechanisms governing disengagement and return: exit hazard is state-dependent but duration-independent, whereas re-entry hazard decays sharply with inactivity length. These findings provide a methodological foundation for lifecycle-oriented mobility analysis and practical guidance for transit operators to identify at-risk users and time retention interventions.
Read the full abstract

All authors Bingxun Wang, Valeria Maria Urbano, Shan He, Yang Chen, Wei Liu, Zhibin Jiang, Piercesare Secchi

Statistical Methodology

Spatial mixed models for assessing environmental exposure effects on the microbiome

The influence of environmental exposures, such as air pollution, on human health has become increasingly recognized. A growing body of evidence suggests that the microbiome may mediate these effects, explaining the relationship between the environment and host biology. However, the impact of environmental exposures on the microbiome is not yet fully understood, and statistical modeling in this context is challenged by complex dependency structures. In particular, microbiome data exhibit spatial dependencies across sampling regions as well as ecological correlations among microbial taxa, which, if ignored, can substantially reduce detection power, leading to missed true signals. We introduce a novel spatial mixed modeling framework for microbiome data that accounts for both region-level spatial dependency and taxon-level ecological dependency using conditional autoregressive priors. Through simulations, we demonstrate that this framework outperforms existing methods that ignore such dependencies, by achieving high detection power in feature selection while maintaining low false positive rates and reduced mean squared error in estimation. Applied to two real studies-data from Food and Microbiome Longitudinal Investigation study and lung microbiome dataset-with fine particulate matter (PM_2.5) exposures, our model identified genera, which are known to be involved in pollution-related health outcomes, as well as novel taxa that may mediate host responses to air pollution. This novel approach offers a powerful and flexible tool for uncovering biologically meaningful associations in complex environmental data.
Read the full abstract

All authors Sooran Kim, Chan Wang, Soyoung Kwak, Fares Darawshy, Alexander Bain, Leopoldo N. Segal, Jiyoung Ahn, Huilin Li

Statistical Computation

Nested Sampling: A Critical and Comprehensive Theoretical Guide

The nested sampling (NS) technique has gained widespread attention, particularly in cosmology and astronomy, due to its ability to efficiently explore high-likelihood regions - a feature akin to an implicit likelihood optimization that underlies its success. While the full theoretical derivation of NS is complex and involves several approximations, the central challenge lies in sampling from the likelihood-constrained priors, which is crucial for its performance. This work provides a comprehensive and detailed exposition of NS derivation, clarifying both its theoretical foundations and practical challenges. We provide a thorough description of the NS procedure, emphasizing both its strengths and potential limitations. The connections between NS, importance sampling (IS), and vertical Monte Carlo are also discussed in detail. In doing so, this work seeks to deepen understanding of the method and to foster the development of future enhancements, novel variants, and more efficient implementations across a wide range of scientific applications. Thus, the main contribution of this work is twofold: it serves both as a tutorial for newcomers to the field and as a critical review for experienced practitioners.
Read the full abstract

Statistical Methodology

Subgroup analysis in randomized controlled trials with binary outcomes: dilution and logic-respecting properties

Subgroup analysis is routinely used in randomized controlled trials to examine whether treatment effects are homogeneous across patient subgroups or differ because of treatment-effect heterogeneity. In this paper, we investigate the properties of the odds ratio and the relative response in subgroup analyses with binary outcomes, extending previous work with new theoretical insights and methodological developments. We establish several new theorems that characterize how the odds ratio for the overall population changes in both magnitude and direction when two subgroups are combined. These results further confirm that the odds ratio is inappropriate as an efficacy measure in this subgroup setting, whereas the relative response is appropriate. We also present the formal relationship between the odds ratio and the relative response, and clarify their differences in terms of the logic-respecting property, that is, whether the overall efficacy lies between the subgroup efficacies, and the dilution property, that is, whether mixing subgroups moves the overall odds ratio toward 1. Although the odds ratio is generally not logic-respecting, it may behave approximately like a logic-respecting efficacy measure under certain conditions. To illustrate our findings, we present an illustrative example based on clinical trial data and discuss its implications for subgroup analysis in randomized controlled trials.
Read the full abstract

Statistics Applications

Distributed Experimental Design: Bayes-optimal Fusion of Local Designs

We develop a decision-theoretic framework for distributed Bayesian experimental design in which local agents evaluate candidate experiments using expected information gain and transmit their local design decisions to a fusion center. Unlike centralized Bayesian design, where all likelihood components and information-gain values are available to a single planner, the fusion center in the distributed setting chooses a global experiment from compressed local recommendations. We derive the Bayes-optimal fusion rule, which selects the experiment with largest conditional expected centralized information gain given the observed local design decisions. This rule is analogous in spirit to optimal fusion rules in distributed detection, but differs fundamentally because the underlying utility is expected information gain and the resulting loss is information-gain regret rather than classification error. We also establish information-loss bounds and identify conditions under which the decision-only fusion rule is asymptotically equivalent to the centralized design. Numerical experiments show that Bayes-optimal fusion closely approximates the centralized oracle, whereas majority voting can be highly suboptimal when a minority of sites carry disproportionate information.
Read the full abstract

Statistical Methodology

On Response-Adaptive Targeting Strategies for Multi-Treatment Experiments

Response-adaptive randomization (RAR) in clinical trials aims to improve ethical and statistical efficiency by dynamically allocating patients to treatments based on observed outcomes. While RAR based on a target optimal allocation have been extensively studied for two-arms settings, their extension to multi-treatment experiments ($K \geq 2$) remains theoretically fragmented, with most existing methods focusing on specific algorithms or restricted target allocations. In this paper, we introduce a unified framework for response-adaptive targeting, the $\alpha$-Rebalancing Targeting Strategies ($\alpha$RTS), which generalize the ERADE two-armed strategy of Hu et al. [2009]. We prove that all designs in this family share fundamental asymptotic properties: strong consistency, asymptotic normality of allocation proportions and treatment effect estimators, and asymptotic efficiency. To address sparse target regimes (where some treatments are asymptotically eliminated), we further propose $\alpha$RTS with Forced Exploration, a variant that guarantees infinite sampling for all treatments while preserving the asymptotic guarantees. Extensive simulations illustrate the finite-sample behavior of $\alpha$RTS variants in a 3-armed context, highlighting in particular the critical role of forced exploration in sparse settings.
Read the full abstract

Statistics Applications

Tail Dependence in EU Carbon Markets: Graphical Models of Extremes for EUA Futures

Understanding how extreme price movements propagate across financial and energy markets is critical for risk management and regulatory design in the EU Emissions Trading System (EU ETS). We apply Hüsler-Reiss graphical models of extremes to a system of 20 daily variables centred on EU allowances futures across Phases 3 and 4 of the EU ETS (2013–2025), with a Gaussian graphical model as the average-dependence baseline. The tail networks are structurally distinct from the average dependence network: substantially denser, organized around different central nodes, and governed by within-sector homophily that binds sector boundaries more tightly than at the average-dependence level. EU allowances futures are peripheral in the standard graphical model but achieve the highest centrality in the tail networks, while equity indices and major FX pairs follow the opposite trajectory. Exponential random graph models confirm equity and FX peripherality in tail networks across all sample periods and identify triadic closure during market downturns as a Phase~3 phenomenon that vanishes in Phase~4. The phase transition restructures the tail network without thinning it: average dependence contracts sharply while tail dependence persists, and crash contagion shifts from clustered to diffuse propagation. These findings have direct implications for hedge construction by compliance entities, stress-test calibration by regulators, and the design of systemic-risk monitoring tools for EU ETS markets.
Read the full abstract

Statistical Methodology

Double zero-inflated spatio-temporal modeling of daily precipitation under detection thresholds

Explaining precipitation behavior at daily scale is important for fine scale understanding of the mechanisms driving precipitation. However, this effort is challenging because of the frequent incidence of zeros. The challenge is amplified by the acknowledged incidence of two types of zeros – absence of precipitation as a dry event and absence of measured precipitation due to detection limits. In this work, we propose a multilevel spatio-temporal model which allows us to distinguish and explain the two types of zeros, as well as to model positive precipitation above the detection limit. The methodology combines a point mass at zero with probability modeled through a probit regression, a Gamma regression for latent positive precipitation amounts, and an observation mechanism subject to threshold-induced censoring. To capture spatial dependencies, Gaussian processes are employed in each regression model. Working within a Bayesian framework, we can obtain a rich range of inference with exact uncertainty. In particular, we provide model-based inference tools to compare and quantify differences between the true precipitation process and its observed counterpart across relevant characteristics. We apply our model to the analysis of daily spring observations at 70 sites over 15 years from the Ebro River Basin in northeastern Spain. Our findings indicate that the threshold strongly affects the occurrence of observed precipitation, especially in humid regions. While its impact on total accumulated amounts is small, it can exert a relevant effect on upper quantiles.
Read the full abstract

All authors Juan Marcen-Gutierrez, Jorge Castillo-Mateo, Alan E. Gelfand, Jesús Asín, Ana C. Cebrián

Machine Learning (Statistics)

Geometrical fairness in graph neural networks

Graph-based learning methods have become increasingly prominent due to their strong performance across diverse applications. Among these, recent frameworks grounded in diffusion processes provide a unifying perspective that extends traditional graph neural network formulations while addressing limitations of standard message-passing mechanisms. Despite these advances, concerns remain regarding the fairness of such models, as they may propagate or amplify biases present in the data. In this work, we introduce a fairness-aware adaptation of graph-based diffusion by modifying the underlying Laplacian operator. Our approach incorporates multiple complementary transformations, including subspace projections, spectral adjustments, and frequency-based filtering, to mitigate bias-related components. Leveraging the intrinsic smoothing properties of graph diffusion, we provide a principled analysis of the resulting behavior and establish theoretical insights into fairness properties. We evaluate the proposed framework on both synthetic and real-world datasets, demonstrating that it achieves competitive performance while improving fairness metrics with limited additional computational cost.
Read the full abstract

All authors Arturo Pérez-Peralta, Sandra Benítez-Peña, Blas Kolic, Rosa E. Lillo

Statistics Theory

Non-asymptotic Tail Bounds for the Kostlan–Shub–Smale Field: Tensor PCA and Spherical $k$-Spin Complexity

This paper builds a hierarchy of explicit, non-asymptotic tail bounds for the supremum of the Kostlan–Shub–Smale (KSS) random field on the sphere, and applies it to two problems: Spiked Tensor PCA and the landscape of the spherical $k$-spin model. For Tensor PCA, we study the non-asymptotic statistical limits of estimating a rank-$R$ symmetric signal tensor of order~$k\ge 3$ and dimension~$d\ge 3$ from a single Gaussian observation at signal-to-noise ratio~$\lambda$, through the profile maximum likelihood estimator, the MLE restricted to normalized rank-$R$ tensors of coherence at least~$\kappa$. Our analysis uses a single reduction: a deterministic geometric inequality (the Tube Method) and a rank-reduction step bound the estimation error by the supremum of the canonical KSS field, which the Kac–Rice formula turns into a Gaussian integral against the expected absolute characteristic polynomial of a shifted Gaussian Orthogonal Ensemble, controlled in turn by the four explicit tail bounds of our hierarchy (three from a Mehta–Fyodorov representation, one from a Ben Arous–Dembo–Guionnet large deviation). The same reduction yields two results, each with explicit constants. For estimation, a finite-$(k,d)$ error bound recovers the asymptotically optimal rate~$\sqrt{d\log k}$ of Perry, Wein and Bandeira, with explicit dependence on the rank~$R$ and the coherence~$\kappa$. For the landscape, a two-sided non-asymptotic bracketing of the annealed complexity of the spherical $k$-spin Hamiltonian recovers the Auffinger–Ben Arous–Černý complexity function in the high-dimensional limit.
Read the full abstract

Statistics Theory · Cross-listed from cs.DS

Spectral recovery of a planted triangle-dense subgraph

Given a simple graph on $n$ vertices and a parameter $k$, the triangle-densest-$k$-subgraph problem is known to be computationally hard in the worst case. To circumvent the computational hardness, we study an average-case model where a triangle-dense subgraph on $k$ vertices is planted in an Erdős-Rényi random graph on $n$ vertices. For the recovery of the planted subgraph, we propose a simple spectral algorithm and a semidefinite program, both of which use a graph matrix whose entries are local signed triangle counts. Theoretical guarantees for these algorithms are established through spectral analysis of the graph matrix. Finally, we provide evidence showing a statistical-to-computational gap analogous to that for the planted clique problem. The computational threshold in terms of the subgraph size $k$ is at least $\sqrt{n}$ in the framework of low-degree polynomial algorithms, while the information-theoretic threshold is at most logarithmic in $n$.
Read the full abstract

Statistical Methodology

Proximal Mediation Analysis with Hidden Recanting Witnesses

Mediation analysis is essential for decomposing the causal effect of a treatment into direct and indirect pathways. However, many practical settings rely on the stringent assumption that recanting witnesses, defined as treatment-induced mediator-outcome confounders, are either absent or fully known a priori. Such a requirement is often untenable, especially when these variables remain unobservable due to measurement difficulties or privacy constraints. In this paper, we leverage proximal causal inference to develop three novel identification strategies to address the challenge of identifying path-specific effects in the presence of unknown recanting witnesses. Building on this, we develop a semiparametric inference framework that derives the efficient influence function and proposes a proximal multiply robust estimator, which remains consistent if at least one set of nuisance models is correctly specified. When all nuisance models are correctly specified and converge at appropriate rates, the estimator is asymptotically normal and achieves the semiparametric efficiency bound. We provide a minimax optimization-based debiased machine learning procedure for point estimation and constructing valid confidence intervals. The performance of the proposed methods is demonstrated by simulation studies and a real data application.
Read the full abstract

Statistics Applications · Cross-listed from physics.soc-ph

Public transit gains and spatially uneven travel demand changes after NYC congestion pricing

New York City implemented the nation's first cordon-based congestion pricing program in January 2025, providing an opportunity to evaluate how system-wide urban mobility responds to large-scale pricing interventions. Because such policies generate spillovers across modes and locations, credible control groups are difficult to construct. We address this challenge using time series foundation models to generate probabilistic counterfactual demand forecasts with calibrated uncertainty. Applying this framework to bus, subway, and aggregate trip volume data, we find that post-policy bus and subway ridership increased significantly relative to expected no-policy demand, while overall travel demand decreased modestly. The effects are spatially heterogeneous: while reductions in overall travel demand are concentrated within the Congestion Relief Zone, transit gains extend beyond Manhattan's core. Socio-demographic analyses further reveal uneven adaptation across neighborhoods, highlighting spatial equity implications. Our framework provides a scalable approach for the uncertainty-aware evaluation of system-wide urban interventions when clean control groups are unavailable.
Read the full abstract

All authors Donghang Li, Dingyi Zhuang, Yunlin Li, Chenan Shen, Nina Cao, Yunhan Zheng, Shenhao Wang, Jinhua Zhao

Statistical Methodology · Cross-listed from cs.LG

FoundCause: Causal Discovery with Latent Confounders from Observational Data

Causal discovery from observational data remains challenging due to the need to recover directed structure and latent confounding without interventions. We propose FoundCause, an amortized causal discovery model trained entirely on synthetic data that maps datasets directly to causal graphs in a single forward pass. By learning from large collections of simulated structural causal models, FoundCause captures transferable statistical patterns that generalize beyond individual datasets. The architecture incorporates several key inductive biases for causal discovery. It uses a permutation-invariant transformer encoder with alternating attention over samples and variables to jointly model cross-variable dependence and per-variable distributions. Pairwise statistical features derived from classical asymmetry measures are injected through statistics-conditioned attention, guiding the model toward known causal signals. A factorized decoder separates edge existence from direction, while a triangular refinement module enables reasoning over higher-order causal motifs such as chains and colliders. In addition, a dedicated confounder module based on learnable latent tokens explicitly models hidden common causes, and the model explicitly handles missing data via its masked input representation. To our knowledge, FoundCause is the first amortized causal discovery approach to explicitly model latent confounding. FoundCause outperforms 11 classical non-amortized methods (e.g., PC, GES, NOTEARS-style optimization) and 4 amortized causal discovery methods on 15 real-world datasets, achieving +9.6% improvement in $F_1$, +1.2% in AUROC, and an 18.9% reduction in structural Hamming distance relative to the strongest non-amortized methods, while performing inference in a single forward pass.
Read the full abstract

Statistical Methodology

Anytime-valid Optimal Policy Identification

We develop an anytime-valid framework for optimal policy identification from logged contextual bandit data. In many applied settings, the analyst wants to select the optimal policy from a candidate policy class $\Pi$, but data are generated by an externally determined logging policy that they do not control. The analyst may also wish to monitor evidence continuously, stopping once the optimal policy is clear rather than committing to a fixed sample size in advance. This paper addresses these challenges by constructing a time-indexed set $S_t$ that retains the true optimal policy set uniformly over time with high probability. The resulting procedure allows the analyst to monitor policy values, eliminate clearly suboptimal policies, and stop at data-dependent times without invalidating inference. When the optimal policy is unique, we define a stopping time for its identification and derive a sample-complexity bound scaling as $O\!\left(\frac{\log |\Pi|+\log\log(1/\Delta_{\min})}{\Delta_{\min}^2}\right)$, where $\Delta_{\min}$ is the gap between the best and second-best policy values. Simulations demonstrate that the anytime-valid approach can yield substantial sample savings relative to fixed-$N$ designs. An application to a large adaptive experiment on reducing misinformation online illustrates how the method provides a dynamic view as evidence on the optimal policy accumulates.
Read the full abstract