Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 22 December

Statistics Applications · Cross-listed from cs.AI

Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight

Reference labels for machine-learning benchmarks are increasingly synthesized with LLM assistance, but their reliability remains underexamined. We audit MedCalc-Bench, a clinical benchmark for medical score computation whose labels were partly derived with LLM assistance, and develop a scalable physician-in-the-loop stewardship pipeline to reassess them. At least 27% of test labels are likely erroneous or incomputable. On a 50-instance subset validated by physicians, our recomputed labels agree with physician ground truth 74% of the time (95% CI, 60-84%) versus 20% for the originals (95% CI, 11-33%). Using original labels to evaluate frontier LLMs underestimates accuracy by 16-23 percentage points. In a controlled reinforcement-learning experiment, a model trained on recomputed labels outperforms one trained on originals by 13.5 percentage points (95% CI, 10.6-16.6%) on physician-labeled instances, and this advantage extends to related medical tasks. LLM-assisted benchmarks can propagate systematic errors into both evaluation and post-training unless actively stewarded.
Read the full abstract

All authors Junze Ye, Daniel Tawfik, Alex J. Goodell, Nikhil V. Kotha, Mark K. Buyyounouski, Mohsen Bayati

Statistics Applications

An Adaptive Graphical Lasso Approach to Modeling Symptom Networks of Common Mental Disorders in Eritrean Refugee Population

Despite the significant public health burden of common mental disorders (CMDs) among refugee populations, their underlying symptom structures remain underexplored. This study uses Gaussian graphical modeling to examine the symptom network of post-traumatic stress disorder (PTSD), depression, anxiety, and somatic distress among Eritrean refugees in the Greater Washington, DC area. Given the small sample size (n) and high-dimensional symptom space (p), we propose a novel extension of the standard graphical LASSO by incorporating adaptive penalization, which improves sparsity selection and network estimation stability under n < p conditions. To evaluate the reliability of the network, we apply bootstrap resampling and use centrality measures to identify the most influential symptoms. Our analysis identifies six distinct symptom clusters, with somatic-anxiety symptoms forming the most interconnected group. Notably, symptoms such as nausea and reliving past experiences emerge as central symptoms linking PTSD, anxiety, depression, and somatic distress. Additionally, we identify symptoms like feeling fearful, sleep problems, and loss of interest in activities as key symptoms, either being closely positioned to many others or acting as important bridges that help maintain the overall network connectivity, thereby highlighting their potential importance as possible intervention targets.
Read the full abstract

All authors Elizabeth B. Amona, Indranil Sahoo, David Chan, Marianne B. Lund, Miriam Kuttikat

Statistical Methodology

Testing for Conditional Independence in Binary Single-Index Models

We wish to test whether a real-valued variable $Z$ has explanatory power, in addition to a multivariate variable $X$, for a binary variable $Y$. Thus, we are interested in testing the hypothesis $\mathbb{P}(Y=1\, | \, X,Z)=\mathbb{P}(Y=1\, | \, X)$, based on $n$ i.i.d.\ copies of $(X,Y,Z)$. In order to avoid the curse of dimensionality, we follow the common approach of assuming that the dependence of both $Y$ and $Z$ on $X$ is through a single-index $X^\top\beta$ only. Splitting the sample on both $Y$-values, we construct a two-sample empirical process of transformed $Z$-variables, after splitting the $X$-space into parallel strips. Studying this two-sample empirical process is challenging: it does not converge weakly to a standard Brownian bridge, but after an appropriate normalization it does. We use this result to construct distribution-free tests.
Read the full abstract

Statistical Methodology

A Markov Chain Modeling Approach for Predicting Relative Risks of Spatial Clusters in Public Health

Predicting relative risk (RR) of spatial clusters is a complex task in public health that can be achieved through various statistical and machine-learning methods for different time intervals. However, high-resolution longitudinal data is often unavailable to successfully apply such methods. The goal of the present study is to further develop and test a new methodology proposed in our previous work for accurate sequential RR predictions in the case of limited lon gitudinal data. In particular, we first use a well-known likelihood ratio test to identify significant spatial clusters over user-defined time intervals. Then we apply a Markov chain modeling ap approach to predict RR values for each time interval. Our findings demonstrate that the proposed approach yields better performance with COVID-19 morbidity data compared to the previous study on mortality data. Additionally, increasing the number of time intervals enhances the accuracy of the proposed Markov chain modeling method.
Read the full abstract

All authors Lyza Iamrache, Kamel Rekab, Majid Bani-Yagoub, Julia Pluta, Abdelghani Mehailia

Statistics Applications

Ant Colony Optimisation applied to the Travelling Santa Problem

The hypothetical global delivery schedule of Santa Claus must follow strict rolling night-time windows that vary with the Earth's rotation and obey an energy budget that depends on payload size and cruising speed. To design this schedule, the Travelling-Santa Ant-Colony Optimisation framework (TSaP-ACO) was developed. This heuristic framework constructs potential routes via a population of artificial ants that iteratively extend partial paths. Ants make their decisions much like they do in nature, following pheromones left by other ants, but with a degree of permitted exploration. This approach: (i) embeds local darkness feasibility directly into the pheromone heuristic, (ii) seeks to minimise aerodynamic work via a shrinking sleigh cross sectional area, (iii) uses a low-cost "rogue-ant" reversal to capture direction-sensitive time-zones, and (iv) tunes leg-specific cruise speeds on the fly. On benchmark sets of 15 and 30 capital cities, the TSaP-ACO eliminates all daylight violations and reduces total work by up to 10% compared to a distance-only ACO. In a 40-capital-city stress test, it cuts energy use by 88%, and shortens tour length by around 67%. Population-first routing emerges naturally from work minimisation (50% served by leg 11 of 40). These results demonstrate that rolling-window, energy-aware ACO has potential applications more realistic global delivery scenarios.
Read the full abstract

Statistical Computation

srvar-toolkit: A Python Implementation of Shadow-Rate Vector Autoregressions with Stochastic Volatility

We introduce srvar-toolkit, an open-source Python package for Bayesian vector autoregression with shadow-rate constraints and stochastic volatility. The toolkit implements the methodology of Grammatikopoulos (2025, Journal of Forecasting) for forecasting macroeconomic variables when interest rates hit the effective lower bound. We provide conjugate Normal-Inverse-Wishart priors with Minnesota-style shrinkage, latent shadow-rate data augmentation via Gibbs sampling, diagonal stochastic volatility using the Kim-Shephard-Chib mixture approximation, and stochastic search variable selection. Core dependencies are NumPy, SciPy, and Pandas, with optional extras for plotting and a configuration-driven command-line interface. We release the software under the MIT licence at https://github.com/shawcharles/srvar-toolkit.
Read the full abstract

Statistical Methodology

Possibilistic Inferential Models for Post-Selection Inference in High-Dimensional Linear Regression

Valid uncertainty quantification after model selection remains challenging in high-dimensional linear regression, especially within the possibilistic inferential model (PIM) framework. We develop possibilistic inferential models for post-selection inference based on a regularized split possibilistic construction (RSPIM) that combines generic high-dimensional selectors with PIM validification through sample splitting. A first subsample is used to select a sparse model; ordinary least-squares refits on an independent inference subsample yield classical t/F pivots, which are then turned into consonant plausibility contours. In Gaussian linear models this leads to coor-dinatewise intervals with exact finite-sample strong validity conditional on the split and selected model, uniformly over all selectors that use only the selection data. We further analyze RSPIM in a sparse p >> n regime under high-level screening conditions, develop orthogonalized and bootstrap-based extensions for low-dimensional targets with high-dimensional nuisance, and study a maxitive multi-split aggregation that stabilizes inference across random splits while preserving strong validity. Simulations and a riboflavin gene-expression example show that calibrated RSPIM intervals are well behaved under both Gaussian and heteroskedastic errors and are competitive with state-of-the-art post-selection methods, while plausibility contours provide transparent diagnostics of post-selection uncertainty.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from astro-ph.CO

Deep Learning for Primordial $B$-mode Extraction

The search for primordial gravitational waves is a central goal of cosmic microwave background (CMB) surveys. Isolating the characteristic $B$-mode polarization signal sourced by primordial gravitational waves is challenging for several reasons: the amplitude of the signal is inherently small; astrophysical foregrounds produce $B$-mode polarization contaminating the signal; and secondary $B$-mode polarization fluctuations are produced via the conversion of $E$ modes. Current and future low-noise, multi-frequency observations enable sufficient precision to address the first two of these challenges such that secondary $B$ modes will become the bottleneck for improved constraints on the amplitude of primordial gravitational waves. The dominant source of secondary $B$-mode polarization is gravitational lensing by large scale structure. Various strategies have been developed to estimate the lensing deflection and to reverse its effects the CMB, thus reducing confusion from lensing $B$ modes in the search for primordial gravitational waves. However, a few complications remain. First, there may be additional sources of secondary $B$-mode polarization, for example from patchy reionization or from cosmic polarization rotation. Second, the statistics of delensed CMB maps can become complicated and non-Gaussian, especially when advanced lensing reconstruction techniques are applied. We previously demonstrated how a deep learning network, ResUNet-CMB, can provide nearly optimal simultaneous estimates of multiple sources of secondary $B$-mode polarization. In this paper, we show how deep learning can be applied to estimate and remove multiple sources of secondary $B$-mode polarization, and we further show how this technique can be used in a likelihood analysis to produce nearly optimal, unbiased estimates of the amplitude of primordial gravitational waves.
Read the full abstract

Statistical Methodology

A Statistical Framework for Understanding Causal Effects that Vary by Treatment Initiation Time in EHR-based Studies

Standard practice in electronic health record (EHR)-based studies evaluating the comparative effectiveness of bariatric surgery relative to no surgery is to estimate and report a constant treatment effect across calendar time. However, real-world treatment strategies can evolve, particularly when comparators include standard of care or surgical procedures where techniques may improve, making it clinically important to ascertain whether efficacy of bariatric surgery has changed over time. Efforts to determine whether treatment efficacy itself is evolving are complicated by changing patient populations, with potential covariate shift in key effect modifiers. Through a comprehensive analysis of EHR data from Kaiser Permanente comparing two bariatric surgical procedures to standard of care, we develop a statistical framework to estimate calendar time-specific average treatment effects and describe both how and why effects vary across treatment initiation time in EHR-based studies. Our approach projects doubly robust, time-specific treatment effect estimates onto candidate marginal structural models and uses model selection to best describe how effects vary by calendar time. We further introduce a novel summary metric, based on standardization, to quantify the degree to which observed effect changes may be attributed to covariate shift in the population receiving treatment, as opposed to changes in underlying treatment efficacy.
Read the full abstract

All authors Luke Benz, Rajarshi Mukherjee, Rui Wang, David Arterburn, Heidi Fischer, Catherine Lee, Susan M. Shortreed, Alexander W. Levis, Sebastien Haneuse

Machine Learning (Statistics) · Cross-listed from cs.LG

Toward Scalable and Valid Conditional Independence Testing with Spectral Representations

Conditional independence (CI) is central to causal inference, feature selection, and graphical modeling, yet it is untestable in many settings without additional assumptions. Existing CI tests often rely on restrictive structural conditions, limiting their validity. Kernel methods using partial covariance operators offer a more principled approach but suffer from limited adaptivity and scalability. In this work, we explore whether representation learning can help address these limitations. Specifically, we focus on representations derived from the singular value decomposition of partial covariance operators and use them to construct a simple test statistic. We also introduce a bi-level contrastive algorithm to learn these representations. Our theory links representation learning error to test performance and establishes asymptotic validity and power guarantees. Experiments on real and synthetic data suggest that this approach offers a principled and statistically grounded path toward scalable CI testing, bridging kernel-based theory with modern representation learning.
Read the full abstract

All authors Alek Fröhlich, Vladimir R. Kostic, Karim Lounici, Daniel Perazzo, Daniel Tiezzi, Massimiliano Pontil

Machine Learning (Statistics) · Cross-listed from econ.GN

Structured Event Representation and Stock Return Predictability

We find that event features extracted by large language models (LLMs) are effective for text-based stock return prediction. Using a pre-trained LLM to extract event features from news articles, we propose a novel deep learning model based on structured event representation (SER) and attention mechanisms to predict stock returns in the cross-section. Our SER-based model provides superior performance compared with other existing text-driven models to forecast stock returns out of sample and offers highly interpretable feature structures to examine the mechanisms underlying the stock return predictability. We further provide various implications based on SER and highlight the crucial benefit of structured model inputs in stock return predictability.
Read the full abstract

Statistical Methodology

A Reduced Basis Decomposition Approach to Efficient Data Collection in Pairwise Comparison Studies

Comparative judgement studies elicit quality assessments through pairwise comparisons, typically analysed using the Bradley-Terry model. A challenge in these studies is experimental design, specifically, determining the optimal pairs to compare to maximize statistical efficiency. Constructing static experimental designs for these studies requires spectral decomposition of a covariance matrix over pairs of pairs, which becomes computationally infeasible for studies with more than approximately 150 objects. We propose a scalable method based on reduced basis decomposition that bypasses explicit construction of this matrix, achieving computational savings of two to three orders of magnitude. We establish eigenvalue bounds guaranteeing approximation quality and characterise the rank structure of the design matrix. Simulations demonstrate speedup factors exceeding 100 for studies with 64 or more objects, with negligible approximation error. We apply the method to construct designs for a 452-region spatial study in under 7 minutes and enable real-time design updates for classroom peer assessment, reducing computation time from 15 minutes to 15 seconds.
Read the full abstract

Machine Learning (Statistics)

Cluster-Based Generalized Additive Models Informed by Random Fourier Features

In developing data-driven modeling methodologies, there is an ongoing need to reconcile the strong predictive performance of opaque black-box models with the transparency required for critical applications. This work introduces an interpretable and computationally tractable regression framework for heterogeneous data by combining response-informed spectral representation learning with localized additive modeling. The method first fits a random Fourier feature regression model and constructs a spectral feature map from the learned amplitudes and adaptively resampled frequencies, so that the representation reflects predictive variation in the data. This representation is then compressed by principal component analysis to obtain a low-dimensional latent embedding, in which a Gaussian mixture model performs soft regime discovery. Within each regime, a cluster-specific generalized additive model captures nonlinear covariate effects through interpretable spline-based univariate smooth functions. The final predictor is formed as a soft mixture of these local additive models, enabling flexible modeling of a nonlinear, heterogeneous structure while preserving interpretability. Numerical experiments across several benchmark regression datasets show that the proposed method consistently improves upon classical globally interpretable baselines while remaining competitive with more flexible black-box models. Overall, the framework provides a unified approach to heterogeneous regression that combines predictive adaptivity with interpretable local covariate effects.
Read the full abstract

Statistics Theory

A hybrid-Hill estimator enabled by heavy-tailed block maxima

When analysing extreme values, two alternative statistical approaches have historically been held in contention: the block maxima method (or annual maxima method, spurred by hydrological applications) and the peaks-over-threshold. Clamoured amongst statisticians as wasteful of potentially informative data, the block maxima method gradually fell into disfavour whilst peaks-over-threshold-based methodologies climbed to the centre stage of extreme value statistics. This paper devises a hybrid method which reconciles these two hitherto disconnected approaches. Appealing in its simplicity, our main result introduces a new universality class of extreme value distributions that discards the customary requirement of a sufficiently large block size for the plausible block maxima-fit to an extreme value distribution. Natural extensions to dependent and/or non-stationary settings are mapped out. We advocate that inference should be drawn solely on larger block maxima, from which practice the mainstream peaks-over-threshold methodology coalesces: the asymptotic properties of the hybrid-Hill estimator herald more than its efficiency, but rather that a fully-fledged unified semi-parametric stream of statistics for extreme values is viable. A reduced-bias off-shoot of the hybrid-Hill estimator provably outclasses the incumbent maximum likelihood estimation that relies on a numerical fit to the entire sample of block maxima.
Read the full abstract

Statistics Theory · Cross-listed from cs.IT

Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models

We propose an orthogonal approximate message passing (OAMP) algorithm for signal estimation in the rectangular spiked matrix model with general rotationally invariant (RI) noise. We establish a rigorous state evolution that precisely characterizes the algorithm's high-dimensional dynamics and enables the construction of iteration-wise optimal denoisers. Within this framework, we accommodate spectral initializations under minimal assumptions on the empirical noise spectrum. In the rectangular setting, where a single rank-one component typically generates multiple informative outliers, we further propose a procedure for combining these outliers under mild non-Gaussian signal assumptions. For general RI noise models, the predicted performance of the proposed optimal OAMP algorithm agrees with replica-symmetric predictions for the associated Bayes-optimal estimator, and we conjecture that it is statistically optimal within a broad class of iterative estimation methods.
Read the full abstract

Statistics Applications · Cross-listed from hep-ex

The asymptotic distribution of the likelihood ratio test statistic in two-peak discovery experiments

Likelihood ratio tests are widely used in high-energy physics, where the test statistic is usually assumed to follow a chi-squared distribution with a number of degrees of freedom specified by Wilks' theorem. This assumption breaks down when parameters such as signal or coupling strengths are restricted to be non-negative and their values under the null hypothesis lie on the boundary of the parameter space. Based on a recent clarification concerning the correct asymptotic distribution of the likelihood ratio test statistic for cases where two of the parameters are on the boundary, we revisit the the question of significance estimation for two-peak signal-plus-background counting experiments. In the high-energy physics literature, such experiments are commonly analyzed using Wilks' chi-squared distribution or the one-parameter Chernoff limit. We demonstrate that these approaches can lead to strongly miscalibrated significances, and that the test statistic distribution is instead well described by a chi-squared mixture with weights determined by the Fisher information matrix. Our results highlight the need for boundary-aware asymptotics in the analysis of two-peak counting experiments.
Read the full abstract

All authors Clara Bertinelli Salucci, Hedvig Borgen Reiersrud, A. L. Read, Anders Kvellestad, Riccardo De Bin

Statistical Methodology

High dimensional matrix estimation through elliptical factor models

Elliptical factor models play a central role in modern high-dimensional data analysis, particularly due to their ability to capture heavy-tailed and heterogeneous dependence structures. Within this framework, Tyler's M-estimator (Tyler, 1987a) enjoys several optimality properties and robustness advantages. In this paper, we develop high-dimensional scatter matrix, covariance matrix and precision matrix estimators grounded in Tyler's M-estimation. We first adapt the Principal Orthogonal complEment Thresholding (POET) framework (Fan et al., 2013) by incorporating the spatial-sign covariance matrix as an effective initial estimator. Building on this idea, we further propose a direct extension of POET tailored for Tyler's M-estimation, referred to as the POET-TME method. We establish the consistency rates for the resulting estimators under elliptical factor models. Comprehensive simulation studies and a real data application illustrate the superior performance of POET-TME, especially in the presence of heavy-tailed distributions, demonstrating the practical value of our methodological contributions.
Read the full abstract

All authors Xinyue Xu, Huifang Ma, Hongfei Wang, Long Feng

Statistics Theory

Simple Cubic Variance Functions on $\R^n$, Part one

The classification of natural exponential families started with the paper \cite {Morri} where Carl Morris unifies six very familiar families by the fact that their variance functions are polynomials of degree less or equal to two. Extension of this classification to $\R^n$ and to degree three is the subject of this paper. Keywords: Actions of the group $GL(n+1,\R)$, classification of natural exponential families, multivariate Lagrange formula. variance functions.
Read the full abstract

Statistical Methodology

Scale-Invariant Robust Estimation of High-Dimensional Kronecker-Structured Matrices

High-dimensional Kronecker-structured estimation faces a conflict between non-convex scaling ambiguities and statistical robustness. The arbitrary factor scaling distorts gradient magnitudes, rendering standard fixed-threshold robust methods ineffective. We resolve this via Scaled Robust Gradient Descent (SRGD), which stabilizes optimization by de-scaling gradients before truncation. To further enforce interpretability, we introduce Scaled Hard Thresholding (SHT) for invariant variable selection. A two-step estimation procedure, built upon robust initialization and SRGD–SHT iterative updates, is proposed for canonical matrix problems, such as trace regression, matrix GLMs, and bilinear models. The convergence rates are established for heavy-tailed predictors and noise, identifying a phase transition where optimal convergence rates recover under finite noise variance and degrade optimally for heavier tails. Experiments on simulated data and two real-world applications confirm superior robustness and efficiency of the proposed procedure.
Read the full abstract

All authors Xiaoyu Zhang, Zhiyun Fan, Wenyang Zhang, Di Wang

Statistical Methodology

Interpolated Quantile Estimation: A Unified Framework Bridging Quantiles and the Mean

This paper develops and analyzes three families of estimators that continuously interpolate between classical quantiles and the sample mean. The construction begins with a smoothed version of the $L_1$ loss, indexed by a location parameter $z$ and a smoothing parameter $h \ge 0$, whose minimizer $\hat q(z,h)$ yields a unified $M$-estimation framework. Depending on how $(z, h)$ is specified, this framework generates three distinct classes of estimators: fixed-parameter smoothed quantile estimators, plug – in estimators of fixed quantiles, and a new continuum of mean – estimating procedures. For all three families we establish consistency and asymptotic normality via a uniform asymptotic equicontinuity argument. The limiting variances admit closed forms, allowing a transparent comparison of efficiency across families and smoothing levels. A geometric decomposition of the parameter space shows that, for fixed quantile level $\tau$, admissible pairs $(z, h)$ lie on straight lines along which the estimator targets the same population quantile while its asymptotic variance evolves. The theoretical analysis reveals two efficiency regimes. Under light-tailed distributions (e.g., Gaussian), smoothing yields a monotone variance reduction. Under heavy-tailed distributions (e.g., Laplace), a finite smoothing parameter $h^{*}(\tau) > 0$ strictly improves efficiency for quantile estimation. Numerical experiments – based on simulated data and real financial returns – validate these conclusions and show that, both asymptotically and in finite samples, the mean-estimating family does not improve upon the sample mean.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage

We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lovász extensions), we propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function. This loss function allows optimal trade-offs between conditional probabilistic coverage and the ”size” of the set, measured by a non-decreasing submodular function. We also propose several extensions that mimic loss functions and criteria for binary classification with asymmetric losses, and show how to naturally obtain sets with optimized conditional coverage. We derive efficient optimization algorithms, either based on stochastic gradient descent or reweighted least-squares formulations, and illustrate our findings with a series of experiments on synthetic datasets for classification and regression tasks, showing improvements over approaches that aim for marginal coverage.
Read the full abstract

Statistics Applications

Unraveling time-varying causal effects of multiple exposures: integrating Functional Data Analysis with Multivariable Mendelian Randomization

Mendelian Randomization is a widely used instrumental variable method for assessing causal effects of lifelong exposures on health outcomes. Many exposures, however, have causal effects that vary across the life course and often influence outcomes jointly with other exposures or indirectly through mediating pathways. Existing approaches to multivariable Mendelian Randomization assume constant effects over time and therefore fail to capture these dynamic relationships. We introduce Multivariable Functional Mendelian Randomization (MV-FMR), a new framework that extends functional Mendelian Randomization to simultaneously model multiple time-varying exposures. The method combines functional principal component analysis with a data-driven cross-validation strategy for basis selection and accounts for overlapping instruments and mediation effects. Through extensive simulations, we assessed MV-FMR's ability to recover time-varying causal effects under a range of data-generating scenarios and compared the performance of joint versus separate exposure effect estimation strategies. Across scenarios involving nonlinear effects, horizontal pleiotropy, mediation, and sparse data, MV-FMR consistently recovered the true causal functions and outperformed univariable approaches. To demonstrate its practical value, we applied MV-FMR to UK Biobank data to investigate the time-varying causal effects of systolic blood pressure and body mass index on coronary artery disease. MV-FMR provides a flexible and interpretable framework for disentangling complex time-dependent causal processes and offers new opportunities for identifying life-course critical periods and actionable drivers relevant to disease prevention.
Read the full abstract

All authors Nicole Fontana, Francesca Ieva, Luisa Zuccolo, Emanuele Di Angelantonio, Piercesare Secchi

Statistics Theory · Cross-listed from math.PR

Sharp Decoupling Inequalities for the Variances and Second Moments of Sums of Dependent Random Variables

Both complete decoupling and tangent decoupling are classical tools aiming to compare two random processes where one has a weaker dependence structure. We give a new proof for the complete decoupling inequality, which provides a lower bound for the sum of dependent square-integrable nonnegative random variables $\sum\limits^n_{i=1} d_i$ \[ \frac{1}{2} \mathbb E \left( \sum\limits^n_{i=1} z_i \right)^2 \leq \mathbb E \left( \sum\limits^n_{i=1} d_i \right)^2, \] where $z_i \stackrel{\mathcal{L}}{=} d_i$ for all $i\leq n$ and $z_i$'s are mutually independent. We will then provide the following sharp tangent decoupling inequalities \[\mathbb Var \left( \sum\limits^n_{i=1} d_i\right) \leq 2 \mathbb Var \left( \sum\limits^n_{i=1} e_i\right),\] and \[\mathbb E \left( \sum\limits^n_{i=1} d_i\right)^2 \leq 2 \mathbb E \left( \sum\limits^n_{i=1} e_i\right)^2 - \left[ \mathbb E \left( \sum\limits^n_{i=1} e_i\right) \right]^2,\] where $\{e_i\}$ is the decoupled sequences of $\{d_i\}$ and $d_i$'s are not forced to be nonnegative. Applications to construct Chebyshev-type inequality and Paley-Zygmund-type inequality, and to bound the second moments of randomly stopped sums will be provided.
Read the full abstract

Statistics Applications

Dyadic Flow Models for Nonstationary Gene Flow in Landscape Genomics

The field of landscape genomics aims to infer how landscape features affect gene flow throughout a population. Most landscape genomic frameworks assume the isolation-by-distance and isolation-by-resistance hypotheses, which propose that genetic dissimilarities between individuals increase as a function of distance and as a function of cumulative landscape resistance, respectively. While these hypotheses are valid in certain settings, other mechanisms may affect gene flow. For example, the gene flow of invasive species may depend on founder effects and multiple introductions. Such mechanisms are not considered in modern landscape genomic models. We extend dyadic models to allow for mechanisms that range-shifting and/or invasive species may experience by introducing dyadic spatially-varying coefficients (DSVCs) defined on pairs of sampled locations. The DSVCs allow the effects of landscape on gene flow to vary across space, capturing nonstationary and asymmetric connectivity. Additionally, we incorporate explicit landscape features as connectivity covariates, which are localized to specific regions of the spatial domain and may function as barriers or corridors to gene flow. Such covariates are central to colonization and invasion, where spread accelerates along corridors and slows across landscape barriers. The proposed framework accommodates colonization-specific processes while retaining the ability to assess landscape influences on gene flow. Our case study of the highly invasive cheatgrass (Bromus tectorum) demonstrates the necessity of accounting for nonstationary gene flow in range-shifting species.
Read the full abstract

All authors Michael R. Schwob, Nicholas M. Calzada, Justin J. Van Ee, Diana Gamba, Rebecca A. Nelson, Megan L. Vahsen, Peter B. Adler, Jesse R. Lasky, Mevin B. Hooten

Statistics Theory

Operator Tail Densities of Multivariate Copulas

Operator regular variation of a multivariate distribution can be decomposed into the operator tail dependence of the underlying copula and the regular variation of the univariate marginals. In this paper, we introduce operator tail densities for copulas and show that an operator-regularly-varying density can be characterized through the operator tail density of its copula together with the marginal regular variation. As an example, we demonstrate that although a Liouville copula is not available in closed form, it nevertheless admits an explicit operator tail-dependence function.
Read the full abstract

Statistical Methodology

A Universal Framework for Factorial Matched Observational Studies with General Treatment Types: Design, Analysis, and Applications

Matching is one of the most widely used causal inference frameworks in observational studies. However, all the existing matching-based causal inference methods are designed for either a single treatment with general treatment types (e.g., binary, ordinal, or continuous) or factorial (multiple) treatments with binary treatments only. To our knowledge, no existing matching-based causal methods can handle factorial treatments with general treatment types. This critical gap substantially hinders the applicability of matching in many real-world problems, in which there are often multiple, potentially non-binary (e.g., continuous) treatment components. To address this critical gap, this work develops a universal framework for the design and analysis of factorial matched observational studies with general treatment types (e.g., binary, ordinal, or continuous). We first propose a two-stage non-bipartite matching algorithm that constructs matched sets of units with similar covariates but distinct combinations of treatment doses, thereby enabling valid estimation of both main and interaction effects. We then introduce a new class of generalized factorial Neyman-type estimands that provide model-free, finite-population-valid definitions of marginal and interaction causal effects under factorial treatments with general treatment types. Randomization-based Fisher-type and Neyman-type inference procedures are developed, including unbiased estimators, asymptotically valid variance estimators, and variance adjustments incorporating covariate information for improved efficiency. Finally, we illustrate the proposed framework through a county-level application that evaluates the causal impacts of work- and non-work-trip reductions (social distancing practices) on COVID-19-related and drug-related outcomes during the COVID-19 pandemic in the United States.
Read the full abstract

All authors Jianan Zhu, Tianruo Zhang, Diana Silver, Ellicott Matthay, Omar El-Shahawy, Hyunseung Kang, Siyu Heng

Machine Learning (Statistics)

On Conditional Stochastic Interpolation for Generative Nonlinear Sufficient Dimension Reduction

Identifying low-dimensional sufficient structures in nonlinear sufficient dimension reduction (SDR) has long been a fundamental yet challenging problem. Most existing methods lack theoretical guarantees of exhaustiveness in identifying lower dimensional structures, either at the population level or at the sample level. We tackle this issue by proposing a new method, generative sufficient dimension reduction (GenSDR), which leverages modern generative models. We show that GenSDR is able to fully recover the information contained in the central $\sigma$-field at both the population and sample levels. In particular, at the sample level, we establish a consistency property for the GenSDR estimator from the perspective of conditional distributions, capitalizing on the distributional learning capabilities of deep generative models. Moreover, by incorporating an ensemble technique, we extend GenSDR to accommodate scenarios with non-Euclidean responses, thereby substantially broadening its applicability. Extensive numerical results demonstrate the outstanding empirical performance of GenSDR and highlight its strong potential for addressing a wide range of complex, real-world tasks.
Read the full abstract

Statistical Methodology

Integrating Prioritized and Non-Prioritized Structures in Win Statistics

Composite endpoints are frequently used as primary or secondary analyses in cardiovascular clinical trials to increase clinical relevance and statistical efficiency. Alternatively, the Win Ratio (WR) and other Win Statistics (WS) analyses rely on a strict hierarchical ordering of endpoints, assigning higher priority to clinically important endpoints. However, determining a definitive endpoint hierarchy can be challenging and may not adequately reflect situations where endpoints have comparable importance. In this study, we discuss the challenges of endpoint prioritization, underscore its critical role in WS analyses, and propose Rotation WR (RWR), a hybrid prioritization framework that integrates both prioritized and non-prioritized structures. By permitting blocks of equally-prioritized endpoints, RWR accommodates endpoints of equal or near equal clinical importance, recurrent events, and contexts requiring individualized shared decision making. Statistical inference for RWR is developed using U-statistics theory, including the hypothesis testing procedure and confidence interval construction. Extensions to two additional WS measures, Rotation Net Benefit and Rotation Win Odds, are also provided. Through extensive simulation studies involving multiple time-to-event endpoints, including recurrent events, we demonstrate that RWR achieves valid type I error control, desirable statistical power, and accurate confidence interval coverage. We illustrate both the methodological and practical insights of our work in a case study on endpoint prioritization with the SPRINT clinical trial, highlighting its implications for real-world clinical trial studies.
Read the full abstract

Sunday 21 December

Statistical Methodology

Testing for latent structure via the Wilcoxon–Wigner random matrix of normalized rank statistics

This paper considers the problem of testing for latent structure in large symmetric data matrices. The goal here is to develop statistically principled methodology that is flexible in its applicability, computationally efficient, and insensitive to extreme data variation, thereby overcoming limitations facing existing approaches. To do so, we introduce and systematically study certain symmetric matrices, called Wilcoxon–Wigner random matrices, whose entries are normalized rank statistics derived from an underlying independent and identically distributed sample of absolutely continuous random variables. These matrices naturally arise as the matricization of one-sample problems in statistics and conceptually lie at the interface of nonparametrics, multivariate analysis, and data reduction. Among our results, we establish that the leading eigenvalue and corresponding eigenvector of Wilcoxon–Wigner random matrices admit asymptotically Gaussian fluctuations with explicit centering and scaling terms. These asymptotic results enable rigorous parameter-free and distribution-free spectral methodology for addressing two hypothesis testing problems, namely community detection and principal submatrix detection. Numerical examples illustrate the performance of the proposed approach. Throughout, our findings are juxtaposed with existing results based on the spectral properties of independent entry symmetric random matrices in signal-plus-noise data settings.
Read the full abstract

Statistics Theory

Model-Agnostic Bounds for Augmented Inverse Probability Weighted Estimators' Wald-Confidence Interval Coverage in Randomized Controlled Trials

Nonparametric estimators, such as the augmented inverse probability weighted (AIPW) estimator, have become increasingly popular in causal inference. Numerous nonparametric estimators have been proposed, but they are all asymptotically normal with the same asymptotic variance under similar conditions, leaving little guidance for practitioners to choose an estimator. In this paper, I focus on another important perspective of their asymptotic behaviors beyond asymptotic normality, the convergence of the Wald-confidence interval (CI) coverage to the nominal coverage. Such results have been established for simpler estimators (e.g., the Berry-Esseen Theorem), but are lacking for nonparametric estimators. I consider a simple but practical setting where the AIPW estimator based on a black-box nuisance estimator, with or without cross-fitting, is used to estimate the average treatment effect in randomized controlled trials. I derive non-asymptotic Berry-Esseen-type bounds on the difference between Wald-CI coverage and the nominal coverage. I also analyze the bias of variance estimators, showing that the cross-fit variance estimator might overestimate while the non-cross-fit variance estimator might underestimate, which might explain why cross-fitting has been empirically observed to improve Wald-CI coverage even if both estimators converge to the same asymptotic normal distribution.
Read the full abstract