Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistical Methodology

stat.ME Watch Statistical Methodology (sign in to use)

Monday 12 October

Statistical Methodology · Cross-listed from math.ST

Fisher Information and Differentiability for $r$-Largest Generalized Extreme-Value Models

The $r$ largest observations of a block form a dependent random vector whose support moves with all three parameters of the $r$-largest generalized extreme-value ($r$GEV) model. For a fixed number $r$ of order statistics, we show that the model is differentiable in quadratic mean if and only if $k_0<1/2$, where $k=-\gamma$ is the shape parameter; the threshold is thus the same as for block maxima, because only the largest or the smallest retained order statistic can reach the moving boundary. In this range the Fisher information is positive definite and decomposes into the block-maximum information plus contributions of the lower order statistics, and it equals the negative expected Hessian, which we prove without using any estimator. The model is then locally asymptotically normal, whereas for $k_0\ge1/2$ root-$n$ local asymptotic normality fails. Numerically, using the five largest order statistics instead of the block maximum alone lowers the efficient asymptotic standard error by about $45\%$ for the shape parameter and by $20$–$24\%$ for the 100-year return level in heavy-tailed cases ($k_0\le-0.2$), but by only about $2\%$ for the return level when $k_0=0.4$.
Read the full abstract

Statistical Methodology

When Treatments Spill Over: A Neural Framework for Spatiotemporal Causal Effect Estimation

Real-world public health policies are often enacted by local jurisdictions, but their effects can ripple through connected populations. For causal evaluation, this creates a difficult spatiotemporal problem: intermittent policy updates create path-level treatments, and because spillover depends on neighboring treatment paths along with the focal-unit's own history, controlling time-varying confounding requires high-dimensional, multimodal information from both sources. To address this challenge, we develop a neural spatiotemporal spillover framework that extends potential-outcome causal analysis to networked policy-systems with unit-level effects and structured interference. The framework indexes each unit's potential outcomes through own-treatment and neighborhood spillover-exposure paths, yielding direct, spillover, and total effects. We introduce an LSTM-based representation-learning encoder that integrates each unit's own and neighborhood histories into compact sequential summaries for downstream modeling and causal estimation. These summaries feed a self-normalized kernel inverse-probability-weighting estimator for continuous spillover exposure. Beyond theoretical guarantees, simulations show improved performance over outcome-regression baselines; in a practically motivated county-level COVID-19 policy study, the framework finds both business-and-economic restrictions and education-and-childcare policies associated with lower subsequent case burden, with larger estimates for business-and-economic restrictions. Overall, it provides a scalable, policy-interpretable framework for evaluating unit-level effects in high-dimensional, multimodal spatiotemporal systems when interventions spill over across space and time.
Read the full abstract

Sunday 11 October

Statistical Methodology

The LR2 Lifetime Distribution: Rate Crossing, Reliability Properties, and Estimation

We study the LR2 one-parameter lifetime distribution obtained by mixing an exponential component of rate $\theta/2$ and a gamma component with shape four and rate $\theta^2$. The exponential mixing probability is $\theta^2/(1+\theta^2)$ for $\theta>0$. We derive the density, survival function, moments, transforms, truncated moments, quantiles, order-statistic distributions, mean residual life, and stress–strength reliability. Identifiability follows from the density at the origin, and strict monotonicity of the mean yields a unique moment estimator. A complete density-turning-point criterion and analytic initial and tail hazard expansions clarify the possible shapes. The component rates cross at one half. The hazard decreases initially for every positive parameter and eventually increases in the gamma-dominated regime; it eventually decreases in the exponential-dominated regime. We develop likelihood estimation, observed information, constrained expectation–maximization, spacing estimation, and inference under regularity conditions, and state the likelihood for independent right censoring. A reproducible Monte Carlo study with eight settings and 4,000 generated samples compares likelihood and moment estimation and evaluates log-scale confidence intervals. The analysis distinguishes mathematical properties from empirical validation and explains the restrictions associated with a fixed working time unit.
Read the full abstract

Statistical Methodology

Small area learning

Small area estimation (SAE) traditionally borrows strength across domains through a prespecified working model. We introduce small area learning (SAL), a learner-agnostic framework that formulates SAE as a multi-target semi-supervised learning problem and treats outcome prediction as a nuisance task. Areas may differ through unrestricted covariate shift and structured conditional shift, with information shared through common and area-specific components. We develop standard and cross-fitted estimators for low-complexity and flexible learners, unifying synthetic, model-based, and fully residual-corrected estimation. For area means, we establish consistency and asymptotic normality and provide variance estimators and confidence intervals. We extend the theory to settings with only aggregated population covariates and characterize the resulting aggregation bias. SAL reveals a bias–variance trade-off: partial residual correction can reduce mean squared error, whereas full correction yields estimators robust to working-model misspecification, potentially at higher variance. Simulations yield RMSE reductions of 34%–49% relative to direct estimation, with the best configuration varying across settings. An empirical application favours a cross-fitted estimator using a flexible learner under the chosen validation criterion. The numerical studies also illustrate how shrinkage can produce overly optimistic confidence intervals at the expense of strong modeling assumptions.
Read the full abstract

All authors Katarzyna Reluga, Nicola Salvati, Setareh Ranjbar, Nikos Tzavidis, Mark van der Laan

Statistical Methodology · Cross-listed from stat.CO

Jensen-Bregman split Gibbs sampling

Split Gibbs sampling (SGS) is a versatile tool to sample from complex posterior distributions, such as those arising in imaging inverse problems. However, existing SGS instances are limited to quadratic variable couplings, restricting their applicability to Gaussian likelihoods. This letter genearalizes SGS to non-Euclidean geometry by introducing a coupling kernel defined through a Jensen-Bregman divergence. This framework not only enables posterior sampling in these settings but also unifies several ad hoc sampling schemes from the literature under a principled theoretical foundation.
Read the full abstract

Statistical Methodology · Cross-listed from math.ST

Adaptive Robust Confidence Sets in High Dimensions

We study adaptive confidence sets for mean vectors of multivariate Gaussians with identity covariance under Huber contamination when the contamination proportion is unknown. We show that an information-theoretically optimal adaptive confidence set can be constructed by simultaneously quantifying uncertainty across all one-dimensional projections. The resulting confidence set achieves the optimal diameter and reveals a substantial adaptation cost compared with the minimax estimation error. Although statistically optimal, this construction is computationally infeasible. We establish a Statistical Query (SQ) lower bound showing that achieving diameter $O(m^{-1/2})$ requires either exponentially many queries or query tolerance $d^{-\Omega(m)}$. Thus, in contrast to robust mean estimation, adaptation to the unknown contamination proportion incurs both a statistical cost and a computational barrier within the SQ framework. Finally, we give a low-degree moment-matching algorithm that constructs an adaptive confidence set using $d^{O(m)}$ samples and time. Its diameter bound contains an additional $O(m^{-1/2})$ term beyond the information-theoretic rate, matching the dependence on $m$ in the SQ lower bound. Together, these results characterize the statistical limits of adaptive robust confidence sets under Huber contamination and quantify a computational barrier within the SQ framework.
Read the full abstract

All authors Ilias Diakonikolas, Chao Gao, Daniel M. Kane, Yuetian Luo

Statistical Methodology

One Goodness-of-Fit Test Is Not Enough: Error-Controlled Localization of Misfit in Logistic Risk Models

A goodness-of-fit test tells whether a logistic risk model fits, but the analyst also needs to know what is wrong: an overall risk that is too high is repaired by updating the intercept, risks that are too extreme by recalibration, a curved relation between score and risk by a flexible recalibration, and misfit within groups of patients only by revising the model. We split any misfit into these four parts, which are orthogonal and match the calibration hierarchy. Each part gets a group of score tests that sees that part only, and the failing parts are named by closed testing. In external validation, in large samples and for small departures, a part is wrongly named with probability at most the chosen level, and the highest part named points to the lightest update that removes the misfit. Checking a model on its development data never sees an error in the overall risk or the calibration slope, and with binary or skewed covariates it can mistake misfit within patient groups for a curved link. External validation sees all four parts, for any prediction model. In simulations it kept its error rate when the model was right and for small departures, and usually pointed to the right update. On intensive-care data its verdicts for one model differed in-sample, on similar patients and after transport.
Read the full abstract

2610.15649 · 11 Oct · 41 pages, 5 figures, 18 tables, including appendices with the proofs and additional studies. Code, per-data-set results and data: doi:10.5281/zenodo.23192535. The procedures are implemented in the R package ebrahim.gof (version 2.10.0, functions localize.gof and localize.external)

Statistical Methodology · Cross-listed from math.ST

Rank-based Model Average Estimator for Average Treatment Effects

Model averaging emerges as an attractive ensemble technique that strategically synthesizes all available information from potential candidate models to enhance estimation accuracy. Despite of having been widely practiced in various areas in biostatistics and economics, its application to causal inference is rather limited. We propose to estimate the average treatment effect (ATE) by forming the weighted combination of the robust rank-based estimators obtained from a series of candidate models, where the model weights, selected via the growing window cross-validation, dynamically adjust the importance of candidate models. The proposed robust rank-based model averaging (RRMA) is not only robust to handle potential outliers and heavy-tailed error distributions but also capable to reduce the risk of model mis-specification and enhance the flexibility and the accuracy of estimation. Theoretically, we establish the desirable asymptotic properties of RRMA, including the asymptotic optimality, consistency and asymptotic distribution. Moreover, motivated by the asymptotic distribution of RRMA, an easy-to-implement confidence interval is also developed for the ATE. Simulation experiments and an empirical application demonstrate the superiority of RRMA over existing competitors in estimation accuracy, coverage probabilities and widths of confidence intervals.
Read the full abstract

Statistical Methodology

Identifying and Measuring Treatment-Effect Heterogeneity across Sites through Within-Site Randomization

Parallel cluster-randomized trials can identify mean effects by site characteristics and risk differences between fixed predictors, while leaving residual heterogeneity and absolute prediction risk (mean squared error of site-effect predictions) unidentified. We acquire missing same-site cross-arm information by randomizing participants within selected sites. Under randomized site selection with compatible participant reference distributions and intervention versions, we characterize necessary and sufficient observation patterns for quadratic targets in a model with unrestricted joint distributions of site response means. From finite independent participant samples, we construct unbiased moments and exact covariances. Under regularity conditions, we prove consistency as informative paired site counts grow, allowing fixed within-site sample sizes. For a fixed predictor, we optimize paired-site selection and unequal cell counts using corrected risk variances, pilot error, and costs. In an International Stroke Trial information-deletion experiment, pilot allocation reduces the conditional variance of reconstructing the complete-information corrected risk estimate by 34.3% relative to uniform allocation with equal expected cost. In 40 Honduras villages, correction reduces estimated residual variation from 0.04081 to 0.00282 in squared log-consumption units for the same fixed explanatory basis; the explained share rises despite a smaller explained component, illustrating how participant sampling noise can inflate estimated residual variation.
Read the full abstract

Statistical Methodology

Deep Neural Mixed Models for Nonlinear Longitudinal Prediction with Subject-Specific Heterogeneity

Longitudinal health data often exhibit within-subject correlation, subject-specific heterogeneity, and complex nonlinear predictor–response relationships that may not be adequately captured by conventional mixed models. We propose a Deep Generalized Neural Network Mixed Model (DGNMM) that extends the GLMM framework by using a deep neural network to learn nonlinear population-level relationships while incorporating directly trainable, regularized subject-specific effects to capture individual heterogeneity. The model is estimated using prediction-oriented stochastic optimization with separate regularization of the neural-network parameters and subject-specific effects. This formulation allows us to investigate when increased nonlinear representation capacity provides additional predictive benefit in the presence of subject-specific heterogeneity. A simulation study varies population-level nonlinearity and subject-specific heterogeneity and compares DGNMM with GLMM, artificial neural network (ANN), Maity–Pal (MP), and GNMM alternatives for continuous and binary longitudinal outcomes. In the real-data application, DGNMM achieved the lowest MSPE for continuous depression and anxiety prediction, while alternative models remained competitive for binary outcomes. Exploratory subject-level analyses showed heterogeneous predictive gains across individuals, but the magnitude of the estimated subject-specific effects was not significantly associated with predictive gain. Overall, increased nonlinear representation capacity was most beneficial for continuous prediction under nonlinear response structures, while greater subject-specific heterogeneity did not systematically yield larger predictive gains.
Read the full abstract

Statistical Methodology

Online Change-Point Detection in Linear Regression with Autoregressive Errors via Streaming Variational Bayes

Large-scale streaming time-series regression requires methods that can detect structural changes while accounting for serially correlated errors and quantifying uncertainty about the current regime. We propose a streaming variational Bayes method for linear regression with autoregressive errors that combines online changepoint detection with sequential posterior inference. The proposed method processes observations in chronological mini-batches and uses a finite-mixture variational approximation to represent separate retention and reset configurations for the regression coefficients and innovation variance, while the autoregressive coefficients are shared across regimes and learned sequentially. This construction yields block-specific changepoint probabilities and posterior uncertainty summaries. Closed-form variational updates and mixture pruning enable scalable sequential inference without repeated full-data refits. In simulations, the proposed method maintains low empirical false-alarm rates under stable regimes, with posterior coverage improving as the sample size increases. Under repeated structural changes, it provides accurate type-specific changepoint detection, small changepoint location errors, and short detection delays relative to frequentist sequential monitoring procedures, while posterior coverage approaches nominal levels after an initial adaptation period. An application to high-frequency Bitcoin volatility data illustrates distinct adaptation of the regression and innovation-variance components while accounting for residual autoregressive dependence.
Read the full abstract

All authors Yujin Jeong, Chaeeun Park, Sihyeon Pyeon, Seonghyun Jeong, Taeyoung Park

Statistical Methodology

Approximation, Consistency, and Prediction in Generalized Neural Network Mixed Models for Longitudinal and Clustered Data

Generalized neural network mixed models (GNMMs) combine neural networks and mixed-effects models to analyze longitudinal and clustered data. Although GNMMs have shown good predictive performance in practice, their statistical properties are not well understood. Therefore, this paper studies several important properties of GNMM and provides theoretical support for its use. First, we show that the neural-network component can approximate an unknown nonlinear fixed-effect function arbitrarily well and establish consistency of the fitted neural- network function for its population target within a given network class. Next, we establish consistency of random-effect estimation under increasing within-subject information and show that the GNMM predictor converges to the true predictor. We then prove consistency of the conditional mean function and show that the excess mean squared prediction error (MSPE) relative to the oracle prediction error converges to zero. We conduct simulation studies to examine nonlinear fixed-effect recovery, random-effect estimation, predictor and conditional mean estimation, and excess prediction error under different numbers of subjects and repeated observations. We also apply GNMM to a longitudinal schizophrenia dataset to predict continuous and binary depression outcomes using smartphone- based behavioral measurements and compare its predictive performance with generalized linear mixed models and conventional artificial neural networks. Both the theoretical and empirical results demonstrate that GNMM provides a flexible framework for modeling nonlinear longitudinal and clustered data. These findings provide a statistical foundation for GNMM prediction and support future methodological developments.
Read the full abstract

All authors Mahmud Hasan, Md Moinul Ahsan, Mathias Nthiani Muia, Mous-Abou Hamadou, Chathurika S Abeykoon

Statistical Methodology

Moment-Calibrated Conformal Prediction with Side Information

For a fixed conformal score function, standard split conformal prediction calibrates a prediction set using the empirical distribution of the conformal scores, without using available side information about the data. We propose a moment-calibrated approach that uses such information to reduce variability across calibration data sets. Side information involving the covariates, the response, or both is expressed as moment restrictions, which are enforced through empirical likelihood weights on the calibration data points. The resulting weighted empirical distribution of the scores determines the weighted empirical quantile without changing the score function. We establish finite-sample bounds on training-conditional and marginal coverage of the moment-calibrated prediction set. We derive asymptotic distributions for training-conditional coverage and average prediction-set size, show that their asymptotic variances are no larger than the corresponding variances under standard split conformal prediction, and characterize when the reductions are strict. In the semi-supervised setting, we identify an optimal moment function of a given covariate representation, develop implementations using prespecified moment functions or a function learned by cross-fitting, and quantify the effect of the unlabeled sample size. We also extend the method to coverage conditions indexed by a class of functions of the covariates through weighted quantile regression. Numerical studies demonstrate the effectiveness of the proposed methods.
Read the full abstract

All authors Guanghui Wang, Liuhua Peng, Liu Liu, Changliang Zou

Saturday 10 October

Statistical Methodology

Stable Causal Estimation with Optimal Transport

In observational studies, the overlap assumption, also called positivity, is frequently strained in practice. Limited overlap destabilizes widely used estimators such as inverse propensity weighting (IPW), whose variance depends on density ratios that explode as propensity scores approach zero or one. To address this challenge, we propose the Stable Transport Estimator (STE), an optimal transport-based approach for estimating average causal effects when overlap holds but strict overlap fails. Unlike IPW, the variance of STE contains no density-ratio factor; it is governed by the transport map itself and stays finite under bounded moments, even when density ratios diverge. By replacing reweighting with a geometry-aware transport map, STE constructs stable counterfactual comparisons without requiring density estimation or propensity score modeling. STE further extends to settings where the overlap condition is violated: when covariate supports overlap only partially, a linear interpolation transport map enables principled extrapolation with explicit bias-variance trade-offs. We establish consistency, asymptotic normality, and a variance characterization that makes explicit the contribution of the estimated transport map to the asymptotic variance. Across synthetic and real datasets, STE demonstrates improved robustness, reduced bias, and markedly more stable variance compared to existing estimators.
Read the full abstract

Statistical Methodology

Precision follows the decision: how well an acceptance probability is known for mean-spread release rules

Many release rules accept a lot when a statistic built from the sample mean and standard deviation stays within a limit, and the probability that a process passes such a rule can be computed exactly for a broad class of parent distributions. This paper asks how well that probability is itself known once the parent has been estimated. Under a stated reference posterior the answer is an interval, narrow where a lot clearly passes or fails and wide where it is contested. In ninety-eight published content-uniformity lots that position, not the sample size, sets the width: among the forty-eight lots assayed at the same ten units, and so equally informed about the spread, the $95\%$ interval runs from essentially zero width to $0.96$. With the operating point held fixed while test and estimation sample grow together, the interval narrows only slowly, the contraction of the posterior being offset by the steepening of the operating characteristic; a calibrated study at the sample sizes of $1207$ occupational exposure assessments shows the same under a half-plane rule. For a pure-spread rule under a Normal parent the interval has an exact chi-square form in the plug-in value and the two sample sizes, extending to the mean-spread wedge with a measured residual and containing Bergum's lower bound as one endpoint. With the parent's posterior formed from the same published summaries through the evaluator's own likelihood, the qualitative pattern survives but shape moves both endpoints and width: on the same lots a strong right skew lowers the pessimistic endpoint by up to a fifth and widens the interval by up to $0.22$. At the content-uniformity operating point studied, under a Normal parent, the symmetric error bar in standard use puts the lower endpoint $0.21$ too high and returns an upper endpoint above one, even with the full posterior covariance.
Read the full abstract

Statistical Methodology

Closed forms all the way up: the sample variance of four observations

The distribution function $F(y)=\Pr(S^2\le y)$ of the sample variance $S^2$ of four independent observations is elementary on its entire support for every polynomial parent density on $[0,1]$ (without loss of generality), and we give it in finite closed form for the uniform, Beta$(2,1)$ and Beta$(2,2)$ parents. The support splits at $y = 1/6$, $2/9$ and $1/4$, with the ceiling at $1/3$, all fixed by geometry: in coordinates aligned with the cube diagonal the event $S^2\le y$ is a ball of radius $\sqrt{3y}$, the cube projects to a rhombic dodecahedron, and these are the levels at which the ball reaches its faces, edges and two kinds of vertices. The divergence theorem reduces every moment of the clipped ball to integrals along circular arcs, so each regime closes in algebraic functions of $\sqrt{y}$ and inverse trigonometric functions. For the uniform parent the only non-obvious constant is $K_g = 12\sqrt2\arctan\sqrt2$, proportional to the mean width of the cuboctahedron; the Beta constants are rational expressions in $\pi$, $\sqrt2$, $\sqrt3$ and $\arctan\sqrt2$, as in the uniform case. At $n = 4$ the normal-theory $\chi^2$ recipe used in acceptance testing misplaces the $99$th percentile of $S^2$ by $50\%$ for the uniform parent. For three observations, polynomial parents also give elementary closed forms (arXiv:2609.25333), but two elementary cases in a row do not guarantee a third, and we make no claim beyond $n = 4$.
Read the full abstract

Statistical Methodology

A Cosine-Projection Framework for Choosing Among Cox HR, D-RMST, and Average-Hazard Tests in Survival Trials

The log-rank/Cox hazard-ratio test remains the standard approach for the primary analysis of randomized trials with time-to-event endpoints and is fully efficient under proportional hazards (PH). Two model-free alternatives that have recently gained attention are the difference in restricted mean survival time (D-RMST), a contrast on the time scale, and the more recently proposed ratio of average hazards (RAH), a contrast on the rate scale. Simulation studies suggest that the RAH-based test can outperform both under delayed treatment effects, but its efficiency has not been characterized analytically, and no unified framework explains how censoring and accrual affect the efficiency ordering among these tests. We show that all three statistics share a weighted-log-rank representation and derive their asymptotic relative efficiencies under local alternatives, including the first analytic efficiency results for the RAH-based test. Geometrically, each test defines a direction in a function space and the anticipated treatment effect defines a signal direction. A test's efficiency relative to the oracle test equals the squared cosine of the angle between these directions, so the asymptotic relative efficiency of any two tests is a ratio of squared cosines. Because censoring and accrual enter through an information weight, the framework shows how design choices can alter, and even reverse, the efficiency ordering. We illustrate the framework using the CheckMate 057 immunotherapy trial and corroborate the theoretical results through simulations spanning proportional- and non-proportional-hazards settings.
Read the full abstract

Statistical Methodology · Cross-listed from math.ST

Energy-Guided Learning for Heat Equation with Noisy Source: Optimal Estimation and Generalization Bounds

Data and the governing partial differential equations (PDEs) of continuous-time dynamical systems provide rich structure for statistical learning. A central challenge is how to integrate data with PDE constraints in a principled way to improve generalization. In this paper, we characterize the statistical estimation and generalization properties of learning solutions to the heat equation with a noisy source function from finite, randomly sampled data. We first derive a novel variational formulation, which we characterize as the minimizer of loss function for an energy regularized and ellipticity debiased Sobolev regression. By formulating the learning task as a supervised problem over space-time domains under physically meaningful norms, our approach goes beyond the traditional supervised learning framework by effectively leveraging auxiliary data—samples of the source function of the heat equation—to design a physics-informed energy regularization. Our analysis reveals that learning in linear parabolic PDEs exhibits intrinsic anisotropy between spatial and temporal regularity, in sharp contrast to the static elliptic problems, leading to fundamentally different estimation and generalization behaviors. Specifically, we derive non-asymptotically rate-optimal estimation errors, as well as generalization error, which explicitly capture the interplay among temporal sampling, spatial resolution, and solution smoothness. Our numerical experiments demonstrate substantial orders of magnitude of reduction in testing errors against various physics-informed neural network methods on various parabolic and hyperbolic PDEs.
Read the full abstract

Statistical Methodology · Cross-listed from stat.AP

A model-assisted design for phase I oncology drug combination trials incorporating backfill

Historically, phase I oncology trials were designed with the focus to find the highest tolerable dose. However, given the FDA's acknowledgment that the highest tolerable dose might not be the dose which is optimal biologically, there is an increased emphasis the development of designs that explore the dose-toxicity and dose-efficacy relationship. Backfill designs have emerged as a popular response to address this need by allocating participants to doses deemed safe while awaiting participants outcomes. Previous designs have explored how to assign participants to doses in backfill designs in a rigorous manner while providing simplicity and performance (a class of designs known as model-assisted) for single agents. Still, a research gap exists in the lack of model-assisted backfill designs for drug combinations. Frequently, the combinations two of cancer agents are explored in phase I trials, with the goal of identifying the highest tolerable combination. This paper incorporates and extends methodology based on previous model-assisted backfill designs to provide clinicians with a simple-to-use backfill design for phase I drug combination trials.
Read the full abstract

Statistical Methodology

Rate-Based Causal Discovery for Heavy-Tailed Models

In many complex systems, such as river networks and financial markets, extreme events propagate from causes to their effects, and the largest observations carry the most information about the underlying causal structure. Existing methods for causal discovery in extremes, however, are tailored to specific mechanisms, such as linear or max-linear models. This work develops causal discovery for heavy-tailed homogeneous structural causal models, a flexible class that contains linear, max-linear and norm-type mechanisms, which may differ from node to node. In these models, the causal structure is encoded by ancestral impulse responses, which measure how a shock at one node is transmitted to its descendants. We propose the extremal minimum ratio estimator of these responses, which uses only observations in which the potential cause is extreme. The estimator is consistent under regular variation alone, and we derive its exact rates of convergence and limiting distributions, which depend on both the causal relation and the type of mechanism. Building on these rates, we construct consistent estimators of the causal order and of the full set of ancestral relations, using a threshold that adapts to the data and requires no knowledge of the tail index. The method is illustrated on simulated data and on river discharges in the upper Danube basin.
Read the full abstract

Statistical Methodology · Cross-listed from math.ST

Recurrent Barycenter Encoder: Prediction, Simulation and Applications to Time Series Analysis

This dissertation has two parts. In the first part we theoretically develop a framework that extends the optimal transport barycenter problem from independent data with observed covariates to processes whose relevant covariate is their own history. Given a stationary stochastic process, our framework shows there exists an invertible transport map that extracts from it an iid component whose distribution is the barycenter of the conditional distributions of the input process on its own past. This extracted iid sequence thus captures the innovation or noise component of the process that is not explained by the process's own past. In the second part of this thesis we implement an effective algorithm to solve this problem and test it against synthetic data generated with some well-known models in time series analysis for which the solution is known in closed-form. We then show how this method can be applied to build a predictive sampler for such processes, and assess the accuracy of this predictive sampler both with the aforementioned synthetic data and with real-world time series such as the S&P 500 and Bitcoin 4-hour returns, and the daily records of the North Atlantic Oscillation index from the NOAA Physical Sciences Laboratory. Finally, we demonstrate an application of the sampler to path simulation with a well-known time series model.
Read the full abstract

Statistical Methodology

Bayesian adaptive design for neural recruitment curves

Nonlinear response models remain ubiquitous across disciplines with recruitment curves describing muscle response to neural stimulation. Conventional acquisition methods typically rely on static uniform sampling, requiring many stimuli, while adaptive threshold-hunting designs such as ML-PEST forfeit accurate characterization of the overall curve. We present a Bayesian adaptive design that uses posterior uncertainty over curve parameters to propose stimuli that maximize information gain across the full recruitment curve. We additionally implement a focused acquisition method that can target arbitrary subset of curve parameters, which we use to evaluate and interpret our design. In simulations, our design outperforms uniform sampling in recovering threshold while matching ML-PEST, and outperforms ML-PEST in predictive curve accuracy while matching uniform. It likewise matches our threshold-focused acquisition, suggesting our design doesn't trade off threshold estimation for recovering the curve. We further show data collected using our design yields greater power when detecting a change in threshold in intervention studies. Our design proposes the next stimulus within 5 seconds, making it feasible for transcranial magnetic stimulation (TMS) experiments. Finally, we validate our design in TMS experiments and provide a library for Python, called onMEP, for online acquisition of neural recruitment curves.
Read the full abstract

All authors Vishweshwar Tyagi, Saihari S. Dukkipati, Daniel Twili, Jonas M. Mikhaeil, Andrew Gelman, Jason B. Carmel, James R. McIntosh

Statistical Methodology

An Algebraic Approach to Constrained K-means Clustering

Clustering is typically performed using only object features, although additional information about relationships between objects may be available and can be incorporated through constraints. Among these, pairwise constraints require two objects to belong to the same cluster or to different clusters. Several extensions of the k-means algorithm accommodate such constraints, but they sometimes fail to find a feasible assignment or incur high computational cost. Motivated by these limitations, we propose an algebraic approach to k-means clustering with pairwise constraints. The algorithm simplifies the constraints using Gröbner bases before optimization, improving the efficiency of cluster assignment. It can also detect whether the constraint set has a feasible assignment before optimization. Comparative experiments with existing algorithms showed that the proposed algorithm achieved comparable or better clustering quality at a lower computational cost, particularly when the numbers of constraints and clusters were small.
Read the full abstract

Statistical Methodology

Comparing randomized trials

Disagreements between randomized trials that address the same causal question pose challenges for the interpretation and synthesis of evidence. We sketch a causal framework for comparing the results of randomized trials that explicitly accommodates the possibility of trial engagement effects. Using counterfactual outcomes defined under joint interventions on treatment assignment and trial-related actions, we compare trial-native treatment effects with the effects transported from one trial to the population of another in a common target population. We show that standard conditions for transporting treatment effects across trial populations are not sufficient to guarantee agreement between transported and trial-native effects. Agreement additionally requires conditions governing how trial-related actions affect outcome across studies - for example, absence of trial engagement effects, equivalence of such effects across trials, or lack of interaction between trial engagement and treatment assignment. Our framework clarifies potential sources of disagreement across trials and extends existing approaches to study comparison to settings in which trial engagement effects may be present.
Read the full abstract

Statistical Methodology

An Unscented PrO Kalman Filter

Predictively-oriented (PrO) filtering for state-space models was proposed in Shen et al. [2026] as a promising route to avoiding over-confident inferences in settings where either the dynamical model, the measurement model, or both, could be misspecified. Our focus is on nonlinear state-space models, where the local linearisation used to construct the PrO filter in Shen et al. [2026] deteriorates as nonlinearities enter either the dynamical model or the measurement model. Our proposed solution is based on unscented transforms, an established tool from classical filtering which provides a more accurate mechanism to propagate Gaussian uncertainty through a nonlinear functional, and which includes the cubature rule as a special case. We introduce two variants: replacing the linearisation in the predict step alone, which we call UKF-PrO, has a computational cost comparable to the existing PrO filter; replacing it in the update step as well, UKF-PrO+, requires a substantially more expensive joint optimisation. Performance is empirically assessed on the Lorenz96 model under conditions of increasing difficulty, in which the state-space model is systematically misspecified; we find that UKF-PrO with the parameters of the cubature Kalman filter offers consistent improvements, which become substantial under strong nonlinearity, whereas the default unscented parameters are unreliable, and that the full update-step extension with the same parameters does not justify its additional cost.
Read the full abstract

Statistical Methodology

The price of a weak shadow variable: inference under nonignorable missingness as relevance vanishes

When an outcome is missing not at random, identification is commonly restored by a shadow variable: a covariate that predicts the outcome but is excluded from the response mechanism. The literature treats the first requirement, relevance, as a yes-or-no condition. We make it continuous, parametrizing it by a scalar that scales the shadow variable's entire contribution to the moment conditions, and ask what inference costs as that scalar vanishes. One functional of the inverse weight governs the answer. We characterize it exactly, show that it fixes the effective sample size, and use it to sharpen the rates of the estimator that avoids the weight. The characterization is concrete. Whether the functional saturates or diverges turns not on boundedness of the outcome but on whether the tilting variable has an atom at its essential supremum: a binary outcome saturates, a uniform one does not. It also bounds how far a scanned set reaches, and that reach grows only as the square root of the logarithm of the sample size, so a thousandfold increase buys one unit of tilt while the set to be scanned widens without limit.
Read the full abstract

Statistical Methodology · Cross-listed from q-bio.QM

Bayesian Uncertainty Quantification of Breast-Cancer Pathology AI Agreement under Reader- and Annotation-Condition-Dependent Reference Labels

Human reference labels used to evaluate medical AI can vary across readers and annotation workflows. We quantified agreement of a fixed pathology AI system with a future qualified reader without treating inferred consensus as error-free biological truth. Linked Evaluation and Unbiased NuCLS multirater data were analyzed using a fixed external HoVer-Net decision vector. The selected frame contained 1,144 anchors, 52 fields of view, five slides/patients, and five primary readers. The target was fixed-AI agreement with a future reader, with descriptive contrast $\Delta=\theta_E-\theta_U$. Analyses included raw agreement and iMRMC benchmarks, reader-only and shared-detectability Bayesian models, structural sensitivities, adversarial simulations, and predictive assessment. Among 5,188 reader-anchor pairs observed in both conditions, 19.62% changed AI-agreement state, while raw agreement was 0.5466 in Unbiased and 0.5473 in Evaluation. The reader-only model gave $\Delta=-0.0008$ (95% CrI, $-0.0033$ to $0.0008$), and the shared-detectability model gave $\Delta\approx0$ ($-0.0140$ to $0.0127$). Across structural perturbations, absolute agreement ranged from 0.420 to 0.646 in Unbiased and 0.409 to 0.640 in Evaluation. Frozen adversarial simulations met the comparative-stability criterion (median $R=0.295$; 14/16 model-scenario cells had $R<1$). Held-out-AI prediction supported shared predictive information, but varied by slide composition. The Evaluation-minus-Unbiased contrast was comparatively stable, whereas absolute agreement was structurally sensitive. These findings do not establish equivalence, causal assistance effects, biological accuracy, or new-patient transportability. Evaluations using variable human references should report reader population, selected frame, observation mechanism, and structural sensitivity.
Read the full abstract

Statistical Methodology

Cointegrated Regressors: Where Folded Concave Penalties Go Beyond LASSO

LASSO is one of the most popular regularization methods for high-dimensional regression models. While cointegration is common in practical applications of predictive regressions with many regressors, the convex penalty of LASSO overly suppresses the coefficients of cointegrated regressors and therefore fails to deliver consistent parameter estimation. We employ folded concave penalties (FCP), including the well-known SCAD and MCP, to address this issue. Using FCP, we establish two asymptotic results: (i) a local oracle property for the penalized regression, and (ii) a global oracle property for regressions with a penalty-level constraint. In particular, the latter guarantees a global solution that consistently estimates the true coefficients and enjoys oracle prediction performance. Simulation studies and an application to predicting the euro area unemployment rate show that FCP improves variable selection and out-of-sample prediction relative to LASSO.
Read the full abstract

All authors Hongqi Chen, Chengwang Liao, Ziwei Mei, Zhentao Shi

Statistical Methodology

Identification and Estimation of Treatment Effects from Network Data with Confounded Social Links

In observational network data, the adoption of a treatment and the formation of social ties often depend on the same characteristics that affect outcomes. We propose a generalized propensity score for each unit's effective treatment, built from a model for treatment assignment and a dyadic model for network formation, which may include latent node heterogeneity. The score is marginalized over the network given covariates rather than conditioned on the realized network, and the estimand is a population effect: the average effect of an exposure level for a unit drawn from the population of units and networks. We relate this estimand to those of methods that condition on the realized network and show what marginalizing costs and buys. It requires network formation to be unconfounded given observed covariates, a stronger assumption than unconfoundedness of treatment given the network. In exchange, it defines effects at exposure levels that poorly connected units cannot reach, and at levels every unit can reach it gives an estimator whose asymptotic variance is no larger than the network-conditional one. We combine the score with an augmented inverse probability weighting estimator, prove asymptotic normality as the number of networks grows with bounded network size, discuss what carries over to a single large sparse network, and apply the method to the effect of a political communication technology on political participation in Uganda.
Read the full abstract

Statistical Methodology

Copula Active Subspaces II: Error Decomposition, A Posteriori Estimation, and Sharpness of the Bounds

The Copula Active Subspaces (CAS) method of Part I estimates a non-Gaussian observation-noise density $\pi_{\boldsymbol{n}}$ from residual samples. Its KL divergence $\mathcal{D}_{\mathrm{KL}}(\pi_{\boldsymbol{n}}\,\|\,\widehat\pi_{\boldsymbol{n}})$ admits an exact decomposition into marginal, projection and reduced-density terms and bounds the posterior KL divergence averaged over the true predictive distribution. We derive a priori bounds for these terms, together with estimates or bounds from residual samples. Weighted nearest-neighbour entropy estimates are used for the marginal and reduced-density terms, with asymptotic standard errors in exact rank coordinates. For the projection error, the score is fitted on one subsample, and its mean squared magnitude outside the retained subspace is estimated on the other. Corrections for score and rank-transform errors then give a theoretical upper bound with probability at least $1-\delta$. Tests on three known noise laws in dimension $d=20$ use the estimated rank transform. In $11$ of $12$ combinations of example and sample size, the sum of the two estimated terms and the trace bound exceeds the reference divergence after averaging over the estimated models. In these tests, score-estimation error is the largest contribution to looseness. The overestimate from the Gaussian log-Sobolev inequality alone is at most a factor of $1.98$.
Read the full abstract