Statistical Methodology
Forecasting multiple density-valued time series
Han Lin Shang
Density-valued time series are common in practice. However, since these functions are non-negative and must integrate to one, they do not form a linear vector space, which complicates the direct application of standard time-series methods. To address these constraints, we introduce two transformations, based on the centered log-ratio transformation and the cumulative distribution function, for modeling and forecasting balanced panel data represented by multiple densities. Both transformations map densities bijectively into an unconstrained function space in which cross-sectional and serial dependence can be modeled directly. Within the resulting unconstrained space, we consider two approaches: multi-population functional time-series modeling to account for correlations among populations, and gap modeling between gender and region. Using Japanese subnational age-specific life-table death counts from 1973 to 2024, we evaluate and compare the one- to 20-step-ahead point and interval forecast accuracy of the two approaches for each transformation, and offer some general recommendations.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.CL
Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation
Thomas Reiter, Christoph Kern, Fedor Miasnikov et al. (+4 more)
Large
language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task designs, three independent runs, and 3,000 tweets labeled for offensive
language and hate
speech. Repeating the same model and task design produced high agreement (median Fleiss' $\kappa = 0.91$). Agreement fell when we changed the task design for the same tweets (median Cohen's $\kappa = 0.76$). Task design and model choice increased the variance of estimated prevalence by factors of 76.7 for offensive
language and 110.6 for hate
speech compared with sampling variance alone. Variation across LLM task designs reached 560-572 basis points, compared with 270-331 basis points across five human instrument versions. Confidence scores did not solve this problem. They tracked repeated model outputs more closely than agreement with human labels, and grouping six tweets in one
prompt lowered mean offensive-
language confidence by 660 basis points. We call the variation caused by task design and model choice instrument uncertainty. Researchers can measure it only by comparing reasonable task designs. Repeating one setup or relying on confidence scores cannot replace that test.
Read the full abstract ▾
All authors Thomas Reiter, Christoph Kern, Fedor Miasnikov, Sofiia Nikolenko, Rob Chew, Stephanie Eckman, Frauke Kreuter
Statistical Methodology
Bayesian Nonparametric Approaches to Ordinal Drought Modeling in the United States
Mostafa Shams, Robert Erhardt and Staci Hepler
Data observed over space and time can exhibit dependence across both dimensions, and
datasets can grow very large in both the number of locations and the number of time periods. This dependence and data size mean that many statistical models cannot be fit at a reasonable computational cost as a result of dense
matrix inversions, large parameter spaces, or
memory and storage challenges. When the data are ordinal, this only adds to the computational complexity of model fitting. Many ordinal models rely on a latent continuous variable designed to capture dependence in a computationally efficient manner, and then partitioned according to cutoff parameters to yield the observed ordinal response data. A very common choice is a latent Gaussian distribution, which can accommodate different dependence structures and permits Gibbs sampling in a
Bayesian framework. Unfortunately, this model can be overly restrictive and fails to provide the flexibility needed to capture a range of ordinal outcomes. In this paper, we demonstrate the use of
Bayesian nonparametric (BNP) methods to enhance the model flexibility of a Gaussian latent model for ordinal drought data observed over space and time, with Dirichlet process priors inducing clustering among time periods within each spatial location. In this work, we model ordinal drought data separately at each spatial location while accounting for temporal dependence, but we do not model spatial dependence across locations. We show that these BNP models often outperform
Bayesian parametric approaches at a reasonable computational cost.
Read the full abstract ▾
Statistical Methodology
A Bayesian framework for multilevel data under model mis-specification
Widemberg S. Nobre, David A. Stephens, Alexandra M. Schmidt et al. (+1 more)
We propose a
Bayesian framework for uncertainty quantification from the perspective that the working model is mis-specified in settings of a multilevel data-generating process. We focus on settings in which the mis-specification fails to match the functional form of the mean structure, and discuss
Bayesian estimation of target parameters under dependence induced by a mismatch between working and data-generating models. The proposal represents a
Bayesian semi-parametric procedure aimed at estimating population-level parameters while accounting for cluster- and unit-level variation in the estimating function. The proposal extends the regular
Bayesian bootstrap to account for cluster- and unit-level variation using multilevel weights from an enriched Dirichlet model. Simulation studies indicate that the proposed approach has good frequentist properties when the data-generating process and the proposed model induce a partially exchangeable sequence associated with the unknown quantity of interest. Applications to radon (Gelman and Hill, 2007), Programme for International Student Assessment 2022 (OECD, 2023), and tuberculosis (Nobre et al., 2023)
datasets are presented for illustrative purposes. The results demonstrate that the proposed method is competitive with variations of multilevel models, with major differences observed in the range of credible intervals, which are justified by the nonparametric assumptions underlying the proposed method.
Read the full abstract ▾
All authors Widemberg S. Nobre, David A. Stephens, Alexandra M. Schmidt, Erica E. M. Moodie
Statistical Methodology
Formulating Cross-World Mediation Estimands Through Single-World Mixtures
Razieh Nabi and David Benkeser
In causal mediation analysis, natural direct and indirect effects are defined through nested counterfactuals that combine the outcome under one exposure level with the mediator value under another and are therefore inherently cross-world. Their canonical identification additionally relies on cross-world independence assumptions. Consequently, both the estimands and their identifying assumptions remain controversial. In this paper, we ask what additional single-world structure would be required to re-express cross-world estimands as single-world quantities, identifiable under single-world assumptions. We show that these quantities can be written as mixtures of controlled single-world effects under assumptions involving a susceptibility marker, a possibly latent baseline variable that encodes the "would-be" mediator value under a reference treatment arm. If observed, the marker would make several implications of the marker restrictions empirically testable. These results provide a transparent single-world formulation of cross-world mediation effects while making explicit the assumptions required. Although these conditions clarify how cross-world estimands can be interpreted within a single-world framework, their practical relevance depends on whether such susceptibility markers can be justified in specific applications. We discuss implications for principal stratification and show in the Supplementary Material how the formulation extends to ordered mediators and settings with exposure-induced mediator-outcome confounding.
Read the full abstract ▾
Statistical Methodology
CVaR anchor regression protects against rare shifts
Malte Londschien
We study prediction in new environments when training data contain rare, large shifts. Anchor regression penalizes the average of the squared mean residual across environments. It protects against shifts in an ellipsoid determined by the second moment of the training shifts. Covering rare shifts may therefore require a large penalty, expanding the ellipsoid in every direction and reducing accuracy on common environments. We propose CVaR anchor regression, which replaces the average of the squared mean residuals with a tail average. Unlike CVaR or GroupDRO applied directly to prediction risks, it does not give environments more weight solely because their noise levels are high. We prove an exact worst-case risk guarantee under a linear structural model that allows for heteroscedastic noise. For discrete environments, decreasing the CVaR tail fraction expands the robustness set from an ellipsoid to a scaled convex hull of the training shifts and their negatives. A separate parameter controls its scale. Examples show how the method can improve protection against rare shifts while retaining accuracy on common environments. We illustrate the method on New York City taxi data.
Read the full abstract ▾
Statistical Methodology · Cross-listed from math.ST
Rolling Conformal Prediction in Sequential Model Training
Chen Cheng, Ruiting Liang and Rina Foygel Barber
We introduce Rolling Conformal Prediction (rolling-CP), a distribution-free predictive inference method for the setting of sequential model training. Specifically, given a data stream $(X_1,Y_1),(X_2,Y_2),\dots$, at each time $n$ the trained model may depend on the observed history $\{(X_i,Y_i)\}_{i<n}$. This setting arises naturally in modern sequential training, including one-pass training over massive
datasets and continual fine-tuning or test-time adaptation of
language models during deployment. Rolling-CP first calibrates each incoming observation against the current predictor and then rolls it into future training. In this way, we avoid the need for data splitting. Remarkably, although the models at times $n=1,2,\dots$ may have entirely different properties and accuracy levels, for exchangeable data it is nonetheless possible to establish a guarantee of marginal coverage, with a familiar universal factor-two guarantee (a worst case guarantee of $1-2\alpha$ coverage, as compared to the target level $1-\alpha$), without any assumptions of stability or any restrictions on the model training process. For i.i.d. data streams, we further prove high-probability training-conditional validity uniformly over time; under stability conditions, coverage guarantees sharpen towards $1-\alpha$. Numerical experiments on sequential regression, multiclass SGD, and one-pass neural-network training further demonstrate the practical effectiveness of rolling-CP.
Read the full abstract ▾
Statistical Methodology
Additive Nonparametric Regression with Spatial and Network Objects
Rajarshi Guhaniyogi, Pritam Dey, Krishnendu Chandra et al. (+2 more)
This article is motivated by an imaging application from the Adolescent
Brain Cognitive Development (ABCD) study, aiming to predict task-based
brain activation maps from t-fMRI using cortical metrics from structural MRI (s-MRI) and
brain connectivity data from resting-state fMRI (rs-fMRI). Hierarchical
Bayesian modeling is well-suited for integrating diverse imaging data and quantifying prediction uncertainty. However, progress in this field is limited due to challenges in designing joint priors that capture the structures and interconnections between different imaging modalities, along with computational complexity and lack of theoretical assurances. To address these challenges, the article introduces a novel regression framework that treats t-fMRI and s-MRI images as functional data, incorporating additive non-linear effects of both network and functional predictors on the functional response. Specifically, we employ Gaussian process (GP) priors on coefficients related to the functional predictors to capture their intricate functional dependencies with the response. Furthermore, a GP prior is assigned to encapsulate the non-linear nodal effects of the network predictor on the response function. The method is supported by theoretical results on predictive accuracy for the functional response, and is empirically validated through simulation studies and analysis of multi-modal neuroimaging data from the ABCD study.
Read the full abstract ▾
All authors Rajarshi Guhaniyogi, Pritam Dey, Krishnendu Chandra, Aaron Scheffler, Bani K. Mallick
Statistical Methodology
Empirical Bayes prepivoting under group invariance: false discovery rate control and moderated t-tests
Nikolaos Ignatiadis and Etienne Roquain
We consider simultaneously testing hypotheses about thousands of units, e.g., genes or
proteins, where each unit yields a handful of replicate measurements and we test whether its mean is zero. A widely used approach in genomics, implemented in the limma software, borrows strength across units to learn the distribution of the unit-specific variances, then computes moderated t-statistics. Here we develop the first procedures using moderated t-statistics that (i) control the false discovery rate (FDR) in finite samples under independence across units and null group invariance, without assuming limma's hierarchical model, and (ii) match the power of an oracle local false discovery rate procedure in a sparse asymptotic regime under limma's working model. Benjamini-Hochberg (BH) with standard t-test p-values has asymptotically zero power in the same regime. Our approach learns the variance distribution from statistics that depend on each unit's data only through its orbit under a compact group of transformations that preserves the null distributions, such as sign flips or orthogonal rotations. We then calibrate the resulting moderated t-statistics against group-transformed statistics pooled across units to obtain compound p-values, which we use with BH and a close variant. For small finite groups, we also construct a Selective SeqStep+ procedure using the same learned scores. Our approach extends to two-sample tests and tests of linear model coefficients.
Read the full abstract ▾
Statistical Methodology
Automatic depth-based local center clustering via $\beta$-integrated local depth and adaptive grouping
Siyi Wang, Alexandre Leblanc and Paul D. McNicholas
Clustering is an unsupervised learning technique that partitions unlabeled data into groups. Most existing methods require user-specified parameters, such as the number of clusters or neighborhood size. Conversely, we propose automatic depth-based local center clustering (A-DLCC), a fully data-driven method that eliminates numerical parameter tuning. A-DLCC uses the $\beta$-integrated local depth to identify stable exemplars, points consistently central across multiple locality levels, termed local centers, which are ranked by their representativeness. Each local center induces a group of similar points, with group-level similarity measured by a proposed nonparametric metric called group-level local similarity. To guide merging, we incorporate the bottleneck path idea from graph theory, which forms the basis of our adaptive merging criterion. Based on this criterion, we design a single agglomeration rule in which a group is either absorbed by a neighbor it reaches better than itself or bonded to a neighbor that both sides find more reachable than their own background, every merge being additionally required to be carried by a contact stronger than a configuration-model null expects. The rule automatically estimates the number of clusters and decides when to stop merging. Experiments on synthetic and real data show that A-DLCC produces interpretable clustering results without parameter tuning.
Read the full abstract ▾
Statistical Methodology
Mark distance correlation functions: from moment-based to distributional mark summary characteristics in spatial point processes
Matthias Eckardt, Mari Myllymäki and Aila Särkkä
With the rapid advancement in data collection devices and storage capacities, we have access to increasing amount of spatial point pattern data where each event location is augmented by multiple, potentially non-scalar marks. Therefore, there is need for efficient analysis techniques to investigate the structural relationships between the marks. In this paper, we recall the distance covariance and distance correlation and adjust them to the marked point process setting. As a result, we introduce a novel class of mark characteristics for single real-valued marks as well as multivariate combinations of marks, including mixtures of integer- and real-valued quantities, and non-scalar marks.
Read the full abstract ▾
Statistical Methodology
Optimal Sequential Annotations for Off-Policy Evaluation
Woojin Chae, Ezinne Nwankwo, Haitong Qin et al. (+1 more)
Offline reinforcement learning and off-policy evaluation evaluates dynamic treatment rules based on retrospectively collected data prior to deployment. In recent AI applications, state and reward information is recorded as complex text or image, which recent AI advancements such as LLM-as-a-judge can label with unknown bias. Expert annotation may be available but at a higher cost. For example, safety classification via cheap but imperfect classifiers vs. expensive expert review. We show how a limited budget for ground-truth data-annotation can be used via doubly-robust OPE with missing rewards, and we optimize variance-optimal annotation probabilities for sequential off-policy evaluation, where the target policy value is estimated from annotated data. We characterize the optimal annotation probabilities for sequential forward-monotone annotation protocols, and provide a feasible batch-adaptive implementation. Our work is motivated by a collaboration with a homelessness services nonprofit that writes casenotes for individuals over time. Our method can be used to unlock trustworthy inference from casenote data and answer new inferential questions such as: how does expanding outreach effort over time affect progress towards a housing application and improvement in housing placement? In simulations and on two real
datasets - casenotes from the nonprofit and human-preference votes from LMArena - we see reductions in RMSE of 34-65% for housing placement and 17-68% for progress towards a housing application at budgets of 40% of full annotation and above, and by 55-62% at every budget on LMArena.
Read the full abstract ▾
All authors Woojin Chae, Ezinne Nwankwo, Haitong Qin, Angela Zhou
Statistical Methodology
Sample-Efficient Multiple Testing with Adaptive Data Collection
Zhanran Lin, Wanteng Ma, Zhimei Ren et al. (+1 more)
This paper studies adaptive experimental design for multiple testing, where an experimenter sequentially chooses which hypothesis to sample. We propose the e-value-based posterior sampling (e-PS) procedure, which uses the empirical average of log e-value increments to guide randomized sampling and applies e-BH to construct rejection sets. Under conditionally valid e-value increments, the procedure controls the false discovery rate at arbitrary stopping times and produces nested rejection sets. We establish high-probability bounds on the number of samples needed to discover all nonnull hypotheses in terms of the growth and concentration of the underlying e-processes. We specialize these bounds to simple-versus-simple, composite-versus-simple, and simple-versus-composite testing. Simulations and experiments using joke ratings and watermarked text illustrate the procedure's power under limited sampling budgets.
Read the full abstract ▾
All authors Zhanran Lin, Wanteng Ma, Zhimei Ren, Yuting Wei
Statistical Methodology
On the Convex Transform Order of Order Statistics
Yijia Li, Maochao Xu and Peng Zhao
Let X1,...,Xn be independent exponential random variables with arbitrary positive, not necessarily equal, rates, and let Y1,\ldots,Yn be independent and identically distributed exponential random variables. We prove that, for every order statistic, the homogeneous order statistic is smaller than its heterogeneous counterpart in the convex transform order. Our results show that, relative to the homogeneous
benchmark, heterogeneity stretches upper
quantiles more strongly than lower
quantiles, while the homogeneous order statistic ages faster in the convex-transform sense. We further extend the comparison to a proportional-hazards family that includes Weibull, Lomax, and Burr XII distributions. We give explicit conditions under which the comparison is preserved and show that the shape restriction is sharp within the Weibull family. These results characterize how component heterogeneity changes the distributional shape of order statistics beyond effects on location or scale.
Read the full abstract ▾
Statistical Methodology
A Projection-Based Approach to Bayesian Age Estimation under Stratigraphic Constraints
Imène Bouafia, Anne Philippe and Guillaume Guérin
In archaeological chronology building, stratigraphic constraints arising from the excavation context are commonly incorporated into
Bayesian models through order-constrained priors. However, these models suffer from well-documented artifacts: estimated durations are inflated, and the bias worsens as more samples are dated. First, we prove that this phenomenon is intrinsic to the prior: when all true ages coincide, the posterior duration converges to the prior support's width. We then propose an unconstrained posterior that does not include stratigraphic knowledge and project it onto the space induced by the stratigraphic order. The projected posterior is the Wasserstein-closest constrained distribution; it can be computed via isotonic regression on a directed acyclic graph and corrects duration inflation unlike constrained priors. Simulation studies and two archaeological case studies demonstrate substantial bias reduction and precision gains.
Read the full abstract ▾
Statistical Methodology
Flexible latent variable models on graphs: Laplace approximated inference for multiview network data
Anna van Es, Eva Cantoni and Davide La Vecchia
We propose a novel and flexible nonlinear approach for dimensionality reduction of large-scale multiview network data and derive its theory. The (linear) predictor incorporates observed
covariates (edge-specific, layer-specific, and global) and Gaussian latent factors. Inference is conducted via the graph Laplace approximated maximum likelihood
estimator. Letting $K$ denote the number of network layers and $n_V$ the number of nodes, we derive asymptotic theory under two regimes: (i) $K \to \infty$ with fixed $n_V$, and (ii) double asymptotics $K, n_V \to \infty$, establishing consistency and asymptotic normality for local and global parameters, with distinct convergence rates. In an application to the
gravity model for commodity trades, we use a zero-adjusted Gamma distribution with latent factors and observable
covariates (e.g.\ distance, tariffs, common
language) to capture excess zeros, skewness, and unobserved heterogeneity. Synthetic and real-data exercises show that our approach outperforms the routinely applied Poisson pseudo-maximum likelihood
estimator with fixed effects. We complement our theoretical and empirical contributions with open-source R/C++ routines and a novel strategy for starting values selection.
Read the full abstract ▾
Statistical Methodology
Engineering safe structures: recent advances in structural reliability modelling
Adwait Sharma, Oindrila Kanjilal and C S Manohar
This paper reviews advances over the last decade in the field of computational structural reliability modelling. The discussion is focused on problems of time-variant/time-invariant reliability at the component level. Three broad classes of problems are considered: (a) time-invariant reliability problems involving static problems, (b) problems of time-variant reliability analysis for deterministically parametered dynamical systems driven by random excitations, and (c) time-variant reliability analysis of randomly parametered dynamical systems subjected to random excitations. Four classes of approaches are discussed: (a) analytical methods based on first/second order reliability analyses (for time-invariant reliability analysis) and level crossing based approaches for time-variant reliability problems, (b) methods based on importance sampling strategies (including the Girsanov transformation based method for dynamical systems), (c) particle and trajectory splitting based methods, and (d) methods that employ machine learning based tools (primarily involving development of surrogate models and active learning strategies) in tackling reliability problems. The focus of the discussions is on methodological advances, and questions related to specific applications are not addressed. The review presents critical discussions on the relative merits of alternative approaches (in terms of computational efficiency, accuracy, scalability, treatment of rare events, ability to handle
geometric complexities linked to failure surface, and suitability for high-dimensional problems) and identifies several directions for future research.
Read the full abstract ▾