Peer Revenue Impact factor: volatile

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistical Methodology

stat.ME Watch Statistical Methodology (sign in to use)

Monday 10 August

Statistical Methodology

A Plasmode Benchmark Can Refute but Cannot Certify: Projection and Regularity in Causal-Inference Simulation

Plasmode simulation - resampling covariates, and often treatment, from a real cohort and regenerating the outcome from a model with an injected effect - is a standard tool for comparing causal-inference estimators. We argue that its validity turns on two separable design choices, and that separating them clarifies a recent debate over resampling versus regenerating the treatment. Our central message limits what such a benchmark can show: a plasmode can refute a universal claim about an estimator but cannot certify one. Refuting needs only that the simulated data genuinely belong to the class over which a guarantee is quoted; certifying also needs the benchmark's difficulty to carry over to the target population, which the analyst cannot verify. Two choices set that difficulty. Regularity is whether the simulated law preserves the overlap that inverse-weighting and targeted-learning theory require. Resampling the real treatment with fine-grained covariates destroys it, and the resulting coverage failure falls on flexible, machine-learning-based estimators, while parametric-nuisance estimators - including inverse-probability weighting - stay near nominal; it disappears once the treatment is regenerated from a bounded propensity or the covariates are coarsened, and cross-fitting removes only part of it. Projection is that the injected outcome model hands a matching estimator the right answer by construction; this survives the regularity fix and can reverse estimator rankings even when overlap is intact. We give a regular-plasmode algorithm, a confirming real-data simulation, and practical recommendations. A plasmode is a filter against estimators, not a warrant for them.
Read the full abstract

Statistical Methodology · Cross-listed from stat.ML

Black-Box Knowledge Transfer across Distinct Feature Sets

Pre-trained black-box predictive functions encode knowledge distilled from massive datasets and extensive computation. However, when the available input features differ from those the black box expects, direct use is infeasible. We introduce a method for transferring predictive knowledge from the black box to a new, heterogeneous input space. Our approach decomposes the target regression function into a transferable component, which the black box can inform, and a non-transferable component, which captures information unique to the new space. We propose a two-step neural network procedure, estimating the transferable component from abundant unlabeled feature pairs that bridge the two input spaces and the non-transferable component from limited labels. We derive prediction risk bounds that improve on those of a non-transfer alternative when the non-transferable component is small or smooth, and the procedure adapts to either case. Under additional conditions, the worst-case risk of our estimator is of strictly smaller polynomial order than the minimax risk of estimation from the labeled data alone. We extend the framework to multiple black boxes, each on its own input space, and show that aggregation can reduce prediction error relative to the best single black box. Simulated and real data demonstrate the practical value of the method.
Read the full abstract

Statistical Methodology

Statistical analysis of block structured latent variable models

This paper studies block structured latent variable models, in which observed variables are grouped into distinct blocks based on their relationships with the underlying latent variables. These block structures are prevalent in various fields such as psychology, education, economics, and genetics. Despite their widespread applications, the fundamental statistical properties of these models remain largely unexplored. In this work, we present a comprehensive statistical analysis of the block structured latent variable models. In particular, we first derive conditions for model identifiability across various block designs. Furthermore, we investigate the maximum likelihood estimation under these identifiability constraints. To accommodate these intricate constraints associated with various block configurations, we introduce a Lagrangian-type formulation for the constrained nonconvex optimization problem and show that its optimum coincides with that of the original problem. This formulation serves as a critical tool for understanding the behavior of the constrained estimator under various block structures. Building on that, we establish sharp non-asymptotic error bounds and asymptotic distributions of the constrained maximum likelihood estimator. We also propose a computational framework to obtain the estimator and establish theoretical properties for the algorithm output. Our theoretical findings are validated through simulation studies and empirical data analyses.
Read the full abstract

Statistical Methodology

COMPACT: Spectral Adjustment Scores from a Complete and Irreducible Causal Criterion

Observational datasets frequently contain many baseline variables, yet investigators estimating causal effects may not know which variables to include in the adjustment set. Confounding information may also be distributed weakly across many variables. Propensity scores can simplify adjustment by reducing high-dimensional covariates to a scalar with binary treatment. Although the propensity score is the coarsest balancing score, this distributional optimality does not imply maximal specificity over causal graphs. We instead examine all causal graphs among a candidate score, treatment, and outcome while allowing latent variables. Under faithfulness, we identify the largest set of unconditional and conditional dependence relations whose truth is invariant to whether treatment causes the outcome, leaving treatment-effect estimation to the downstream analysis. This criterion defines the maximally specific graph class expressible through these relations. We then develop the proposed algorithm, which operationalizes the criterion through a generalized eigenvalue problem whose score space targets the span of a balancing coordinate and an outcome-guided coordinate. We show that sufficiently informative proxies can recover this span without direct observation of the adjustment variables, characterize the resulting estimation and causal errors, and establish bootstrap validity for the complete procedure. Simulations and a real-data application demonstrate superior performance over several alternatives.
Read the full abstract

Statistical Methodology

Cohen's f or Mean Standardized Differences? Assessing Covariate Balance with Multivalued Treatments

Assessing covariate balance across more than two treatment groups has no established omnibus standard: the prevailing practice averages, or takes the maximum of, pairwise standardized mean differences (SMD), while Cohen's f - the classical generalization of Cohen's d to more than two groups - offers an alternative grounded in an established effect-size framework, but the two have not been formally compared. We extend both to arbitrary weighted and covariate-adjusted models via a community-contributed Stata command, esizereg, and validate them in a simulation study of three, four, and six treatment groups under correctly specified and misspecified generalized-propensity-score weighting, correlating each statistic against downstream treatment-effect bias. Cohen's f tracks estimation bias comparably to mean absolute SMD, both pooled (r = 0.93) and within weighting arm (r approximately 0.79); maximum absolute SMD is generally weakest. This ranking was unchanged under a deliberately adversarial nonlinear/interaction outcome model. Cohen's f and the SMD-based statistics are not numerically comparable: we derive an exact representation of f as a size-weighted quadratic function of the pairwise SMDs and prove the minimum attainable ratio between f and mean absolute SMD under equal group weights, so conventional SMD thresholds should not apply directly to f. We recommend reporting f alongside its per-level and pairwise decomposition.
Read the full abstract

Statistical Methodology

Structured dispersion modelling for mortality data using a Conway–Maxwell–Poisson specification

Mortality models that attempt to capture dispersion typically assume a fixed dispersion structure, an assumption that is rarely satisfied in practice and that can lead to miscalibrated uncertainty and poor predictive performance. In this paper, we introduce a flexible framework for explicitly modelling dispersion in mortality data using the Conway–Maxwell–Poisson (CMP) distribution, which accommodates underdispersion, equidispersion, and overdispersion within a unified specification. Rather than imposing a global dispersion parameter, the framework allows both the type and degree of dispersion to vary by age and over time, thus capturing structural heterogeneity that simpler models may overlook. A Bayesian formulation treats dispersion as unknown, with prior structures that coherently propagate parameter, process, and distributional uncertainty. Estimation is carried out via Markov chain Monte Carlo (MCMC) methods. Using empirical death data for males in England and Wales, we show that variability in mortality counts differs substantially across ages and across time periods. This has meaningful implications for the calibration of longevity risk and the pricing of annuity products.
Read the full abstract

Statistical Methodology

Joint return levels of maximum temperature and minimum relative humidity by combining copulas with an extreme value framework for bimodal data

Climate change has become a growing concern, particularly in regions experiencing increasingly frequent extreme events. In Brasília, the capital of Brazil, significant shifts in climate patterns have drawn attention, including episodes of intense heat, unusually cold weather, and prolonged dry periods, often accompanied by wildfires. These phenomena directly affect the population and local ecosystems, requiring detailed analyses to understand their causes and effects. In this work, the return period for the joint distribution of maximum air temperature and minimum relative humidity in Brasília is determined using a copula-based approach. First, the dependence structure between the variables is modeled through a copula, while the marginal distributions are fitted using the novel methodology for modelling extreme values in complex systems. Different copula families, including rotated versions, were evaluated using information criteria, leading to the selection of a model that adequately captured the dependence structure, including asymmetry and tail behavior. The selected model also reproduced the multimodal pattern observed in the joint density, a feature consistent with the possible presence of multiple climate regimes. In the context of climate data modelling, the bivariate return level was determined using a conditional approach to assess the occurrence of extreme scenarios characterized by high temperatures combined with low relative humidity. This analysis makes it possible to quantify the expected frequency of such events, providing valuable information for environmental risk monitoring, the planning of preventive measures, and the development of climate change adaptation strategies in the region.
Read the full abstract

All authors Beatriz G da Cruz Albernaz, Cira E G Otiniano, Carolyne Soares de Brito, Fidel E C Morales, Enzo Porto Brasil

Statistical Methodology

Bandable Cumulant Tensors: Optimal Estimation and Applications in Non-Gaussian Data Modeling

Higher-order cumulants capture the non-Gaussian dependence that covariance misses, but they are hard to use in high dimensions. An order-$d$ cumulant tensor has $p^d$ entries, and the plug-in sample cumulant is generally not even rate-optimal under the tensor spectral norm. For ordered data, both difficulties admit one remedy: assuming that higher-order interactions decay away from the main tensor diagonal, we introduce a bandable cumulant class and a tapered sample cumulant estimator that computes only $O(pk^{d-1})$ local entries at bandwidth $k$ and never forms the full tensor. Under exponential-type tail conditions, we prove nonasymptotic spectral-norm bounds that separate tapering bias from stochastic error, and a minimax lower bound over the same class that matches the leading bias and stochastic terms of the upper bound; for sub-Gaussian observations, the tapered estimator attains the minimax rate whenever $n\gg(k+\log p)^{d-1}$ at the oracle bandwidth $k$, with the ambient dimension entering only through $\log p$. Localization also suppresses the higher-order fluctuations behind this suboptimality, so tapering plays a stronger role here than in bandable covariance estimation. The spectral-norm guarantee transfers directly to downstream tasks, yielding plug-in error bounds for cumulant Yule–Walker estimation in autoregressive models, minimum-distance estimation in moving-average models, and matched-filter source localization in sensor arrays. Simulations corroborate the theory, and real-data analyses of RR-interval, air-quality, and Neuropixels recordings illustrate the resulting stability gains.
Read the full abstract

All authors Runshi Tang, Anru R. Zhang, Yuefeng Han, Wei Biao Wu

Statistical Methodology · Cross-listed from math.ST

Optimal Inference with Black-box Predictions

Powerful black-box predictive models have motivated many proposals for combining observed data with predictions to perform valid statistical inference. Despite this progress, the field lacks a unifying principle that explains how hypothesis tests should integrate data and predictions in a way that is both valid and efficient. In this work, we address this gap in the high-dimensional Gaussian sequence model. We characterize the information-theoretic limits of inference with black-box predictions when their accuracies are known and, for orthogonal predictions, when they are unknown. Building on these characterizations, we develop practical hypothesis tests that adapt to the unknown accuracies of the predictions while benefiting from strong alignment among them.
Read the full abstract

All authors Lucas Kania, Abhinav Chakraborty, Edward Kennedy, Larry Wasserman, Sivaraman Balakrishnan

Statistical Methodology

Asymptotics of Nonparametric Estimation under General Non-monotone MAR Missingness: A Nonparametric Maximum Likelihood Approach

Missing data constitute a pervasive challenge in empirical research. Consequently, there is an ever-growing number of methods designed to address this challenge, with multiple imputation and inverse probability weighting the dominant strategies. Despite this, theoretical guarantees remain limited, particularly in the challenging case of non-monotone missing at random (MAR). When guarantees exist, they are often confined to simplified settings such as missing completely at random, monotone or block-wise missingness, or rest on restrictive assumptions about the missingness mechanism. In this paper, we utilize the theory of sieve maximum likelihood to establish a general rate of convergence under MAR that requires no modeling of the missingness mechanism and no restriction on the configuration of missing patterns, beyond MAR itself and a natural positivity condition. Applying this result to density estimation, we show that the complete-data density can be estimated at the minimax rate over a Hölder class, up to a logarithmic factor, for any prescribed smoothness level. The missingness does not affect the rate and enters only through a constant. The estimator is approximated in practice by a simple expectation-maximization (EM) algorithm operating on the incomplete data directly. In simulations, it performs comparably to the kernel density estimator supplied with the complete data across a wide range of missingness levels.
Read the full abstract

Statistical Methodology

Covariate Informed Identification of Heterogeneity and Outliers in Longitudinal Data

We often observe heterogeneity in longitudinal data, where the mean and variance for certain profiles meaningfully differ from the rest. Some profiles may also exhibit outliers at a limited number of measurements. Using a standard mixed effects model, which assumes homogeneity, can lead to overestimating the residual variance and inefficient estimation. In this work, we identify and account for three sources of heterogeneity in longitudinal data: incompatible mean trajectories, increased residual variance, and outliers at individual measurements. Our Bayesian mixture model incorporates binary indicators of heterogeneity for each of these features, modeled through logistic regression using covariates. We perform statistical inference using Markov chain Monte Carlo and implement model selection to evaluate the inclusion of various heterogeneous components. Simulations demonstrate that our model can accurately identify heterogeneity and produce efficient estimates of the fixed effects parameters. We further validate our approach using the DHEAS hormone data from the SWAN study.
Read the full abstract

All authors Anish Mukherjee, Jeremy T. Gaskins, Jeffrey M. Albert, Tanujit Dey, Sounak Chakraborty

Statistical Methodology · Cross-listed from math.ST

Dynamic $e$-closure for online hypotheses with any-time-valid evidence: closure principles and projective mergers

Many modern testing problems are sequential along two axes: new hypotheses may arrive over time, while evidence for hypotheses already under consideration continues to evolve and may be inspected at arbitrary stopping times. We develop dynamic $e$-closure for this setting. At a global stopping time the active true-null intersection is random. Future-extension coherence allows its certificate to be compared with that of a fixed terminal intersection, yielding simultaneous stopped-FDR control. If the certificates are also time-monotone, the resulting closure controls simultaneous SupFDR and is setwise persistent. Conversely, every procedure satisfying either criterion is contained in a dynamic closure generated by canonical normalized-loss processes. For pointwise mergers, fixed-dimensional admissibility is equivalent to ordinary arbitrary-dependence $e$-merging. Coherence across horizons then forces a single globally summable weight sequence and exact neutrality under padding by the $e$-value one; on a countably infinite hypothesis universe, this rules out nontrivial symmetric mergers in the admissible pointwise class. The theory extends from FDP to bounded losses that are monotone in the possible true-null configuration and local in the reported action. We also give a coherence counterexample, persistent constructions, and a globally valid shared-control model.
Read the full abstract

Statistical Methodology · Cross-listed from math.OC

Safe Start: Configuring Optimization Algorithms for Decision-Making under Extreme Risks

We consider stochastic optimization where the goal is not only to optimize an average-case objective, but also to mitigate the occurrence of rare catastrophic events. This problem is motivated by safety-aware decision-making and AI training. We first argue that, in the presence of a simulation model, natural attempts to integrate variance reduction into optimization, even executed in a reasonable adaptive fashion, encounter fundamental challenges in guaranteeing realistic runtime when using common stochastic gradient descent algorithms. This challenge arises from the extreme sensitivity of tail-based objectives with respect to the decision variables, which renders a dichotomic failure of convergence regardless of what step size we select. We offer remedies based on a new notion of safe start that allows for efficient finite-time error control, and show how the sampling complexity scales favorably under the combination of safe start and variance reduction. We illustrate our methodologies on examples in portfolio optimization and robust classification with neural networks.
Read the full abstract

Statistical Methodology

The Earth Moves, But So Does the Bias: Systematic Upward Bias of the Wasserstein (Earth Mover's) Distance and Permutation-Based Null Calibration

The Earth Mover's Distance (EMD) is gaining increasing interest among political scientists for assessing similarity in preference distributions. However, there remains a risk of finite-sample upward bias induced by sampling variation in empirical probability measures, which is under-recognized by existing studies. This problem is especially severe in high-dimensional or sparse settings, including conjoint distributions that serve as an illustrative example in this paper. As political scientists are broadening their use cases of EMD, this paper cautions against interpreting standard bootstrap uncertainty bounds as a correction for the upward bias of empirical EMD. It proposes a permutation-based null calibration framework for more robust hypothesis testing. As a non-parametric approach, it frees researchers from making directional or distributional shape assumptions. While alternative estimators require these rigid assumptions to correct for upward bias, political science data often fail to meet them in practice. Through four sets of Monte Carlo simulations, this paper demonstrates the utility of this framework. The proposed approach also applies more generally to empirical comparisons of two probability distributions defined on a common metric space, provided that the ground distance between support points is substantively meaningful.
Read the full abstract

Statistical Methodology

Clustering Informed Inverse Probability Weighting Strategies for Causal Effect Estimation in Observational Studies

Inverse probability weighting (IPW) is widely used to estimate causal effects in observational studies but depends on adequate propensity-score specification. We compare three strategies for addressing treatment assignment heterogeneity: standard IPW, clustering augmented IPW with cluster specific propensity score models, and a global propensity score model including estimated cluster membership as a covariate. Through simulations with and without latent cluster structure and under correctly specified and omitted covariate propensity score models, we evaluate bias, mean squared error (MSE), and confidence interval coverage across sample sizes of 100 to 500. Both cluster informed strategies reduced bias and MSE from omitted covariate misspecification relative to standard IPW, but neither uniformly dominated: clustering augmented IPW achieved lower MSE when latent cluster structure was present, whereas the global model generally provided lower bias and better coverage at smaller sample sizes. We also apply the methods to 966 breast cancer patients treated with carboplatin, using generalized propensity scores to estimate the dose response relationship between treatment cycles and hypersensitivity reaction risk. Standard and clustered analyses produced similar pooled estimates, while clustering additionally provided subgroup specific estimates and diagnostic profiles. Overall, cluster informed strategies may improve robustness to propensity score misspecification, with relative performance depending on subgroup structure, sample size, and inferential priorities.
Read the full abstract

All authors Ruohui Chen, Scott Zuo, Whitney Stevens, Seth Pollack, Wenna Xi, Lucia Petito, Lihui Zhao, Hui Zhang

Statistical Methodology · Cross-listed from econ.EM

Bias-robust causal inference for panel data

We develop a bias-robust causal inference method for observational panel data settings. Such methods typically impute untreated outcomes, so counterfactual error passes straight into the estimated treatment effect while conventional standard errors ignore it. We adapt bias-aware minimax methods, developed for estimating regression coefficients in factor-model panels, to a causal target: the average effect on the treated, which has to be imputed and may vary across units and periods. The estimator corrects the imputed counterfactual with weighted untreated residuals and reports intervals with an explicit allowance for the error that remains. In simulations the proposed method holds nominal coverage where alternatives such as the generalized synthetic control have almost none, especially when the factor rank is underfitted, at the cost of wider intervals. By applying the developed methodology to real data the estimated effect remains significant for counterfactual errors nearly twice the size that the design's placebos typically exhibit.
Read the full abstract

Statistical Methodology

Why the unrestricted weighted least squares should be routinely reported in medical meta-analyses

The unrestricted weighted least squares (UWLS) meta-analysis estimator of mean effect is an alternative to the conventional random-effects model (RE). It is a weighted least squares regression estimator that can be represented as a multiplicative random-effects model. UWLS has been shown to fit medical research better than RE as measured by AIC/BIC widely across Cochrane Database of Systematic Reviews (CDSR). The independence of UWLS's mean and heterogeneity estimators provide small-sample advantages that RE does not possess. Large small-sample biases and uncertainty in RE's heterogeneity variance estimates explain most of RE's relatively poor fit along with RE's boundary problem, where RE's heterogeneity variance is estimated to be zero. We prove that UWLS almost always has superior fit at RE's boundary with uncommon exceptions. UWLS has also been found to have generally superior statistical properties: bias, MSE, and coverage relative to RE across 1,665 simulation designs compiled from four published studies authored by different teams of researchers. A recent study in this journal replicated UWLS's superior goodness of fit widely across both the CDSR and a new set of simulations. Due to reporting and interpretation errors, this recent study calls for the continued use of RE as the default meta-analysis estimator with limited applications of UWLS. We address this recent study's concerns and show why UWLS should be routinely reported in medical meta-analyses.
Read the full abstract

All authors T.D. Stanley, John P.A. Ioannidis, Maximilian Maier, Hristos Doucouliagos, Willem M. Otte, František Bartoš

Statistical Methodology · Cross-listed from stat.AP

Amplitude-Phase Analysis of the COVID-19 Point Process and the Early Countermeasures

We investigate how governmental restrictions relate to the spread and temporal dynamics of COVID-19 early in the pandemic. We model daily infection data from each US state as realisations of a point process, taking the random intensity measure to be the latent object of interest and, crucially, allowing these realisations to vary not only in magnitude but also in the temporal dynamics. By non-parametrically separating these amplitude and phase variations, we examine how government restrictions relate to each source of variability, relating the infection curves to the Oxford Stringency Index, which we treat as a measure on the same time window. Employing Wasserstein PCA, we analyse the temporal variability of both the infections and the restrictions. We then use the resulting scores, together with the scalars representing the overall stringency budget and the total infection count, as inputs to a linear vector-on-vector regression model. Our findings suggest that, when considering the separate contributions of amplitude and phase variability, earlier implementation of restrictions is associated with flatter infection curves. By contrast, we do not find significant evidence of an association between stringency and total infection counts, nor between the overall stringency budget and the infection outcomes considered.
Read the full abstract

Statistical Methodology

A Spatial Fay-Herriot Model when the Auxiliary Variables are based on Non-traditional Data

Small area estimation methods combine direct survey estimates with model-based predictions to produce reliable estimates of population quantities. When covariates are measured with error, as often occurs when auxiliary information is coming from big data sources, standard area-level approaches such as the traditional Fay-Herriot model can perform poorly if this measurement error is ignored, potentially yielding biased estimates. Building on the extension proposed by Ybarra and Lohr to account for measurement error in covariates, we further develop this modelling framework by incorporating spatial dependence between areas. We propose a spatial measurement error Fay-Herriot model that jointly accounts for covariate measurement error and spatial correlation, enabling information borrowing across relevant neighbouring areas while properly adjusting for the additional uncertainty introduced by the covariates. We derive the properties of the proposed model and its parameter estimators and outline a parametric bootstrap procedure for estimating the mean squared error of the resulting empirical best linear unbiased predictor. A simulation study examines model performance under a range of measurement error scenarios. The proposed approach is then applied to European Social Survey data, supplemented with big data auxiliary information, to estimate regional attitudes towards climate change in Spain.
Read the full abstract

Statistical Methodology

Local conformal prediction for individual causal effects

Standard CATE estimators become inadequate under strong treatment-effect heterogeneity: confidence intervals for conditional means need not cover individual counterfactual effects. We propose an Individualized Causal Prediction (ICP) framework that constructs finite-sample valid conformal prediction intervals for the individual causal effect of a specific query unit. The method localizes calibration to a causally relevant neighborhood using cosine similarity weighted by Causal Forest variable importance, augments small local samples synthetically, and calibrates intervals with doubly robust AIPW conformity scores satisfying Neyman orthogonality. Under standard identifying assumptions (SUTVA and strong ignorability) and an outcome-independent calibration-set selection condition, the resulting intervals attain marginal coverage at the nominal level. The local design also supports approximately conditional coverage by making calibration scores more representative of the query unit. Experiments on a high-heterogeneity synthetic dataset and the IHDP benchmark demonstrate that local strategies improve point accuracy over global baselines while maintaining nominal or above-nominal coverage.
Read the full abstract

Statistical Methodology

Conditional Regime Analog Forecasting with Trajectories: A Nonparametric Framework for Multivariate Probabilistic

We propose Conditional Regime Analog Forecasting with Trajectories (CRAFT), a nonparametric framework for multivariate probabilistic time-series prediction. The method constructs paired backward and forward trajectory profiles from cumulative multi-horizon returns, learns recurrent low-dimensional regime labels in both spaces using singular value decomposition, change-point segmentation, and segment clustering, and estimates a backward-to-forward conditional regime correspondence. Forecast distributions are obtained by sampling historically realized future trajectory profiles according to a composite compatibility score that combines future-regime correspondence with similarity to the current backward trajectory profile. Unlike parametric vector autoregressions or Gaussian state-space models, CRAFT preserves empirical cross-sectional and multi-horizon dependence by resampling complete future paths. We describe the estimator, its diagnostics, and a reproducible simulation benchmark comparing CRAFT with direct analog resampling, SVD analogs, unconditional bootstrap, OLS VAR bootstrap, and random-forest forecasts.
Read the full abstract

Statistical Methodology

Relative Variability Estimation for the Power Lindley Distribution with Progressive Type-I Interval Censored Data

The measures of relative variability, such as the coefficient of variation, are highly utilized tools of inference in several interdisciplinary fields such as reliability, finance, quality control, and biological sciences. Inferential studies of these measures are highly limited for lifetime models, such as the flexible power Lindley distribution, particularly under progressive type-I interval censoring. In this article, frequentist approaches including the midpoint approximation, maximum likelihood estimation, method of moments, bootstrap, and nonlinear least squares methods, and Bayesian inference using slice sampling are applied for estimation of these measures of relative variability for the power Lindley distribution under progressive type-I interval censoring. Confidence intervals are also constructed based on asymptotic theory, bootstrap and Bayesian paradigm. A discussion on choosing optimal monitoring intervals is also highlighted. A comprehensive simulation study is conducted to evaluate the performance of the proposed estimators across various censoring plans and sample sizes. A real data application illustrates the practical utility of the proposed methodologies. The results indicate that the Bayesian framework generally exhibits superior performance in both point and interval estimation.
Read the full abstract

Statistical Methodology · Cross-listed from stat.AP

Beyond Zipf's Law: Equifinality and Mechanistic Inference from Scaling Laws

Scaling laws summarize complex systems through low-dimensional regularities, but the same marginal law can arise from different stochastic dynamics. We examine this ambiguity for Zipf rank–frequency scaling. An i.i.d. finite-Zipf process, a persistent Markov chain, and canonical sample-space reduction (SSR) are constructed to have exactly the same stationary marginal, $p_j=(jH_V)^{-1}$, while a latent-scale mixture produces a similar marginal through aggregation. The first three models therefore hold the population rank distribution fixed while changing temporal organization. Markov dependence shifts the finite-sample distribution of fitted exponents; at moderate persistence, a block-adjusted effective sample size reproduces most of this shift. Excess lag-1 mutual information detects serial dependence relative to a shuffle null, whereas transition direction separates reversible Markov persistence from the directional contraction of SSR. Conditioning on latent scale reveals the aggregation mechanism. Fit-window, alphabet, and sequence-boundary analyses quantify sensitivity to the observation design. These examples separate the population marginal, the finite-sample behavior of a fitted exponent, and temporal structure. Matching a scaling law is therefore a compatibility condition, not a mechanism identifier: discrimination requires observables on which candidate processes make different predictions.
Read the full abstract

Statistical Methodology · Cross-listed from stat.AP

From Risk Prediction to Risk Mechanisms: A Multi-Resolution Causal Representation for Road Safety and Motor Insurance

Operational risk models can estimate event frequencies precisely while leaving the underlying mechanisms weakly resolved. We study this resolution mismatch using motor insurance and road safety. A revisable DAG represents trip-level crash generation; a separate predictive layer links annual rating information to latent driving states; and an observation process maps crashes into recorded liability claims. Compatibility sets collect the structural and crash-to-claim laws that reproduce an observed annual contrast under stated restrictions. External studies enter only through explicit bridge assumptions and sensitivity bounds. The framework therefore distinguishes sampling uncertainty from uncertainty about structure, observation, and study-to-target correspondence. Two limited examples illustrate the gap. A sublinear mileage relation constrains an aggregate accident rate per unit distance but not its mechanism. In French motor-liability data, the 18-20 versus 40-49 claim-frequency relativity is 3.388 in a model including vehicle and geographic variables and 1.235 when the same model also conditions on a medium-resolution bonus-malus score. These are different predictive functionals, not successive causal adjustments. A Spanish culpability estimate is used only to show how a strong cross-study bridge would restrict a toy bookkeeping region. The framework does not estimate the full DAG; it makes explicit which assumptions are needed before an annual predictive contrast can support a mechanism-specific risk statement.
Read the full abstract

Statistical Methodology

Estimating the average treatment effect under limited overlap via Polynomial Approximation and Extrapolation

Estimating the average treatment effect (ATE) remains a fundamental challenge in observational studies in the presence of poor or limited covariate overlap. Although the inverse probability weighting (IPW) estimator is a widely used approach for estimating the ATE, its performance can deteriorate substantially when overlap is limited, often resulting in increased finite sample bias and unreliable confidence intervals. One common strategy is to shift attention from the original target estimand, the ATE, to alternative estimands such as a class of weighted ATEs that are less sensitive to extreme propensity scores; however, doing so changes the scientific question of interest. In this manuscript, we propose a novel ATE estimator that preserves the original target estimand, the ATE, while improving robustness to limited overlap. A key idea is that this class of estimands can be represented by a polynomial function of a hyperparameter characterizing the estimands. Exploiting this structure, the proposed method computes IPW estimators for a sequence of such estimands, models these estimates using a polynomial regression, and extrapolates to recover the ATE. We show that the estimator has consistency and asymptotic normality under weaker overlap conditions than required for the standard IPW estimator. Simulation studies demonstrate that the proposed method improves estimation accuracy and interval performance in settings with limited overlap. In addition to its theoretical and empirical advantages, the proposed approach has a clear interpretation and is easy to implement using standard statistical software.
Read the full abstract

Statistical Methodology

Sparse Variable Sharpening in High-Dimensional Kernel Density Estimation

High-dimensional kernel density estimation suffers from the curse of dimensionality. This study proposes a hybrid density estimator defined as the product of a joint density over a pre-specified subset of dimensions and marginal densities for the remaining variables, instead of estimating the full high-dimensional density directly. Under this framework, given the observed data, we select a subset of variables for joint density construction to improve estimation accuracy, while modeling the remaining variables through their respective marginal densities. This construction involves a trade-off between the approximation error induced by simplifying the dependence structure and the variance reduction achieved by lowering the dimension of the joint density. We employ a genetic algorithm to efficiently identify such variable subsets. Theoretical and numerical results demonstrate that the proposed estimator can outperform conventional full-dimensional kernel density estimation when this trade-off is appropriately balanced.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Online Learning of Scale Parameters in Score-Driven Filters

A score-driven filter multiplies its scaled log-likelihood score by a scale parameter. We call this coefficient the gain and learn it online. Given the current state and realised scaled score, each admissible gain selects a reachable next state and predictive density. A scalar gain moves along a line; diagonal gains control coordinatewise transmission and may change direction. We evaluate gain selection using a one-step predictive Kullback-Leibler objective. In the scalar unscaled case, the negative consecutive-score product is a stochastic gradient; the positive product used in accelerated recursions is a descent direction. Positive scalar score scaling changes only the effective learning rate. Strictly increasing, continuously differentiable gain links with positive derivative induce mirror-descent geometry, while persistence adds a Bregman pull towards a reference gain. Under convexity, compactness, integrability, and schedule conditions, projected and discounted mirror updates satisfy dynamic-regret bounds relative to time-varying, current-information comparators. Simulations isolate score scaling, link geometry, persistence, and coordinatewise gains. Across twelve equity indices, the bounded discounted-logistic gain records a lower out-of-sample mean negative log score than the constant gain in eleven markets, although market-level evidence is mixed. It also avoids the extreme transients of the numerically capped exponential-link benchmark.
Read the full abstract

Statistical Methodology · Cross-listed from hep-ex

Silent coverage failures in rare-event searches and a degeneracy index that predicts them

Searches for new physics in low-background experiments infer a non-negative signal strength from few events and often report an upper limit. Nominal frequentist coverage requires both a valid interval construction and an adequate data model. We study how controlled model departures affect lower- and upper-endpoint coverage for six interval procedures in an exact Poisson counting experiment, dark-matter recoil spectra, and a neutrinoless-double-beta-decay peak search. We introduce the Poisson–Fisher degeneracy index $\mathcal I_{\mathrm{PF}}(\delta\nu;\vartheta_0)=(\beta,\gamma)$, which maps a specified expected-count deformation, after projection onto the complete fitted tangent space, to $\beta$, the signed fitted signal shift in profiled standard-error units, and $\gamma$, the Poisson–Fisher norm of the unabsorbed residual. Locally, the sign of $\beta$ identifies the threatened endpoint, while larger $\gamma$ implies greater detectability by the saturated-Poisson goodness-of-fit test used here at a fixed $5\%$ type-I error rate. Across the studied deformations, positive signal-like bias degrades discovery-side coverage while making upper limits conservative; negative signal bias from overestimated signal efficiency can make the upper endpoint undercover. Calibration under the nominal simulator does not protect against misspecification of that simulator relative to the data-generating process. A plausible deformation with large $|\beta|$ and small $\gamma$ may therefore evade diagnosis and should be represented by a nuisance constrained with auxiliary information or included in a defensible envelope. In the exactly collinear constrained-nuisance benchmark, modelling the deformation restores coverage at the evaluated grid points, at a quantifiable cost in interval sensitivity.
Read the full abstract

Statistical Methodology · Cross-listed from cs.RO

Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spaces

Reliable uncertainty representation is essential for deploying autonomous systems that interact with their environment, as robots must reason about how uncertainty arising from both stochasticity and model mismatch is impacted by contacts with obstacles (e.g., when navigating through a cluttered environment or inserting a part into an assembly). We propose Calibrated Particle-sets for Trans-dimensional Uncertainty Representation (CaPTURe), a geometry-aware, conformal prediction-based algorithm that generates probabilistically valid prediction regions of the unknown future system configuration using particle-based models of arbitrary fidelity. While calibrated uncertainty predictions are essential for safe and efficient planning, analytical or learned motion models are often inaccurate - due to limited data, simplifying assumptions, unmodeled effects, etc. - which can lead to unsafe executions or task failure. Additionally, when a robot contacts an obstacle, the distribution of its future configurations can become multimodal or disjoint, or lie along manifolds of lower intrinsic dimension than the space of possible robot configurations. Our method uses a calibration dataset of system transitions to locally calibrate motion uncertainty estimates, constructing regions guaranteed to contain the future robot configuration at a user-set probability. Our calibration procedure captures how motion uncertainty varies between contact-rich and contactless motions, leading to sufficient coverage in both cases. We evaluate our method on two simulated planning tasks: controlling a marble around a labyrinth and performing tight-tolerance peg-in-hole insertion with a manipulator. Compared to relevant baselines, CaPTURe achieves the user-specified coverage requirement both in and out of contact and achieves up to a 30% absolute improvement in task success rate over the best baseline.
Read the full abstract

Statistical Methodology

Bayesian ACCESS for Understanding Latent Epidemic Trajectories from Publicly Released Suppressed Data: Application to U.S. Opioid-related Overdose Mortality

Publicly released health statistics play a central role in characterizing temporal trends and identifying structural changes in population health. However, disclosure limitation through suppression of small cell counts, as implemented in systems such as the Centers for Disease Control and Prevention Wide-ranging ONline Data for Epidemiologic Research (CDC WONDER), produces partially observed count data that complicate statistical inference. These challenges are particularly acute for rare outcomes and subgroup analyses, where suppression is widespread and varies across geographic regions, demographic populations, and time. We propose Bayesian ACCESS (Autoregressive Change-point and Clustering Estimation for Suppressed Count Series), a Bayesian hierarchical framework for inference on latent epidemic trajectories and their structural changes from disclosure-limited health statistics. The proposed model directly represents suppressed count data through a suppression-aware observation model, jointly infers multiple temporal change points and latent trajectories, and borrows information across related geographic and demographic populations through Bayesian nonparametric clustering while preserving meaningful heterogeneity. We apply Bayesian ACCESS to opioid-related overdose mortality data from CDC WONDER for U.S. states from 1999 to 2024. The analysis identifies distinct subgroup-specific epidemic trajectories and structural changes that would be difficult to characterize using publicly released health statistics without explicitly accounting for data suppression.
Read the full abstract