Statistical Methodology · Cross-listed from stat.ML
PICPIs: Prediction-Interval-Conditional Prediction Intervals
Xuelin Yang, Baihe Huang, Yilong Hou et al. (+2 more)
A classical question in statistics is which observable quantities to condition on when drawing inferences about unobservable targets. For conformal prediction in nonparametric uncertainty quantification, standard marginal validity offers limited resolution at the prediction values on which decisions are based, and fully conditional guarantees with respect to the
covariates are provably unattainable. We address this gap by introducing a prediction-based conditioning framework that we refer to as Prediction-Interval-Conditional Prediction Intervals (PICPIs). Formally, a PICPI is an interval $I$ satisfying a self-consistency condition: $$\mathbb{E} [Y \mid p(X) \in I] \in I,$$ for predictive model $p$, contextual
covariate $X$, and outcome $Y$. Thus, an interval simultaneously defines a stratum of prediction values and certifies that the mean outcome in that stratum lies in the same interval. This self-consistency condition yields data-adaptive strata without altering the original prediction. Such intervals can be constructed using practical
algorithms. Under regularity of the prediction distribution, the constructed intervals cover all but an arbitrarily small fraction of prediction values and have widths that decrease at rate $n^{-1/3}$, up to logarithmic factors and the prediction error. Moreover, identifying these locally calibrated intervals can, in turn, inform downstream decision-making. We derive inference procedures for PICPIs in probabilistic prediction and multi-class classification, accompanied by theoretical guarantees. Empirical results are provided that compare PICPIs with existing interval-based baselines.
Read the full abstract ▾
All authors Xuelin Yang, Baihe Huang, Yilong Hou, Guido Imbens, Michael I. Jordan
Statistical Methodology · Cross-listed from cs.LG
Concept Drift from a Causal Perspective
Eduardo V. L. Barboza, Jean Paul Barddal, Robert Sabourin et al. (+1 more)
Concept drift is a common phenomenon in real-world data streams, in which changes in the data-generating distribution can degrade predictive model performance. Most existing definitions characterize drift as changes in the joint distribution $P(\mathbf{x}, y)$, without distinguishing which component of the data-generating process has changed. In this work, we introduce a causal perspective on concept drift based on Structural Causal Models (SCMs). We propose a taxonomy that categorizes drift events by their causal origin, including changes in exogenous variables, endogenous mechanisms, confounders, and target-generating processes. Building on this framework, we develop an SCM-based data stream generator that simulates controlled mechanism-level drift events. Our experiments empirically characterize the distributional effects of each drift type and show that drifts with different causal origins induce distinct patterns of distribution shift and predictive behavior. Furthermore, by integrating causal discovery methods, we use our framework to construct data streams grounded in real-world dependency structures, enabling more realistic and informative evaluation scenarios. We also demonstrate that leveraging the generated data can improve downstream performance. These results highlight the importance of accounting for causal structure when studying and evaluating adaptive learning methods, and establish a foundation for causally-aware evaluation in non-stationary environments.
Read the full abstract ▾
All authors Eduardo V. L. Barboza, Jean Paul Barddal, Robert Sabourin, Rafael M. O. Cruz
Statistical Methodology
Closed forms and open obstructions: the sample variance of three observations
Remus Osan, Kevin T. Chu and Ron Yu
The exact distribution of the sample variance for $n=3$ is known since Rietz (1931) for the uniform parent, and Royen (2007, 2008) gave a
Fourier series for any bounded continuous parent. Neither settles which parents admit a finite closed form, nor which special functions it forces. In coordinates aligned with the cube diagonal the variance constraint becomes a cylinder and the cube cross-section a polygon with $S_3$
symmetry, hexagonal over the central band of the diagonal and triangular near either corner; the CDF is the volume of their intersection, the parent density entering as a weight. A closure hierarchy is then organized by the minimal function class containing the parent density:
polynomial parents on any bounded interval always close in elementary terms, a
theorem for the whole class; in the negative direction a single explicit parent already suffices, and for a rational one we prove the CDF is not elementary, the obstruction being an irreducible dilogarithmic part. Beyond those two
theorems the hierarchy is a set of example-specific obstructions rather than a classification: for an algebraic parent the radial first-kind differential is shown non-elementary on a genus-two curve, and the exponential row is a
conjecture supported by the Bessel structure of its radial integral. For the uniform parent we obtain a two-piece formula bifurcating at $Y=1/4$, where the variance disk first circumscribes the hexagonal cross-section at the cube center. For the singular arcsine parent we derive both endpoint laws in closed form and give a six-term approximation accurate to about $10^{-3}$, an accuracy Royen's universal series reaches at about a hundred terms.
Read the full abstract ▾
Statistical Methodology
The Informational Content in Lepto-Variance and Its Relation to Higher Moments
Vassilis Polimenis
Lepto-regression is defined as the machine learning process of constructing a Regression Tree of a target feature on itself. It is a novel, model-free method potentially revealing information on important sample structure properties. But it is yet not clear what the informational content of lepto-variance is and how it is related to other well-known statistics of a sample. One significant finding is that 58% of the historical US stock return variability is 1-bit lepto-variance that can not be explained by any financial factor. The central question investigated in this paper is to use small normal N(0, 1) drawn samples to explore how the 1-bit sample lepto-variance and lepto-ratio relate to sample variance, skewness and excess kurtosis. Using a large sample simulation, the lepto ratio of a normal is found to converge to 36.3%. For smaller normally distributed simulated N(0, 1) samples, while lepto-variance itself is highly correlated to sample variance, lepto-variance as a fraction of total variance is highly correlated to excess kurtosis. Both lepto-variance and lepto-ratio are orthogonal to sample skew. Another finding is that while lepto-ratio is strongly correlated to lepto-variance it remains orthogonal to sample variance.
Read the full abstract ▾
Statistical Methodology
Calibration Count Reuse: Validity Does Not Determine Efficiency
Rudra Chopra
Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity scores: transferring one count from another class to the scored class must not improve its conformity. A leave-self-out full conformal reference proves the criterion without requiring normalization or preservation of same-class score order. For a common separable transformation, universal exchangeable validity is equivalent to being nondecreasing in the count, provided $K\alpha \geq 1$; normalized multiplicative weights obey the complementary nonincreasing condition. Additive penalties are covered under the stated information restrictions. Efficiency has no parallel ordering: two iid constructions make the same valid rule improve or worsen expected size at unchanged coverage. An expanded 55-rule study finds no resolved advantage from selected live-count rules over uniform weights. Image studies identify undercoverage under iid resampling, including at numerical convergence. Separately, execution of the released Conf-OT pipeline on its DTD and Aircraft
benchmark subsets produces near-nominal median coverage under fixed stratified counts. The native results are reported separately from the iid analyses, without treating a
benchmark observation as a universal guarantee. The findings separate validity, classifier confidence, numerical convergence, and population-specific efficiency.
Read the full abstract ▾
Statistical Methodology
Bounding the Risk Difference Bias Given Incomplete Parameters for Externally Adjusting Uncontrolled Confounding
Onyebuchi A. Arah
Bias due to uncontrolled confounding can be analyzed using several external adjustment formulas. Like other bias expressions, the risk difference bias formula involving an uncontrolled polytomous confounder requires only three externally obtainable parameters: confounder prevalences among the exposed and unexposed and the confounder-outcome risk difference. This paper presents two-parameter bounds on the risk difference bias expression when only one of the three required bias parameters is unavailable. Each bound is sharp over the full range of the unknown parameter. Some numerical examples with graphs are provided.
Read the full abstract ▾
Statistical Methodology
Selective Inference for CART with Binary Outcomes
Tomoshige Nakamura
Binary classification trees select subgroups using the same outcomes later used to assess their differences. We develop finite-sample conditional tests of a common success probability within a parent selected by deterministic Gini CART. The construction retains all eligible cutpoints and conditions on the selected split, its ancestor path, the parent success total, and outside outcomes. The resulting uniform label fiber gives an exact count distribution, while a reversible parallel Monte Carlo construction yields super-uniform inclusive and exactly uniform tie-randomized p-values for any prespecified finite run budget. Within a two-child constant-risk model, the selected count law is an exponential family with information equal to its conditional count variance. We separate information loss from computational limitations and exhibit a selected fiber disconnected under single-label swaps. Simulations with 200 or 400 observations, ten independent or correlated predictors, and trees of depth three show conservative inclusive tests and nontrivial power for large risk differences. At 400 observations and a generating risk difference of 0.4, randomized rejection conditional on reaching the prespecified third-level target is 51–71%, while selection followed by rejection occurs in 11–21% of
datasets. Smaller signals remain difficult to detect, and a representative fivefold increase in computation gives little power improvement. The guarantee concerns parent homogeneity, or equality of two constant child risks, and does not cover equality of heterogeneous regional averages.
Read the full abstract ▾
Statistical Methodology
Structured Screen-and-Select for Ultra-High-Dimensional Variable Selection
Nilotpal Sanyal and Padmore N. Prempeh
Ultra-high-dimensional data, with p far exceeding n, are common in genomics and biomedical research. Marginal screening can miss active predictors with weak marginal signals, especially under strong predictor correlation. We develop Structured Screen-and-Select Variable Selection (S3VS), an iterative framework that combines outcome-based screening with correlation-based local predictor sets. At each iteration, S3VS identifies leading variables, forms local sets through predictor associations, applies a model-specific selector, aggregates selected and nonselected variables, and updates the candidate set and, when appropriate, the outcome representation. The framework allows flexible leading-variable, local-set, and aggregation rules, with implementations for linear, generalized linear, accelerated failure-time, and Cox models. For a specified one-step linear configuration, we establish sure screening under conditions on proxy coverage, correlation separation, within-set retention, and active-preserving aggregation. Simulations compare full and first-iteration S3VS with one-pass SIS procedures in linear, logistic, and Cox settings. S3VS can improve variable recovery or prediction when correlated predictors provide useful proxy information, although gains depend on predictor structure and selector choice. In ovarian-cancer data, full S3VS with clinical variables showed the strongest internal discrimination and early prediction, whereas SIS–Cox–LASSO with clinical variables showed the strongest external discrimination. Neither molecular approach consistently reduced prediction error, and no gene was selected in all five outer folds. S3VS provides a flexible framework for exploiting predictor dependence before model-specific selection. The method is implemented in the CRAN R package S3VS.
Read the full abstract ▾
Statistical Methodology · Cross-listed from q-bio.PE
Phylogenetic Inference and the Stickiness of Fréchet Means, via Precise Asymptotics of an Embedded Random Walk
Adam Quinn Jaffe
A well-known phenomenon in statistical analyses of populations of phylogenetic trees in the Billera-Holmes-Vogtmann space is that the
topology of the Fréchet mean tree can contain multifurcations (i.e., internal nodes with more than two children), which raises the practical question of whether this reflects a population-level branching structure (hard polytomy) or merely sampling variability in the data (soft polytomy). This is an instance of the more general phenomenon of "stickiness" in non-Euclidean statistics, whereby the sample Fréchet mean in certain non-positively curved stratified spaces becomes permanently trapped in a lower-dimensional stratum. In this work, we identify a particular multidimensional random walk embedded within the Fréchet mean process, and we show that the time at which stickiness occurs is determined by the largest last-passage time above zero of the coordinates of this random walk. Using this representation, we develop a fully nonparametric procedure for estimating the probability that trifurcations in a sample Fréchet mean tree will bifurcate at some future time if more observations are collected. Lastly, we apply our methodology to a problem in phylogenetics where we consider whether an observed trifurcation in the species tree of primates, glires, and tree shrews is genuinely trifurcated at the population level.
Read the full abstract ▾
Statistical Methodology
On the use of meaningful score regions to interpret treatment effects on continuous clinical outcome assessments
Andrew Trigg and Fraser D. Bocell
Clinical outcome assessments (COAs), such as patient-reported outcomes (PROs), are faced with an interpretability issue due to a variety of score metrics being employed. Therefore, evidence linking COA scores to interpretable patient experiences is necessary to judge the extent to which the results of COA-based clinical trial endpoints are meaningful. The fourth Patient-Focused Drug Development guidance, published in draft form by the FDA in 2023, proposes the use of meaningful score regions (MSRs) to divide COA scores into more easily interpretable categories (e.g. severity-based MSRs of 'none', 'mild', 'moderate' and 'severe'). A recommendation is to compare the magnitude of estimated treatment effects for continuous COA scores (e.g. the difference in mean change from baseline between two treatment groups) to the maximum MSR width. In this article we explore this recommendation further, through historical evidence and simulations, and conclude that using the maximum MSR width as a
benchmark may fail to detect meaningful treatment effects in practice. We suggest alternative approaches to evaluate treatment effects against MSRs, describing the expected outcome within each arm in terms of the probabilities of MSR membership.
Read the full abstract ▾
Statistical Methodology · Cross-listed from stat.CO
Bayesian Calibration with Functional Outputs Using Elastic Partial Matching
Paul Casteras, Julien Bect, Josselin Garnier et al. (+1 more)
Calibrating a simulation model involves estimating its parameters by comparing model outputs with experimental data, so that simulation results faithfully reproduce the experimental observations. When the outputs are functions of time, there are multiple ways to quantify the discrepancy between experimental and simulated curves. A recent approach based on elastic functional data analysis decomposes a functional output into two components: a function temporally aligned to a template, and the corresponding warping function. This decomposition splits the problem into two independent calibration tasks, thereby addressing functional misalignment. However, it assumes that experimental and simulated curves share the same temporal support, an assumption often violated in practice when initial or end times are themselves uncertain or depend on the calibration parameters. In this work, we reinterpret the decomposition step as an approximation to a more general
Bayesian calibration problem that incorporates an error term on the time axis. This perspective allows us to naturally extend the framework to a broader family of time warpings with varying initial or end times, using partial elastic
alignment. We illustrate the method on a synthetic test case, comparing it with existing
Bayesian calibration methods and demonstrating improved surrogate performance and error modeling. We then apply the proposed approach to the calibration of an equation of state (a thermodynamic equation relating the state variables of a material).
Read the full abstract ▾
All authors Paul Casteras, Julien Bect, Josselin Garnier, Gwenaël Salin
Statistical Methodology
Statistical Depth for Interval Data: A Closeness-Based Approach and Its Applications
Xiaozheng Chen and Wenlin Dai
With the rapid development of technology, interval data have been increasingly widely used in various fields, as they can effectively carry information on measurement uncertainty and original variability. Reasonable ranking and outlier screening of interval data are crucial for exploring their value, and statistical depth theory is an important tool to achieve this goal. Existing statistical depth methods are mostly defined for ordinary point-valued data, which are difficult to adapt to the characteristics of interval data and cannot meet the needs of their ranking and outlier detection. Therefore, this paper proposes a statistical depth suitable for interval data and conducts relevant application verification.
Read the full abstract ▾
Statistical Methodology · Cross-listed from stat.AP
Overcoming Model Misspecification in Bayesian Inference of Molecular Signalling Networks
Yinghan Li, Heishiro Kanagawa, Francesca Romana Crucinio et al. (+2 more)
Bayesian inference of molecular signalling networks usually relies on tractability of the marginal likelihood, enabling the set of possible networks to be efficiently explored. As such, linear models with independent errors and conjugate priors are routinely used. However, the dynamics of molecular signalling are nonlinear, and relevant confounders are often unobserved; failure to account for these complexities will almost certainly lead to over-confident inferences in the standard
Bayesian framework. To confront this reality, we develop a post-
Bayesian approach to inference of molecular signalling networks, guided by the principle that uncertainty should not vanish when the statistical model is misspecified, even in the infinite-data limit. Technically, we extend the predictively-oriented (PrO) posterior of McLatchie et al. (2025) to the setting of latent variable models, empirically investigating the properties of PrO posteriors in the challenging network inference context.
Read the full abstract ▾
All authors Yinghan Li, Heishiro Kanagawa, Francesca Romana Crucinio, Matthew A. Fisher, Chris. J. Oates
Statistical Methodology
Filtering and Smoothing with Score-Driven Models
Giuseppe Buccheri, Giacomo Bormetti, Fulvio Corsi et al. (+1 more)
Score-driven models are, by construction, purely predictive filters. When such models are read as filters for an underlying latent process, rather than as data generating processes, this is a restriction rather than a feature, because the contemporaneous and the future observations also carry information on the current state. Starting from the observation that the Kalman filter and smoother recursions for linear Gaussian models can be written in terms of the score of the conditional log-likelihood, and that the predictive step then takes the form of a score-driven recursion, we generalize score-driven models along this direction by deriving an update filter and a smoother, which exploit the contemporaneous observation and the whole sample respectively, together with the corresponding conditional variances. In extensive Monte Carlo analyses the update filter lowers the mean square error of the predictive filter by 3.5% to 8.8%, and the smoother by 32% to 44%, with the ordering holding in every replication; confidence bands built on the associated conditional variances attain their nominal coverage, whereas bands that ignore filtering uncertainty capture less than a third of it. Empirically, we demonstrate the benefits of employing score-driven models as filters rather than as purely predictive processes, showing that the resulting smoothed estimates align more closely with realized quantities than their predictive counterparts.
Read the full abstract ▾
All authors Giuseppe Buccheri, Giacomo Bormetti, Fulvio Corsi, Fabrizio Lillo
Statistical Methodology
Tensor Completion using Subspace Information
Jingyang Li and Michael K. Ng
Tensor completion has attracted significant
attention in both applications and theoretical research. Under standard uniform sampling, existing
polynomial-time guarantees generally require more observations than the number of degree of freedom, motivating the study of a possible statistical-to-computational gap in highly missing regimes. Fortunately, in many practical scenarios, side information is available, which can provide valuable insights to mitigate these challenges. In this paper, we introduce an
algorithm called Tensor Completion using Subspace Information (TCSI) that incorporates side information through an estimated subspace. Our approach first extracts the subspace from the available side information and then reformulates tensor completion as a
matrix regression problem. We provide a theoretical analysis showing that, when accurate subspace information is available, the required sample complexity is reduced to nearly linear order in the uncoupled ambient dimensions, removing the coupled-mode dimension from the leading term. Leveraging the estimated subspace information, we obtain a less stringent sufficient signal-to-noise ratio requirement than those in several existing passive-uniform-sampling guarantees. Under additional mild conditions, we obtain a sharper statistical error bound. Our theoretical findings are supported by numerical simulations. We apply TCSI to the reconstruction of global Total Electron Content (TEC) maps and observe lower reconstruction errors than the compared methods in our experiments.
Read the full abstract ▾
Statistical Methodology
Conformalized Safe Feasible Sets in Uncertain Decision Systems
Yajie Bao, Yinjie Min, Haojie Ren et al. (+1 more)
Safety-critical decision systems often require a downstream optimizer to choose from an unknown feasible set determined by an unobserved label $Y$. Given a context $X$, the goal is to construct a safe subset $D(X)$ contained in the oracle feasible set $A(X,Y)$ with probability at least $1-\alpha$. Existing conformal approaches typically construct a prediction set of the unobserved label $Y$ and retain decisions that are safe for every value in this set. Although valid, this requires a stronger intermediate event than set inclusion. We propose Directed Inclusion Safety Calibration (DISC), a conformal framework that directly controls the probability of this inclusion event by reducing its verification to a scalar critical-inclusion score. Given a pretrained nested family of candidate feasible sets, DISC assigns each labeled observation the smallest nestedness level at which the corresponding subset is contained in $A(X,Y)$, and constructs the safe feasible set using an empirical
quantile at test-time. With data exchangeability, this yields a finite-sample, distribution-free inclusion guarantee. Under two practical set families, we show that DISC produces a safe feasible set containing that obtained by the corresponding calibration baseline. We further develop optimization-based score computation and decision-aware procedures for learning subset families. Experiments across continuous and structured decision problems show that DISC achieves the target inclusion guarantee while producing larger feasible regions.
Read the full abstract ▾
All authors Yajie Bao, Yinjie Min, Haojie Ren, Changliang Zou
Statistical Methodology
A Three-Way Testing Framework for Quantifying Epistemic Calibration Uncertainty in SBI
Luben M. C. Cabezas, Pedro L. C. Rodrigues and Rafael Izbicki
Current experimental scientists increasingly rely on simulation-based inference (SBI) to invert complex models with intractable likelihoods. A primary goal in these settings is to obtain credible regions with valid coverage. While recent model-agnostic conformal calibration methods have succeeded in constructing credible sets with prescribed local
Bayesian coverage, their approximate nature introduces inherent epistemic uncertainty in the calibration process. In this work, we propose a novel tool for diagnosing calibration uncertainty. Our approach is based on a simple three-way hypothesis testing procedure. We demonstrate how this tool can be used to effectively assess necessary simulation budgets for calibration sets and analyze the epistemic uncertainty associated with cutoff estimation.
Read the full abstract ▾
Statistical Methodology · Cross-listed from stat.AP
Transporting summary measures of relative effects from randomised trials to the treated patient population: an application to breast cancer endocrine therapy
Bonnie E. Shook-Sa, Paul N. Zivich, Carolyn Taylor et al. (+4 more)
Randomised trials often report relative treatment effects, such as risk ratios and hazard ratios, for trial populations. Clinical decision-making, however, often benefits from estimates of absolute treatment effects in the population eligible for treatment. Trial participants may not represent this target population well, and restrictions on access to individual participant trial data can further complicate absolute effect estimation. Routine care data are often representative of the target population but may be subject to uncontrolled confounding. We consider estimation of the average treatment effect on the treated (ATT), an absolute measure, by combining a representative sample of treated routine care patients with summary measures (i.e., estimated risk or hazard ratios) from either a randomised trial or a meta-analysis of trials. Under marginal or conditional transportability assumptions, the ATT is shown to be identifiable. The implications of collapsibility of the effect measure on transportability are discussed, and plug-in
estimators of the ATT are presented. Simulation studies are used to assess finite sample performance of the
estimators in a range of settings. The proposed methods are applied to estimate the ATT of endocrine therapy on 15-year breast cancer mortality using results from a meta-analysis of randomised trials and England's National Disease Registration Service.
Read the full abstract ▾
All authors Bonnie E. Shook-Sa, Paul N. Zivich, Carolyn Taylor, David Dodwell, Jake Probert, Sarah C Darby, Paul McGale
Statistical Methodology
Quantifying Long-Range Dependence in Object-Valued Time Series
Won-Ki Seo, Jiazhen Xu and Han Lin Shang
Object-valued time series, including distributions, covariance
matrices, networks, and compositions, lack the linear structure required by conventional autocovariance-based definitions of long
memory. We develop an intrinsic framework for defining and estimating long
memory in metric spaces of negative type. An isometric
embedding yields a centered
Hilbert-valued process, and the trace of its lag-covariance operator provides a signed, additive measure of temporal dependence. Crucially, this trace equals the difference between the marginal mean pairwise distance and expected lagged distance, so the framework and its
estimators use only distances between the original objects. We define the
memory parameter through the nonsummable decay of this trace and show that it coincides with the usual parameter in
Hilbert-valued settings. Estimation uses Bartlett aggregates of distance-based lag measures. Estimating the common marginal distance from the same dependent sample induces a common-centering bias in these aggregates. We derive its finite-sample form and propose iterated block-difference corrections, log-ratio and multi-bandwidth log-slope
estimators, and a localized self-consistency refinement. We establish consistency, document substantial bias reduction in simulations, and find long-
memory evidence in foreign-exchange return distributions and U.S. electricity-generation compositions.
Read the full abstract ▾
Statistical Methodology · Cross-listed from stat.ML
Model-Agnostic Feature Selection via LOCO-Guided Adaptive Minipatch Sampling
Xuhui Liu and Lili Zheng
Black-box machine learning models increasingly deliver strong predictions, but extracting useful information from them, such as a set of important features, remains challenging. Existing model-agnostic methods primarily estimate feature importance or conduct inference on it rather than directly selecting features, whereas many feature selection methods are model-specific or rely on the model-X assumption. We introduce LOCO-guided Adaptive Minipatch Sampling (LAMPS), a model-agnostic ensemble framework that uses any black-box regression
algorithm as its base learner to select features important for predicting the response. The base learner need only produce predictions and need not perform feature selection itself. LAMPS operates within a minipatch ensemble framework that subsamples both observations and features, allowing leave-one-
covariate-out (LOCO) feature importance scores to be easily computed. It adaptively concentrates minipatch sampling on features with high LOCO scores while maintaining exploration. The resulting sampling probabilities rapidly separate signal from noise features after a few iterations, enabling selection through simple thresholding. We establish that LAMPS achieves exact feature selection in high-dimensional settings, provided that the base predictive models are sufficiently well trained on average. Extensive experiments on synthetic and real data show that LAMPS outperforms state-of-the-art feature selection methods, with particularly strong performance in the presence of correlated features.
Read the full abstract ▾