Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistical Methodology

stat.ME Watch Statistical Methodology (sign in to use)

Friday 18 September

Statistical Methodology

Orthogonal Policy Learning with Ordinal Outcomes

Policy learning methods based on conditional average treatment effects can obscure subpopulation heterogeneity when applied to ordinal outcomes. We develop a policy learning framework for ordinal outcomes with heterogeneous utilities for individuals who strictly benefit from treatment and those who do not. Since the probability of strict benefit is only partially identified without further conditions, we adopt a minimax strategy which minimizes the worst-case regret over the identification region. To estimate the resulting nonsmooth objective, we combine smooth approximations with Neyman orthogonalization to remove first-order bias from nuisance estimation. We also derive an excess worst-case regret bound over a restricted policy class. The proposed method is validated through extensive simulations and an application to the 2022 Survey of Income and Program Participation (SIPP) data.
Read the full abstract

Statistical Methodology

Stochastic Epidemic Model Criticism with Neural Evidence Estimation

Neural techniques for simulation-based inference (SBI) have become popular in scientific domains featuring complex simulation models. These methods train neural networks on simulated data to infer unknown parameters; their reliability depends on the simulator faithfully representing reality. In infectious disease modeling, multiple models encoding different hypotheses often provide plausible explanations of observed data. Model selection is a difficult problem when likelihoods are unavailable, as is the case with many common stochastic transmission models. We propose a model criticism methodology designed for SBI, Neural Evidence Estimation (NEE), that integrates model selection and model misspecification detection. NEE learns a model's marginal likelihood (a.k.a. evidence) directly from simulations using normalizing flows, thereby avoiding noisy or complex Monte Carlo estimators. Through synthetic experiments, we demonstrate that NEE reliably selects the correct data generating process among multiple models and flags poorly specified models. We apply NEE to a real-world modeling problem concerning competing mechanisms for influenza reinfection in a multiple wave epidemic. This work argues for the importance of principled model criticism in SBI and infectious disease modeling.
Read the full abstract

Statistical Methodology

Adaptive online kernel changepoint detection

We propose an adaptive online kernel-based changepoint detection method for streaming data that is capable of detecting a broad range of changes in the underlying data distribution. The method maintains a recursively-weighted reproducing kernel Hilbert space representation of observations and adaptively updates the forgetting factor through a gradient-based procedure driven by a maximum mean discrepancy-type statistic between the current observation and the weighted empirical distribution of the past. This self-tuning mechanism allows the detector to adapt its effective memory and responsiveness to changes in the underlying process. Simulation results demonstrate that the proposed method achieves strong detection performance across a wide range of distributional changes. Further, our proposed approach maintains constant computational and storage cost through recursive updates, and is very computationally efficient in comparison to competing methods. Experiments on both simulated data and benchmark real-world datasets show improved performance over several other leading kernel-based methods.
Read the full abstract

Statistical Methodology · Cross-listed from math.ST

Asymptotics and finite sample bounds for prediction and smoothing in Wright-Fisher hidden Markov models

We study signal prediction and smoothing in hidden Markov models whose latent signal is a multi-type Wright-Fisher diffusion and whose discrete-time observations are categorical. We consider a repeated-sampling regime with finitely many inspection times and large within-time samples. Under parent-independent mutation, the Bayesian predictive and smoothing laws admit exact representations as mixtures of Dirichlet distributions. We show that the one-step predictive and local smoothing laws converge in total variation to the corresponding Wright-Fisher transition and bridge laws determined by the neighboring population frequencies, thereby justifying plug-in approximations based on empirical proportions. For prediction, we derive explicit finite-sample bounds for this approximation and for bounded predictive functionals such as diversity, together with a finite-sample concentration bound for the marginal smoother. The joint posterior at the inspection times concentrates at the target frequencies and satisfies a Bernstein-von Mises limit with distinct interior and boundary regimes, with Gaussian and Gamma limits on different scales. The results allow arbitrary dependence across inspection times. A key technical ingredient is a fixed-interval tail bound for the coalescent-based dual death process.
Read the full abstract

All authors Luigi M. Malgieri, Filippo Ascolani, Matteo Giordano, Matteo Ruggiero

Statistical Methodology

Bridging Local and Population Causal Effects: A Proximal Instrumental Variable Approach

Instrumental variable (IV) methods address treatment endogeneity, but with non-compliance and heterogeneous treatment effects a binary instrument generally identifies the local average treatment effect (LATE) among compliers rather than the population average treatment effect (ATE). When treatment effects and compliance probabilities are heterogeneous and dependent through latent factors, the ATE need not be identified by IV variation alone. We develop a proximal instrumental variable framework that uses proxies for these factors to adjust for the compliance weighting in LATE and recover the ATE. We show that the ATE is identified through either an outcome bridge capturing latent treatment effect heterogeneity or a compliance bridge representing inverse compliance probabilities, and combining the two yields a doubly robust representation. We derive the efficient influence function under known and unknown instrument propensities. For estimation, we propose regularized kernel minimax bridge estimators, with orthogonalized conditional moments for estimated propensity, and a three-way sequential cross-fitting scheme. We establish projected bridge-risk bounds, and give conditions for asymptotic normality and semiparametric efficiency. The framework provides a proxy-based route from local IV effects to population causal effects under latent treatment effect and compliance heterogeneity.
Read the full abstract

Statistical Methodology · Cross-listed from cs.SI

Auditing bipartite motif interpretations: a worked example with conservation checks and open-path decomposition

Motif profiles of bipartite agent-object networks, such as tourist-site visits and customer-item transactions, are read as evidence about structural roles and about differences between networks, often without asking what the two degree sequences already fix. In a simple bipartite graph the induced k-fan count on one node type is a sum of degree combinations, so it has zero variance under a null that preserves both degree sequences. We apply this known result to a reconstructed tourism rating network of 17 tourists, 80 sites and 637 edges, the sole inferential worked example, and, as a provenance-limited illustration, to published motif-instance aggregates over 36 monthly luxury customer-item networks. The four fan classes are exact functions of the degree sequences: in the tourism network the raw fan counts and the size-3 two-fan ratio (84.7% fan-out) restate those sequences. The published luxury counts require at least 89,502 customer-item edges against 26,451 reported transactions, so their 99.8% fan-in is reported as a descriptive value only. Against a hard bipartite configuration null, the four-cycle count is degree-consistent (z about +1.0) and the open path is deficient (z about -5.8) by 2,243 instances, 2.4% of the null mean; the deficit survives every leave-one-tourist-out re-run (z -4.5 to -8.1). An exact identity splits it at the point estimate into 64.4% mixing and 35.6% four-cycle, but that split is not an attribution: the observed mixing term lies below all 500 null samples, the two components are almost collinear under the null (r = 0.968), and the mixing share ranges from 36.6% to 116.5% under leave-one-tourist-out deletion. The deficit is extreme relative to the sampled null, while its class-level interpretation is undetermined and unstable. We give a four-step pre-interpretation check and a reference implementation.
Read the full abstract

Statistical Methodology

RKHS-Based Inference for Nonlinear Granger Causality via Conditional Centering

Granger causality is commonly formulated through linear prediction in vector autoregressive models, limiting its ability to detect nonlinear predictive relationships. We propose a reproducing kernel Hilbert space (RKHS)-based test for nonlinear Granger non-causality in conditional mean for nonlinear autoregressive processes. The key idea is a conditional-centering decomposition of the target regression function into an own-history component and an orthogonal component capturing the additional predictive contribution of the potential source history. Non-causality is characterized by the vanishing of the latter component. The own-history component is estimated by kernel ridge regression, and the residuals are embedded in a second RKHS using a conditionally centered kernel. This yields an RKHS-valued residual moment whose squared norm forms the test statistic. We establish a weighted chi-square null limit and consistency against fixed alternatives for the population-centered statistic. An empirically centered version is shown to retain the null limit and enables spectral calibration of critical values and $p$-values without resampling. Simulation studies demonstrate accurate size control and power against nonlinear alternatives, and real-data applications illustrate the usefulness of the method for detecting nonlinear predictive relationships in time series.
Read the full abstract

Statistical Methodology

A Novel Convolution-Based Stratified Attribute Estimator for QRE Determination in R&D Tax Credit Studies

The One Big, Beautiful Bill Act reinstates immediate expensing of domestic research expenses, reducing the tax burden and incentivizing reinvestment in U.S.-based research and development including software development across a wide range of industries. In this context, accurate and defensible estimation of qualified research expenses (QREs) is of high importance. Statistical sampling provides a practical framework for estimating QREs for a well-defined population of business components (sampling frame) documented in accordance with Internal Revenue Code Section 41 (Form 6765). IRS Revenue Procedure 2011-42 permits both attribute and variable statistical methods, although the latter (stratified mean and difference estimators) are often regarded as the standard approach to QRE estimation. In this paper, we compare attribute and variable statistical methods within a simulation study and demonstrate that attribute methods offer several theoretical and practical advantages. A key concern with the simple attribute estimator is the possibility of upward bias in QRE determination arising from a preponderance of high potential QRE (pQRE), non-qualified projects in the sampling frame. We address this concern by introducing a stratified sampling design that ensures adequate representation of high pQRE projects in the sample. We demonstrate that a convolution-based stratified attribute estimator produces a valid one-sided 95% lower confidence bound on total QREs across a range of sampling frame structures.
Read the full abstract

Statistical Methodology

Integrating Multivariate Adaptive Regression Splines into Small Area Estimation for Nonlinear Poverty Modeling in Java Island

Small area estimation (SAE) modeling is often used to address the unreliability of estimates in areas with limited survey samples. However, standard SAE models (Fay-Herriot) can only accommodate linear patterns, whereas in real cases, including poverty, many patterns are nonlinear. This study proposes the implementation of a supervised learning model, multivariate adaptive regression splines (MARS), integrated within the SAE framework (MARS-SAE) to address the challenge of unreliable direct estimation, particularly in nonlinear cases. The hyperparameter tuning results yielded the best model with the following hyperparameter combinations: maximum basis function (Max BF) = 10, maximum interaction (MI) = 2, minimum observation (MO) = 1, penalty = 3, with general cross validation (GCV) = 6.128. MARS-SAE demonstrated very strong performance; empirically, the model proved more efficient at reducing estimation error due to its higher relative efficiency (RE) compared to the SAE Fay-Herriot model, as well as better reduction of relative standard error (RSE) than the comparison model, and it yielded a lower relative root mean squared error (RRMSE) compared to the SAE Fay-Herriot model (13.64% vs. 13.75%). MARS-SAE has been proven to capture nonlinear patterns very well while still maintaining interpretability, making it an effective tool for the predictive and diagnostic analyses of phenomena. Thus, this research can serve as an important reference for readers regionally and in Southeast Asia, given that Java Island is a strategic region for Indonesia, the largest economic power in the ASEAN region.
Read the full abstract

Statistical Methodology

Consistency of Optimal Matching-based Clustering for Mixtures of Markov Chains

We study clustering of categorical sequences using the Optimal Matching (OM) distance under finite mixtures of finite-state Markov chains. We show that the normalized OM distance between two independent chains converges almost surely to a deterministic population quantity, concentrates exponentially around its finite-horizon mean, and admits an $O(\sqrt{\log n/n})$ convergence rate when the two chains have the same transition kernel. These population quantities yield a natural separation condition: the largest within-component limit must be smaller than the smallest between-component limit. Under this condition, hierarchical clustering with any bracketed linkage and Partitioning Around Medoids consistently recover the latent mixture partition. We also propose a consistent estimator of the number of components based on empirical OM distance profiles. The results extend to finite-state hidden Markov models and multichannel categorical observations. Overall, they provide a statistical justification for standard OM-based clustering methods for categorical time series.
Read the full abstract

Statistical Methodology · Cross-listed from stat.ML

Riemannian Simultaneous Inference for Tangent Vector Field Regression

We consider nonparametric tangent vector field regression on a Riemannian manifold without boundary. Because responses at different points lie in different tangent spaces, the proposed kernel estimator first parallel transports nearby responses to the target tangent space and then forms a volume-corrected local average. We first derive its uniform second-order bias, finite-bandwidth covariance, and stochastic rate. For simultaneous inference, the tangent norm is written as a supremum over the unit tangent bundle. Exact covariance whitening gives a unit-variance Gaussian field whose correlation length is of order $h$ along the base manifold and of order one along the fibre. Its local covariance geometry leads to a Gumbel limit with an explicit intrinsic constant. Combining this limit with Gaussian approximation and cross-fitted covariance estimation yields a feasible simultaneous confidence tube for the regression field. We further discuss improved finite-sample inference with bandwidth selection and high-order bias corrections. Simulations on various manifolds support the proposed inference procedure. A randomized reconstruction of global wind data illustrates how the tube's cross-sections describe spatially varying uncertainty.
Read the full abstract

Statistical Methodology

Optimal study interior partitioning (OSIP): An optimization approach to observational study design under fine balance constraints

Covariate matching between treatment and control groups remains a foundational methodology in observational causal inference, designed to replicate the covariate balance of randomized controlled trials. Despite substantial methodological advancements, applied researchers continue to face an inherent trade-off between achieving rigorous balance and preserving the statistical power of the matched sample. We introduce a novel framework for a matching method named Optimal Study Interior Partitioning (OSIP). OSIP frames observational matching as a joint optimization problem governed simultaneously by a maximum cardinality constraint on the number of strata, and a strict fine balance threshold on the matching carried out in each strata. OSIP is based on two-stage optimization algorithm. In both stages it considers a projection of the (high-dimensional) data into one-dimensional space by evaluating an estimated propensity score for each unit. The method is based on enforcing a partition of the input to subsets of consecutive intervals of estimated propensity scores. The novelty arises because we consider this task as a global optimization problem. In the first stage, we employ an optimal partitioning algorithm with respect to an auxiliary (one-dimensional) goal function. In the second phase, we go back to the original distance function and apply heuristics approaches locally optimizing the solution from the first stage. We compare our framework with established methodologies from the literature across well-studied empirical benchmarks. We use the standard benchmark of Lindner cardiovascular dataset to demonstrate that OSIP provides a highly efficient sweet spot solution for the corresponding multi-criteria goal. This conclusion is verified by considering also other benchmark datasets.
Read the full abstract

Statistical Methodology

A Partial Fay-Herriot model for Small Area Estimation: Estimating district-level consumption in Mozambique

This paper proposes a new small area estimation approach that integrates Partial Least Squares within the Fay-Herriot model to address the challenges posed by high-dimensional and highly correlated auxiliary variables sets. The resulting Partial Fay-Herriot (PFH) estimator constructs supervised components that maximize their association with the target variable, enhancing model stability and predictive efficiency. Monte Carlo simulations demonstrate that PFH estimator achieves lower mean squared error than the standard Fay-Herriot estimator and outperforms principal components-based alternatives while relying on fewer latent dimensions. The methodology is applied to the estimation of district-level per capita consumption in Mozambique, where the survey data source is complemented by large set of correlated census variables. The resulting estimates highlight pronounced geographic heterogeneity and uncover spatial clusters of deprivation. Overall, the findings show that the proposed supervised dimension-reduction approach represents an effective and easily interpretable tool for producing reliable indicators in high-dimensional contexts.
Read the full abstract

Statistical Methodology

Testing Conditional Stochastic Dominance via Copula Derivatives

Comparing two populations at the same physical covariate value requires more than conditional means or isolated target-point decisions: researchers may need evidence about an entire conditional-distribution ordering over a continuum, even when covariate margins differ. This paper makes that common-value comparison estimable under an explicit structure–flexibility tradeoff and turns the resulting surface into simultaneous evidence for first-order stochastic dominance. Population-specific margins map the common covariate value into each group, while a fitted copula-derivative representation links conditional distributions across the region. Uniform inference propagates uncertainty from both the margins and dependence model through a one-sided statistic with unknown binding locations. Under correct specification within a finite copula class, smoothness and trimming conditions, and a uniquely best candidate family, the procedure admits uniform control and consistent calibration. Simulations show increasing rejection as alternatives become more distinguishable, alongside model-selection sensitivity and small-sample size distortion. In a descriptive PSID application, the high–low parental-education comparison satisfies the two-direction criterion after multiplicity adjustment, whereas adjacent education-group comparisons remain inconclusive. The framework therefore supports region-wide distributional comparison while making its structural and inferential boundaries explicit.
Read the full abstract

Statistical Methodology

Phasing out Monte Carlo: exact acceptance probabilities for mean-spread release rules

A large family of regulated release decisions accepts a production lot if and only if the pair (sample mean, sample standard deviation) of $n$ units falls in a fixed plane region. Content uniformity (USP <905>), capability release, variables sampling and percent-within-limits highway acceptance all take this form. The acceptance probability under a hypothesized unit-level law is not, so far as we are aware, evaluated exactly in current practice: for a general parent the joint finite-$n$ law of $(\bar X,s)$ is a constrained $n$-fold integral (Craig 1932 at $n=3,4$; Springer 1953 at general $n$) which admits no elementary reduction. Yet that probability is fixed by the law of the additive pair $(T_1,T_2)=(\sum_{i=1}^{n} X_i,\sum_{i=1}^{n} X_i^2)$, whose characteristic function is the $n$-th power of a one-observation transform. One two-dimensional Fourier inversion therefore delivers it for an arbitrary smooth parent - exactly, as an identity, and numerically on a grid whose dimension stays two whatever $n$ is - a deterministic alternative to both the simulation and the Normality assumption. The identity is exact; what we evaluate is a finite-grid inversion of it, whose error we report per example rather than bound. At $n=3$ the Uniform parent yields the joint law of $(\bar X,s)$, the acceptance probability and the capability law in closed form; we derive these, use them as the anchor of a validation ladder, and then work three rules end to end under non-normal parents at their prescribed sample sizes ($n=10$-$30$). Against the exact answer the Normal-theory incumbent errs by enough to change the risk a plan is designed against, and errs in both directions along the operating curve, so no single constant corrects it. In general only the full distribution suffices.
Read the full abstract

Statistical Methodology

Model-based estimation and imputation with torus missing values

This paper addresses the problem of parameter estimation and model-based imputation for multivariate circular data lying on a p-dimensional torus in the presence of missing values. Actually, the periodic nature of the sample space invalidates conventional imputation techniques designed for Euclidean data. Then, we propose a general framework for maximum likelihood estimation under the wrapped elliptically symmetric family of distributions, with particular interest in the multivariate wrapped normal distribution, when the missing data mechanism is ignorable. The methodology leverages the conditional properties of the elliptically symmetric distributions on the unwrapped space, embedding the imputation of missing torus data into an Expectation-Maximization algorithm that treats both the wrapping coefficients and the missing entries as latent variables. Derivation of both the E and M steps is detailed and a working algorithm is discussed. Imputation methods are also taken into account. The finite-sample performance of the maximum likelihood estimator under ignorable missingness is assessed through Monte Carlo simulations under the wrapped normal specification. The methodology is also illustrated on data with artificially introduced missingness.
Read the full abstract

Statistical Methodology

Fuzzy entropic triple k-means

This paper proposes fuzzy entropic triple k-means (FE3KM), an entropy-regularized fuzzy partitioning method for three-way data arrays that simultaneously clusters objects, variables, and occasions. Unlike standard fuzzy clustering, which controls fuzziness through a nonlinear exponent m, FE3KM embeds memberships linearly within a Least-Squares (LS) objective, using row-stochastic membership matrices and entropy regularization to represent uncertainty in cluster assignments. This linear structure is what enables an exact decomposition of total deviance into within- and between-cluster components, further attributable to each mode (objects, variables, occasions) and to individual clusters within each mode. This is an interpretive property unavailable to exponent-m fuzzy models. We derive Alternating Least-Squares algorithms for FE3KM, including an accelerated variant for large arrays, and formally relate the method to tandem clustering procedures and to Tucker3/three-mode partitioning models. A simulation study assesses recovery accuracy, robustness to noise and fuzziness mis-specification, and performance against tandem and three-way clustering baselines. An application to a benchmark three-way TV ratings dataset demonstrates how FE3KM recovers interpretable object-variable-occasion structures and quantifies the contribution of each cluster and mode to overall data variability.
Read the full abstract

Statistical Methodology

Estimating heterogeneous treatment effects from randomised trials: a comparison of the risk modelling and treatment effect modelling approaches

Introduction Risk modelling (RM) and treatment effect modelling (EM) are two approaches to build models to estimate heterogeneous treatment effects from randomized control trials (RCT). RM is a two-stage approach; it estimates a predicted baseline risk at the first stage and then includes it as the only effect modifier in a regression model in the second stage. EM is a full treatment interaction multivariable model. Both approaches have theoretical advantages and limitations, but a thorough comparison including a simulation study is missing. Methods In a theoretical review of the two approaches, we present their underlying assumptions and theoretical advantages and disadvantages. Based on our theoretical review, we design simulation scenarios for RCTs with a dichotomous outcome and evaluate the performance of both approaches with respect to the root mean square error and the bias in the predicted risk difference. Results In the theoretical part we argue that baseline risk is a treatment effect modifier in many clinical situations. RM is a dimensionality reduction approach which, however, makes strong assumptions about the role of prognostic factors modifying the treatment effect. EM's greater flexibility is a possible advantage when the sample size is large. In most simulated scenarios RM performs better than EM, even when the assumptions underlying RM are not fully met. The advantage of RM diminishes as sample size increases. Conclusion When choosing between RM and EM the available sample size and the plausibility of their underlying assumptions should be considered.
Read the full abstract

All authors Frederik Luca Philipona, Marta Mainetti, Orestis Efthimiou, Georgia Salanti

Statistical Methodology · Cross-listed from stat.ML

Improving the Predictive Performance of Bootstrap Aggregating by Dirichlet Resampling

We revisit Breiman's observation that reducing inter-tree correlation without weakening individual trees can improve random forests. Building on this principle, we introduce two variants: Dirichlet-Multinomial Bagging Random Forest (DM) and Dirichlet-Weighted Random Forest (DW). Both modulate sample reweighting via a concentration parameter $\alpha>0$. We provide a simple theoretical criterion that clarifies when these variants behave indistinguishably from standard random forests, and we use it to guide a lightweight tuning strategy. In a controlled evaluation on public classification benchmarks, DM and DW are consistently competitive and often stronger than other random-forest (RF) baselines, with negligible additional runtime.
Read the full abstract

Statistical Methodology

Robust tests for log-normal lifetimes under cyclic-stress accelerated tests and interval monitoring

Some products are, by nature, designed to operate under stress conditions that repeatedly cycle between two levels, such as batteries undergoing repeated charge and discharge, or components exposed to recurring thermal or pressure cycles. When such products are tested under accelerated conditions, the applied stress is naturally elevated in the same cyclic manner rather than held constant or increased in a single step. Cyclic-stress accelerated life testing (CyALT) reflects this by alternating the applied stress between specified levels. Units are often inspected only at fixed times, so the data consist of interval failure counts rather than exact lifetimes. Hypothesis testing plays a crucial role in this setting. It can assess whether the applied stress significantly affects lifetime, or validate the parameter values used to design the experiment. Classical tests rely on the maximum likelihood estimator, which is highly sensitive to contamination. Since interval probabilities are estimated jointly across stress conditions, a single anomalous failure count can distort the entire parameter vector and severely affect the test's significance level and power. This paper develops Wald-type and Rao-type test statistics for log-normal CyALT lifetimes under interval monitoring, based on the weighted minimum density power divergence estimator. Both tests are derived for simple and composite null hypotheses. A Monte Carlo study shows that classical tests lose control of the significance level rapidly under contamination, while the robust tests stay close to nominal with only a modest loss of power under clean data. An application to air-conditioner evaporator lifetime data further illustrates the proposed tests.
Read the full abstract

Statistical Methodology · Cross-listed from econ.EM

Summary Indices in Treatment Effect Estimation

This paper studies the practice of combining multiple outcomes into a summary index to estimate a causal effect. For common estimators and index constructions, the estimate equals a weighted sum of the estimated effects on the components, with weights that are implicit and rarely reported. The paper derives the weights and shows that, for inverse-covariance-weighted indices, they can be negative and unrestricted in magnitude, so the index effect can have the opposite sign to every component effect. The paper proposes two procedures for valid inference on the index effect: a variance estimator that accounts for the data-dependent weights, and a shifted t-test that requires no such correction. Conventional t-tests of the null of no effect remain valid. Contrary to common claims, summary indices do not generally improve power. Three published studies illustrate the results.
Read the full abstract

Statistical Methodology

Robust Dual-Regularized Variable Selection under Outlier Contamination

Real data often contain unusual observations that can exert disproportionate effects on variable selection, especially in complex predictor settings. We propose a two-stage sparse median outer product of gradients (smOPG) method for variable selection in single index models with outlier contamination. We first estimate sparse local gradients via \(\ell_1\)-penalized local median regression and then recover the active predictor set from a rank-one sparse approximation of the resulting gradient matrix using regularized singular value decomposition. The combination of median regression and local weighting provides robustness to both response outliers and leverage points. Extensive simulations across varying dimensions and contamination mechanisms demonstrate the favorable variable selection performance of smOPG relative to existing methods. Applications to air pollution and genomic data demonstrate practical utility, while theory establishes active-set recovery without requiring selection consistency of individual local regressions.
Read the full abstract

Statistical Methodology

Asymptotic Anytime-Valid Quantile Inference under Local Differential Privacy

Sequential quantile inference is difficult under local differential privacy because every record is randomized before reaching the analyst and the limiting quantile variance depends on an unknown density. We develop an online procedure that combines randomized response with dynamically chained parallel stochastic gradient descent (P-SGD). The resulting Polyak–Ruppert estimator admits a strong Gaussian approximation. A cross-chain quadratic statistic, computed entirely from private iterates, consistently estimates the limiting variance without a separate online density estimator. These results yield asymptotic confidence sequences and, under polynomial chain growth, asymptotic time-uniform coverage. Arm-wise constructions support locally private quantile best-arm identification, time-uniform simple-regret bounds, and sequential A/B tests of quantile treatment effects. Simulations and salary-data analyses illustrate the finite-sample behavior and practical use of the proposed methods.
Read the full abstract

Statistical Methodology

Where quantum and quantum-like methods earn their place in behavioural-trial analysis: three simulation pilots across the RCT pipeline

Behavioural randomized controlled trials (RCTs) are the empirical workhorse of behavioural science, yet quantum computing has reached the field only through opinion-dynamics demonstrations and decision-theory formalism, leaving the RCT analysis pipeline unexamined. I ask, stage by stage, where quantum or quantum-like computation earns its place against a strong classical baseline, and report three controlled simulation pilots across the pipeline. In a power-calculation pilot anchored on a 97-outcome preprocessing multiverse of real behavioural RCTs, quantum amplitude estimation recovers enumerated multiverse-significance fractions with absolute error 20-30 times smaller than classical sampled Monte Carlo at matched query budgets on the two non-degenerate outcomes, conditional on efficient state preparation. In an interference pilot, an entanglement-structured estimator recovers a four-way spillover that a pairwise model cannot represent at any sample size. In an adaptation pilot, a quantum-inspired contextual bandit is decisively beaten by a correctly specified classical baseline. The three results - one conditional win, one representational win, one honest loss - yield a decision rule for when quantum methods are worth adopting, piloting, or deferring, together with a five-gap research agenda.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Reliability-Centered Evaluation of Sparse Longitudinal CT Lesion-Size Forecasting with Conformal Interval Calibration and Gompertz-Inspired Regularization

Sparse longitudinal CT follow-up limits lesion-size forecasting when only a few prior observations are available. We constructed a five-visit DLT-derived same-lesion trajectory benchmark from DeepLesion and Deep Lesion Tracker (DLT), yielding 205 trajectories from 129 patients. We compared an exploratory conventional sparse-to-final analysis with a primary fixed visit-index horizon design predicting the common log change from T3 to T4 while progressively adding earlier observations, evaluating predictive accuracy, uncertainty reliability, post-hoc conformal interval calibration, subgroup performance, and Gompertz-inspired trajectory regularization. The evaluated methods showed partially overlapping point-prediction accuracy but distinct uncertainty behavior. Mean held-out RMSE across ten training seeds was 0.4726, 0.4305, 0.4499, and 0.4513 for m = 1, 2, 3, 4, indicating the lowest mean RMSE at m = 2; additional history did not improve RMSE. At m = 4, raw Cohort-Level Feature GP coverage was near the 95% nominal level, whereas MC Dropout, Deep Ensemble, and residual-scale intervals were conservative. Patient-level conformal calibration generally produced near-nominal or conservative coverage at the cost of wider intervals. Patient-grouped development cross-validation selected lambda* = 0 for the Gompertz-inspired term. A global population reference frequently opposed lesion-level change directions, and prediction difficulty varied across anatomical subgroups. Overall, additional historical observations provided limited predictive benefit once the prediction horizon was controlled, while predictive accuracy, uncertainty reliability, and trajectory consistency did not necessarily improve together, and should be evaluated jointly in sparse longitudinal imaging.
Read the full abstract

Thursday 17 September

Statistical Methodology

Bayesian additive regression trees for evaluating treatment benefit predictors using observational data

A treatment benefit predictor (TBP) is an algorithm that maps a patient's characteristics to their putative benefit from a given treatment, which can be used to inform treatment decisions. However, a TBP must be evaluated in the target population before being adopted for patient care. When only observational data are available, evaluating TBPs requires standard causal identification assumptions, as treatment assignment in such settings is not random. We obtain the posterior distributions of predictive performance measures to evaluate prespecified TBPs using observational data, by taking advantage of Bayesian additive regression trees (BART). We illustrate the evaluation of TBPs using selected measures and graphical visualizations: the concentration of benefit ($C_b$) index and the moderate calibration curve. Simulation studies of binary and continuous outcomes settings, including balanced and imbalanced treatment allocation in the binary setting, establish the validity of the proposed approach. In a case study, we use this approach to assess a TBP for systemic antibiotic therapy for patients with chronic obstructive pulmonary disease (COPD). We show that the constructed TBP does not in fact make calibrated predictions, because it both makes optimistic predictions of risks and exaggerates the risk reduction due to treatment. We conclude that flexible Bayesian approaches have the potential to assess TBPs, offering opportunities for flexible model specifications, adjustment for confounding, and uncertainty characterization.
Read the full abstract

All authors Yuan Xia, Mohsen Sadatsafavi, Jeenat Mehareen, Paul Gustafson

Statistical Methodology

A New Generalized Birnbaum-Saunders Regression Model: Inference, Diagnostics, and Applications

The Birnbaum-Saunders (BS) distribution has become a widely used model for positive and asymmetric continuous data. Several extensions of the BS distribution have been proposed, including regression formulations that relate its parameters to covariates. In this paper, we introduce a new regression model for the Generalized Birnbaum-Saunders (GBS) distribution, an extension that has received limited attention in the literature despite offering a stronger physical interpretation. The proposed framework follows the structure of Generalized Additive Models for Location, Scale, and Shape (GAMLSS), allowing covariate effects to be incorporated into all three parameters of the distribution, thereby providing greater flexibility for modeling data. In particular, one of the model parameters has a direct interpretation as the median of the response variable, facilitating the interpretation of covariate effects on the conditional median. Parameter estimation is performed via maximum likelihood, and hypothesis tests are proposed based on the Wald test. The proposed regression model is implemented in R through the gamlss package, allowing access to a broad range of tools for model fitting, diagnostics, and assessment. Monte Carlo simulation studies are conducted to investigate the finite-sample performance of the maximum likelihood estimators. The results show that the estimators exhibit desirable properties, with bias decreasing and efficiency increasing as sample size grows. Additionally, the behavior of the Wald test is investigated, showing good performance for moderate to large sample sizes. Finally, two applications to real data illustrate the practical usefulness of the proposed GBS regression model, demonstrating that it is a flexible alternative for modeling positive and asymmetric data compared to the BS model
Read the full abstract

All authors Matheus B. Milhomem, Terezinha K. A. Ribeiro, Michelli Barros, Eriton B. Santos

Statistical Methodology

Aggregated Posterior Predictive Checks for Generative Modeling

Latent variable generative models are commonly fit using simple priors over latent variables, but draws from these priors often fail to produce realistic data. This failure is due to a mismatch between the prior and the aggregated posterior, the distribution of latent variables induced by the fitted model and the data. This mismatch is often viewed as evidence that the prior is misspecified and should be replaced. Alternatively, in modern generative models, a two-stage strategy is increasingly used where first, the model is fit, and second, the aggregated posterior is estimated (van den Oord et al.,2017; Rombach et al., 2022.). Synthetic data are then obtained by sampling from this aggregated posterior instead of the prior. To check such procedures, we introduce the aggregated posterior predictive check (APPC). Theoretically, we establish sufficient conditions under which the APPC is asymptotically calibrated. For probabilistic principal component analysis, we show that the APPC can remain calibrated under a misspecified latent prior when pervasive factors permit recovery of the signal space. Experiments with variational autoencoders show that aggregated posterior sampling improves generation for heavy-tailed and clustered data relative to Gaussian prior sampling while performing comparably to models with more flexible latent priors.
Read the full abstract

Statistical Methodology

Identification problem and quasi-maximum likelihood estimation for matrix-variate CP-factor models

Matrix-valued time series, arising in diverse fields such as economics, neuroscience, and recommender systems, have become increasingly prominent in modern data analysis. Among various modeling frameworks, the matrix-variate CP-factor model represents an important and widely applicable class for capturing low-rank structures in matrix time series. In this paper, we provide a unified treatment for the identification problem of CP factor models using both analytic and algebraic tools. In particular, we characterize the parameter space as the union of two subspaces, one identifiable and the other non-identifiable. We show that existing estimation methods only apply to an open proper subset of the identifiable subspace. In contrast, we propose a quasi-maximum likelihood estimation (QMLE) procedure for CP factor models, which allows consistent estimation on the whole identifiable subspace. Moreover, we show that the estimated loading matrices by QMLE achieve a faster convergence rate compared to existing approaches. A simulation study and a real application are conducted to demonstrate the finite-sample performance of the proposed method.
Read the full abstract

Statistical Methodology

Conditional Independence Testing in Time Series

We consider the problem of testing Granger causality in time series, specifically, whether the future outcome $Y_{t+1}$ and the exposure history $\bar X_t$ are conditionally independent given the history of $\bar Y_t$ and a set of confounding variables $\bar Z_t$ up to time $t$. This testing procedure distinguishes true causal effects from associations driven by common external processes, supporting reliable decision-makings in applications such as finance, neuroscience, and climate science. While traditional approaches assume a linear vector autoregressive (VAR) model and are vulnerable to misspecification, we instead address a model-free version of the problem. We propose nonlinearly regressing both the outcome and exposure on the joint history of $Y$ and $Z$, and calculating a test statistic based on the sample covariance of residuals, we call the Generalised Temporal Covariance Measure (GTCM). To account for heteroscedasticity and improve power against local alternatives, we incorporate variance weights and employ a data-adaptive test based on polynomial lag expansions.The type I error control of the test relies on the relatively weak assumption that user-chosen regression procedures estimate conditional means at a sufficiently fast rate that is slow enough to accommodate nonparametric settings. By further assuming stability of the regression procedures and weak dependence in the time series, we can utilise the entire dataset to estimate the conditional means without splitting the time series into subsets.
Read the full abstract