Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Thursday 16 April

Statistics Theory

On a Probability Inequality for Order Statistics with Applications to Bootstrap, Conformal Prediction, and more

“Behind every limit theorem, there is an inequality” said Kolmogorov. We say “for every inequality, there is an approximate inequality under approximate regularity conditions.” Suppose $X, X'$ are independent and identically distributed random variables. Then $X \le X'$ with a probability of at least $1/2$, irrespective of the underlying (common) distribution. One can ask what happens to the probability if $X, X'$ are independent but not identically distributed. It should be approximately $1/2$ if the distributions are approximately equal. Similarly, what if the random variables are dependent? It should, again, be approximately $1/2$ if the random variables are approximately independent. We explore an extension of this probability inequality involving order statistics and develop approximate versions of such an inequality under violations of independence and identical distribution assumptions. We further show that this inequality can be used as a basis to prove asymptotic validity of bootstrap/subsampling, finite-sample validity of conformal prediction, permutation tests, and asymptotic validity of rank tests without group invariance. Specifically, in the context of resampling inference, our results can be seen as a finite-sample instantiation of some results by Peter Hall and yield an alternative “cheap bootstrap” that applies to high-dimensional data.
Read the full abstract

Statistical Methodology

A Bayesian Approach to Unit-level Dependent Multi-type Survey Data

The American Community Survey (ACS) Public Use Microdata Sample (PUMS) provides access to a wide range of unit-level survey data consisting of correlated Gaussian and binomial distributed survey responses along with associated survey weights. As such, we propose a Bayesian hierarchical framework for jointly modeling unit-level Gaussian and binomial survey data. The model introduces a shared area-level random effect to capture dependence across responses. Informative sampling is addressed using a pseudo-likelihood construction, and Polya-Gamma data augmentation provides an efficient conjugate Gibbs sampler, enabling scalable inference for large survey datasets. Through empirical simulations based on ACS PUMS data, we show that the joint model achieves notable reductions in mean squared error and improved interval scores compared to univariate and design-based estimators. Applying the method to the 2023 Illinois PUMS data, we find that the joint model yields small-area estimates similar to those from the univariate model and the Horvitz-Thompson estimator, but with smaller posterior variances. The computational cost associated with the joint model is also comparable to that of the univariate binomial model. Combined with the empirical simulation results, these findings demonstrate the practical advantages of the proposed approach.
Read the full abstract

All authors Zewei Kong, Paul A. Parker, Jonathan R. Bradley, Scott H. Holan

Machine Learning (Statistics)

Amortized Optimal Transport from Sliced Potentials

We propose a novel amortized optimization method for predicting optimal transport (OT) plans across multiple pairs of measures by leveraging Kantorovich potentials derived from sliced OT. We introduce two amortization strategies: regression-based amortization (RA-OT) and objective-based amortization (OA-OT). In RA-OT, we formulate a functional regression model that treats Kantorovich potentials from the original OT problem as responses and those obtained from sliced OT as predictors, and estimate these models via least-squares methods. In OA-OT, we estimate the parameters of the functional model by optimizing the Kantorovich dual objective. In both approaches, the predicted OT plan is subsequently recovered from the estimated potentials. As amortized OT methods, both RA-OT and OA-OT enable efficient solutions to repeated OT problems across different measure pairs by reusing information learned from prior instances to rapidly approximate new solutions. Moreover, by exploiting the structure provided by sliced OT, the proposed models are more parsimonious, independent of specific structures of the measures, such as the number of atoms in the discrete case, while achieving high accuracy. We demonstrate the effectiveness of our approaches on tasks including MNIST digit transport, color transfer, supply-demand transportation on spherical data, and mini-batch OT conditional flow matching.
Read the full abstract

Machine Learning (Statistics)

MinShap: A Shapley-Based Framework for Feature Redundancy

Shapley values provide a flexible framework for attributing feature contributions to model predictions, but they are not naturally suited for feature selection: a feature may receive a positive attribution even when it is redundant given the remaining variables. In this paper, we introduce MinShap, a general framework for identifying important or non-redundant features through conditional importance functionals $VI_j^S$. Rather than averaging feature contributions across conditioning sets, MinShap aggregates them using the minimum, thereby testing whether a feature remains relevant under every conditioning context. We show that, under a simple null monotonicity condition, the minimum aggregation exactly characterizes feature redundancy and yields a principled feature selection criterion. This perspective provides a unified framework for statistical feature selection and representation-based interpretability while retaining the stability advantages of Shapley-style aggregation. We develop scalable algorithms with statistical guarantees, establish connections to multiple-testing procedures, and demonstrate through theory and experiments that MinShap produces more accurate and stable feature selection than existing model-agnostic approaches.
Read the full abstract

Statistical Methodology

Cellwise Robust Twoblock Dimension Reduction

Cellwise Robust Twoblock (CRTB) is introduced, the first cellwise robust method for simultaneous dimension reduction of multivariate predictor and response blocks, in both a dense and a sparse variable-selecting variant. Classical robust methods protect against casewise outliers by downweighting or removing entire observations, a strategy that becomes inefficient – and eventually breaks down – when contamination is scattered across individual cells rather than concentrated in whole rows. CRTB combines a column-wise pre-filter for cellwise outlier detection with model-based imputation of flagged cells inside an iteratively reweighted M-estimation loop, retaining the clean cells of partially contaminated rows instead of discarding the observation. An efficient algorithm is provided that uses the classical twoblock SVD as a warm start and converges in a handful of IRLS iterations at a moderate computational cost. The method resists settings where more than $50\%$ of rows contain contaminated cells while retaining comparable efficiency on clean data. A simulation study confirms these properties and shows that CRTB additionally recovers the underlying cellwise outlier pattern with high fidelity and, in the sparse setting, the correct set of informative variables. Two compelling examples illustrate CRTB's practical utility. In each of these, CRTB is shown to be conducive to results that are highly interpretable in the respective domains in the presence of cellwise outliers. As a by-product, the corresponding cells are identified with high fidelity.
Read the full abstract

Statistical Methodology

On the Conservativeness of Robust Variance Estimators in Propensity Score Weighted Cox Models

In propensity score weighted analysis, robust variance that does not account for weight estimation is commonly used. In propensity score weighted Cox models (CoxPSW), the robust variance is known to be conservative when weights for the average treatment effect (ATE) are used, but it remains unclear whether this conservativeness also holds for other weighting schemes. This study evaluated the performance of the robust variance in CoxPSW when weights other than ATE are applied. We conducted an asymptotic comparison between the robust variance and a variance estimator that accounts for weight estimation under non-ATE weights. Their performance was further evaluated through simulation studies and real data analysis. The analytical results, simulations, and real data analysis indicated that the robust variance is not necessarily conservative in CoxPSW when weights other than ATE are used. These findings suggest that variance estimators that account for weight estimation should be used when applying non-ATE weights in CoxPSW.
Read the full abstract

All authors Hiroya Morita, Shunichiro Orihara, Fumitaka Shimizu, Masataka Taguri

Statistical Methodology

Adaptive Multi-Prior Lasso for High-Dimensional Generalized Linear Models

Incorporation of external information into high-dimensional modeling for gene expression data has been shown, both theoretically and empirically, to substantially enhance performance. Such external information, sometimes referred to as prior information or priors, has become increasingly accessible from multiple sources, yet its reliability may vary considerably. Existing approaches often integrate these priors without sufficiently accounting for their quality, which may result in unsatisfactory or even misleading results. To effectively and selectively exploit such priors, we propose adaptive Multi-Prior Lasso, a novel regularization approach that simultaneously identifies reliable prior sources and integrates them to improve model performance. For high-dimensional generalized linear models (GLMs), an adaptive data-driven weight is assigned to each prior, so that more reliable sources are emphasized while less credible ones are downweighted. Theoretical guarantees are established, and the proposed method is shown through extensive simulations to improve estimation, prediction, and variable selection. An application to TCGA breast cancer gene expression data further illustrates the practical value of the proposed method, showing that incorporating prior information from PubMed published studies improves model performance.
Read the full abstract

All authors Fuzhi Xu, Weijuan Liang, Shuangge Ma, Qingzhao Zhang

Statistics Applications

Capturing Aleatoric Uncertainty in Climate Models

Internal climate variability arises from the climate system's inherently chaotic dynamics. Quantifying it is essential for climate science, as it enables risk-based decision-making and differentiates between externally forced change and internal fluctuations. In statistical terms, natural variability corresponds to aleatoric uncertainty, i.e., irreducible stochastic variability. Despite this close conceptual alignment, the link between internal climate variability and aleatoric uncertainty has not yet been formalized. We establish a theoretical link by showing that member-to-member differences in single-model large ensembles provide a direct representation of aleatoric uncertainty. To quantify the spatio-temporal structure of aleatoric uncertainty, we employ generalized additive models. The proposed framework is validated through comparison with ERA5-Land reanalysis data, demonstrating that ensemble-derived estimates reproduce key spatial and temporal patterns of real-world variability. Applied to the water balance over the Iberian Peninsula, our approach reveals coherent variability structures and pronounced regional heterogeneity. We find a decline in variability in drought-prone regions and seasons, a pattern that strengthens under +3 {\deg}C global warming, implying an increased risk of persistent summer drought conditions. Beyond this application, the framework is climate-model agnostic and transferable to other variables and spatial scales, providing a statistical basis for quantifying internal climate variability as aleatoric uncertainty.
Read the full abstract

All authors Cornelia Gruber, Henri Funk, Magdalena Mittermeier, Helmut Küchenhoff, Göran Kauermann

Statistical Methodology

Ranked-choice conjoint experiments

Forced-choice conjoint designs have become a staple method in the experimentalist's toolkit. However, the forced-choice outcome is neither always consistent with the types of choices individuals make in real political contexts, nor is it statistically efficient. In this paper, we formalize how ranked outcomes can be integrated into the conjoint framework. We provide a proof that rank-expanded estimators are equivalent to conventional AMCE, a theoretical account of how additional profiles increase the efficiency of conjoint designs, and design-based tests for the transitivity and independence of irrelevant alternatives assumptions that underpin the expansion. Across two pre-registered survey experiments–the first comparing forced-choice and ranked-choice designs across candidate and policy domains, and the second varying the number of ranked profiles–we find that ranked-choice conjoints yield substantively similar but more precise AMCE estimates, shrinking standard errors by 12-13% with one additional profile and up to 55% with six profiles per vignette. Based on efficiency–validity trade-offs, we recommend K = 4 profiles for most applications. We provide an accompanying open-source R package, cjrank, that implements rank expansion, AMCE estimation, efficiency diagnostics, and the assumption tests described in this paper.
Read the full abstract

Statistics Theory

On general weighted cumulative residual (past) extropy of extreme order statistics

Weighted extropy has recently emerged as a flexible information measure for quantifying uncertainty, with particular relevance to order statistics. In this paper, we introduce and study a weighted cumulative analogue of extropy, extending the framework of weighted cumulative residual and cumulative past entropies to extreme order statistics. Specifically, we define the general weighted cumulative residual extropy (GWCREx) for the smallest order statistic and the general weighted cumulative past extropy (GWCPEx) for the largest order statistic, along with their dynamic versions. We show that these weighted measures and their dynamic counterparts uniquely characterize the underlying distribution. Moreover, we establish new characterization results for two widely used reliability models: the generalized Pareto distribution and the power distribution. The proposed framework provides a unified information-theoretic tool for analysing extreme lifetimes in reliability engineering and survival analysis.
Read the full abstract

Statistical Computation

Theta-regularized Kriging: Modelling and Algorithms

To obtain more accurate model parameters and improve prediction accuracy, we proposed a regularized Kriging model that penalizes the hyperparameter theta in the Gaussian stochastic process, termed the Theta-regularized Kriging. We derived the optimization problem for this model from a maximum likelihood perspective. Additionally, we presented specific implementation details for the iterative process, including the regularized optimization algorithm and the geometric search cross-validation tuning algorithm. Three distinct penalty methods, Lasso, Ridge, and Elastic-net regularization, were meticulously considered. Meanwhile, the proposed Theta-regularized Kriging models were tested on nine common numerical functions and two practical engineering examples. The results demonstrate that, compared with other penalized Kriging models, the proposed model performs better in terms of accuracy and stability.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning

You are a robot and you live in a Markov decision process (MDP) with a finite or an infinite number of transitions from state-action to next states. You got brains and so you plan before you act. Luckily, your roboparents equipped you with a generative model to do some Monte-Carlo planning. The world is waiting for you and you have no time to waste. You want your planning to be efficient. Sample-efficient. Indeed, you want to exploit the possible structure of the MDP by exploring only a subset of states reachable by following near-optimal policies. You want guarantees on sample complexity that depend on a measure of the quantity of near-optimal states. You want something, that is an extension of Monte-Carlo sampling (for estimating an expectation) to problems that alternate maximization (over actions) and expectation (over next states). But you do not want to StOP with exponential running time, you want something simple to implement and computationally efficient. You want it all and you want it now. You want TrailBlazer.
Read the full abstract

Statistics Applications

Mapping Subnational Vulnerability to Inadequate Micronutrient Intake using a Bayesian Small Area Estimation Framework

Inadequate dietary micronutrient intake is a significant risk factor for deficiency and remains a major global health challenge. Nutrition programmes and interventions are most effective when targeted to populations at greatest risk. Household Consumption and Expenditure Surveys (HCES) are a widely available source of dietary data; however, they are often not powered for estimation below the first administrative level, limiting their utility for geographically targeted interventions. To address this, we applied Bayesian Small Area Estimation (SAE) methods to estimate the prevalence of apparent inadequate intake at the second administrative level. Three approaches were considered: a cluster level Beta binomial model and two area level models (mean smoothing and joint smoothing). Models were evaluated using a Rwanda HCES survey that supports inference at this scale. All models were implemented in a fully Bayesian framework to propagate uncertainty. Simulation results in Rwanda showed that the cluster level Beta binomial model achieved the strongest performance, while the area level joint smoothing model was the most reliable alternative among models accounting for survey design. Based on these results, models were applied to Senegal and Nigeria. In Senegal, second administrative level estimates captured meaningful subnational variation, reduced uncertainty relative to direct estimates, and remained consistent with first administrative level benchmarks. In Nigeria, despite smaller sample sizes and survey design constraints, modelled estimates reduced extreme uncertainty and showed good agreement with first administrative level estimates. This study demonstrates that Bayesian SAE methods can be applied to HCES data to generate reliable fine scale estimates of inadequate micronutrient intake, supporting localised nutrition interventions.
Read the full abstract

All authors Sahoko Ishida, Mohammed Osman, Ziyao Cui, Uchenna Agu, Emily Becher, Gabriel Battcock, Daniel Hernandez, Duccio Piovani, Frances Knight, Seth Flaxman, Kevin Tang

Machine Learning (Statistics)

Unsupervised feature selection using Bayesian Tucker decomposition

In this paper, we proposed Bayesian Tucker decomposition (BTuD) in which residual is supposed to obey Gaussian distribution analogous to linear regression. Although we have proposed an algorithm to perform the proposed BTuD, the conventional higher-order orthogonal iteration can generate Tucker decomposition consistent with the present implementation. Using the proposed BTuD, we can perform unsupervised feature selection successfully applied to various synthetic datasets, global coupled maps with randomized coupling strength, and gene expression profiles. Thus we can conclude that our newly proposed unsupervised feature selection method is promising. In addition to this, BTuD based unsupervised FE is expected to coincide with TD based unsupervised FE that were previously proposed and successfully applied to a wide range of problems.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits

We study downlink beam and rate adaptation in a multi-user mmWave MISO system where multiple base stations (BSs), each using analog beamforming from finite codebooks, serve multiple single-antenna user equipments (UEs) with a unique beam per UE and discrete data transmission rates. BSs learn about transmission success based on ACK/NACK feedback. To encode service goals, we introduce a satisficing throughput threshold $\tau_r$ and cast joint beam and rate adaptation as a combinatorial semi-bandit over beam-rate tuples. Within this framework, we propose SAT-CTS, a lightweight, threshold-aware policy that blends conservative confidence estimates with posterior sampling, steering learning toward meeting $\tau_r$ rather than merely maximizing. Our main theoretical contribution provides the first finite-time regret bounds for combinatorial semi-bandits with satisficing objective: when $\tau_r$ is realizable, we upper bound the cumulative satisficing regret to the target with a time-independent constant, and when $\tau_r$ is non-realizable, we show that SAT-CTS incurs only a finite expected transient outside committed CTS rounds, after which its regret is governed by the sum of the regret contributions of restarted CTS rounds, yielding an $O((\log T)^2)$ standard regret bound. On the practical side, we evaluate the performance via cumulative satisficing regret to $\tau_r$ alongside standard regret and fairness. Experiments with time-varying sparse multipath channels show that SAT-CTS consistently reduces satisficing regret and maintains competitive standard regret, while achieving favorable average throughput and fairness across users, indicating that feedback-efficient learning can equitably allocate beams and rates to meet QoS targets without channel state knowledge.
Read the full abstract

All authors Emre Özy{\i}ld{\i}r{\i}m, Bar{\i}ş Yayc{\i}, Umut Eren Akturk, Cem Tekin

Machine Learning (Statistics)

Best of both worlds: Stochastic & adversarial best-arm identification

We study bandit best-arm identification with arbitrary and potentially adversarial rewards. A simple random uniform learner obtains the optimal rate of error in the adversarial scenario. However, this type of strategy is suboptimal when the rewards are sampled stochastically. Therefore, we ask: Can we design a learner that performs optimally in both the stochastic and adversarial problems while not being aware of the nature of the rewards? First, we show that designing such a learner is impossible in general. In particular, to be robust to adversarial rewards, we can only guarantee optimal rates of error on a subset of the stochastic problems. We give a lower bound that characterizes the optimal rate in stochastic problems if the strategy is constrained to be robust to adversarial rewards. Finally, we design a simple parameter-free algorithm and show that its probability of error matches (up to log factors) the lower bound in stochastic problems, and it is also robust to adversarial ones.
Read the full abstract

All authors Yasin Abbasi-Yadkori, Peter L. Bartlett, Victor Gabillon, Alan Malek, Michal Valko

Machine Learning (Statistics)

Scalable Model-Based Clustering with Sequential Monte Carlo

In online clustering problems, there is often a large amount of uncertainty over possible cluster assignments that cannot be resolved until more data are observed. This difficulty is compounded when clusters follow complex distributions, as is the case with text data. Sequential Monte Carlo (SMC) methods give a natural way of representing and updating this uncertainty over time, but have prohibitive memory requirements for large-scale problems. We propose a novel SMC algorithm that decomposes clustering problems into approximately independent subproblems, allowing a more compact representation of the algorithm state. Our approach is motivated by the knowledge base construction problem, and we show that our method is able to accurately and efficiently solve clustering problems in this setting and others where traditional SMC struggles.
Read the full abstract

All authors Connie Trojan, Pavel Myshkov, Paul Fearnhead, James Hensman, Tom Minka, Christopher Nemeth

Machine Learning (Statistics)

Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness

We study a classification problem with three key challenges: pervasive informative missingness, the integration of partial prior expert knowledge into the learning process, and the need for interpretable decision rules. We propose a framework that encodes prior knowledge through an expert-guided class-conditional model for one or more classes, and use this model to construct a small set of interpretable goodness-of-fit features. The features quantify how well the observed data agree with the expert model, isolating the contributions of different aspects of the data, including both observed and missing components. These features are combined with a few transparent auxiliary summaries in a simple discriminative classifier, resulting in a decision rule that is easy to inspect and justify. We develop and apply the framework in the context of seismic monitoring used to assess compliance with the Comprehensive Nuclear-Test-Ban Treaty. We show that the method has strong potential as a transparent screening tool, reducing workload for expert analysts. A simulation designed to isolate the contribution of the proposed framework shows that this interpretable expert-guided method can even outperform strong standard machine-learning classifiers, particularly when training samples are small.
Read the full abstract

All authors Shahar Cohen, David M. Steinberg, Yael Radzyner, Yochai Ben Horin

Machine Learning (Statistics) · Cross-listed from cs.LG

Gating Enables Curvature: A Geometric Expressivity Gap in Attention

Multiplicative gating is widely used in neural architectures and has recently been applied to attention layers to improve performance and training stability in large language models. Despite the success of gated attention, the mathematical implications of gated attention mechanisms remain poorly understood. We study attention through the geometry of its representations by modeling outputs as mean parameters of Gaussian distributions and analyzing the induced Fisher–Rao geometry. We show that ungated attention operator is restricted to intrinsically flat statistical manifolds due to its affine structure, while multiplicative gating enables non-flat geometries, including positively curved manifolds that are unattainable in the ungated setting. These results establish a geometric expressivity gap between ungated and gated attention. Empirically, we show that gated models exhibit higher representation curvature and improved performance on tasks requiring nonlinear decision boundaries whereas they provide no consistent advantage on tasks with linear decision boundaries. Furthermore, we identify a structured regime in which curvature accumulates under composition, yielding a systematic depth amplification effect.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Zeroth-Order Optimization at the Edge of Stability

Zeroth-order (ZO) methods are widely used when gradients are unavailable or prohibitively expensive, including black-box learning and memory-efficient fine-tuning of large models, yet their optimization dynamics in deep learning remain underexplored. In this work, we provide an explicit step size condition that exactly captures the (mean-square) linear stability of a family of ZO methods based on the standard two-point estimator. Our characterization reveals a sharp contrast with first-order (FO) methods: whereas FO stability is governed solely by the largest Hessian eigenvalue, mean-square stability of ZO methods depends on the entire Hessian spectrum. Since computing the full Hessian spectrum is infeasible in practical neural network training, we further derive tractable stability bounds that depend only on the largest eigenvalue and the Hessian trace. Empirically, we find that full-batch ZO methods operate at the edge of stability: ZO-GD, ZO-GDM, and ZO-Adam consistently stabilize near the predicted stability boundary across a range of deep learning training problems. Our results highlight an implicit regularization effect specific to ZO methods, where large step sizes primarily regularize the Hessian trace, whereas in FO methods they regularize the top eigenvalue.
Read the full abstract

All authors Minhak Song, Liang Zhang, Bingcong Li, Niao He, Michael Muehlebach, Sewoong Oh

Statistics Applications

Evacuation destination choices during Hurricane Ian: A direct demand modeling approach

Hurricanes are causing unprecedented damage to the natural environment, infrastructure, and communities. Understanding evacuation behavior is essential for improving emergency preparedness. Past studies have relied on surveys and interviews, which are prone to recall bias. Additionally, they urge incorporating social vulnerability in evacuation research, emphasizing its impact on evacuation capability and destination choice. This study addresses these gaps by analyzing evacuation behavior using mobile device location data from Hurricane Ian, one of Florida's deadliest hurricanes, and directly incorporating variables from the Social Vulnerability Index (SVI) into a zone-to-zone (census tract level) evacuation demand model. We find that vehicle availability, residence in group quarters, road density, and English proficiency have significant effects on evacuation demand, shaping both the ability to evacuate from origin tracts and the attractiveness of destination tracts. Travel impedance, measured as distance, also plays a significant role, with evacuees substantially less likely to travel longer distances.
Read the full abstract

All authors Alessandra Recalde, Luyu Liu, Xiaojian Zhang, Sangung Park, Shangkun Jiang, Xilei Zhao

Statistical Methodology

Model Checking for Regressions Based on Weighted Residual Processes with Diverging Number of Predictors

The integrated conditional moment (ICM) test is a classical and widely used method for assessing the adequacy of regression models. Although it performs well in fixed-dimension settings, its behavior changes dramatically when the predictor dimension diverges: in such regimes, the limiting null and alternative distributions of the ICM statistic degenerate to fixed constants. Moreover, when the number of predictors diverges, the commonly used wild bootstrap no longer approximates the null distribution of the ICM statistic well, leading to size distortion and substantial power loss. To address these challenges, we propose a new specification test based on weighted residual processes for evaluating the parametric form of the regression mean function in high-dimensional settings where the number of predictors increases with the sample size. We establish the asymptotic properties of the test statistic under the null hypothesis and under global and local alternatives. The proposed test maintains the nominal significance level and can detect local alternatives that deviate from the null hypothesis at the parametric rate $1/\sqrt{n}$. Furthermore, we propose a smooth residual bootstrap to approximate the limiting null distribution and establish its validity in high-dimensional settings. Two simulation studies and a real-data example are conducted to evaluate the finite-sample performance of the proposed test.
Read the full abstract

Machine Learning (Statistics)

Differentially Private Conformal Prediction

Conformal prediction (CP) has attracted broad attention as a simple and flexible framework for uncertainty quantification through prediction sets. In this work, we study how to deploy CP under differential privacy (DP) in a statistically efficient manner. We first introduce differential CP, a non-splitting conformal procedure that avoids the efficiency loss caused by data splitting and serves as a bridge between oracle CP and private conformal inference. By exploiting the stability properties of DP mechanisms, differential CP establishes a direct connection to oracle CP and inherits corresponding validity behavior. Building on this idea, we develop Differentially Private Conformal Prediction (DPCP), a fully private procedure that combines DP model training with a private quantile mechanism for calibration. We establish the end-to-end privacy guarantee of DPCP and investigate its coverage properties under additional regularity conditions. We further study the efficiency of both differential CP and DPCP under empirical risk minimization and general regression models, showing that DPCP can produce tighter prediction sets than existing private split conformal approaches under the same privacy budget. Numerical experiments on synthetic and real datasets demonstrate the practical effectiveness of the proposed methods.
Read the full abstract

All authors Jiamei Wu, Ce Zhang, Zhipeng Cai, Jingsen Kong, Bei Jiang, Linglong Kong, Lingchen Kong

Machine Learning (Statistics) · Cross-listed from cs.LG

CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization

Lion optimizer is a popular learning-based optimization algorithm in machine learning, which shows impressive performance in training many deep learning models. Although convergence property of the Lion optimizer has been studied, its generalization analysis is still missing. To fill this gap, we study generalization property of the Lion via algorithmic stability based on the mathematical induction. Specifically, we prove that the Lion has a generalization error of $O(\frac{1}{N\tau^T})$, where $N$ is training sample size, and $\tau>0$ denotes the smallest absolute value of non-zero element in gradient estimator, and $T$ is the total iteration number. In addition, we obtain an interesting byproduct that the SignSGD algorithm has the same generalization error as the Lion. To enhance generalization of the Lion, we design a novel efficient Cautious Lion (i.e., CLion) optimizer by cautiously using sign function. Moreover, we prove that our CLion has a lower generalization error of $O(\frac{1}{N})$ than $O(\frac{1}{N\tau^T})$ of the Lion, since the parameter $\tau$ generally is very small. Meanwhile, we study convergence property of our CLion optimizer, and prove that our CLion has a fast convergence rate of $O(\frac{\sqrt{d}}{T^{1/4}})$ under $\ell_1$-norm of gradient for nonconvex stochastic optimization, where $d$ denotes the model dimension. Extensive numerical experiments demonstrate effectiveness of our CLion optimizer.
Read the full abstract

Statistical Methodology

HASOD: A Hybrid Adaptive Screening-Optimization Design for High-Dimensional Industrial Experiments

Industrial experimentation requires both factor screening to identify critical variables and response optimization to find optimal operating conditions. Traditional approaches treat these as separate phases, necessitating costly sequential experimentation and full experimental redesign between phases. This paper introduces HASOD (Hybrid Adaptive Screening-Optimization Design), a novel three-phase sequential framework that simultaneously addresses factor identification and response surface optimization within a unified adaptive structure. Phase 1 employs a modified Definitive Screening Design with an enhanced Cumulative Weighted Effect Screening Statistic (CWESS) incorporating interaction detection via ElasticNet regression. Phase 2 adaptively selects augmentation strategies – from full factorial to Response Surface Methodology designs – based on critical factors identified in Phase 1. Phase 3 applies Gaussian process-based global optimization with uncertainty-guided refinement near the predicted optimum. We prove that CWESS asymptotically separates active from inactive factors, providing classification consistency guarantees absent from most screening methodologies. Across six test scenarios, HASOD achieves 97.08% factor detection accuracy – 13.75 percentage points above traditional sequential methods (83.33%) – and significantly outperforms all eight competitor methods (p < 0.001). HASOD yields improved prediction performance (mean error: 3.61) while maintaining >=90% detection across all scenarios including interaction-heavy systems. The framework requires an average of 41.5 experimental runs – a 43% increase over traditional approaches – yet delivers superior detection accuracy with dramatically reduced prediction error. HASOD offers a theoretically grounded, unified framework that eliminates sequential redesign without sacrificing predictive capability.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Generative Augmented Inference of LLM-generated Data for Market Research: Theory and Empirical Evidence

Marketing research often relies on parameters estimated from costly human-generated data, such as conjoint survey responses, purchase decisions, and field experiment outcomes. Recent advances in large language models (LLMs) and other AI systems offer inexpensive auxiliary data, but introduce a new challenge: AI outputs are not direct observations of the target outcomes, but could involve high-dimensional representations with complex and unknown relationships to human labels. Conventional methods leverage AI predictions as direct proxies for true labels, which can be inefficient or unreliable when this relationship is weak or misspecified. We propose Generative Augmented Inference (GAI), a general framework that incorporates AI-generated outputs as informative features for estimating models of human-labeled outcomes. GAI uses an orthogonal moment construction that enables consistent estimation and valid inference with a flexible, nonparametric relationship between LLM-generated outputs and human labels. We establish asymptotic normality and a key dominance result: under random labeling, GAI is optimal within a unified class of debiased estimators-including human-data-only estimators and state-of-the-art debiasing methods-and delivers strict improvements under a mild informativeness condition. Even when the labeled sample is not representative of the target population, an extended variant of GAI still dominates the weighted human-data-only estimator. Empirically, GAI outperforms benchmarks across diverse marketing research settings. In a conjoint analysis, it halves estimation error and reduces human labeling requirements by over 75%. In a pricing study, it consistently outperforms alternative estimators when all methods receive identical auxiliary inputs. In a health insurance study, it saves over 90% of labels while preserving decision accuracy.
Read the full abstract

All authors Cheng Lu, Mengxin Wang, Dennis J. Zhang, Heng Zhang

Statistical Methodology

Bayesian sparse principal coordinates analysis with delta-tolerant linear approximation for microbiome data

Principal coordinates analysis (PCoA) is a standard exploratory tool for microbiome beta-diversity studies, but its axes are defined by pairwise dissimilarities and therefore do not directly identify the taxa driving an ordination. We propose Bayesian sparse principal coordinates analysis (BSPCoA), a post hoc framework that approximates the leading principal coordinates by a sparse linear surrogate in the observed taxa. A delta-tolerance diagnostic quantifies the discrepancy between the classical ordination and its best linear surrogate, clarifying when taxon-level interpretation is well supported. We place three-parameter beta normal global-local priors on the surrogate coefficients to induce row sparsity, obtain posterior uncertainty, and select influential taxa. The method reduces to sparse principal component analysis under Euclidean distance, while remaining applicable to ecologically meaningful dissimilarities such as Bray–Curtis and Hellinger distances. We conduct simulation studies to demonstrate that BSPCoA provides an approximately linear representation of the dominant ordination geometry while enhancing interpretability in sparse microbiome settings. In the Hadza gut microbiome data, the method produces an ordination close to that of classical PCoA while highlighting a parsimonious set of taxa associated with seasonal variation.
Read the full abstract

All authors Hsin-Hsiung Huang, Ruitao Liu, Liangliang Zhang, Shao-Hsuan Wang

Statistics Applications · Cross-listed from cs.LG

An unsupervised decision-support framework for multivariate biomarker analysis in athlete monitoring

Purpose. Athlete monitoring is constrained by small cohorts, heterogeneous biomarker scales, limited feasibility of repeated sampling, and the lack of reliable injury ground truth. These limitations reduce the interpretability and utility of traditional univariate and binary risk models. This study addresses these challenges by proposing an unsupervised multivariate framework to identify latent physiological states in athletes using real data. Methods. We propose a modular computational framework that operates in the joint biomarker space, integrating preprocessing, clinical safety screening, unsupervised clustering, and centroid-based physiological interpretation. Profiles are learned exclusively from amateur soccer players during a competitive microcycle. Synthetic data augmentation evaluates robustness and scalability. Ward hierarchical clustering supports monitoring and etiological differentiation, while Gaussian Mixture Models (GMM) enable structural stability analysis in high-dimensional settings. Results. The framework identifies coherent profiles that distinguish mechanical damage from metabolic stress while preserving homeostatic states. Synthetic data augmentation demonstrates feasibility and detection of latent silent risk phenotypes typically missed by univariate monitoring. Structural analyses indicate robustness under augmentation and higher-dimensional settings. Conclusion. The framework enables interpretable identification of latent physiological states from multivariate biomarker data without injury labels. By distinguishing mechanisms and revealing silent risk patterns not captured by conventional monitoring, it provides actionable insights for individualized athlete monitoring and decision making.
Read the full abstract

All authors Fernando Barcelos Rosito, Sebastião De Jesus Menezes, Simone Ferreira Sturza, Adriana Seixas, Muriel Figueredo Franco

Statistical Methodology

Bayesian Node-Level Outlier Detection for Graph Signals

This paper proposes a fully Bayesian framework for node-level outlier detection in graph signals, where measurements are observed on the nodes of an underlying graph. Unlike traditional outlier detection methods, our approach accounts for the relational dependencies induced by the graph, identifying outliers that disrupt the underlying smoothness. We model the observed signal as a combination of a graph-smooth component, captured via an intrinsic Gaussian Markov random field (IGMRF) prior, and a sparse outlier component modeled by a spike-and-slab prior. A key advantage of the proposed method is its ability to provide principled uncertainty quantification by estimating the posterior probability that each node is an outlier, rather than enforcing a deterministic binary decision. To facilitate posterior inference, we develop an efficient Gibbs sampling algorithm. We demonstrate the effectiveness of the proposed method through simulation studies on various graph structures, as well as a real data analysis of PM2.5 levels in California, exploring their relationship with wildfire occurrences.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.AI

Improving Machine Learning Performance with Synthetic Augmentation

Synthetic augmentation is increasingly used to mitigate data scarcity in financial machine learning, yet its statistical role remains poorly understood. We formalize synthetic augmentation as a modification of the effective training distribution and show that it induces a structural bias–variance trade-off: while additional samples may reduce estimation error, they may also shift the population objective whenever the synthetic distribution deviates from regions relevant under evaluation. To isolate informational gains from mechanical sample-size effects, we introduce a size-matched null augmentation and a finite-sample, non-parametric block permutation test that remains valid under weak temporal dependence. We evaluate this framework in both controlled Markov-switching environments and real financial datasets, including high-frequency option trade data and a daily equity panel. Across generators spanning bootstrap, copula-based models, variational autoencoders, diffusion models, and TimeGAN, we vary augmentation ratio, model capacity, task type, regime rarity, and signal-to-noise. We show that synthetic augmentation is beneficial only in variance-dominant regimes, such as persistent volatility forecasting-while it deteriorates performance in bias-dominant settings, including near-efficient directional prediction. Rare-regime targeting can improve domain-specific metrics but may conflict with unconditional permutation inference. Our results provide a structural perspective on when synthetic data improves financial learning performance and when it induces persistent distributional distortion.
Read the full abstract

All authors Mel Sohm, Charles Dezons, Sami Sellami, Oscar Ninou, Axel Pincon