Statistics Theory
On a Probability Inequality for Order Statistics with Applications to Bootstrap, Conformal Prediction, and more
Manit Paul and Arun Kumar Kuchibhotla
“Behind every limit
theorem, there is an inequality” said Kolmogorov. We say “for every inequality, there is an approximate inequality under approximate regularity conditions.” Suppose $X, X'$ are independent and identically distributed random variables. Then $X \le X'$ with a probability of at least $1/2$, irrespective of the underlying (common) distribution. One can ask what happens to the probability if $X, X'$ are independent but not identically distributed. It should be approximately $1/2$ if the distributions are approximately equal. Similarly, what if the random variables are dependent? It should, again, be approximately $1/2$ if the random variables are approximately independent. We explore an extension of this probability inequality involving order statistics and develop approximate versions of such an inequality under violations of independence and identical distribution assumptions. We further show that this inequality can be used as a basis to prove asymptotic validity of bootstrap/subsampling, finite-sample validity of conformal prediction, permutation tests, and asymptotic validity of rank tests without group invariance. Specifically, in the context of resampling inference, our results can be seen as a finite-sample instantiation of some results by Peter
Hall and yield an alternative “cheap bootstrap” that applies to high-dimensional data.
Read the full abstract ▾
Statistical Methodology
A Bayesian Approach to Unit-level Dependent Multi-type Survey Data
Zewei Kong, Paul A. Parker, Jonathan R. Bradley et al. (+1 more)
The American Community Survey (ACS) Public Use Microdata Sample (PUMS) provides access to a wide range of unit-level survey data consisting of correlated Gaussian and binomial distributed survey responses along with associated survey weights. As such, we propose a
Bayesian hierarchical framework for jointly modeling unit-level Gaussian and binomial survey data. The model introduces a shared area-level random effect to capture dependence across responses. Informative sampling is addressed using a pseudo-likelihood construction, and Polya-Gamma data augmentation provides an efficient conjugate Gibbs sampler, enabling scalable inference for large survey
datasets. Through empirical simulations based on ACS PUMS data, we show that the joint model achieves notable reductions in mean squared error and improved interval scores compared to univariate and design-based
estimators. Applying the method to the 2023 Illinois PUMS data, we find that the joint model yields small-area estimates similar to those from the univariate model and the Horvitz-Thompson
estimator, but with smaller posterior variances. The computational cost associated with the joint model is also comparable to that of the univariate binomial model. Combined with the empirical simulation results, these findings demonstrate the practical advantages of the proposed approach.
Read the full abstract ▾
All authors Zewei Kong, Paul A. Parker, Jonathan R. Bradley, Scott H. Holan
Machine Learning (Statistics)
Amortized Optimal Transport from Sliced Potentials
Minh-Phuc Truong and Khai Nguyen
We propose a novel amortized optimization method for predicting optimal transport (OT) plans across multiple pairs of measures by leveraging Kantorovich potentials derived from sliced OT. We introduce two amortization strategies: regression-based amortization (RA-OT) and objective-based amortization (OA-OT). In RA-OT, we formulate a functional regression model that treats Kantorovich potentials from the original OT problem as responses and those obtained from sliced OT as predictors, and estimate these models via least-squares methods. In OA-OT, we estimate the parameters of the functional model by optimizing the Kantorovich dual objective. In both approaches, the predicted OT plan is subsequently recovered from the estimated potentials. As amortized OT methods, both RA-OT and OA-OT enable efficient solutions to repeated OT problems across different measure pairs by reusing information learned from prior instances to rapidly approximate new solutions. Moreover, by exploiting the structure provided by sliced OT, the proposed models are more parsimonious, independent of specific structures of the measures, such as the number of atoms in the discrete case, while achieving high accuracy. We demonstrate the effectiveness of our approaches on tasks including MNIST digit transport, color transfer, supply-demand transportation on spherical data, and mini-batch OT conditional flow matching.
Read the full abstract ▾
Machine Learning (Statistics)
MinShap: A Shapley-Based Framework for Feature Redundancy
Chenghui Zheng and Garvesh Raskutti
Shapley values provide a flexible framework for attributing feature contributions to model predictions, but they are not naturally suited for feature selection: a feature may receive a positive attribution even when it is redundant given the remaining variables. In this paper, we introduce MinShap, a general framework for identifying important or non-redundant features through conditional importance functionals $VI_j^S$. Rather than averaging feature contributions across conditioning sets, MinShap aggregates them using the minimum, thereby testing whether a feature remains relevant under every conditioning context. We show that, under a simple null monotonicity condition, the minimum aggregation exactly characterizes feature redundancy and yields a principled feature selection criterion. This perspective provides a unified framework for statistical feature selection and representation-based interpretability while retaining the stability advantages of Shapley-style aggregation. We develop scalable
algorithms with statistical guarantees, establish connections to multiple-testing procedures, and demonstrate through theory and experiments that MinShap produces more accurate and stable feature selection than existing model-agnostic approaches.
Read the full abstract ▾
Statistical Methodology
Cellwise Robust Twoblock Dimension Reduction
Sven Serneels
Cellwise Robust Twoblock (CRTB) is introduced, the first cellwise robust method for simultaneous dimension reduction of multivariate predictor and response blocks, in both a dense and a sparse variable-selecting variant. Classical robust methods protect against casewise outliers by downweighting or removing entire observations, a strategy that becomes inefficient – and eventually breaks down – when contamination is scattered across individual
cells rather than concentrated in whole rows. CRTB combines a column-wise pre-filter for cellwise outlier detection with model-based imputation of flagged
cells inside an iteratively reweighted M-estimation loop, retaining the clean
cells of partially contaminated rows instead of discarding the observation. An efficient
algorithm is provided that uses the classical twoblock SVD as a warm start and converges in a handful of IRLS iterations at a moderate computational cost. The method resists settings where more than $50\%$ of rows contain contaminated
cells while retaining comparable efficiency on clean data. A simulation study confirms these properties and shows that CRTB additionally recovers the underlying cellwise outlier pattern with high fidelity and, in the sparse setting, the correct set of informative variables. Two compelling examples illustrate CRTB's practical utility. In each of these, CRTB is shown to be conducive to results that are highly interpretable in the respective domains in the presence of cellwise outliers. As a by-product, the corresponding
cells are identified with high fidelity.
Read the full abstract ▾
Statistical Methodology
On the Conservativeness of Robust Variance Estimators in Propensity Score Weighted Cox Models
Hiroya Morita, Shunichiro Orihara, Fumitaka Shimizu et al. (+1 more)
In propensity score weighted analysis, robust variance that does not account for weight estimation is commonly used. In propensity score weighted Cox models (CoxPSW), the robust variance is known to be conservative when weights for the average treatment effect (ATE) are used, but it remains unclear whether this conservativeness also holds for other weighting schemes. This study evaluated the performance of the robust variance in CoxPSW when weights other than ATE are applied. We conducted an asymptotic comparison between the robust variance and a variance
estimator that accounts for weight estimation under non-ATE weights. Their performance was further evaluated through simulation studies and real data analysis. The analytical results, simulations, and real data analysis indicated that the robust variance is not necessarily conservative in CoxPSW when weights other than ATE are used. These findings suggest that variance
estimators that account for weight estimation should be used when applying non-ATE weights in CoxPSW.
Read the full abstract ▾
All authors Hiroya Morita, Shunichiro Orihara, Fumitaka Shimizu, Masataka Taguri
Statistical Methodology
Adaptive Multi-Prior Lasso for High-Dimensional Generalized Linear Models
Fuzhi Xu, Weijuan Liang, Shuangge Ma et al. (+1 more)
Incorporation of external information into high-dimensional modeling for gene expression data has been shown, both theoretically and empirically, to substantially enhance performance. Such external information, sometimes referred to as prior information or priors, has become increasingly accessible from multiple sources, yet its reliability may vary considerably. Existing approaches often integrate these priors without sufficiently accounting for their quality, which may result in unsatisfactory or even misleading results. To effectively and selectively exploit such priors, we propose adaptive Multi-Prior Lasso, a novel regularization approach that simultaneously identifies reliable prior sources and integrates them to improve model performance. For high-dimensional generalized linear models (GLMs), an adaptive data-driven weight is assigned to each prior, so that more reliable sources are emphasized while less credible ones are downweighted. Theoretical guarantees are established, and the proposed method is shown through extensive simulations to improve estimation, prediction, and variable selection. An application to TCGA breast cancer gene expression data further illustrates the practical value of the proposed method, showing that incorporating prior information from PubMed published studies improves model performance.
Read the full abstract ▾
All authors Fuzhi Xu, Weijuan Liang, Shuangge Ma, Qingzhao Zhang
Statistics Applications
Capturing Aleatoric Uncertainty in Climate Models
Cornelia Gruber, Henri Funk, Magdalena Mittermeier et al. (+2 more)
Internal
climate variability arises from the
climate system's inherently chaotic dynamics. Quantifying it is essential for
climate science, as it enables risk-based decision-making and differentiates between externally forced change and internal fluctuations. In statistical terms, natural variability corresponds to aleatoric uncertainty, i.e., irreducible
stochastic variability. Despite this close conceptual
alignment, the link between internal
climate variability and aleatoric uncertainty has not yet been formalized. We establish a theoretical link by showing that member-to-member differences in single-model large ensembles provide a direct representation of aleatoric uncertainty. To quantify the spatio-temporal structure of aleatoric uncertainty, we employ generalized additive models. The proposed framework is validated through comparison with ERA5-Land reanalysis data, demonstrating that ensemble-derived estimates reproduce key spatial and temporal patterns of real-world variability. Applied to the water balance over the Iberian Peninsula, our approach reveals coherent variability structures and pronounced regional heterogeneity. We find a decline in variability in drought-prone regions and seasons, a pattern that strengthens under +3 {\deg}C global warming, implying an increased risk of persistent summer drought conditions. Beyond this application, the framework is
climate-model agnostic and transferable to other variables and spatial scales, providing a statistical basis for quantifying internal
climate variability as aleatoric uncertainty.
Read the full abstract ▾
All authors Cornelia Gruber, Henri Funk, Magdalena Mittermeier, Helmut Küchenhoff, Göran Kauermann
Statistical Methodology
Ranked-choice conjoint experiments
Thomas S. Robinson, Mats Ahrenshop and Spyros Kosmidis
Forced-choice conjoint designs have become a staple method in the experimentalist's toolkit. However, the forced-choice outcome is neither always consistent with the types of choices individuals make in real political contexts, nor is it statistically efficient. In this paper, we formalize how ranked outcomes can be integrated into the conjoint framework. We provide a proof that rank-expanded
estimators are equivalent to conventional AMCE, a theoretical account of how additional profiles increase the efficiency of conjoint designs, and design-based tests for the transitivity and independence of irrelevant alternatives assumptions that underpin the expansion. Across two pre-registered survey experiments–the first comparing forced-choice and ranked-choice designs across candidate and policy domains, and the second varying the number of ranked profiles–we find that ranked-choice conjoints yield substantively similar but more precise AMCE estimates, shrinking standard errors by 12-13% with one additional profile and up to 55% with six profiles per vignette. Based on efficiency–validity trade-offs, we recommend K = 4 profiles for most applications. We provide an accompanying open-source R package, cjrank, that implements rank expansion, AMCE estimation, efficiency diagnostics, and the assumption tests described in this paper.
Read the full abstract ▾
Statistics Theory
On general weighted cumulative residual (past) extropy of extreme order statistics
Santosh Kumar Chaudhary, Sarikul Islam and Nitin Gupta
Weighted extropy has recently emerged as a flexible information measure for quantifying uncertainty, with particular relevance to order statistics. In this paper, we introduce and study a weighted cumulative analogue of extropy, extending the framework of weighted cumulative residual and cumulative past
entropies to extreme order statistics. Specifically, we define the general weighted cumulative residual extropy (GWCREx) for the smallest order statistic and the general weighted cumulative past extropy (GWCPEx) for the largest order statistic, along with their dynamic versions. We show that these weighted measures and their dynamic counterparts uniquely characterize the underlying distribution. Moreover, we establish new characterization results for two widely used reliability models: the generalized Pareto distribution and the power distribution. The proposed framework provides a unified information-theoretic tool for analysing extreme lifetimes in reliability engineering and survival analysis.
Read the full abstract ▾
Statistical Computation
Theta-regularized Kriging: Modelling and Algorithms
Xuelin Xie and Xiliang Lu
To obtain more accurate model parameters and improve prediction accuracy, we proposed a regularized Kriging model that penalizes the hyperparameter theta in the Gaussian
stochastic process, termed the Theta-regularized Kriging. We derived the optimization problem for this model from a maximum likelihood perspective. Additionally, we presented specific implementation details for the iterative process, including the regularized optimization
algorithm and the
geometric search cross-validation tuning
algorithm. Three distinct penalty methods, Lasso, Ridge, and Elastic-net regularization, were meticulously considered. Meanwhile, the proposed Theta-regularized Kriging models were tested on nine common numerical functions and two practical engineering examples. The results demonstrate that, compared with other penalized Kriging models, the proposed model performs better in terms of accuracy and stability.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
Jean-Bastien Grill, Michal Valko and Rémi Munos
You are a
robot and you live in a Markov decision process (MDP) with a finite or an infinite number of transitions from state-action to next states. You got
brains and so you plan before you act. Luckily, your roboparents equipped you with a generative model to do some Monte-Carlo planning. The world is waiting for you and you have no time to waste. You want your planning to be efficient. Sample-efficient. Indeed, you want to exploit the possible structure of the MDP by exploring only a subset of states reachable by following near-optimal policies. You want guarantees on sample complexity that depend on a measure of the quantity of near-optimal states. You want something, that is an extension of Monte-Carlo sampling (for estimating an expectation) to problems that alternate maximization (over actions) and expectation (over next states). But you do not want to StOP with exponential running time, you want something simple to implement and computationally efficient. You want it all and you want it now. You want TrailBlazer.
Read the full abstract ▾
Statistics Applications
Mapping Subnational Vulnerability to Inadequate Micronutrient Intake using a Bayesian Small Area Estimation Framework
Sahoko Ishida, Mohammed Osman, Ziyao Cui et al. (+8 more)
Inadequate dietary micronutrient intake is a significant risk factor for deficiency and remains a major global health challenge. Nutrition programmes and interventions are most effective when targeted to populations at greatest risk. Household Consumption and Expenditure Surveys (HCES) are a widely available source of dietary data; however, they are often not powered for estimation below the first administrative level, limiting their utility for geographically targeted interventions. To address this, we applied
Bayesian Small Area Estimation (SAE) methods to estimate the prevalence of apparent inadequate intake at the second administrative level. Three approaches were considered: a cluster level Beta binomial model and two area level models (mean smoothing and joint smoothing). Models were evaluated using a Rwanda HCES survey that supports inference at this scale. All models were implemented in a fully
Bayesian framework to propagate uncertainty. Simulation results in Rwanda showed that the cluster level Beta binomial model achieved the strongest performance, while the area level joint smoothing model was the most reliable alternative among models accounting for survey design. Based on these results, models were applied to Senegal and Nigeria. In Senegal, second administrative level estimates captured meaningful subnational variation, reduced uncertainty relative to direct estimates, and remained consistent with first administrative level
benchmarks. In Nigeria, despite smaller sample sizes and survey design constraints, modelled estimates reduced extreme uncertainty and showed good agreement with first administrative level estimates. This study demonstrates that
Bayesian SAE methods can be applied to HCES data to generate reliable fine scale estimates of inadequate micronutrient intake, supporting localised nutrition interventions.
Read the full abstract ▾
All authors Sahoko Ishida, Mohammed Osman, Ziyao Cui, Uchenna Agu, Emily Becher, Gabriel Battcock, Daniel Hernandez, Duccio Piovani, Frances Knight, Seth Flaxman, Kevin Tang
Machine Learning (Statistics)
Unsupervised feature selection using Bayesian Tucker decomposition
Y-h. Taguchi and Yoh-ichi Mototake
In this paper, we proposed
Bayesian Tucker decomposition (BTuD) in which residual is supposed to obey Gaussian distribution analogous to linear regression. Although we have proposed an
algorithm to perform the proposed BTuD, the conventional higher-order orthogonal iteration can generate Tucker decomposition consistent with the present implementation. Using the proposed BTuD, we can perform unsupervised feature selection successfully applied to various synthetic
datasets, global coupled maps with randomized coupling strength, and gene expression profiles. Thus we can conclude that our newly proposed unsupervised feature selection method is promising. In addition to this, BTuD based unsupervised FE is expected to coincide with TD based unsupervised FE that were previously proposed and successfully applied to a wide range of problems.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits
Emre Özy{\i}ld{\i}r{\i}m, Bar{\i}ş Yayc{\i}, Umut Eren Akturk et al. (+1 more)
We study downlink beam and rate adaptation in a multi-user mmWave MISO system where multiple base stations (BSs), each using analog beamforming from finite codebooks, serve multiple single-
antenna user equipments (UEs) with a unique beam per UE and discrete data transmission rates. BSs learn about transmission success based on ACK/NACK feedback. To encode service goals, we introduce a satisficing throughput threshold $\tau_r$ and cast joint beam and rate adaptation as a combinatorial semi-bandit over beam-rate tuples. Within this framework, we propose SAT-CTS, a lightweight, threshold-aware policy that blends conservative confidence estimates with posterior sampling, steering learning toward meeting $\tau_r$ rather than merely maximizing. Our main theoretical contribution provides the first finite-time regret bounds for combinatorial semi-bandits with satisficing objective: when $\tau_r$ is realizable, we upper bound the cumulative satisficing regret to the target with a time-independent constant, and when $\tau_r$ is non-realizable, we show that SAT-CTS incurs only a finite expected transient outside committed CTS rounds, after which its regret is governed by the sum of the regret contributions of restarted CTS rounds, yielding an $O((\log T)^2)$ standard regret bound. On the practical side, we evaluate the performance via cumulative satisficing regret to $\tau_r$ alongside standard regret and
fairness. Experiments with time-varying sparse multipath channels show that SAT-CTS consistently reduces satisficing regret and maintains competitive standard regret, while achieving favorable average throughput and
fairness across users, indicating that feedback-efficient learning can equitably allocate beams and rates to meet QoS targets without channel state knowledge.
Read the full abstract ▾
All authors Emre Özy{\i}ld{\i}r{\i}m, Bar{\i}ş Yayc{\i}, Umut Eren Akturk, Cem Tekin
Machine Learning (Statistics)
Best of both worlds: Stochastic & adversarial best-arm identification
Yasin Abbasi-Yadkori, Peter L. Bartlett, Victor Gabillon et al. (+2 more)
We study bandit best-arm identification with arbitrary and potentially
adversarial rewards. A simple random uniform learner obtains the optimal rate of error in the
adversarial scenario. However, this type of strategy is suboptimal when the rewards are sampled stochastically. Therefore, we ask: Can we design a learner that performs optimally in both the
stochastic and
adversarial problems while not being aware of the nature of the rewards? First, we show that designing such a learner is impossible in general. In particular, to be robust to
adversarial rewards, we can only guarantee optimal rates of error on a subset of the
stochastic problems. We give a lower bound that characterizes the optimal rate in
stochastic problems if the strategy is constrained to be robust to
adversarial rewards. Finally, we design a simple parameter-free
algorithm and show that its probability of error matches (up to log factors) the lower bound in
stochastic problems, and it is also robust to
adversarial ones.
Read the full abstract ▾
All authors Yasin Abbasi-Yadkori, Peter L. Bartlett, Victor Gabillon, Alan Malek, Michal Valko
Machine Learning (Statistics)
Scalable Model-Based Clustering with Sequential Monte Carlo
Connie Trojan, Pavel Myshkov, Paul Fearnhead et al. (+3 more)
In online clustering problems, there is often a large amount of uncertainty over possible cluster assignments that cannot be resolved until more data are observed. This difficulty is compounded when clusters follow complex distributions, as is the case with text data. Sequential Monte Carlo (SMC) methods give a natural way of representing and updating this uncertainty over time, but have prohibitive
memory requirements for large-scale problems. We propose a novel SMC
algorithm that decomposes clustering problems into approximately independent subproblems, allowing a more compact representation of the
algorithm state. Our approach is motivated by the knowledge base construction problem, and we show that our method is able to accurately and efficiently solve clustering problems in this setting and others where traditional SMC struggles.
Read the full abstract ▾
All authors Connie Trojan, Pavel Myshkov, Paul Fearnhead, James Hensman, Tom Minka, Christopher Nemeth
Machine Learning (Statistics)
Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness
Shahar Cohen, David M. Steinberg, Yael Radzyner et al. (+1 more)
We study a classification problem with three key challenges: pervasive informative missingness, the integration of partial prior expert knowledge into the learning process, and the need for interpretable decision rules. We propose a framework that encodes prior knowledge through an expert-guided class-conditional model for one or more classes, and use this model to construct a small set of interpretable goodness-of-fit features. The features quantify how well the observed data agree with the expert model, isolating the contributions of different aspects of the data, including both observed and missing components. These features are combined with a few transparent auxiliary summaries in a simple discriminative classifier, resulting in a decision rule that is easy to inspect and justify. We develop and apply the framework in the context of seismic monitoring used to assess compliance with the Comprehensive
Nuclear-Test-Ban Treaty. We show that the method has strong potential as a transparent screening tool, reducing workload for expert analysts. A simulation designed to isolate the contribution of the proposed framework shows that this interpretable expert-guided method can even outperform strong standard machine-learning classifiers, particularly when training samples are small.
Read the full abstract ▾
All authors Shahar Cohen, David M. Steinberg, Yael Radzyner, Yochai Ben Horin
Machine Learning (Statistics) · Cross-listed from cs.LG
Satwik Bathula and Anand A. Joshi
Multiplicative gating is widely used in neural architectures and has recently been applied to
attention layers to improve performance and training stability in large
language models. Despite the success of gated
attention, the mathematical implications of gated
attention mechanisms remain poorly understood. We study
attention through the geometry of its representations by modeling outputs as mean parameters of Gaussian distributions and analyzing the induced Fisher–Rao geometry. We show that ungated
attention operator is restricted to intrinsically flat statistical
manifolds due to its affine structure, while multiplicative gating enables non-flat geometries, including positively curved
manifolds that are unattainable in the ungated setting. These results establish a
geometric expressivity gap between ungated and gated
attention. Empirically, we show that gated models exhibit higher representation
curvature and improved performance on tasks requiring nonlinear decision boundaries whereas they provide no consistent advantage on tasks with linear decision boundaries. Furthermore, we identify a structured regime in which
curvature accumulates under composition, yielding a systematic depth amplification effect.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Zeroth-Order Optimization at the Edge of Stability
Minhak Song, Liang Zhang, Bingcong Li et al. (+3 more)
Zeroth-order (ZO) methods are widely used when gradients are unavailable or prohibitively expensive, including black-box learning and
memory-efficient fine-tuning of large models, yet their optimization dynamics in deep learning remain underexplored. In this work, we provide an explicit step size condition that exactly captures the (mean-square) linear stability of a family of ZO methods based on the standard two-point
estimator. Our characterization reveals a sharp contrast with first-order (FO) methods: whereas FO stability is governed solely by the largest Hessian
eigenvalue, mean-square stability of ZO methods depends on the entire Hessian spectrum. Since computing the full Hessian spectrum is infeasible in practical neural network training, we further derive tractable stability bounds that depend only on the largest
eigenvalue and the Hessian trace. Empirically, we find that full-batch ZO methods operate at the edge of stability: ZO-GD, ZO-GDM, and ZO-Adam consistently stabilize near the predicted stability boundary across a range of deep learning training problems. Our results highlight an implicit regularization effect specific to ZO methods, where large step sizes primarily regularize the Hessian trace, whereas in FO methods they regularize the top
eigenvalue.
Read the full abstract ▾
All authors Minhak Song, Liang Zhang, Bingcong Li, Niao He, Michael Muehlebach, Sewoong Oh
Statistics Applications
Evacuation destination choices during Hurricane Ian: A direct demand modeling approach
Alessandra Recalde, Luyu Liu, Xiaojian Zhang et al. (+3 more)
Hurricanes are causing unprecedented damage to the natural environment, infrastructure, and communities. Understanding evacuation behavior is essential for improving emergency preparedness. Past studies have relied on surveys and interviews, which are prone to recall bias. Additionally, they urge incorporating social vulnerability in evacuation research, emphasizing its impact on evacuation capability and destination choice. This study addresses these gaps by analyzing evacuation behavior using mobile device location data from Hurricane Ian, one of Florida's deadliest hurricanes, and directly incorporating variables from the Social Vulnerability Index (SVI) into a zone-to-zone (census tract level) evacuation demand model. We find that vehicle availability, residence in group quarters, road density, and English proficiency have significant effects on evacuation demand, shaping both the ability to evacuate from origin tracts and the attractiveness of destination tracts. Travel impedance, measured as distance, also plays a significant role, with evacuees substantially less likely to travel longer distances.
Read the full abstract ▾
All authors Alessandra Recalde, Luyu Liu, Xiaojian Zhang, Sangung Park, Shangkun Jiang, Xilei Zhao
Statistical Methodology
Model Checking for Regressions Based on Weighted Residual Processes with Diverging Number of Predictors
Yue Hu, Haiqi Li and Xintao Xia
The integrated conditional moment (ICM) test is a classical and widely used method for assessing the adequacy of regression models. Although it performs well in fixed-dimension settings, its behavior changes dramatically when the predictor dimension diverges: in such regimes, the limiting null and alternative distributions of the ICM statistic degenerate to fixed constants. Moreover, when the number of predictors diverges, the commonly used wild bootstrap no longer approximates the null distribution of the ICM statistic well, leading to size distortion and substantial power loss. To address these challenges, we propose a new specification test based on weighted residual processes for evaluating the parametric form of the regression mean function in high-dimensional settings where the number of predictors increases with the sample size. We establish the asymptotic properties of the test statistic under the null hypothesis and under global and local alternatives. The proposed test maintains the nominal significance level and can detect local alternatives that deviate from the null hypothesis at the parametric rate $1/\sqrt{n}$. Furthermore, we propose a smooth residual bootstrap to approximate the limiting null distribution and establish its validity in high-dimensional settings. Two simulation studies and a real-data example are conducted to evaluate the finite-sample performance of the proposed test.
Read the full abstract ▾
Machine Learning (Statistics)
Differentially Private Conformal Prediction
Jiamei Wu, Ce Zhang, Zhipeng Cai et al. (+4 more)
Conformal prediction (CP) has attracted broad
attention as a simple and flexible framework for uncertainty quantification through prediction sets. In this work, we study how to deploy CP under differential
privacy (DP) in a statistically efficient manner. We first introduce differential CP, a non-splitting conformal procedure that avoids the efficiency loss caused by data splitting and serves as a bridge between oracle CP and private conformal inference. By exploiting the stability properties of DP mechanisms, differential CP establishes a direct connection to oracle CP and inherits corresponding validity behavior. Building on this idea, we develop Differentially Private Conformal Prediction (DPCP), a fully private procedure that combines DP model training with a private
quantile mechanism for calibration. We establish the end-to-end
privacy guarantee of DPCP and investigate its coverage properties under additional regularity conditions. We further study the efficiency of both differential CP and DPCP under empirical risk minimization and general regression models, showing that DPCP can produce tighter prediction sets than existing private split conformal approaches under the same
privacy budget. Numerical experiments on synthetic and real
datasets demonstrate the practical effectiveness of the proposed methods.
Read the full abstract ▾
All authors Jiamei Wu, Ce Zhang, Zhipeng Cai, Jingsen Kong, Bei Jiang, Linglong Kong, Lingchen Kong
Machine Learning (Statistics) · Cross-listed from cs.LG
CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization
Feihu Huang, Guanyi Zhang and Songcan Chen
Lion optimizer is a popular learning-based optimization
algorithm in machine learning, which shows impressive performance in training many deep learning models. Although convergence property of the Lion optimizer has been studied, its generalization analysis is still missing. To fill this gap, we study generalization property of the Lion via algorithmic stability based on the mathematical induction. Specifically, we prove that the Lion has a generalization error of $O(\frac{1}{N\tau^T})$, where $N$ is training sample size, and $\tau>0$ denotes the smallest absolute value of non-zero element in gradient
estimator, and $T$ is the total iteration number. In addition, we obtain an interesting byproduct that the SignSGD
algorithm has the same generalization error as the Lion. To enhance generalization of the Lion, we design a novel efficient Cautious Lion (i.e., CLion) optimizer by cautiously using sign function. Moreover, we prove that our CLion has a lower generalization error of $O(\frac{1}{N})$ than $O(\frac{1}{N\tau^T})$ of the Lion, since the parameter $\tau$ generally is very small. Meanwhile, we study convergence property of our CLion optimizer, and prove that our CLion has a fast convergence rate of $O(\frac{\sqrt{d}}{T^{1/4}})$ under $\ell_1$-norm of gradient for nonconvex
stochastic optimization, where $d$ denotes the model dimension. Extensive numerical experiments demonstrate effectiveness of our CLion optimizer.
Read the full abstract ▾
Statistical Methodology
HASOD: A Hybrid Adaptive Screening-Optimization Design for High-Dimensional Industrial Experiments
Kumarjit Pathak
Industrial experimentation requires both factor screening to identify critical variables and response optimization to find optimal operating conditions. Traditional approaches treat these as separate phases, necessitating costly sequential experimentation and full experimental redesign between phases. This paper introduces HASOD (Hybrid Adaptive Screening-Optimization Design), a novel three-phase sequential framework that simultaneously addresses factor identification and response surface optimization within a unified adaptive structure. Phase 1 employs a modified Definitive Screening Design with an enhanced Cumulative Weighted Effect Screening Statistic (CWESS) incorporating interaction detection via ElasticNet regression. Phase 2 adaptively selects augmentation strategies – from full factorial to Response Surface Methodology designs – based on critical factors identified in Phase 1. Phase 3 applies Gaussian process-based global optimization with uncertainty-guided refinement near the predicted optimum. We prove that CWESS asymptotically separates active from inactive factors, providing classification consistency guarantees absent from most screening methodologies. Across six test scenarios, HASOD achieves 97.08% factor detection accuracy – 13.75 percentage points above traditional sequential methods (83.33%) – and significantly outperforms all eight competitor methods (p < 0.001). HASOD yields improved prediction performance (mean error: 3.61) while maintaining >=90% detection across all scenarios including interaction-heavy systems. The framework requires an average of 41.5 experimental runs – a 43% increase over traditional approaches – yet delivers superior detection accuracy with dramatically reduced prediction error. HASOD offers a theoretically grounded, unified framework that eliminates sequential redesign without sacrificing predictive capability.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Generative Augmented Inference of LLM-generated Data for Market Research: Theory and Empirical Evidence
Cheng Lu, Mengxin Wang, Dennis J. Zhang et al. (+1 more)
Marketing research often relies on parameters estimated from costly human-generated data, such as conjoint survey responses, purchase decisions, and field experiment outcomes. Recent advances in large
language models (LLMs) and other AI systems offer inexpensive auxiliary data, but introduce a new challenge: AI outputs are not direct observations of the target outcomes, but could involve high-dimensional representations with complex and unknown relationships to human labels. Conventional methods leverage AI predictions as direct proxies for true labels, which can be inefficient or unreliable when this relationship is weak or misspecified. We propose Generative Augmented Inference (GAI), a general framework that incorporates AI-generated outputs as informative features for estimating models of human-labeled outcomes. GAI uses an orthogonal moment construction that enables consistent estimation and valid inference with a flexible, nonparametric relationship between LLM-generated outputs and human labels. We establish asymptotic normality and a key dominance result: under random labeling, GAI is optimal within a unified class of debiased
estimators-including human-data-only
estimators and state-of-the-art debiasing methods-and delivers strict improvements under a mild informativeness condition. Even when the labeled sample is not representative of the target population, an extended variant of GAI still dominates the weighted human-data-only
estimator. Empirically, GAI outperforms
benchmarks across diverse marketing research settings. In a conjoint analysis, it halves estimation error and reduces human labeling requirements by over 75%. In a pricing study, it consistently outperforms alternative
estimators when all methods receive identical auxiliary inputs. In a health insurance study, it saves over 90% of labels while preserving decision accuracy.
Read the full abstract ▾
All authors Cheng Lu, Mengxin Wang, Dennis J. Zhang, Heng Zhang
Statistical Methodology
Bayesian sparse principal coordinates analysis with delta-tolerant linear approximation for microbiome data
Hsin-Hsiung Huang, Ruitao Liu, Liangliang Zhang et al. (+1 more)
Principal coordinates analysis (PCoA) is a standard exploratory tool for microbiome beta-diversity studies, but its axes are defined by pairwise dissimilarities and therefore do not directly identify the taxa driving an ordination. We propose
Bayesian sparse principal coordinates analysis (BSPCoA), a post hoc framework that approximates the leading principal coordinates by a sparse linear surrogate in the observed taxa. A delta-tolerance diagnostic quantifies the discrepancy between the classical ordination and its best linear surrogate, clarifying when taxon-level interpretation is well supported. We place three-parameter beta normal global-local priors on the surrogate coefficients to induce row sparsity, obtain posterior uncertainty, and select influential taxa. The method reduces to sparse principal component analysis under Euclidean distance, while remaining applicable to ecologically meaningful dissimilarities such as Bray–Curtis and Hellinger distances. We conduct simulation studies to demonstrate that BSPCoA provides an approximately linear representation of the dominant ordination geometry while enhancing interpretability in sparse microbiome settings. In the Hadza gut microbiome data, the method produces an ordination close to that of classical PCoA while highlighting a parsimonious set of taxa associated with seasonal variation.
Read the full abstract ▾
All authors Hsin-Hsiung Huang, Ruitao Liu, Liangliang Zhang, Shao-Hsuan Wang
Statistics Applications · Cross-listed from cs.LG
An unsupervised decision-support framework for multivariate biomarker analysis in athlete monitoring
Fernando Barcelos Rosito, Sebastião De Jesus Menezes, Simone Ferreira Sturza et al. (+2 more)
Purpose. Athlete monitoring is constrained by small cohorts, heterogeneous biomarker scales, limited feasibility of repeated sampling, and the lack of reliable injury ground truth. These limitations reduce the interpretability and utility of traditional univariate and binary risk models. This study addresses these challenges by proposing an unsupervised multivariate framework to identify latent physiological states in athletes using real data. Methods. We propose a modular computational framework that operates in the joint biomarker space, integrating preprocessing, clinical safety screening, unsupervised clustering, and centroid-based physiological interpretation. Profiles are learned exclusively from amateur soccer players during a competitive microcycle. Synthetic data augmentation evaluates robustness and scalability. Ward hierarchical clustering supports monitoring and etiological differentiation, while Gaussian Mixture Models (GMM) enable structural stability analysis in high-dimensional settings. Results. The framework identifies coherent profiles that distinguish mechanical damage from metabolic stress while preserving homeostatic states. Synthetic data augmentation demonstrates feasibility and detection of latent silent risk phenotypes typically missed by univariate monitoring. Structural analyses indicate robustness under augmentation and higher-dimensional settings. Conclusion. The framework enables interpretable identification of latent physiological states from multivariate biomarker data without injury labels. By distinguishing mechanisms and revealing silent risk patterns not captured by conventional monitoring, it provides actionable insights for individualized athlete monitoring and decision making.
Read the full abstract ▾
All authors Fernando Barcelos Rosito, Sebastião De Jesus Menezes, Simone Ferreira Sturza, Adriana Seixas, Muriel Figueredo Franco
Statistical Methodology
Bayesian Node-Level Outlier Detection for Graph Signals
Seongmin Kim and Kyusoon Kim
This paper proposes a fully
Bayesian framework for node-level outlier detection in graph signals, where measurements are observed on the nodes of an underlying graph. Unlike traditional outlier detection methods, our approach accounts for the relational dependencies induced by the graph, identifying outliers that disrupt the underlying smoothness. We model the observed signal as a combination of a graph-smooth component, captured via an intrinsic Gaussian Markov random field (IGMRF) prior, and a sparse outlier component modeled by a spike-and-slab prior. A key advantage of the proposed method is its ability to provide principled uncertainty quantification by estimating the posterior probability that each node is an outlier, rather than enforcing a deterministic binary decision. To facilitate posterior inference, we develop an efficient Gibbs sampling
algorithm. We demonstrate the effectiveness of the proposed method through simulation studies on various graph structures, as well as a real data analysis of PM2.5 levels in California, exploring their relationship with wildfire occurrences.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.AI
Improving Machine Learning Performance with Synthetic Augmentation
Mel Sohm, Charles Dezons, Sami Sellami et al. (+2 more)
Synthetic augmentation is increasingly used to mitigate data scarcity in financial machine learning, yet its statistical role remains poorly understood. We formalize synthetic augmentation as a modification of the effective training distribution and show that it induces a structural bias–variance trade-off: while additional samples may reduce estimation error, they may also shift the population objective whenever the synthetic distribution deviates from regions relevant under evaluation. To isolate informational gains from mechanical sample-size effects, we introduce a size-matched null augmentation and a finite-sample, non-parametric block permutation test that remains valid under weak temporal dependence. We evaluate this framework in both controlled Markov-switching environments and real financial
datasets, including high-frequency option trade data and a daily equity panel. Across generators spanning bootstrap, copula-based models, variational autoencoders,
diffusion models, and TimeGAN, we vary augmentation ratio, model capacity, task type, regime rarity, and signal-to-noise. We show that synthetic augmentation is beneficial only in variance-dominant regimes, such as persistent
volatility forecasting-while it deteriorates performance in bias-dominant settings, including near-efficient directional prediction. Rare-regime targeting can improve domain-specific metrics but may conflict with unconditional permutation inference. Our results provide a structural perspective on when synthetic data improves financial learning performance and when it induces persistent distributional distortion.
Read the full abstract ▾
All authors Mel Sohm, Charles Dezons, Sami Sellami, Oscar Ninou, Axel Pincon