Statistical Methodology · Cross-listed from math.ST
Asymptotic Null Distributions of Moran's $I$ and Assortativity in Large Networks
Karin Ait Braham, Louis-Paul Rivest and Thierry Duchesne
This study investigates the asymptotic behavior of two dependence measures defined on networks, Moran's $I$ statistic and Newman's assortativity, under the null hypothesis that a Gaussian node attribute $Y$ is independent of the network structure. We demonstrate that the structure of the network directly affects the convergence rate to normality of these measures as the size of the network increases. We further establish that, in some instances, the mean values of these dependence measures under the null hypothesis remain non-negligible asymptotically and must therefore be explicitly accounted for when calculating the test statistics. Applications to a variety of simulated and real networks also reveal that the normal approximation performs well only when the network is not strongly heterogeneous. Network
topology determines both the convergence rate to normality and whether the limiting distribution is Gaussian. In dense networks whose degree heterogeneity does not vanish, we further show that assortativity can fail to be a valid test statistic even though Moran's $I$ remains well behaved, whereas a dominating node invalidates both.
Read the full abstract ▾
Statistical Methodology
When does conformal calibration need censoring weights? Cause-of-failure prediction sets under competing risks
Sunny Yang and Weiyan Zhao
Split conformal prediction sets for competing-risks labels at a fixed horizon require calibration labels that right censoring can leave unobserved. Complete-case calibration guarantees coverage for the label-complete subpopulation, but its population coverage can deviate in either direction, even at the true class probabilities. We study how selection changes the score distribution near the population
quantile. At the true score in our main simulation family, with independent draws, complete-case calibration covers 0.8723 at a nominal 0.900 when 22% of subjects are event-free at the horizon. In 4 of 48 further designs using per-draw normalisation, estimating cause incidences as one minus the exponential of the negative of the cause-specific Nelson-Aalen cumulative hazards, with the event-free probability as the clipped and renormalised remainder, puts complete-case coverage at least four standard errors below nominal. In these designs, weights from a correctly specified censoring model keep coverage near nominal. Extending the argument of Yi et al. (2025) to a cause label, we establish a finite-sample coverage lower bound with an explicit penalty for censoring-model error. Misspecifying the censoring model can also lead to under-coverage.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
Measuring Gift Card Program Incrementality via Causal Data Fusion
Justin Whitehouse, William Betz, Yan Zhang et al. (+3 more)
Businesses regularly offer gift card programs to drive customer spending and increase engagement. A central question is how much incremental revenue these programs generate, and which channels drive it most efficiently. Measuring the incremental revenue associated with a gift card program is a challenging problem in causal inference, requiring a firm to infer how much each customer would have spent if they never received a gift card. Observational data on past customer purchasing behavior reveal possession of a gift card only when a customer makes a purchase, thus leaving a customer's treatment status systematically censored. In this paper, we develop a novel data fusion approach to overcome this missing data challenge. We identify and estimate incrementality by combining a large observational
dataset with a smaller experimental
dataset from a different population. Our approach relies on a mild transferability condition, which posits that the conditional relative treatment effect of gift card receipt on the decision to purchase is invariant across the two populations. We develop a flexible, machine learning-based
estimator for the incremental revenue and establish its asymptotic normality. We apply our
estimator across both first- and third-party channels through which Airbnb distributes gift cards, finding heterogeneity in incrementality across segments of the population. In particular, we find not only that third-party channels are more incremental than first-party ones, but also that "self-gifters" (i.e., customers likely to have purchased their own gift cards) are more incremental than the broader population.
Read the full abstract ▾
All authors Justin Whitehouse, William Betz, Yan Zhang, Peter Coles, Ramesh Johari, Vasilis Syrgkanis
Statistical Methodology
Fridge: focused fine-tuning of ridge regression for personalize predictions
Kristoffer Herland Hellton and Nils Lid Hjort
Statistical prediction methods typically require some form of fine-tuning of tuning parameter(s), with $K$-fold cross-validation as the canonical procedure. For ridge regression there exist numerous procedures, but common for all, including cross-validation, is that one single parameter is chosen for all future predictions. We propose instead to calculate a unique tuning parameter for each individual for which we wish to predict an outcome. This generates an individualized prediction by focusing on the vector of
covariates of a specific individual. The focused ridge – fridge – procedure is introduced with a two-part contribution: 1) first we define an oracle tuning parameter minimizing the mean squared prediction error of a specific
covariate vector, 2) then we propose to estimate this tuning parameter by using plug-in estimates of the regression coefficients and error variance parameter. The procedure is extended to logistic ridge regression by utilizing parametric bootstrap. For high-dimensional data, we propose to use ridge regression with cross-validation as the plug-in estimate, and simulations show that fridge gives smaller average prediction error than ridge with cross-validation for both simulated and real data. We illustrate the new concept for both linear and logistic regression models in two applications of personalized medicine: predicting individual risk and treatment response based on gene expression data. The method is implemented in the R package "fridge".
Read the full abstract ▾
Statistical Methodology
Elastic kernel Ridge regression, with applications in phonetics
Beatrice Matteo, Almond Stoecker, Davide Pigoli et al. (+1 more)
Predicting the shapes of entire curves requires accounting for nonlinear geometry and unknown
alignments between curves. We develop elastic kernel ridge regression, a nonparametric method for planar curve responses with scalar, multivariate, or functional
covariates. Using square-root velocity representations, we formulate penalized conditional Fréchet mean estimation in a shape space invariant to translation, rotation, scaling, and reparametrization. A vector-valued reproducing kernel
Hilbert space provides a flexible nonlinear link through the spherical exponential map. We use an alternating
algorithm to align each observed curve to its current fitted value, and updates the regression function. To facilitate this, we provide a new Euclidean quasi-Newton solver that exploits scale invariance of the reparametrization objective; this accelerates
alignment while retaining accuracy in a numerical comparison. Simulations demonstrate the benefits of estimating
alignment within the regression and respecting spherical geometry. Applied to vocal tract contours from a real-time
magnetic resonance imaging recording, the method recovers missing frames and reconstructs shape trajectories from downsampled data. A
speech inversion proof of concept on the same recording predicts tongue shapes from acoustic features, illustrating the method's potential. The method is implemented in the R package sphereg2.
Read the full abstract ▾
All authors Beatrice Matteo, Almond Stoecker, Davide Pigoli, Shahin Tavakoli
Statistical Methodology · Cross-listed from stat.ML
High-Dimensional Statistical Inference for Sparse Support Vector Machines
Peng Zeng and Hanwen Huang
Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample size and number of features grow proportionally. The main challenge is the nonsmooth hinge loss, which prevents direct application of debiasing arguments developed for smooth classification losses. We overcome this difficulty by representing the $L_1$-penalized support vector machine (SVM) as a linear program and identifying the hinge-loss subgradient through its dual variables. This yields a computationally accessible debiased
estimator whose coordinates are asymptotically Gaussian under the proportional asymptotic regime. The resulting distributional characterization provides confidence intervals and hypothesis tests for individual features and enables false-discovery-rate-controlled variable selection. Extensive simulations examine calibration, power, and variable-selection performance under a range of covariance structures, including strongly correlated designs. An analysis of high-dimensional breast cancer gene-expression data illustrates how the proposed inference can distinguish statistically significant features from variables selected by the original sparse SVM.
Read the full abstract ▾
Statistical Methodology
Covariate-dependent Nonparametric $g$-modeling for regression via infinite Mixture-of-Expertizing class
Akira Okazaki and Keisuke Yano
Empirical Bayes $g$-modeling captures unit-level heterogeneity by estimating a latent prior distribution from observed data. In the existing formulations, however, the prior is shared by all units. In this paper, we develop a
covariate-dependent g-modeling framework for regression in which the entire prior distribution of the regression coefficients is allowed to depend on
covariates. We formulate the estimation of the prior as nonparametric maximum likelihood estimation (NPMLE) of the
covariate-dependent prior, and show that the unrestricted problem is ill-posed. To resolve this, we introduce the infinite Mixture-of-Expertizing class of conditional priors, under which the NPMLE is precisely a softmax-gated Mixture of Experts (MoE) whose number of experts is not fixed in advance but is determined by the data. Building on a first-order optimality condition, we propose two exemplar-based estimation
algorithms that select experts automatically, together with a post-hoc aggregation of experts for interpretation. On the theoretical side, we show that every conditional prior in the class is Lipschitz continuous in the
covariates, and that aggregated softmax gates can approximate any continuous gate function. The effectiveness of the proposed NPMLE is shown through application to synthetic
datasets and real
datasets.
Read the full abstract ▾
Statistical Methodology
Bayesian heterogeneous copula mixtures with nonparametric margins: consistency, identifiability and tail asymmetry in physical fitness data
Yujian Liu
Finite mixtures of Clayton, Gumbel, Frank and Gaussian copulas can describe dependence that differs between the upper and lower tails. We study a two-stage
Bayesian analysis in which ranks or kernel estimates replace the margins and the copula likelihood is evaluated at the resulting pseudo-observations. This pseudo-posterior is strongly consistent for the copula density whenever the log density admits a logarithmic boundary envelope. Both stages may use the same data, margins may be standardized within observed strata, and neither smoothness nor identifiability is required. The envelope holds for finite mixtures of Gaussian, Student, Clayton, Gumbel and Frank copulas in any fixed dimension; tail-dependence coefficients and conditional tail probabilities are therefore consistently estimated. We further prove that Clayton, Gumbel, Frank and Gaussian copulas are jointly finitely linearly independent, which makes mixture weights and components identifiable and consistently estimated. In simulations the pseudo-posterior matches multi-start maximum pseudo-likelihood in large samples. It is more stable in small samples and near independence (every component close to the independence copula), where tail coefficients are learned long before the weights and a marginal Metropolis sampler is up to twice as efficient as data augmentation. Two physical fitness
datasets show mirror-image asymmetries. Among 8772 university students, sprint and jump performance are coupled mainly at the top. Among 5336 adults in a national health survey, low grip strength and low daily activity cluster together, increasingly with age, whereas high values do not. A Gaussian copula misses both patterns in held-out data.
Read the full abstract ▾
Statistical Methodology
Where Do Two Populations of Persistence Diagrams Differ? Calibrated Local Inference at a Fixed Budget
Pramita Bagchi, Edward Bae, Atish Mitra et al. (+3 more)
Many two-sample tests for populations of persistence diagrams assess global differences without identifying the regions of the birth-death plane that contribute to them. We study simultaneous inference for local mean contrasts when the number of available diagrams is fixed. They are differences in expected weighted feature mass within $\ell_\infty$ neighborhoods at several centers and radii. We estimate these contrasts using additive landmark responses. A Gaussian multiplier bootstrap calibrates simultaneous confidence intervals while allowing unequal group covariances. The neighborhoods whose intervals exclude zero form a map with approximate family-wise error control, and selecting a subset of original intervals for display preserves their joint coverage guarantee. On the simultaneous coverage event, every reported neighborhood lies within twice its radius of the support of the mean-measure difference. A
geometric result gives sufficient radius conditions for a displaced feature to produce a nonzero contrast. A comparison of sufficient detection thresholds quantifies the tradeoff between reducing the number of tested coordinates and reserving observations for an independent pilot. In simulations with 40 to 120 diagrams per class, the bands achieved 94%-98% simultaneous coverage under both the strict null and equal means with unequal covariances. In the latter setting, a permutation maximum and the pooled-t implementation of the two-stage persistence-image test of Moon and Lazar rejected in up to 32% and 26% of runs, respectively. In the fixed-budget simulations, spending a third of the observations on a pilot to choose landmarks or radii located changes less often than a prespecified grid at a single radius. On the MUTAG
benchmark, the localized region concentrates on rings of fused-ring systems, an exploratory reading.
Read the full abstract ▾
All authors Pramita Bagchi, Edward Bae, Atish Mitra, Alexander D. Silberman, Žiga Virk, Sushovan Majhi
Statistical Methodology · Cross-listed from stat.ML
Anytime-valid simulation-based hypothesis testing
Patrick Forré and Lydia Brenner
For a given data distribution $(X_t)_{t \in \mathbb{N}} \sim Q$ i.i.d., we investigate the hypothesis testing problem: $H_0: Q = P_0$ vs. $H_1: Q = P_1$, for two different model probability distributions $P_0$ and $P_1$. In contrast to the standard setting, where analytic densities $p_0$ and $p_1$ are given, here, we consider the density-free setting, where we only have access to i.i.d. simulations $(Z^0_t)_{t \in \mathbb{N}} \sim P_0$ and $(Z^1_t)_{t \in \mathbb{N}} \sim P_1$. For this simulation-based hypothesis testing setting, we construct an e-test martingale, resulting in a sequential test with anytime-valid type-I error guarantees, approximate growth optimality, geometrically decaying type-II error bounds, and asymptotic power one. Most ingredients used in our constructions are variants of well known concepts. The value of this paper lies in the compact presentation of an effective, anytime-valid solution for the density-free simulation-based sequential hypothesis testing case.
Read the full abstract ▾
Statistical Methodology
The Missing Corner: When Sharp Causal Bounds Fail to Compose
Yicheng Qi and Xiyi Xiong
Sharp bounds on two potential-outcome means need not combine into sharp bounds on their difference. We characterize exactly when composition is guaranteed across environments sharing a joint potential-outcome law. For binary treatment and outcome, fix strict selection-probability bounds and interior treatment propensities. Subtracting the full model's sharp mean intervals gives sharp average-treatment-effect bounds for every compatible observed law if and only if the absolute selection bounds coincide across environments. Under this
alignment, the complete mean region is rectangular; its endpoints and attaining causal models require $O(K)$ arithmetic operations for $K$ environments. For every mismatch, failure occurs on a relatively open set of positive observed tables at the prescribed propensities and persists under small joint drift. We give computable perturbation certificates and an exact family whose gap is first order in the mismatch. A separate finite-confounding family shows when the lost joint information changes an identified causal sign. These results distinguish local marginal sharpness from joint attainability after combining environments.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
The Noise Is the Signal: Correlated Sampling Error Is Rank-Informative for Proxy Metric Selection
Sandro Provenzano
North-
star metrics such as customer lifetime value are often too slow and noisy to decide a short A/B test. Teams therefore rely on a proxy metric, commonly chosen by how closely its effects tracked the north
star's across past experiments. Validating that choice, or any method for making it, is hard: the only
benchmark is the noisy north
star, and the number of available past experiments is limited. In addition, proxy and north-
star effects are estimated on the same customers, so their sampling errors are correlated. Recent work at major experimentation platforms removes this shared error as contamination, improving estimates of the true-effect covariance. Choosing a proxy, however, is a ranking problem, and a better estimate need not give a better ranking. We measure agreement free of shared error by estimating the two effects on disjoint random halves of each experiment's customers. In an archive of 262 experiments and 69 candidate proxies, the shared error ranks the candidates in a similar order to this agreement (Spearman correlation 0.65): it carries information about proxy quality. The more of it a correction removes, the worse the ranking because removal discards part of the signal but leaves the main sources of ranking noise, the noisy north
star and the limited number of experiments, untouched. Archive-calibrated simulations, in which the correct ranking is known, confirm this even when every correction receives the true sampling covariance. Held-out real experiments, evaluated on disjoint customer halves so that shared error cannot bias the comparison, closely reproduce the predicted ordering (Spearman correlation 0.93). Correction can still pay off with more experiments, but the number needed rises steeply with the north
star's noise. We map this crossover and give platform teams three inexpensive checks for deciding from their own archive whether and how strongly to correct.
Read the full abstract ▾
Statistical Methodology
TEASE: Targeted elastic spatial envelope
Mitchell L. Krock
Envelope regression is a crucial part of multivariate linear modeling, leveraging separation into material and immaterial parts to provide parsimonious data reduction. Rekabdarkolaee et al. (2020) construct spatial envelope, a novel multivariate Gaussian process, extended by May et al. (2022) to linear coregionalization envelope. Our proposal (TEASE) combines spatial envelope and full-scale basis graphical lasso (LeDuc et al., 2025), leveraging targeted graphical elastic net (Kovács et al., 2021) for immaterial precision
matrices. Material part is a multivariate multiresolution Gaussian Markov random field (Kleiber et al., 2019; Caringi and Secchi, 2026). Enhanced envelope penalty (Kwon and Zou, 2025) regularizes regression coefficients. Demonstrations are on
climate data.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.DC
Beyond Marginal Monitoring: Distributed Joint-Distribution Testing for Data Concept Drift in Large Scale E-Commerce Operations
Cagdas Pullu, Mahmut Emir Arslan, Bugra Balkac et al. (+4 more)
Concept drift threatens production machine learning, yet the empirical behavior of multivariate two-sample drift detectors at scale remains under-characterized. Existing
benchmarks rarely address the hundreds of millions of rows and high-cardinality features typical of industrial-operational
datasets. We evaluate five multi-column two-sample tests (marginal, projection-based, and kernel
embedding methods) across three complementary environments: the Harvard Dataverse, a validated Failing Loudly reproduction (mean absolute error between 0.030 and 0.053), and a novel synthetic-injection
benchmark on the 137.5-million-row Trendyol collection-ranking feature table. Testing four drift types across two severity-scope regimes, we demonstrate that distributed Maximum Mean Discrepancy with Random
Fourier Features on Apache Spark scales robustly. Averaged over the four drift types in the strong regime and under a calibrated threshold, it achieves a Pearson correlation of r = 0.940 with expected drift magnitude, an 80.4% true positive rate, and a 3.2% false positive rate. Conversely, the per-dimension Kolmogorov-Smirnov test failed due to statistic saturation from ID-like columns under asymmetric sampling, establishing a critical constraint for large-scale sampling design. At weak configurations (realized-flip fractions of at most 0.57%), detectors struggled to reliably discriminate, highlighting the need for future intensity-grid power analyses to distinguish fundamental sensitivity bounds from scalable threshold shifts.
Read the full abstract ▾
All authors Cagdas Pullu, Mahmut Emir Arslan, Bugra Balkac, Aylin Ondersev Balta, Cihangir Celal Palaci, Fikri Cem Yilmaz, Altan Cakir
Statistical Methodology
The Geometry of Existence and Uniqueness of Maximum Likelihood Estimation in Categorical Response Models
Lukas Sablica, Kurt Hornik and Thomas Rusch
Nonexistence of the maximum likelihood estimate (MLE) under separation is treated as a solved problem for binary logistic regression and as a scattered collection of model-specific results everywhere else. We show that it is one phenomenon with one criterion. In a latent polyhedral categorical response model, every observed outcome corresponds to a polyhedral event in latent variables whose faces shift linearly with the parameter. Random-utility choice, cumulative-link, ranking, multivariate binary and ordinal, sequential, adjacent-category logit, and fixed-score stereotype models belong to this family. The likelihood sees each observed factor only through the columns of its threshold map, the structure vectors. A finite MLE exists if and only if the pooled structure vector set has overlap. Sufficiency requires only continuity, necessity requires only strictly threshold-increasing probabilities, and neither likelihood concavity, exchangeability, nor full design rank is needed. Positive strictly log-concave latent densities and threshold-identifiable polyhedra then yield uniqueness on the estimable span. A single linear program determines whether overlap holds, and convex cone geometry measures the dimension of separation. An application using cumulative-link models for willingness to share health data identifies observations and model terms associated with nonexistence and illustrates how the diagnostics inform model revision and sensitivity analysis.
Read the full abstract ▾
Statistical Methodology
What Should We Measure Next? Finding Identification Strategies by Refining Mechanisms
Mikko Väänänen, Fanyu Cui, Carlos Cinelli et al. (+1 more)
Canonical approaches in causal inference treat model specification as fixed, assuming that researchers directly translate all relevant domain knowledge into a causal model, which can then be used to deduce its logical implications. Yet, in practical applications, model specification is often an iterative process, and involves exploration and introspection: of the many aspects of the phenomenon one could investigate, which ones actually matter for the identification of the causal effect of interest? In this paper we study the problem of iterative identification in partially specified causal models. We focus on determining where observing variables that intercept a direct effect or a confounding path between two variables could enable identification in semi-Markovian models. We give necessary and sufficient graphical conditions for when observing such variables can render an unidentifiable query identifiable, together with an efficient
algorithm for locating all such opportunities in a given causal diagram. Our results can help analysts better navigate the model space by drawing
attention to the parts of their substantive knowledge that could result in a successful identification strategy.
Read the full abstract ▾
All authors Mikko Väänänen, Fanyu Cui, Carlos Cinelli, Santtu Tikka
Statistical Methodology
Collaborative representations for targeted causal inference under outcome selection
Johan de Aguas
Estimating causal effects with missing outcomes requires learning outcome, exposure, and selection models. Flexible propensity learners can predict treatment or missingness well while worsening target estimation by emphasizing instruments, weak-overlap regions, or variation unrelated to outcome-regression bias. We propose a collaborative representation-based targeted learning method for recovered average treatment effects under outcome selection. The method learns low-dimensional exposure and selection representations from cross-fitted pseudo-outcomes that encode outcome-regression drift. A finite candidate library alternates targeted outcome updates with representation updates, and inner cross-validation selects representation complexity, collaboration strength, and regularization using a target-aware risk. We give high-level sufficient conditions for collaborative robustness, consistency, and asymptotic linearity after outer validation-fold targeting. Simulations show finite-sample bias reduction, with explicit bias-variance trade-offs across configurations and relative to competing
estimators
Read the full abstract ▾
Statistical Methodology
The Topp-Leone XLindley Distribution: Properties, Estimation and Applications to Lifetime Data
Saileshwari M, Rajesh Moharana and Kousik Maiti
In this article, we use a family of distributions developed by Topp-Leone to create a novel lifetime two-parameter distribution. The Topp-Leone XLindley distribution is the name used for it. We study several types of statistical and mathematical characteristics of this distribution, such as reliability functions, moments, moment-generating function,
quantile function, and the Renyi
entropy. For the purpose of estimating parameters under the proposed distribution model, this simulation study is conducted to evaluate the performance of the maximum likelihood estimates for the parameters of the Topp-Leone XLindley distribution. A comprehensive simulation investigation is carried out to evaluate the performance of the proposed methods. Furthermore, a practical data set of bank customer waiting times has been investigated for the purpose of illustration.
Read the full abstract ▾
Statistical Methodology
Scope-Restricted Backtracking Counterfactuals
Johan de Aguas and Martin Jullum
Interventional counterfactuals evaluate statements such as "had $A$ been $a$" by replacing the assignment for $A$ while keeping the unit's exogenous background fixed. Backtracking counterfactuals reverse this logic: they preserve the structural assignments and instead vary background conditions across worlds. This semantics is natural for diagnostic explanation, but full backtracking may be too permissive because every exogenous coordinate can vary. We introduce scope-restricted backtracking counterfactuals, in which the modeler specifies which coordinates may vary while the remainder are shared across worlds. The scope determines the admissible carriers of the contrast, while a cross-world kernel governs the plausibility of their alternative values. We present model-based inference for general scope-restricted backtracking counterfactuals, derive sharp marginal bounds for a specified explanatory query, and establish frontier and compatibility conditions under which this explanatory query admits a modified-value policy representation. A linear-Gaussian example distinguishes local-noise, ancestor, irrelevant-noise, and infeasible explanations across scopes, while an application to the UCI Adult
benchmark illustrates kernel specification, sensitivity, and how much of the explanation is carried by the cross-world coupling itself.
Read the full abstract ▾
Statistical Methodology · Cross-listed from q-fin.RM
A Functional Representation of Credit Behavior for Probability of Default Modeling
Jonas Brunholm, Bjarne Højgaard, Thomas D. Nielsen et al. (+1 more)
This paper proposes a framework for modeling probability of default via functional data analysis. By representing a series of credit variables as functions, we investigate whether intra-monthly information improves default predictability in linear models. We further show how a range of widely used variables, among them available funds, utilization rate, and overdraft, can all be derived from three quantities observed over time. Namely, 1) the type of each account, 2) the balance of the account, and 3) the size of the credit limit on the account. Retaining these quantities as continuous-time processes, rather than reducing them to monthly aggregated values, yields a continuous faithful representation of the borrower. When analyzing these three processes, we discovered that recurring events associated with the ordinal position among banking days created strong cyclical patterns. We therefore develop a relative time framework that aligns the recurring events across borrowers, ensuring that borrowers possess the same cyclical pattern, regardless of real time. We assess the framework using functional logistic regression. This approach accommodates the continuous representation while remaining closely related to a logistic regression model commonly used in credit risk practice, due to strict regulatory constraints. We show that, when equipped with an effective functional representation of transactional trajectories, the proposed model attains predictive performance on par with XGBoost while consistently outperforming logistic regression. Importantly, the model balances predictive performance of a machine learning model with the interpretability of linear default models, potentially enabling financial institutions to use the model, even under strict regulation.
Read the full abstract ▾
All authors Jonas Brunholm, Bjarne Højgaard, Thomas D. Nielsen, Orimar Sauri
Statistical Methodology
State-space representation of early-life child mortality dynamics: linking event history analysis and orbit-based models
Wendy Lefotlha, David Sherwell and Maria Vivien Visaya
Understanding early-life child mortality requires methods that capture both statistically significant risk factors and the structure of individual life trajectories. Event History Analysis (EHA) estimates associations between
covariates and mortality risk but may obscure rare or high-dimensional configurations of risk. Orbit Theory (OT) represents individuals as exact trajectories in a combinatorial state space, preserving full multivariate structure. We analyse longitudinal data across 12 variables for 31,081 children from the Agincourt Health and Demographic Surveillance System (South Africa, 1998-2008) using both methods. We show three things EHA alone cannot. First, maternal migration is structurally pivotal in the transition dynamics leading to child death despite being statistically non-significant in EHA. Second, maternal refugee status, while statistically significant, generates no trajectory dynamics and is correctly excluded from any structural reduction. Third, a three-variable subsystem (child status, mother status, mother migration) recovers EHA's principal findings within a reduced state space of at most 48 states while exposing small high-risk clusters invisible to regression-based analysis. Maternal death is the dominant EHA predictor; maternal migration organises 87% of the observable sequential transitions into child-death states. These are not contradictory: maternal death rarely appears as a preceding state because its role is terminal rather than sequential. Early-life mortality can be read at two levels simultaneously - statistically, through effect estimates, and structurally, through observed transitions.
Read the full abstract ▾
Statistical Methodology
Design-Driven Inference for Online Experiments: Jointly Optimal Evidence Collection, Testing, and Estimation
Jun Yu, Wenbiao, Zhao et al. (+2 more)
Online multifactorial experiments requires early stopping for practically negligible effects while controlling false stopping under continuous monitoring and retaining precise treatment-effect estimation. We develop a design-driven framework that chooses allocation and the evidence rule jointly, treating allocation as evidence collection and testing and estimation as complementary uses of the same evidence. For multifactor experiments with nuisance block effects and treatment-by-block interactions, we show that block-orthogonal designs remove nuisance contamination from the treatment score and form a complete class for worst-case evidence growth. Under a fixed information budget, an isotropic allocation and an explicit radial e-value jointly attain the minimax rate optimal for directionally unknown alternatives. The same allocation is universally optimal for maximum likelihood estimation of treatment effects, simultaneously achieving A-, D-, and E-optimality and establishing double optimality for testing and estimation. Batchwise replication yields an anytime-valid e-process, retains minimax rate optimality for cumulative conditional worst-case evidence growth, and preserves universal estimation optimality for the prespecified complete experiment. Simulations and a large
language model
prompt experiment illustrate gains in stopping efficiency and estimation precision.
Read the full abstract ▾
All authors Jun Yu, Wenbiao, Zhao, Lixing, Zhu
Statistical Methodology · Cross-listed from stat.ML
Trustworthy Method Comparison with AI Judges: Estimation and Design under Order, Batch, and Aggregation Effects
Tianxi Li and Jie Ding
Large
language models (LLMs) are increasingly used as judges for automated AI evaluation. A common practice is to randomize
prompt sequences and average the resulting scores, but its statistical validity remains unclear. We show that LLM evaluation mechanisms can be approximated by a class of Markov generalized linear mixed models (GLMMs), supported by out-of-sample predictions across three major commercial LLMs. Using a first-order Markov GLMM, we study leaderboard ranking and group comparison. For leaderboard ranking, randomize-and-average selection is consistent under a mild separation condition, and a Williams square design can improve efficiency when item qualities are close. For group comparison, naive averaging can yield inconsistent conclusions about differences in group-level quality because of the response model's nonlinearity. Empirical results further support the validity of the proposed model-based inference beyond the first-order theory, including settings with higher-order sequence
memory. We illustrate the approach in an application where AI judges compare two graphical model estimation methods.
Read the full abstract ▾
Statistical Methodology
Measuring Negotiation Traits with Collaborative Choice Data: A Two-Stage Latent Variable Model
Tongxin Zhang, Yunxiao Chen, Jiangang Hao et al. (+4 more)
Consider a collaborative decision-making setting in which team members with different reward structures need to make a joint decision. Repeated collaborative decisions can reflect individual differences in team members' negotiation ability. However, extracting such information is challenging because each observed response arises from a collaborative process involving multiple participants and their reward structures. In this paper, we propose a two-stage probabilistic measurement model for structured collaborative choice tasks. In the first stage, known member-specific rewards and latent participant negotiation traits jointly determine team-option utilities, from which a latent team choice is generated. In the second stage, individual responses are modeled conditional on the latent team choice, with possible deviations that depend on participants' own rewards. Participant-level
covariates can also be incorporated through a structural model. A simulation study shows satisfactory recovery of the participant traits and model parameters across different sample sizes and item sizes. An application to data from simulated collaborative negotiation tasks shows choice prediction accuracy well above the random-choice baseline and meaningful associations between the estimated traits and external criterion variables. The proposed framework provides a psychometric approach to measuring participant-level negotiation traits with collaborative choice data.
Read the full abstract ▾
All authors Tongxin Zhang, Yunxiao Chen, Jiangang Hao, Emily Kerzabi, Patrick Kyllonen, Irini Moustaki, Tao Xin