Statistics Applications · Cross-listed from cs.HC
The Judge Knows When It Knows: Calibrated Abstention for LLM-Based A/B-Test Prediction
Tyler Dooskin and Squoosh Technical Staff
Can a
multimodal LLM predict which version of a web page will win a real A/B test from screenshots alone? We report the most complete answer we are aware of, from six weeks of pre-registered experiments on real conversion tests: mostly no – and the exceptions are identifiable in advance. On 330 real A/B tests a Gemini 3 Flash judge reaches Cohen's kappa = 0.14, but on the trustworthy (statistically significant) half of the labels the evidence is inconclusive (kappa = 0.11, CI includes zero). We show that 44% of the "ground-truth" labels in a leading CRO agency's catalog come from non-significant tests, and that the judge agrees more with the unreliable labels than the reliable ones – a shared prior between labeler and model, not prediction. Every standard improvement lever (a 2.8x more expensive frontier model,
prompt redesign, stimulus fidelity, change-type priors) fails its pre-registered gate. The judge's confident calls are different: a vote-margin gate isolates a subset (49% coverage) reaching kappa = 0.31 on significant labels. We measure the mechanism directly – judges differing in model or
prompt agree with each other at kappa = 0.74-0.88 while agreeing with real outcomes at only ~0.2, so a 16-vote panel carries about 2 effective independent votes – and we reproduce it in humans: 15 CRO experts agree with each other (inter-rater kappa = 0.53) but score at chance against real outcomes (kappa ~ 0). Consensus, human or model, is reproducible, persuasive, and not evidence. We release our pre-registrations, locked gates, negative results, statistical harness, human responses, and a claims ledger in which every number carries an evidence tier.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CY
Simulating Eutopia: Revisiting Long-term Fairness with Outcomes, Performativity, and Dynamics
Vedant Palit, Udvas Das, Brahim Driss et al. (+1 more)
As AI-driven Decision Makers (ADMs) influence our socioeconomic reality, their roles in both enhancing efficiency and amplifying the social biases have drawn significant
attention. In this paper, we revisit the nuances of long-term
fairness achievable by an ADM– specifically, through the lens of a loan approver inducing a population-level wealth dynamics. The literature generally (a) considers passive environments, i.e. the decisions of an ADM does not change the population's behaviour, and (b) measures bias in terms of disparity in the instantaneous decisions rather than downstream equity. Modern ADMs challenge both the notions. To address these caveats, we first formalise the wealth dynamics induced by a loan approving ADM interacting with a multi-demographic population as a Performative Markov Decision Process. Then, we mitigate the absence of such a performative test-bed by developing Eutopia: a lending-process simulator enabled with a novel performative data generator to learn long-term fair strategies. With Eutopia, we test reinforcement learning
algorithms with different
fairness-aware utilities dependent on approval decisions and downstream wealth. Results show that performative dynamics-aware learning with
fairness-aware utility that incorporates the downstream outcomes induce better long-term equity and inclusivity.
Read the full abstract ▾
All authors Vedant Palit, Udvas Das, Brahim Driss, Debabrota Basu
Statistical Methodology
When Does Trial-Real-World Data Fusion Improve Precision? Model Auditing and Selection-Aware Inference for Adaptive-TMLE
M. Ehsan Karim
Augmenting a randomized controlled trial (RCT) with real-world data (RWD) promises greater efficiency, but how much a given fusion delivers, and how to attach honest uncertainty to that gain, are rarely characterized. Using adaptive targeted maximum likelihood estimation (A-TMLE) as a worked example of an
estimator that learns a working model and then debiases it, we develop three reproducible tools for reliable evidence from combined trial and real-world data. First, a report card that makes the data-adaptively learned bias model auditable: on simulated data it measures how well the model recovers the true enrollment-effect surface and attributes the
estimator's variance to its structural parts. Second, a map of when fusion helps versus hurts, benchmarked against a matched trial-only
estimator; the efficiency gain is driven mainly by the magnitude of the real-world bias rather than its functional complexity (a dominance an exact population-oracle variance identity explains), it crosses break-even near a moderate bias and erodes as the trial grows, so the advantage is finite-sample, not super-efficiency. Third, selection-aware inference for the gain, treated as a data-adaptive estimand: the naive standard error undercovers, and among ten candidate standard errors only a block jackknife achieved consistently near- or above-nominal coverage, though conservatively. Across six fusions of three openly available trials (a biomedical HIV trial, a public-health trial, and a job-training trial), only one interval clears one, and only marginally; in the rest, fusion has not earned an efficiency claim over the RCT alone. On real data the toolkit therefore functions mainly as a guardrail: the learned-model dimension is a stress diagnostic, not a proxy for ground truth, and the block-jackknife interval decides whether fusion or the RCT-only analysis should be primary.
Read the full abstract ▾
Statistical Methodology
A Bayesian Joint Model for Multiple Point Processes with Application to Presence-Only Data
Laura C. M. Teixeira, Dani Gamerman, Vinicius Peripato et al. (+1 more)
Joint modeling of multiple point processes is relevant in applications where relationships among processes are of interest, such as in ecological and archaeological studies. Statistical inference becomes particularly challenging when multiple processes are analyzed jointly and the observed data correspond to presence-only patterns, which are subject to preferential sampling and partial observability. This paper proposes a
Bayesian joint model for multiple point processes, with application to the presence-only setting. The dependence between processes is explicitly incorporated into the probabilistic specification of the model using
Bayesian networks. Direct use of the likelihood leads to intractable likelihood functions. Latent data processes are then introduced so that the augmented likelihood function becomes tractable and can be exactly evaluated. This formulation also enables direct inference on the number and the spatial distribution of unobserved occurrences of any of the point patterns. Inference is carried out using Markov chain Monte Carlo with blocked Gibbs sampling. Simulation studies demonstrate that the proposed inferential scheme is able to recover the true model parameters. The proposed model is applied to real presence-only data of archaeological sites and tree species from Amazonia, as part of the study of the effect that pre-Columbian Indigenous presence might have on the occurrences of relevant tree species. The results are consistent with the findings reported in the literature. They also illustrate how the proposed model enables inference on the existence and on the magnitude of the relation between processes, in addition to their association with environmental
covariates.
Read the full abstract ▾
All authors Laura C. M. Teixeira, Dani Gamerman, Vinicius Peripato, Carolina Levis
Statistical Methodology
Analytic Standard Errors for Latent Gaussian Discrete-Valued Multivariate Time Series
Christopher M. Crawford, Marie-Christine Düker, Younghoon Kim et al. (+3 more)
Unlike their continuous-valued counterparts, there are no universally preferred methodologies for modeling discrete-valued time series. This is especially problematic in fields such as psychology and education, where repeated-measures data often take the form of count, dichotomous, and ordered categorical variables. To address the need for flexible methodology for analyzing discrete-valued time series data, a copula-style multivariate model defined through deterministic functions of a latent stationary Gaussian vector series has been proposed. This model has several promising features, including the ability to accommodate a wide variety of marginal distributions within the same model while also allowing for the most flexible autocorrelation structure possible. We extend this framework by deriving analytic standard errors to facilitate inference on the latent Gaussian dynamics. In so doing, we establish the joint asymptotic normality of
estimators of the parameters governing the latent Gaussian series and the marginal distributions. The performance of these analytic standard errors is examined in a simulation study and an empirical application.
Read the full abstract ▾
All authors Christopher M. Crawford, Marie-Christine Düker, Younghoon Kim, Vladas Pipiras, Barbara L. Fredrickson, Zachary F. Fisher
Statistical Methodology
Targeted maximum likelihood estimation for longitudinal two-stage designs with outcome subsampling
Kirsten E. Landsiedel, Maya L. Petersen and Mark J. van der Laan
We consider efficient estimation of causal parameters in longitudinal two-stage designs with outcome subsampling, motivated by resampling designs in HIV-related mortality studies. In these studies, many participants become lost to follow-up; resampling designs address this by tracing a subset of lost individuals to ascertain their outcomes. Analyses often use inverse-probability-weighted Kaplan-Meier (wKM)
estimators that discard longitudinal
covariate information and suffer from efficiency losses. We note that resampling designs are an instance of a broader class: two-stage designs with outcome subsampling, in which a first stage collects some data on all participants and a second stage collects outcome information on a selected subset. This connection motivates two
estimators. First, drawing on inverse probability of censoring weighted targeted maximum likelihood estimation (IPCW-TMLE) for two-stage designs, we develop its longitudinal extension, IPCW longitudinal TMLE (IPCW-LTMLE) and show that estimating and targeting the known second-stage sampling weights yields variance reductions of up to 36% over the use of known sampling probabilities. Second, given that inverse weighting sacrifices efficiency, we propose an LTMLE that incorporates the second-stage sampling indicator as an intervention node in the sequential regression framework, returning to plug-in estimation and avoiding inverse weighting entirely. Simulations across sample sizes show that LTMLE achieves up to 73% lower variance than wKM with known sampling weights, with reductions of 30-50% common across settings, while IPCW-LTMLE achieves consistent gains of 20-35%. We further demonstrate that cross-fitted variance estimation is essential for valid inference: standard variance
estimators yield confidence interval coverage as low as 76%, while our cross-fitted variants consistently restore coverage to nominal levels.
Read the full abstract ▾
Machine Learning (Statistics)
Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms
Ye Tian, Mengchu Li and Marco Avella Medina
Integrating information across related tasks can improve estimation and prediction in transfer, multi-task, and
federated learning, but contamination and heterogeneity make robust borrowing challenging. We study a contaminated multi-task empirical risk minimization (ERM) framework in which an $\epsilon$ fraction of $K$ tasks, each with sample size $n$, may be arbitrarily contaminated while the remaining tasks are heterogeneous. Our goal is to estimate both the global minimizer of the average risk and the clean task-specific minimizers, thereby combining robustness and personalization. In the Gaussian mean model, we show that several common paradigms, including adaptive and robust regularization around a shared center, global
matrix regularization, decomposition-based regularization, and score-based outlier-task detection, all suffer from a worst-case contamination error of order $\epsilon\sqrt{d/n}$, which is suboptimal compared to the lower bound $\epsilon/\sqrt{n}$. This identifies a dimension-dependent barrier for these approaches. We then establish minimax lower bounds for a general heterogeneous ERM setting and propose a computationally efficient filtering-based robust multi-task gradient descent method. Under local strong convexity, smoothness, and sub-Gaussian gradient assumptions, the proposed method attains high-probability upper bounds matching the minimax rates up to logarithmic factors over a broad regime. In particular, it removes the extra $\sqrt{d}$ contamination dependence of many regularization-based methods and score-based outlier detection, while achieving personalization to local tasks under strong heterogeneity. Simulations and a real-data analysis demonstrate strong robustness and personalization relative to a broad range of
benchmark methods.
Read the full abstract ▾
Machine Learning (Statistics)
Benign Overfitting Does Not Occur in Diffusion Models
Tyler Farghly, Benjamin Dupuis, Alain Durmus et al. (+1 more)
Benign overfitting and double descent have come to shape our understanding of generalization in deep learning, establishing that overfitting is not only compatible with good generalization but can actively benefit it.
Diffusion models share much of the machinery of standard deep learning, so it is natural to assume that they also exhibit these properties. In this work, we show that this assumption is largely incorrect. We first establish fundamental impossibility results showing that, unless the sample size grows exponentially with the data dimension, overfitting and good generalization cannot occur simultaneously. Consequently, the population loss follows a classical U-shaped curve in model complexity rather than exhibiting double descent. Analyzing a simplified setting, we identify a key difference between regression and score matching: regression benefits from an
alignment between the target and the empirical covariance; score matching admits no such
alignment, leaving overfitting irreparably harmful. We further identify implicit regularization stemming from time-smoothness of the score and early stopping during training as mechanisms that prevent such overfitting and verify our findings with high-dimensional image generation experiments. Our results reveal that generalization in
diffusion models is governed by mechanisms distinct from those of traditional regression, motivating the development of new theory.
Read the full abstract ▾
All authors Tyler Farghly, Benjamin Dupuis, Alain Durmus, Umut Simsekli
Statistics Applications · Cross-listed from cs.AI
Online Safety Monitoring for LLMs
Mona Schirmer, Metod Jazbec, Alexander Timans et al. (+3 more)
Despite
alignment training, LLMs remain prone to generating unsafe outputs at deployment time. Monitoring outputs online and raising an alarm when safety can no longer be assumed is therefore critical. We study a simple real-time monitor that turns a verifier signal from an external model into an alarm decision by thresholding, with the threshold calibrated via risk control. In experiments on mathematical
reasoning and red teaming
datasets, we show that this simple design is competitive with more advanced monitors based on sequential hypothesis testing.
Read the full abstract ▾
All authors Mona Schirmer, Metod Jazbec, Alexander Timans, Christian Naesseth, Maja Waldron, Eric Nalisnick
Statistics Applications
Masked complex non-decimated wavelet features for patient-level classification of contrast-enhanced mammography
Sara Antonijevic and Brani Vidakovic
Contrast-enhanced spectral mammography (CESM) acquires two images of each breast, a low-energy image and a recombined contrast image, but two questions central to building a classifier on them remain unsettled: whether the two image types carry comparable malignancy signal, and how a patient's several images should be combined into a single decision. Both are hard to answer reliably, because most published CESM classifiers split cross-validation folds at the image level, letting images of the same patient fall in both training and test sets and inflating reported performance. We pair a masked complex non-decimated wavelet feature bank with an elastic-net logistic classifier, evaluated under repeated patient-grouped nested cross-validation with patient-cluster bootstrap inference on the CDD-CESM
dataset (1,880 images, 308 patients); under this leakage-free evaluation the inflation from testing on previously seen patients is negligible. On normal-versus-malignant detection, the two acquisitions are statistically indistinguishable in patient-level AUC under the proposed evaluation framework. Under single-image fusion the contrast image reaches a patient-level AUC of 0.874 (95% CI 0.827-0.918) and the low-energy image is statistically indistinguishable from it, yet the two encode malignancy through disjoint, interpretable channels: phase coherence on the low-energy image and magnitude distribution on the contrast image. The framework matches a pretrained ResNet-50 representation at the patient level, but whereas the frozen deep representation is not directly interpretable at the level of individual predictors, every predictor in the wavelet representation carries an explicit physical meaning. The result is a transparent, leakage-free baseline against which future CESM classifiers can be measured.
Read the full abstract ▾
Statistical Methodology
Inference for Group Interaction Experiments
Jiawei Fu, Cyrus Samii and Ye Wang
A common experimental research design is one in which individuals are randomly allocated into groups that then interact under different group-level treatment conditions. We develop design-based inference for such "group interaction" experiments, covering scenarios in which groups are either fixed or randomly formed and in which potential outcomes are either fixed relative to others' group assignments or subject to interference. For each scenario, we characterize the causal estimand that the design targets and the inferential strategy appropriate to it. Working in a sparse-sampling asymptotic regime, we show that cluster-robust inference remains consistent and accounts for dependencies from various sources when interference is present, delivering valid inference on marginalized exposure effects. When interference is absent and groups are formed randomly, the design reduces to an individually randomized experiment, and individual-level heteroskedasticity-robust inference suffices for the average treatment effect. Our results on the asymptotic distribution of commonly used
estimators rely on a novel coupling strategy that may be useful for design-based inference in other complex experiments.
Read the full abstract ▾
Machine Learning (Statistics)
Beyond Cultural Persona Bias: Response Geometry as a Measure for Language Models and People
Yuan Yuan
Researchers are increasingly prompting
language models with cultural personas so that they answer questionnaires as diverse human groups would. However, studies often report that these persona responses do not predict the models' behavior. We argue that whether the persona is Western or non-Western, the model's answers carry a cultural preference, and that this preference masks the variables that could predict its behavior. We examine the prompted preference at two levels: aggregate Big Five scores and response geometry, the correlation
matrix between dimensions within each respondent, represented as a symmetric positive-definite (SPD)
matrix. In DeepSeek-V3 and GPT-4o, when the item order is fixed, the prompted preference appears at both levels. Randomizing item order reduces it in the response geometry, while leaving it in the Big Five scores: the Big Five still separate the two personas at $95$–$97\%$, while response geometry falls to chance. In randomized administration, the response geometry still contains information about the respondent. In DeepSeek-V4-Flash, SPD features are associated with whether the same instance sorts Asian American public figures as “foreign” in a text IAT ($p = .047$), while the Big Five do not. In human studies conducted using randomized item order, the response geometry identifies people from test-retest sessions in $81$ – $85\%$, against chance $0.4\%$, and predicts sex and, weakly, age. In addition, SPD features also predict complex human behavior (e.g., cumulative GPA, $R^2 = 0.28$, permutation $p = .003$), where the Big Five do not.
Read the full abstract ▾
Statistical Methodology
Merging of Bayes and quasi-Bayes empirical Bayes procedures for Poisson compound decisions
Stefano Favaro and Sandra Fortini
The Poisson compound decision problem is a long-standing problem in statistics, in which empirical Bayes methods are used to estimate Poisson means under a mixture model. We study this problem from the viewpoint of $g$-modeling, comparing two nonparametric strategies for estimating the unknown mixing distribution: a
Bayesian empirical Bayes strategy, based on the Dirichlet process posterior, and a quasi-
Bayesian empirical Bayes strategy, based on Newton's
algorithm. The latter is computationally attractive, but its relationship with the
Bayesian strategy requires theoretical justification. Under a Poisson mixture model with a “true”, or oracle, mixing distribution, we establish concentration rates for the marginal probability mass functions induced by the
Bayesian and quasi-
Bayesian estimates. These rates are then translated into rates of decay for the corresponding regrets, interpreted as excess Bayes risks, and used to prove a frequentist merging result between the
Bayesian and quasi-
Bayesian empirical Bayes strategies. We also extend the analysis to the multidimensional Poisson compound decision problem. Numerical experiments on synthetic data illustrate that the quasi-
Bayesian strategy achieves accuracy comparable to the
Bayesian strategy, while requiring substantially fewer computational resources, especially in the multidimensional setting.
Read the full abstract ▾
Statistical Methodology
Cross-Audit Projection for Model Risk Prediction
Yijian Huang
For training-data-based model risk prediction, $K$-fold cross-validation~(CV) is widely used to mitigate the well-known over-optimism of the empirical risk and is often regarded as reliable. However, for binary classification via empirical risk minimization, our numerical studies reveal a surprising phenomenon: $K$-fold CV may perform poorly in estimating class-specific risks, even worse than the empirical
estimator. We perform a higher-order asymptotic analysis showing that $K$-fold CV may converge at a slower rate, whereas the empirical
estimator exhibits a second-order asymptotic bias that explains its over-optimism. These findings motivate a novel two-step procedure for model risk prediction, termed cross-audit projection (CAP). The cross-audit step adopts the same resampling scheme as $K$-fold CV to estimate over-optimism in subsamples, while the asymptotic-theory-informed projection step adjusts for the reduced sample size in bias correction of the empirical risk. The resulting CAP
estimator is first-order asymptotically equivalent to the empirical risk while achieving second-order asymptotic unbiasedness. An accompanying inference procedure is also developed. Simulation studies support theoretical advantages of CAP and demonstrate favorable finite-sample performance. An application to breast cancer detection further illustrates the proposed method.
Read the full abstract ▾
Statistics Applications
Instrumented difference-in-differences under case-control sampling
Tran Trong Khoi Le, Emilie Sbidian and Tat-Thang Vo
Case-control designs are fundamental in epidemiology for the efficient study of rare outcomes. Although instrumental variable (IV) methods have been extended to this setting to address unmeasured confounding, they typically rely on the exclusion restriction assumption, which may be violated when the IV candidates directly affect the outcome through pathways independent of the exposure. In this paper, we propose a novel instrumented difference-in-differences (iDiD) approach tailored to case-control designs. Grounded in structural mean modeling, the proposed method accommodates IV candidates that have time-invariant direct effect on the outcome. When retrospective case-control
datasets are collected, the candidate can still be used as a valid instrument on the trend scale when selection bias induced by retrospective sampling is efficiently taken into account. We assess finite-sample performance of this method through extensive simulations, then apply it to evaluate the risk of serious infection of biologic treatments for psoriasis, using French national claim database.
Read the full abstract ▾
Statistics Applications
Value-of-Information Analysis for External Validation of Risk Prediction Models in Multicenter Studies and Systematic Reviews
Laure Wynants, Kim Zhipei Wang, Sabine Grimm et al. (+5 more)
External validation studies have finite sample sizes, creating uncertainty about whether a prediction model's Net Benefit (NB) exceeds default strategies' NB. The expected value of perfect information (EVPI) quantifies consequences of uncertainty. Current EVPI methods focus on single studies, ignoring between-center heterogeneity. We extend EVPI and expected value of partial perfect information (EVPPI) to account for between-cluster heterogeneity in multicenter studies and meta-analyses. We distinguish between the global and local optimal strategy and between observed and unobserved clusters. We define EVPIglobal, EVPIcluster_j, EVPIcluster, and EVPPIcluster,prevalence, implemented in the MetaNB R package, and illustrate them using a systematic review across 36 centers of the ADNEX model for ovarian cancer diagnosis. Assuming one global decision regarding ADNEX adoption, there is no need for further data to confirm ADNEX is superior overall (EVPIglobal 0). Meta-analysis borrows information across observed clusters, resulting in consistent local superiority of ADNEX and nonzero but typically lower EVPIcluster_j than when considering local data alone. There is 0.03 probability default strategies are superior in unobserved centers. Eliminating uncertainty on performance and prevalence in each (EVPIcluster) would gain 1134 net avoided false positives (FP) per year, assuming 350000 tumors annually with 20% malignancies. Determining only local prevalence with certainty (EVPPIcluster, prevalence) would gain net 158 avoided FP per year. EVPI extensions disentangle sources of uncertainty and quantify the need for further validation to determine the global or locally optimal strategy. Considering uncertainty and heterogeneity in clinical utility across clusters is essential to decide whether additional validation studies are warranted.
Read the full abstract ▾
All authors Laure Wynants, Kim Zhipei Wang, Sabine Grimm, Andrea Gabrio, Andrew Vickers, Ewout Steyerberg, Ben Van Calster, Mohsen Sadatsafavi
Statistical Methodology
MATCH: Multiplier-Assisted Tests for Conditional Hypotheses in Non-Euclidean Data
Leheng Cai, Xu Guo and Qirui Hu
We propose a new procedure MATCH (Multiplier-Assisted Tests for Conditional Hypotheses) to test whether the non-Euclidean data match the target model, which is a general framework for significance and specification testing in Fréchet regression. MATCH covers global significance, partial significance, and the adequacy of global Fréchet regression, providing a unified way to compare unrestricted conditional Fréchet means with restricted alternatives. One of the key challenges is that the ordinary held-out loss difference is first-order degenerate under the null: the oracle losses coincide, and plug-in statistics is dominated by nuisance estimation error. MATCH uses sample splitting and independent random multipliers on held-out losses to create a nondegenerate Gaussian leading term without residuals or tangent-space coordinates. To improve data use and stability, we further develop cross-fitted tests and repeated cross-fitting with p-value merging. We establish asymptotic null validity, consistency under fixed alternatives, and local power guarantees. Simulations for distributional, symmetric positive-definite (SPD)
matrix-valued, and spherical responses support the theoretical findings, and applications to county-level household income distributions and North Atlantic tropical-cyclone locations demonstrate the practical use of the proposed tests.
Read the full abstract ▾
Statistical Methodology
Goodness of Fit Tests Based on Joint Densities of Multiple Sample Statistics
Roman Guchenko
We propose goodness-of-fit tests based on simulated confidence sets for joint distributions of multiple sample statistics, focusing on absolutely continuous null distributions with known parameters. One class of tests uses hyperrectangular confidence sets for principal components of order statistics and related statistic vectors. Extending earlier work on horizontal and vertical confidence bands for cumulative distribution functions, these tests are compared with some classical, Zhang, and related graphical tests. Simulations show that the proposed procedures are competitive with, and often more powerful than, existing methods. We also study the geometry of principal-component-based statistics; under a normal null distribution, the first principal component corresponds to the sample mean, while the second is related to a linear analogue of variance. A second class of tests uses confidence sets of arbitrary shape constructed through highest density regions. Unlike earlier kernel-density-based approaches, we use a k-nearest-neighbor method for detecting highest density regions, which is better suited to higher-dimensional statistic vectors. We study tests based on order statistics, empirical distribution function values, moments, and combinations of classical goodness-of-fit statistics. The resulting procedures are powerful against a wide range of alternatives. We also outline a two-sample extension via permutation tests based on joint distributions of several statistics and compare moment-based versions with energy-distance permutation tests. Finally, we discuss transformations other than the probability integral transform, showing that mapping data to another target distribution, such as the standard normal, can be advantageous when powerful tests are available for that distribution.
Read the full abstract ▾
Statistics Theory
Aggregation with Exponential Weights is Optimal in Expectation
Mikael Møller Høgsgaard, Patrick Rebeschini and Tobias Wegel
The aggregation with exponential weights (AEW)
estimator is not fully understood in the basic setting of model selection aggregation with squared loss. In particular, whether it is minimax-rate optimal in expectation for large enough fixed temperatures and under random design has been an open problem since its introduction, which was explicitly posed by Lecué and Mendelson (2013). In this paper, we settle this problem by showing that without requiring a Bernstein-type assumption, the AEW indeed achieves the excess risk $T \log (M) / (n+1)$ in expectation, whenever the temperature $T$ satisfies $(L^2/T)\exp(B/T)\leq \mu /2$. Here, the number of dictionary elements is $M$, the
estimator has observed $n$ i.i.d. samples from any distribution, and the loss is assumed to be bounded by $B$, $L$-Lipschitz continuous and $\mu$-strongly convex. For squared loss, we show that $T\geq 4 b^2$ suffices when the predictions and labels are $[0,b]$-valued. Because AEW is known to be suboptimal in expectation for temperatures below some constant, this shows that AEW has a sharp phase transition when the temperature is large enough but constant, as conjectured by Lecué and Mendelson.
Read the full abstract ▾
Machine Learning (Statistics)
An Additive MLP-GNN Framework for Characterizing Chemical and Structural Contributions to Aqueous Solubility
Sampreeti Bhattacharya and Arkaprava Roy
Aqueous solubility is a key property in early-stage drug discovery, but most predictive models merge physicochemical descriptors and molecular graph information into a single representation, obscuring whether a prediction is driven by global chemistry, molecular structure, or both. We present an additive deep-learning framework that keeps these two sources of information separate throughout training: physicochemical descriptors are encoded by a multilayer perceptron (the chemical branch) and molecular graph
topology by a graph neural network (the structural branch), with the two outputs combined only at the prediction stage through an additive model with an optional multiplicative interaction. This design provides a direct decomposition of chemical and structural components that can be examined separately after training. Furthermore, pretraining on the larger AqSolDB
dataset and fine-tuning on the smaller BigSolDB2
dataset substantially improve accuracy and reduce run-to-run variations, indicating generalizability of the learned features from the data-rich settings. We further interpret the fitted model using best linear projections of the branch outputs, molecule-level
embedding summaries across solubility classes, and atom-level GNNExplainer masks aggregated over functional groups. These analyses show that the chemical branch aligns with familiar physicochemical descriptors, while the structural branch captures graph-topological and functional-group patterns associated with solubility. Across both
datasets, the framework attains competitive predictive performance while making the distinct roles of chemical and structural information more transparent.
Read the full abstract ▾
Machine Learning (Statistics)
Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction
Yurui Zheng and Ying Jin
Predictions are increasingly used to guide high-stakes decisions, from treatment selection to policy making. To ensure reliability with imperfect predictions, uncertainty quantification methods such as conformal prediction build prediction sets with coverage guarantees. However, statistical validity alone does not immediately determine the decisions to take, nor the optimality thereof. This gap is especially delicate in counterfactual settings where the outcome that materializes depends on the action taken, so uncertainty cannot be specified independently of the decision rule. We develop a decision-theoretic framework for uncertainty-informed counterfactual decisions. We identify a novel notion of policy-coupled coverage – namely, coverage of the realized outcome under the action induced by the prediction sets themselves – as the optimal and lossless interface between uncertainty and action. It plays three roles. First, it justifies acting via a natural max-min rule as minimax-optimal under distributional ambiguity. Second, optimizing prediction sets under policy-coupled coverage is equivalent both to a stronger universal-coverage formulation and to the direct risk-averse optimization over policies and utility certificates; this equivalence yields the explicit form of the population-optimal prediction sets. Third, it admits a two-stage procedure, Policy-Coupled Risk-Averse Conformal Prediction (PC-RACP), that approximates these optimal sets with rigorous finite-sample coverage. Simulations and a real email-marketing experiment confirm that PC-RACP delivers higher utility than existing approaches while maintaining valid coverage, and that ignoring the counterfactual structure of the decision problem is suboptimal for both validity and utility.
Read the full abstract ▾
Statistical Methodology
Conformal Bayes for Two-Sided Censored Gaussian Regression under Label Shift
Seungjin Choi
Prediction under label shift becomes nonstandard when responses are censored. In a two-sided censored Gaussian model, latent values below $L$ and above $U$ are recorded at the boundary values, so the observed predictive distribution is mixed, with atoms at $L$ and $U$ and a continuous density on $(L,U)$. In this paper we develop conformal Bayes for this mixed-space setting by combining posterior predictive tilting with weighted conformal calibration. Under a two-sided Tobit Gaussian
Bayesian prediction head with a Laplace posterior approximation, the tilted predictive distribution has left-atom, interior, and right-atom components, with a three-term closed-form normalizer. The resulting prediction set is a mixed highest density region that can combine boundary atoms with an interior interval and can reduce to atom-only sets under strong censoring. The main technical issue is that latent label shift does not directly give an ordinary density ratio on the observed censored scale. A latent exponential tilt induces tail-averaged atom weights at the censored boundaries, while the interior ratio remains density based. This yields a mixed observed-space calibration weight with two atom ratios and one interior density ratio. The weight corrects the calibration measure, while predictive tilting gives target-adapted mixed-HDR geometry. Synthetic experiments show that weighted tilted conformal Bayes restores marginal coverage with smaller sets than weighted source-score calibration, while revealing a trade-off between marginal coverage and component-wise behavior across atoms and interior observations.
Read the full abstract ▾
Statistics Theory
A note on "The volume of random simplices from elliptical distributions in high dimension"
Shan Xizheng and Li Yanpeng
Recent work by Gusakova et al. (
Stochastic Process. Appl. 164 (2023) 357-382) has shown a central and a stable limit
theorem for the logarithmic volume of random simplices and random convex bodies under an elliptical framework in the high dimensional regime, that is, if p and n tend to infinity in such a way that the ratio tends to \gamma within (0,1). A technical condition (Equation (2.6) of Assumption (B) therein) requires that the population
matrix AA* is close in Frobenius norm to a multiple of the identity
matrix, which is rather restrictive and rules out various settings for statistical application, such as spiked models and dependent structure models. In this note we offer a general relaxation of this condition, which arrives at a reasonable condition and covers numerous scenarios, as well as consequences for the volume of general random simplices and random convex bodies. In particular, our results covers the Toeplitz/AR(1) covariance structures studied by Jiang and Pham (Ann. Stat. 53 (2025) 907-928), giving a concrete application of our
theorem to high-dimensional dependent covariance models.
Read the full abstract ▾
Statistics Applications
Quaternion Nondecimated Wavelet Descriptors for Multiclass Breast Histology Classification
Sara Antonijevic and Brani Vidakovic
Breast histology images carry diagnostic information in color, texture, orientation, and tissue architecture across a range of scales. In H&E microscopy this information is inherently chromatic and is not fully recovered when the red, green, and blue (RGB) channels are reduced to grayscale or transformed as independent scalar images. We propose an interpretable quaternion nondecimated wavelet framework for breast histology classification. Each RGB image is encoded as a pure quaternion field, and a quaternion nondecimated wavelet transform in two dimensions (QNDWT2D) produces multiscale, directional, color-coupled coefficient fields on the original image grid, keeping color as a single vector quantity rather than three separate channels. From these coefficients we build interpretable feature families summarizing stain balance, wavelet energy, amplitude heterogeneity, quaternion phase concentration, color-axis geometry, directional anisotropy, orientation
entropy, and scale-dependent energy decay, each tied to a histopathological property such as
nuclear density or glandular organization. We evaluate the descriptors on the BreAst Cancer Histology (BACH) challenge, a balanced four-class set of normal, benign, in situ, and invasive tissue, using a radial-kernel support vector machine (SVM) with repeated nested cross-validation. The descriptors yield balanced recognition across classes, with errors concentrated among adjacent categories while normal and invasive are rarely reversed. Permutation importance shows that directional, phase-concentration, anisotropy, scale, and amplitude-variability groups all contribute, indicating that the classifier draws on genuine quaternion and multiscale geometry rather than global color alone. The framework uses no pretrained networks, learned filters, or external databases, offering a reproducible, interpretable baseline for computational pathology.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
Predictive Conformal Slip Monitoring: An Empirical Evaluation of Rolling Split Conformal Prediction for Pre-Incident Traction Loss Detection
Varshith Roy Kotla
Conventional traction control architectures intervene only after the adhesion limit of a tire has already been breached. This paper investigates whether Rolling Split Conformal Prediction , monitoring the
volatility of non-conformity residuals from a per-driver Random Forest model of expected slip behavior , can serve as a statistically grounded pre-incident warning signal, ahead of gross traction loss. Unlike an earlier internal draft of this work, the evaluation reported here corrects a confound in the slip proxy (vehicle speed is included as an explicit model feature, not left implicit in the target's denominator), uses every racing lap for each driver rather than only the fastest lap, and is scored against real, timestamped incident labels extracted from FIA Race Control Messages and track-limits lap deletions rather than narrated post-hoc. The result is negative: across 19 drivers and 55,563 test-phase telemetry samples, the rolling-
volatility detector achieves a mean precision of essentially 0.0 and mean recall of 0.0 against 14 ground-truth incidents, while flagging on average 15.3% of all samples as anomalous , too high a false-alarm rate for any early-warning use. A static 95th-percentile threshold baseline performs no better in any way that would justify the added complexity of the conformal-
volatility formulation. Residual autocorrelation diagnostics show the split-conformal exchangeability assumption is violated for every driver (Ljung-Box p < 0.001, n = 19/19), which is one plausible driver of the high false-alarm rate. We report this as a methodologically rigorous negative finding, diagnose its likely causes, and outline what a genuinely predictive version of this approach would require.
Read the full abstract ▾