Machine Learning (Statistics)
Kriging via variably scaled kernels
Gianluca Audone, Francesco Marchetti, Emma Perracchione et al. (+1 more)
Classical Gaussian processes and Kriging models are commonly based on stationary kernels, whereby correlations between observations depend exclusively on the relative distance between scattered data. While this assumption ensures analytical tractability, it limits the ability of Gaussian processes to represent heterogeneous correlation structures. In this work, we investigate variably scaled kernels as an effective tool for constructing non-stationary Gaussian processes by explicitly modifying the correlation structure of the data. Through a scaling function, variably scaled kernels alter the correlations between data and enable the modeling of targets exhibiting abrupt changes or discontinuities. We analyse the resulting predictive uncertainty via the variably scaled kernel power function and clarify the relationship between variably scaled kernels-based constructions and classical non-stationary kernels. Numerical experiments demonstrate that variably scaled kernels-based Gaussian processes yield improved reconstruction accuracy and provide uncertainty estimates that reflect the underlying structure of the data
Read the full abstract ▾
All authors Gianluca Audone, Francesco Marchetti, Emma Perracchione, Milvia Rossini
Statistical Computation
Fast multitask Gaussian processes, with application to surrogate modeling of the quark-gluon plasma
Aleksei G. Sorokin, Pieterjan Robbe, Yen-Chun Liu et al. (+2 more)
Gaussian processes (GPs) are broadly used for the surrogate modeling of computer experiments with reliable uncertainty quantification. Our motivating application comes from the study of the
quark-
gluon plasma (QGP), an extreme state of
nuclear matter that filled the universe shortly after the Big Bang. To reliably infer properties of the QGP, multiple surrogate models need to be constructed for related particle collision simulation systems (i.e., multiple "tasks"). While there is a body of literature on multitask GPs (MTGPs), such models can be computationally expensive with large
datasets: they require $\mathcal{O}(N^3)$ work and $\mathcal{O}(N^2)$ storage for model training, where $N$ is the total number of samples over all tasks. To address this, we propose a new fast MTGP approach, which pairs structured low-discrepancy design points, such as Sobol' points, with special kernel forms for efficient and exact model fitting. This pairing of a kernel and potentially different design points of different sizes across tasks provides a structured Gram
matrix, e.g., a circulant block
matrix, which we exploit via a novel
algorithm for efficient Gram
matrix inversion and determinant computation. Our
algorithm reduces training costs to $\mathcal{O}(N \log N)$ work and $\mathcal{O}(N)$ storage in the case of equal sample sizes for each tasks. In the worst case of severely unbalanced sample sizes across tasks, our
algorithm may require up to $\mathcal{O}(N^2)$ work and storage, with continuous interpolation between these best and worst cases depending on the balance of sample sizes across tasks. An open-source Python implementation is made available in the FastGPs package (https://alegresor.github.io/fastgps/). We demonstrate the effectiveness of our fast MTGP approach on a range of simulation experiments and on our motivating QGP application.
Read the full abstract ▾
All authors Aleksei G. Sorokin, Pieterjan Robbe, Yen-Chun Liu, Simon Mak, Fred J. Hickernell
Statistics Theory
Breakdown properties of optimal transport maps: general transportation costs
Alberto Gonzalez-Sanz and Marco Avella Medina
Two recent works, Avella-Medina and González-Sanz (2026) and Passeggeri and Paindaveine (2026), studied the robustness of the optimal transport map through its breakdown point, i.e., the smallest fraction of contamination that can make the map take arbitrarily aberrant values. Their main finding is the following: let $P$ and $Q$ denote the target and reference measures, respectively, and let $T$ be the optimal transport map for the squared Euclidean cost. Then, the breakdown point of $T(u)$, when $P$ is perturbed and $Q$ is fixed, coincides with the Tukey depth of $u$ relative to $Q$. In this note, we extend this result to general convex cost functions, demonstrating that the cost function does not have any impact on the breakdown point of the optimal transport map. Our contribution provides a definitive characterization of the breakdown point of the optimal transport map. In particular, it shows that for a broad class of regular cost functions, all transport-based
quantiles enjoy the same high breakdown point properties.
Read the full abstract ▾
Statistics Applications
Prior-Data Fitted Networks for Causal Inference: a Simulation Study with Real-World Scenarios
Francisco Mourao, David Hajage, Daria Bystrova et al. (+4 more)
Prior-Data Fitted Networks (PFNs) represent a paradigm shift in tabular data prediction. We present the principles of this new paradigm and evaluate two PFNs for estimating the average treatment effect (ATE) of a binary treatment on a binary outcome, using simulated clinical scenarios based on real-world data. We assessed TabPFN combined with causal inference procedures (g-computation and inverse probability of treatment weighting), and CausalPFN, a PFN that directly provides an ATE estimate with a credible interval. Confidence intervals for the TabPFN-based methods were derived using bootstrap resampling. We found that computation times for TabPFN were prohibitive for routine causal inference, particularly because of the need for bootstrapping to yield confidence intervals. Moreover, g-computation with TabPFN produced a highly biased
estimator, partially corrected by fitting separate models for each treatment group (T-learner). CausalPFN, by contrast, was computationally efficient but exhibited poor coverage of its 95% credible interval for the ATE, due to both estimation bias and inadequate uncertainty quantification. Beyond automating model specification, some PFN variants - like CausalPFN - attempt to automate causal modeling. In the settings we evaluated, CausalPFN performed poorly. However, new
algorithms of this kind continue to be developed, and their application to causal inference tasks requires further investigation.
Read the full abstract ▾
All authors Francisco Mourao, David Hajage, Daria Bystrova, Bertrand Bouvarel, Nathanaël Lapidus, Fabrice Carrat, Benjamin Glemain
Statistical Methodology
Time Partitioning in Target Trial Emulation
Harold Tankpinou Zoumenou, Simon Ferreira, Charles Assaad et al. (+3 more)
In target trial emulation, time partitioning enables researchers to handle time-varying confounders and immortal time bias with appropriate methods. Based on two clinical scenarios, this study aimed to explore issues related to time partitioning and to provide guidance for trial emulation. After formalizing the research question within the framework of structural causal models, we show how a given time partitioning may be too fine or too coarse depending on the clinical context. When the partitioning is too fine, the dimensionality of the model is unnecessarily high. When the partitioning is too coarse, the resulting causal structure may hinder effect estimation. We also show that cloning-censoring-weighting may not be valid when treatment influences outcome within study periods, and we confirm this through simulations. In conclusion, we provide practical guidance for actively specifying an appropriate time partitioning in trial emulation, rather than using the available data resolution as a default.
Read the full abstract ▾
All authors Harold Tankpinou Zoumenou, Simon Ferreira, Charles Assaad, Nathanael Lapidus, Daria Bystrova, Benjamin Glemain
Machine Learning (Statistics)
Nuri Mert Vural, Alberto Bietti, Mahdi Soltanolkotabi et al. (+1 more)
Modern large
language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering.
Transformers are central to this capability because they can encode information during training and retrieve it at inference. Existing theoretical analyses typically study
transformers under idealized assumptions such as infinite data or orthogonal
embeddings. In realistic settings, however, models are trained on finite
datasets with non-orthogonal (random)
embeddings. We address this gap by analyzing a single-layer
transformer with random
embeddings trained with (empirical) gradient descent on a simple
token-
retrieval task, where the model must identify an informative
token within a length-$L$ sequence and learn a one-to-one mapping from
tokens to labels. Our analysis tracks the “early phase” of gradient descent and yields explicit formulas for the model's storage capacity – revealing a multiplicative dependence between sample size $N$,
embedding dimension $d$, and sequence length $L$. We validate these scalings numerically and further complement them with a lower bound for the underlying statistical problem, demonstrating that this multiplicative scaling is intrinsic under non-orthogonal
embeddings.
Read the full abstract ▾
All authors Nuri Mert Vural, Alberto Bietti, Mahdi Soltanolkotabi, Denny Wu
Machine Learning (Statistics) · Cross-listed from cs.CE
Data-efficient Bayesian-guided design selection from large candidate sets: Application to hyperelastic stochastic metamaterials
Hooman Danesh and Henning Wessels
From a pool of admissible designs, we aim to identify a structure that achieves a target macroscopic stress response. For each candidate, the response is obtained from a high-fidelity oracle, such as expensive computational homogenization or experiments. We consider cases in which (i) the geometry cannot be conveniently parameterized, rendering gradient-based optimization inapplicable, and (ii) brute-force evaluation of all candidates is infeasible due to costly oracle queries. To tackle this challenge, we propose a
Bayesian-guided design selection framework. The dimensionality of design variants is reduced through statistical feature engineering, and the resulting low-dimensional descriptors are mapped to effective hyperelastic constitutive parameters using a multi-output Gaussian process surrogate. The surrogate is trained using uncertainty-driven active learning with only a limited number of high-fidelity oracle evaluations. The surrogate shortlists promising candidates, and since its accuracy is inherently limited, the final selection of the optimal design is performed through high-fidelity oracle evaluations within the shortlist. In numerical test cases, we consider a design set of 50,000 candidate structures. Active learning requires labeling less than half a percent of the entire candidate set.
Bayesian-guided design selection reaches a prescribed error threshold with only a handful of oracle evaluations in most cases.
Read the full abstract ▾
Statistical Computation
SEMMS with Random Effects: A Mixed-Model Extension for Variable Selection in Clustered and Longitudinal Data
Haim Bar and Martin T. Wells
SEMMS (Scalable Empirical-Bayes Model for Marker Selection) is a variable-selection procedure for generalized linear models that uses a three-component normal mixture prior on regression coefficients. In its original form, SEMMS assumes that all observations are independent. Many real-world
datasets, however, arise from repeated-measures or clustered designs in which observations within the same subject are correlated. Ignoring this correlation inflates the apparent residual variance and can severely degrade variable-selection performance. We extend SEMMS to accommodate random intercepts, random slopes, or both, via an alternating coordinate-ascent
algorithm. After each round of fixed-effect variable selection, the subject-level best linear unbiased predictors (BLUPs) are updated with lmer (Gaussian) or glmer (non-Gaussian); the fixed-effect step then operates on the random-effect-adjusted response. We describe the
algorithm, evaluate its performance in three Gaussian simulation studies spanning a range of signal strengths, random-effect magnitudes, and sample/predictor-space regimes, and present a semi-synthetic real-data example. We further extend the framework to non-Gaussian families (Poisson, binomial) via an IRLS working-response adaptation: at each outer iteration the fixed-effects step uses the RE-adjusted working response computed from the current glmer fitted values rather than the raw response. When the fixed-effect signal is strong relative to the random-effect variance, both the original and extended procedures perform comparably. When the random-effect variance dominates – the scenario most likely to cause plain SEMMS to fail – the mixed-model extension recovers the exact true predictor set in 93% of simulated
datasets (Gaussian), 61% (Poisson), and 65% (binomial), compared with 1%, 45%, and 39% for plain SEMMS respectively.
Read the full abstract ▾
Statistics Applications
A Utility Score Framework for Dose Optimization Studies with Binary Efficacy-Safety Endpoints: Sample Size Determination and Bias Characterization
Xuemin Gu, Cong Xu, Lei Xu et al. (+1 more)
The FDA's Project Optimus initiative emphasizes patient-centered dose selection in oncology that balances efficacy and safety. We develop a framework for randomized dose optimization studies that uses clinically interpretable utility scores to integrate binary efficacy and safety endpoints and select the optimal dose for a follow-on confirmatory trial. The framework provides: (i) a systematic method for eliciting utility scores that reflect clinical priorities; (ii) closed-form sample size formulas to achieve prespecified Probabilities of Correct Selection (PCS) under clinically relevant scenarios; and (iii) analytical expressions characterizing the propagation of selection-induced bias to confirmatory trials, including time-to-event endpoints correlated with the selection endpoint. Extensive simulations (10^6 replications per scenario) confirm that the sample size methods achieve target PCS and that the bias and Type I error formulas closely match empirical estimates. An R package DoseOptDesign and an interactive Shiny application are publicly available.
Read the full abstract ▾
All authors Xuemin Gu, Cong Xu, Lei Xu, Ying Yuan
Statistical Methodology
Besag-Clifford e-values for unnormalized testing
Alexander Dombowsky, Barbara E. Engelhardt and Aaditya Ramdas
Unnormalized probability distributions are frequently used in machine learning for modeling complex data generating processes. Though Markov chain Monte Carlo (MCMC)
algorithms can approximately sample from unnormalized distributions, intractability of their normalizing constants renders likelihood ratio testing infeasible. We propose to use the parallel method of Besag and Clifford to generate samples that are exchangeable with the data under the null, to then generate valid e-values for any number of iterations or algorithmic steps. We show that as the number of samples grows, these Besag-Clifford e-values constructed using the unnormalized likelihood ratio are actually log-optimal up to a multiplicative term that diminishes with the mixing time of the Markov chain. Additionally, averaging over the output of multiple chains retains validity while increasing the e-power. We extend Besag-Clifford e-values to the general problem of unnormalized test statistics, which allows application to composite hypotheses, uncertainty quantification, generative model evaluation, and sequential testing. Through simulations and an application to
galaxy velocity modeling, we empirically verify our theory, explore the impact of autocorrelation and mixing, and evaluate the performance of Besag-Clifford e-values.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
When Stability Fails: Hidden Failure Modes Of LLMS in Data-Constrained Scientific Decision-Making
Nazia Riasat
Large
language models (LLMs) are increasingly used as decision-support tools in data-constrained scientific workflows, where correctness and validity are critical. However, evaluation practices often emphasize stability or reproducibility across repeated runs. While these properties are desirable, stability alone does not guar- antee agreement with statistical ground truth when such references are available. We introduce a controlled behavioral evaluation framework that explicitly sep- arates four dimensions of LLM decision-making: stability, correctness,
prompt sensitivity, and output validity under fixed statistical inputs. We evaluate multi- ple LLMs using a statistical gene prioritization task derived from differential ex- pression analysis across
prompt regimes involving strict and relaxed significance thresholds, borderline ranking scenarios, and minor wording variations. Our ex- periments show that LLMs can exhibit near-perfect run-to-run stability while sys- tematically diverging from statistical ground truth, over-selecting under relaxed thresholds, responding sharply to minor
prompt wording changes, or producing syntactically plausible gene identifiers absent from the input table. Although sta- bility reflects robustness across repeated runs, it does not guarantee agreement with statistical ground truth in structured scientific decision tasks. These findings highlight the importance of explicit ground-truth validation and output validity checks when deploying LLMs in automated or semi-automated scientific work- flows.
Read the full abstract ▾
Statistics Applications
A Portfolio-Anchored Frequency-Severity Risk Index for Trip and Driver Assessment Using Telematics Signals
Jongtaek Lee, Andrei Badescu and X. Sheldon Lin
In this paper, we propose a novel frequency-severity joint trip-level risk index that combines the frequency of abnormal driving patterns with a severity component reflecting how extreme such behavior is relative to a portfolio-level baseline. Severity is quantified through an inverse-probability penalty that increases with the rarity of observed tail extremes, rather than being interpreted as a claim size. Based on high-frequency telematics data, we construct a multi-scale representation of longitudinal acceleration using the maximal overlap discrete wavelet transform (MODWT), which preserves localized driving patterns across multiple time scales. To capture severity as tail rarity, we model the portfolio distribution using a Gaussian-Uniform mixture with a layered tail structure, where Gaussian components describe typical driving behavior and the tail is partitioned into ordered severity layers that reflect increasing extremeness. We develop a likelihood-based estimation procedure that makes inference feasible for this mixture model. The resulting severity layers are then used to construct multi-layer tail counts (MLTC) at the trip level, which are modeled within a Poisson-Gamma framework to yield a closed-form posterior risk index that jointly reflects frequency and severity. This conjugate structure naturally supports sequential updating, enabling the construction of dynamically evolving driver-level risk profiles. Using the UAH-DriveSet controlled
dataset, we demonstrate that the proposed index enables reliable discrimination across behavioral driving states, identification of high-risk trips, and coherent ranking of drivers, yielding a purely behavior-driven risk measure suitable for actuarial ratemaking and potentially mitigating
fairness concerns associated with traditional
covariates.
Read the full abstract ▾
Statistical Methodology
On the Equivalence between Neyman Orthogonality and Pathwise Differentiability
Yuxi Chen, Edward H. Kennedy and Sivaraman Balakrishnan
It has been frequently observed that Neyman orthogonality, the central device underlying double/debiased machine learning (Chernozhukov et al., 2018), and pathwise differentiability, a cornerstone concept from semiparametric theory, often lead to the same debiased
estimators in practice. Despite the widespread adoption of both ideas, the precise nature of this equivalence has remained elusive, with the two concepts having been developed in largely separate traditions. In this work, we revisit the semiparametric framework of van der Laan and Robins (2003) and identify an implicit regularity assumption on the relationship between target and nuisance parameters – a local product structure – that allows us to establish a formal equivalence between Neyman orthogonality and pathwise differentiability. We also show that the two directions of this equivalence impose fundamentally different structural requirements. Finally, we illustrate the theory through three detailed examples of estimating the average treatment effect and expected density in a nonparametric model, as well as the slope in a partially linear model. This helps clarify the relationship between these two foundational frameworks and provides a useful reference for practitioners working at their intersection.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
Longitudinal Risk Prediction in Mammography with Privileged History Distillation
Banafsheh Karimian, Soufiane Belharbi, Alexis Guichemerre et al. (+3 more)
Longitudinal mammography screening has become an important source of information for improving future breast cancer risk prediction. However, the performance of current longitudinal mammography models degrades when prior examinations are unavailable at inference, creating a structured privileged-information setting in which temporal context is available during training but absent at deployment. We propose Single-Exam Mammography risk prediction with privileged History
Distillation (SEM-HD), a framework that uses longitudinal history as privileged information available only during training to preserve the predictive benefits of longitudinal modeling while requiring only the current screening examination at deployment. During training, the student relies on the current examination to predict latent representations of prior visits, while horizon-specific teachers provide additional supervision from the observed longitudinal history. Together, latent history prediction and teacher
distillation preserve the temporal modeling structure of longitudinal predictors under current-exam-only inference. We validate SEM-HD on three longitudinal mammography cohorts, the CSAW-CC, EMBED, and OMI-DB, using the
transformer-based Longitudinal Mammography Risk (LoMaR) and recurrent Visual
Memory Recurrent
Attention (VMRA) backbones. Under current-exam-only inference, SEM-HD consistently improves long-horizon AUC and pAUC over longitudinal models evaluated without history, particularly in the clinically relevant low false-positive-rate region. It also recovers much of the performance gap with respect to full-history inference across
datasets and backbones. Ablations further show that these gains are not reproduced by masking or heuristic history imputation. The strongest performance is achieved by combining patient-specific latent history prediction with distilled temporal risk supervision.
Read the full abstract ▾
All authors Banafsheh Karimian, Soufiane Belharbi, Alexis Guichemerre, Luke McCaffrey, Mohammadhadi Shateri, Eric Granger
Statistics Theory · Cross-listed from math.PR
On the Uniqueness of Fréchet Means for Polytope Norms
Roan Talbut, Andrew McCormack and Anthea Monod
Fréchet means are a popular type of average for non-Euclidean
datasets, defined as those points which minimise the average squared distance to a set of data points. We consider the behaviour of sample Fréchet means on normed spaces whose unit ball is a polytope; this setting is rarely covered by existing literature on Fréchet means, which focuses on smooth spaces or spaces with bounded
curvature. We study the geometry of the set of Fréchet means over polytope normed spaces, with a focus on dimension and probabilistic conditions for uniqueness. In particular, we provide a
geometric characterisation of the threshold sample size at which Fréchet means have a positive probability of being unique, and we prove that this threshold is at most one more than the dimension of our space. We are able to use this
geometric characterisation to compute the unique Fréchet mean sample threshold in the case of the $\ell_\infty$ and $\ell_1$ norms.
Read the full abstract ▾
Machine Learning (Statistics)
Learnability with Partial Labels and Adaptive Nearest Neighbors
Nicolas A. Errandonea, Santiago Mazuelas, Jose A. Lozano et al. (+1 more)
Prior work on partial labels learning (PLL) has shown that learning is possible even when each instance is associated with a bag of labels, rather than a single accurate but costly label. However, the necessary conditions for learning with partial labels remain unclear, and existing PLL methods are effective only in specific scenarios. In this work, we mathematically characterize the settings in which PLL is feasible. In addition, we present PL A-$k$NN, an adaptive nearest-neighbors
algorithm for PLL that is effective in general scenarios and enjoys strong performance guarantees. Experimental results corroborate that PL A-$k$NN can outperform state-of-the-art methods in general PLL scenarios.
Read the full abstract ▾
All authors Nicolas A. Errandonea, Santiago Mazuelas, Jose A. Lozano, Sanjoy Dasgupta
Statistics Applications
Differential Privacy for Network Connectedness Indices
Tom A. Rutter, Yuxin Liu and M. Amin Rahimian
Researchers increasingly use data on social and economic networks to study a range of social science questions, but releasing statistics derived from networks can raise significant
privacy concerns. We show how to release network connectedness indices that quantify assortative mixing across node attributes under edge-adjacent differential
privacy. Standard
privacy techniques perform poorly in this setting both because connectedness indices have high global sensitivity and because a single node's attribute can potentially be an input to connectedness in thousands of
cells, leading to poor composition. Our method, which is straightforward to apply, first adds noise to node attributes, then analytically debiases downstream statistics, and finally applies a second layer of noise to protect the presence or absence of individual edges. We prove consistency and asymptotic normality of our
estimators for both discrete and continuous labels and show our method works well in simulations and on real networks with as few as 200 nodes collected by social scientists.
Read the full abstract ▾
Statistical Methodology
Low-Complexity and Consistent Graphon Estimation from Multiple Networks
Roland Boniface Sogan and Tabea Rebafka
Recovering the random graph model from an observed collection of networks is known to present significant challenges in the setting, where the networks do not share a common node set and have different sizes. More specifically, the goal is the estimation of the graphon function that parametrizes the nonparametric exchangeable random graph model. Existing methods typically suffer from either limited accuracy or high computational complexity. We introduce a new histogram-based
estimator with low algorithmic complexity that achieves high accuracy by jointly aligning the nodes of all graphs, in contrast to most conventional methods that order nodes graph by graph. Consistency results of the proposed graphon
estimator are established. A numerical study shows that the proposed
estimator outperforms existing methods in terms of accuracy, especially when the
dataset comprises only small and variable-size networks. Moreover, the computing time of the new method is considerably shorter than that of other consistent methodologies. Additionally, when applied to a graph neural network classification task, the proposed
estimator enables more effective data augmentation, yielding improved performance across diverse real-world
datasets.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Unbiased and Biased Variance-Reduced Forward-Reflected-Backward Splitting Methods for Stochastic Composite Inclusions
Quoc Tran-Dinh and Nghia Nguyen-Trung
This paper develops new variance-reduction techniques for the forward-reflected-backward splitting (FRBS) method to solve a class of possibly nonmonotone
stochastic composite inclusions. Unlike unbiased
estimators such as mini-batching, developing
stochastic biased variants faces a fundamental technical challenge and has not been utilized before for inclusions and fixed-point problems. We fill this gap by designing a new framework that can handle both unbiased and biased
estimators. Our main idea is to construct
stochastic variance-reduced
estimators for the forward-reflected direction and use them to perform iterate updates. First, we propose a class of unbiased variance-reduced
estimators and show that increasing mini-batch SGD, loopless-SVRG, and SAGA
estimators fall within this class. For these unbiased
estimators, we establish a $\mathcal{O}(1/k)$ best-iterate convergence rate for the expected squared residual norm, together with almost-sure convergence of the iterate sequence to a solution. Consequently, we prove that the best oracle complexities for the $n$-finite-sum and expectation settings are $\mathcal{O}(n^{2/3}\epsilon^{-2})$ and $\mathcal{O}(\epsilon^{-10/3})$, respectively, when employing loopless-SVRG or SAGA, where $\epsilon$ is a desired accuracy. Second, we introduce a new class of biased variance-reduced
estimators for the forward-reflected direction, which includes SARAH, Hybrid SGD, and Hybrid SVRG as special instances. While the convergence rates remain valid for these biased
estimators, the resulting oracle complexities are $\mathcal{O}(n^{3/4}\epsilon^{-2})$ and $\mathcal{O}(\epsilon^{-5})$ for the $n$-finite-sum and expectation settings, respectively. Finally, we conduct two numerical experiments on AUC optimization for imbalanced classification and policy evaluation in reinforcement learning.
Read the full abstract ▾