Machine Learning (Statistics)
The Role of Uncertainty in Assessing the Fairness of Machine Learning Models
Francesca Panero, Ernst C. Wit and Marco Scutari
Machine learning models are widely used in clinical applications, social media, law enforcement and critical infrastructure. Verifying whether their outputs are biased against disadvantaged groups or individuals is crucial to ensuring they are fair and allowing their use in such settings. A rigorous risk assessment of possible
fairness violations requires quantifying the uncertainty associated with selecting and estimating such models. Yet, this is rarely done in the literature, which focuses on identifying a single model with a suitable trade-off between predictive accuracy and
fairness. In this paper, we move beyond point estimation and discuss frequentist and
Bayesian approaches to uncertainty quantification for fair machine learning, with practical examples and implications for simulated and real data.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
Zihao Yang, Zijia Wang and Zhiqiu Huang
Multimodal large
language models often capture visual-linguistic correlations but struggle to predict how local visual interventions propagate and affect downstream answers. We introduce InfluenceField, an intervention-aware latent field inserted between the visual encoder and
language decoder. It lifts patch features into a continuous spatial representation, propagates directed influence over multiple steps, and predicts local intervention effects through a shared transition operator. Training jointly optimizes
language modeling, cross-environment invariance, counterfactual rollout supervision, and structural regularization. For a nonlinear finite-basis population model, we show that target-aligned interventional supervision, together with a one-step separation condition on the transition, restricts admissible representations to within-location reparameterizations, so that the directed dependency graph of the full transition is recovered exactly. A linear specialization gives an exact partial-coverage characterization and a finite-loss stability bound, and the field analysis derives the spatial profile of coefficient interventions together with a shared-channel calibration result. On CausalVQA, InfluenceField improves overall accuracy over its backbone by 13.1 percentage points, with the largest gains on the planning and hypothetical categories. Capacity-matched baselines and structural controls attribute the gains in robustness and factual-counterfactual consistency to the causal objectives rather than to added capacity.
Read the full abstract ▾
Machine Learning (Statistics)
SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws
Konstantinos Christopher Tsiolis, Denny Wu, Christos Thrampoulidis et al. (+1 more)
We study the training dynamics of multiclass logistic regression on high-dimensional Gaussian mixture models with a large number of classes and establish precise scaling laws governing the cross-
entropy risk under gradient-based optimization. We show that learning proceeds sequentially across classes, from most to least frequent. When the class priors follow a power law distribution, the risk dynamics decompose into three phases: an initial plateau until the first class is learned, a power-law decay regime during which sequential learning occurs, and a final convergence regime. We then analyze how model capacity interacts with optimization under a fixed compute budget. When the effective dimension is restricted via projection onto leading principal components, the risk decomposes into a capacity term (a power law in the retained dimension) and an optimization term (a power law in training time). Optimizing this tradeoff yields a compute-optimal scaling law for logistic regression, with explicit prescriptions for model size and training time as functions of compute. These results extend theoretical scaling laws from linear regression to multiclass classification, while connecting to empirical scaling laws observed in large-scale neural networks.
Read the full abstract ▾
All authors Konstantinos Christopher Tsiolis, Denny Wu, Christos Thrampoulidis, Murat A. Erdogdu
Statistics Theory
Theoretical Foundations of Ordinal Spherical Multidimensional Scaling
Elizabeth Coda and Ery Arias-Castro
There has been general interest in spherically constrained
embeddings as data with an inherently circular or spherical structure arise in a number of applications. While many methods have been proposed on the metric side, little work has been done on the ordinal side in terms of methodology or theory. Here, we focus on the fundamental question of uniqueness of ordinal spherical MDS: Given a realizable setting in which underlying objects lie on the unit sphere and all that is known are their ordinal comparisons of the form `object $i$ is more similar to object $j$ than object $k$', is it possible to uniquely recover the original objects, up to an orthogonal transformation, in the large-sample limit? We answer affirmatively, both in the setting just described, as well as in the settings of spherical external unfolding (aka lateration) and spherical internal unfolding (aka preference mapping) in their ordinal variants.
Read the full abstract ▾
Statistics Applications · Cross-listed from astro-ph.IM
Neural Posterior Estimation for Tomographic Weak Lensing Mass Mapping
Tim White, Shreyas Chandrashekaran, Camille Avestruz et al. (+2 more)
Weak gravitational
lensing shear and convergence trace the distribution of baryonic and dark matter across space, making them a powerful probe of
cosmic structure. Inferring shear and convergence from images is a challenging inverse problem. The prevailing approach to this task estimates shear from weighted averages of
galaxy ellipticities, calibrates these estimates to account for systematic biases, and transforms them to reconstruct convergence, a multistage procedure that requires substantial computational resources and meticulous handling of statistical uncertainties. As an alternative, we propose a probabilistic approach to field-level weak
lensing inference in which we train a deep neural network to directly map a multiband image to a variational distribution over the underlying tomographic shear and convergence fields. This neural posterior estimation (NPE) procedure implicitly marginalizes over nuisance variables in the cosmological forward model and does not require evaluating the likelihood function. It is also amortized, so it enables rapid posterior inference for astronomical surveys once the neural network is trained. When evaluated on synthetic images from the LSST-DESC DC2 Simulated Sky Survey, NPE produces well-calibrated variational distributions for shear and convergence that are consistent with the ground truth. We describe how maps sampled from these variational distributions could be used in a subsequent simulation-based inference procedure to approximate the posterior distribution over cosmological parameters.
Read the full abstract ▾
All authors Tim White, Shreyas Chandrashekaran, Camille Avestruz, Jeffrey Regier, the LSST Dark Energy Science Collaboration
Statistics Theory · Cross-listed from math.PR
Markov Chain CLTs: Resolving Open Problems
Austin Brown, Jeffrey S. Rosenthal and Quan Zhou
Markov chain central limit
theorems (CLTs) and their associated variances are very important for implementing Markov chain Monte Carlo
algorithms among other applications. Häggström and Rosenthal (2007) presented various results regarding the equality of different formulae for this variance and also posed seven open problems. We resolve all seven in this paper. For stationary, ergodic and reversible chains, we prove that whenever the normalized partial sums satisfy a $\sqrt{n}$-CLT, the variance limit is finite if the function is square-integrable, otherwise undefined. Moreover, failure of the $\sqrt{n}$-CLT forces the normalized partial sums to be non-tight. We also show that Roberts' holding-probability condition precludes a CLT even without assuming reversibility or square-integrability. Finally, we develop a general principle that expresses
Fourier coefficients as the autocovariances of an ergodic nonreversible Markov chain.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
Non-asymptotic bounds for the average singular value of a complex Gaussian matrix
Luis Daniel Abreu and Pratik Patil
Let $G_{d}$ be a $d \times d$
matrix with independent standard complex Gaussian entries, let $\alpha_{\mathbb{C}}(d)$ be the expected average singular value of $G_{d}/\sqrt{d}$, and set $\Delta_d := \alpha_{\mathbb{C}}(d)-\alpha_{\mathbb{C}}(d+1)$. The statistic $\alpha_{\mathbb{C}}(d)$ admits a variational representation as an expected normalized maximum over the
unitary group and governs approximation guarantees for the little Grothendieck problem over the
unitary group and related
unitary registration problems. We obtain a strictly positive lower bound and an upper bound for $\Delta_d$, both valid in every dimension, together with corresponding bounds for $\alpha_{\mathbb{C}}(d)$ around the Marchenko–Pastur limit. These bounds match the sharp leading behavior of complete asymptotic expansions for both quantities, whose coefficients are explicitly computable. The proof combines a three-term recurrence for $Y_d = d^{3/2}\alpha_{\mathbb{C}}(d)$, obtained from its continuous dual Hahn representation, along with singularity analysis of the underlying Laguerre moment generating function.
Read the full abstract ▾
Statistics Applications
Two-resolution state modelling of intermittent online gambling activity during Sweden's temporary deposit-cap period
Sam Andersson, Timo Koski, Helga Westerlind et al. (+3 more)
Intermittent online gambling records create two distinct representation problems: calendar-time analyses must retain represented days with no recorded play, whereas analyses of active behaviour must preserve variation within active episodes. We pair calendar-time and active-bout models in a two-resolution analysis of 14.2 million player-days from a single operator during 2019-2023. The calendar-time layer retains assignable represented days in five observed categories and propagates a pre-period reference. The active-episode layer fits a
Bayesian mixed-emission hidden semi-Markov model to turnover, playing time, approved deposits and session counts. In the reference four-regime fit, two regimes share a rounded model-implied turnover median of 153.5 EUR, while their model-implied playing-time and approved-deposit medians differ by factors of 4.6 and 5.7, respectively, and their expected total session counts by a factor of 2.8. This separation remains under an ordinary hidden Markov model, two routing perturbations and a residual-dependence sensitivity; the tested turnover-only fits were diagnostically unstable. Over the temporary Swedish deposit-cap period, deposits and turnover per represented player-day were roughly 55% below the propagated reference and active-category occupancy was 7.8% lower. The largest negative deviations in a prespecified Sweden-assigned cohort occurred among high pre-policy deposit-intensity customers. These are operator-specific descriptive comparisons, not causal policy effects.
Read the full abstract ▾
All authors Sam Andersson, Timo Koski, Helga Westerlind, Keenan Lyon, Per Carlbring, Olof Molander
Statistics Theory
Geometric Fluctuations of the $\sin\Theta$ Distance in High-Dimensional Principal Subspace Estimation
Yanlin Hu, Xiao Han and Qing Yang
We investigate the
geometric fluctuations of principal subspaces for high-dimensional covariance
matrices through the squared Frobenius $\sin\Theta$ distance between the sample and population eigenspaces associated with the $r_p$ largest
eigenvalues. An explicit first-order expansion and a central limit
theorem are established for this subspace distance. The theory allows the subspace dimension to diverge subject to $r_p=o(n)$, where $n$ is the sample size. It also permits a diverging spectral norm of the population covariance
matrix, population spikes of different orders, and repeated or closely spaced spikes. This sharp characterisation captures features of the subspace estimation error that are not reflected in existing perturbation bounds. As applications, we derive an explicit asymptotic expansion for the expected PCA excess risk and a refined error bound for distributed PCA. In both cases, existing upper bounds can increase with the spiked-block condition number when some leading spikes become stronger, whereas our results show that the corresponding estimation errors need not increase and may instead decrease. Numerical experiments reproduce this contrasting behaviour and demonstrate the finite-sample accuracy of our theoretical findings.
Read the full abstract ▾
Statistical Methodology
Retrospective Causal Attribution under Case-Control Sampling
Zijian Sui, Hong Zhang, Jinfeng Xu et al. (+1 more)
The probability of necessity PN quantifies the probability that an exposed individual who experienced an outcome would not have experienced it in the absence of exposure. Case-control studies are an important resource for investigating etiologic questions, but their sampling design can introduce selection bias and complicate the statistical inference for PN.In this paper, we develop a nonparametric framework for identification and efficient estimation of PN under case-control sampling. With an externally supplied population outcome prevalence, we derive an exact identification formula that identifies PN under standard causal assumptions and monotonicity and yields a valid lower bound without monotonicity. For rare outcomes, we derive a more tractable approximation that requires no external prevalence information and prove that its approximation error vanishes at the order of the population outcome prevalence. We further establish the semiparametric efficiency theory for the exact and approximate functionals, propose asymptotically efficient
estimators, and construct confidence intervals for the corresponding targets. The proposed approach has potential applications in biomedical and epidemiological studies where causal attribution is investigated using retrospective data.
Read the full abstract ▾
All authors Zijian Sui, Hong Zhang, Jinfeng Xu, Min Zeng
Statistics Theory
Statistical Inference for Persistence Diagrams via Landmark Embeddings: Minimax Theory and Finite Approximation
Pramita Bagchi, Sushovan Majhi, Atish Mitra et al. (+1 more)
Hilbert-space
embeddings enable inference for populations of persistence diagrams, but separation between individual diagrams need not survive population averaging. We develop a framework for inference on population mean
embeddings, with particular
attention to the additive landmark representations PLACE and PALACE. Treating each diagram as one independent observation, we apply
Hilbert-space limit theory to obtain covariance
estimators, two-sample tests, and confidence balls under suitable moment conditions, without requiring a lower-distortion bound. For additive
embeddings, we identify the population mean as an
embedding of the mean counting measure and show that
geometric separation of these measures alone cannot guarantee uniform testing power. We then introduce a model with latent template diagrams, missing features, and location perturbations. Under common or feature-specific prevalence conditions, a diagram-level lower-distortion certificate yields explicit lower bounds on population mean separation. These margins provide finite-sample uniform power guarantees, and an additional information-divergence comparison gives matching sample-complexity bounds over restricted scale ranges. Confidence sets yield lower bounds on transport separation of population mean measures and exclusion guarantees for specified structured alternatives. We also quantify how orthogonal truncation changes the certified signal and the approximation allowance needed for confidence sets targeting the full
embedding, relating sample size, retained coordinates, and template separation. Simulations examine calibration, power, and coverage, and an analysis of resting-state connectivity from the Autism
Brain Imaging Data Exchange illustrates the procedures.
Read the full abstract ▾
All authors Pramita Bagchi, Sushovan Majhi, Atish Mitra, Žiga Virk
Statistical Methodology
Modeling Brain MRI Using Persistent Homology and Multilevel Functional Data Analysis
Shashipraba N. K. Rajakaruna, Asim K. Dey and A. Alexandre Trindade
Persistent homology provides a multiscale representation of biomedical images by capturing higher-order topological features that reflect their underlying structural organization. However, the resulting topological summaries are typically used as predictors or features for classification and group comparisons rather than treated as primary variables of interest. We construct a generalized multilevel functional framework for analyzing persistent-homology summaries as longitudinal functional responses in repeated three-dimensional structural
magnetic resonance imaging (MRI). Specifically, we represent topological features using Betti curves and model these curves as count-valued functional responses. A negative-binomial distribution accommodates the discrete and potentially overdispersed nature of Betti counts, while the multilevel formulation accounts for the dependence induced by repeated measurements and separates between-subject and within-subject sources of functional variation. Functional principal component analysis further evaluates the dominant modes of variation at each level. A
Bayesian approach is used for joint estimation of the functional regression and multilevel functional principal components. We apply this modeling framework to longitudinal structural MRI data from the OASIS-2 study to investigate associations between
brain topology and demographic and clinical characteristics, including age, gender, follow-up time, and dementia severity. The results demonstrate that the framework can capture
covariate-associated variation across the filtration continuum while evaluating distinct sources and patterns of longitudinal variation across homology dimensions.
Read the full abstract ▾
Statistical Computation
Thermodynamic Cyclic Processes with Markov Samplers in Bayesian Inference
Heinrich von Campe and Bjoern Malte Schaefer
The concept of Markov chain Monte Carlo (MCMC) cycles, an analogy to cyclic processes in heat engines, is presented in order to examine
Bayesian inference problems. In this effort, we develop adaptive ensemble schedulers that allow the tuning of external parameters of a
Bayesian canonical ensemble during an MCMC run, realising the MCMC cycles in practice. We run these cycles on different statistical models. As a fundamental insight, we find (both theoretically and in practice) that such systems can produce a non-zero net work output if and only if the considered model is non-Gaussian. As such, they may serve as a measure of non-Gaussianity in
Bayesian inference, which we test on an example from
supernova cosmology.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
Robust A/B Decisions
Max H. Farrell, Malika Korganbekova and Sanjog Misra
A/B tests are standard in firm decision making. In the standard pipeline, experimental data is converted to a deployment decision by applying a t-test of the difference in means (the lift) and deploying the treatment if lift is positive and statistically significant. This common workflow answers the wrong question. We argue that firms need a decision rule for economic payoffs in the future deployment environment, not a test of equality in the experimental sample. We develop an ambiguity-averse decision framework in which each arm is evaluated by its ambiguity-penalized value over distributions close to the experimental outcome distribution. The resulting rule has a simple closed form thanks to the Donsker-Varadhan representation and it requires only the outcome data from a standard A/B test plus one interpretable parameter governing trust in the experiment. Our rule is thus no more difficult to implement than a t-test. A mean-variance approximation shows how the rule penalizes variability, while a connection to utility maximization shows it to be a certainty equivalent. We are able to perform a real-world evaluation of our proposed rule in the context of digital marketing using an archive of 552 advertising experiments from an anonymous US-based online platform. The proposed rule substantially reduces regret relative to conventional hypothesis testing. The results show that economically conservative, distribution-aware deployment rules can outperform statistical-significance rules in digital experimentation.
Read the full abstract ▾
Statistics Applications
Structural Analysis of a Dynamic Multilayer Network via Matrix Autoregressive Models: A Case Study of International Interactions between Countries
Camila Pinzón, Mario Arrieta and Juan Sosa
Dynamic and multilayer networks have been widely studied separately, but their joint analysis remains comparatively underdeveloped. Because the relational information of a dynamic multilayer network can be represented as a tensor at each time point $t$, each layer can be summarized through a set of structural statistics, yielding a
matrix-valued observation and, consequently, a
matrix-valued time series. To exploit this structure, we propose the use of
matrix autoregressive (MAR) models, which simultaneously characterize temporal dependence across relational layers and structural statistics. We apply this framework to the ICEWS
dataset, which records international interactions among countries under four relational domains and therefore naturally defines a dynamic multilayer network. The results indicate that negative verbal interactions (Verbal-) play a prominent role in the subsequent structural reconfiguration of the material-interaction layers, while mean strength exhibits the strongest temporal persistence and reciprocity the broadest cross-statistic influence. These findings illustrate the usefulness of MAR models for providing a parsimonious and interpretable characterization of temporal and cross-layer dependence in dynamic multilayer networks.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
Translation of Black-Box Clinical Prediction Models into Standalone Transparent Nomograms: Temporal External Validation in Heart Transplantation
Henry Pigot, Paulo J. G. Lisboa, Sandra Ortega-Martorell et al. (+3 more)
We convert black-box clinical prediction models for tabular data into standalone nomograms that can be audited term by term. PRiSM (Partial Responses in Structured Models) takes the shape of each effect and interaction from the source model, not merely which variables mattered, and lets the outcome select and weight them. We tested this in 50,356 heart transplant recipients, with validation in a later era than training. Nomograms from all 5 source models - a public clinical risk score, logistic regression, neural networks, random forests and extreme gradient boosting - met a prespecified noninferiority criterion for discrimination before any further simplification, and generally preserved calibration and clinical net benefit. Those from the 3 machine-learning models showed no detectable difference in discrimination from de novo generalized additive and explainable boosting models, exceeded neural additive models, and carried fewer terms than the explainable boosting model. PRiSM is released as an open-source Python package.
Read the full abstract ▾
All authors Henry Pigot, Paulo J. G. Lisboa, Sandra Ortega-Martorell, Ivan Olier, Joseph Mahon, Johan Nilsson
Statistics Applications · Cross-listed from physics.chem-ph
Simulation-Supervised Foundation Models for Retention Time Prediction in High-Performance Liquid Chromatography beyond Experimental Data Coverage
Stephen Wu, Yufeng Han, Yasuhiro Mito et al. (+5 more)
Accurate prediction of high-performance
liquid chromatography (HPLC) retention times (RTs) across diverse molecules and chromatographic methods remains challenging because experimental training data cover only a limited region of chemical and method spaces. Here, we develop FUSE-RT (Foundation model Unifying Simulation and Experimental supervision for Retention Time), a multitask foundation model that integrates RT data from 179 chromatographic methods and adapts to unseen molecules and methods using limited target-domain data. To extend transferability beyond experimental coverage, we introduce simulation-to-real (Sim2Real) transfer learning, in which molecular representations learned from large-scale computational data are transferred to experimental RT prediction. Specifically, we use PolyOmics, comprising 39 properties for approximately 21,400 molecules generated by molecular dynamics and density-functional theory calculations, as auxiliary supervision. We evaluate generalization under molecular, method, and joint molecular–method distribution shifts. Simulation-derived supervision substantially improves transfer beyond the experimental molecular domain, particularly under pronounced coverage gaps and few-shot adaptation. Moreover, RT-prediction error decreases systematically with increasing simulation-data size, following a significant power-law relationship. These results establish Sim2Real transfer as a scalable strategy for extending RT prediction beyond the finite coverage of experimental chromatographic data.
Read the full abstract ▾
All authors Stephen Wu, Yufeng Han, Yasuhiro Mito, Yoshiyuki Watabe, Yoshihiro Hayashi, Hikaru Takaya, Takuya Kubo, Ryo Yoshida
Statistics Applications · Cross-listed from cs.LG
Statistical versus machine learning-based spatial interpolation of post-processed ensemble weather forecasts
Mária Lakatos
Statistical post-processing improves ensemble weather forecasts, but generating calibrated predictions at locations without observations remains challenging. This study compares statistical and machine-learning-based methods for post-processing ECMWF 2-m temperature and 10-m wind speed forecasts at observed and unobserved stations in Germany. We consider EMOS-based approaches, distributional regression networks,
Transformers, and graph neural networks under both limited and extended predictor settings. For temperature, we also investigate linear forecast combinations and propose an altitude-aware linear pool (ALP). The results show that post-processing improves upon the raw ensemble in most settings, but no single method performs best across all variables, station groups, and evaluation metrics. The proposed ALP provides a small but significant improvement over the standard linear pool at unobserved locations.
Read the full abstract ▾
Statistical Methodology
Quasi-randomization-based inference for multivariate Mann-Whitney effects under random missingness
Dennis Dobler, Jörg-Tobias Kuhn, Lubna Amro et al. (+1 more)
Marginal Mann-Whitney effects are widely used across various fields of research, and extensions of this estimand have been developed in many directions in statistical methodology. In this paper, we focus on an extensions for repeated measurements and factorial designs subject to randomly missing data. In a previous work by Rubarth et al. (2022a), asymptotically correct tests were developed under the assumption of deterministic missing indicators. In contrast, the approach in the present paper accounts for the
stochastic nature of missing values under realistic mechanisms. Thus, the involved covariance
matrix incorporates the true variability of missing data. The combination with a quasi-randomization procedure using random permutations within each data point yields asymptotically exact tests and a generally improved type-I error control. Additionally, the tests control the type-I error for finite sample sizes in the special case of exchangeable sampling distributions. Simulations across a wide range of settings demonstrate the benefits of the proposed method in small samples, also for different missingness mechanisms. A real data analysis about school children learning math illustrates several practical aspects of the tests' application.
Read the full abstract ▾
All authors Dennis Dobler, Jörg-Tobias Kuhn, Lubna Amro, Paavo Sattler
Statistical Methodology
Estimation and Inference for Latent Markov Models by Fourier Recursions
Yanqi Huang, Chenxu Li and Qiwei Yao
This paper proposes a new theoretically exact
Fourier recursion framework for a broad class of latent Markov models (LMMs), encompassing models widely used across a broad range of fields in economics. It can be viewed as a counterpart of the celebrated Kalman filter for non-Gaussian and nonlinear LMMs. Closed-form recursive updates of
Fourier coefficients jointly deliver filtering, likelihood evaluation, and simultaneously accumulate the score and Hessian online. We introduce a unified truncated implementation that ensures uniform error control and numerical stability, preventing approximation errors from accumulating through the recursion. We establish asymptotic properties of the feasible maximum likelihood
estimator for LMMs, provide a recursion-based consistent
estimator for the Fisher information and discuss the possible model misspecification. These results provide the first general asymptotic theory for feasible approximate maximum likelihood estimation in LMMs. Simulations demonstrate its accuracy and stability, while an application to U.S. bankruptcy data recovers a persistent latent bankruptcy-pressure process.
Read the full abstract ▾
Machine Learning (Statistics)
Distributed Lag Neural Additive Models
Calle Helmersson, Shivang Pandey, Leonardo Olivetti et al. (+1 more)
We introduce Distributed Lag Neural Additive Models (DLNAMs), neural-additive analogues of Distributed Lag Non-linear Models (DLNMs) for learning nonlinear effects distributed over lags. DLNAMs replace a prespecified spline cross-basis with neural components that learn exposure–lag response surfaces, avoiding choices of basis family, dimension, and
knot placement while preserving additive interpretability and familiar distributed-lag summaries. Exp-centered input layers, smooth activations, and learned subnetwork mixtures produce smooth, locally adaptive representations; pointwise uncertainty combines a conditional last-layer Laplace approximation with between-member ensemble variation. In simulations, DLNAMs generally outperformed DLNM comparators, including penalized and treed variants, in recovering known response functions, with lower bias, stronger boundary recovery, and better-calibrated cumulative intervals; gains were largest for more demanding functions. The architecture performed consistently across sample sizes, outcome families, lag horizons, and jointly fitted multi-exposure settings, retaining recovery performance as exposures were added; fit-specific changes were largely confined to optimization, and applications recovered established empirical patterns.
Read the full abstract ▾
All authors Calle Helmersson, Shivang Pandey, Leonardo Olivetti, Elena Raffetti
Statistics Applications
Bayesian Emulation of Multi-fidelity Earth System Modelling Using Hierarchical Gaussian Processes
Xiaoyu Xiong, Louise Kimpton, Huiyi Yang et al. (+3 more)
Multi-fidelity Earth system models provide simulations at different levels of complexity and computational cost, but exhaustive exploration of the parameter space at the highest fidelity is often prohibitively expensive. Multi-fidelity emulators can reduce this burden by combining abundant lower-fidelity simulations with limited high-fidelity evaluations. We compare four Gaussian-process-based multi-fidelity approaches: the Kennedy–O'Hagan autoregressive model (K&O), hierarchical kriging (HK),
Bayesian hierarchical emulation for multi-level models (BayHEm), and multi-fidelity deep Gaussian processes (MF-DGP). We evaluate the methods using two contrasting applications: a three-fidelity tsunami simulator and a two-fidelity implementation of the Joint UK Land Environment Simulator (JULES). Performance is assessed using leave-one-out predictive accuracy, uncertainty representation, design requirements, and computational characteristics. In the tsunami application, BayHEm gives the lowest normalised root mean square error (NRMSE = 0.031) and highest SCORE (3.025), while MF-DGP performs worse than the single-fidelity baseline when only 10 high-fidelity simulations are available. In the JULES application, MF-DGP gives the lowest NRMSE (0.079) and highest SCORE (3.032), with all 30 held-out high-fidelity observations lying within their nominal 95% predictive intervals. These contrasting results show that no single multi-fidelity emulator is uniformly superior. Instead, method choice should reflect the complexity of the inter-fidelity relationship, the amount of high-fidelity information available, and the importance placed on predictive accuracy and uncertainty quantification.
Read the full abstract ▾
All authors Xiaoyu Xiong, Louise Kimpton, Huiyi Yang, Mian Xu, James Salter, Peter Challenor
Statistics Theory
Glivenko–Cantelli Theorems for Integrated Volatility Functionals in Pure-Jump Semimartingales with an Application to Cryptocurrency Markets
Dachuan Chen, Jia Li, Tian Xie et al. (+1 more)
We develop a two-step procedure for estimating integrated
volatility functionals, defined through the occupation measure of the latent spot
volatility process, when the asset price is a pure-jump semimartingale. In the first step, block-based
estimators formed from absolute powers of high-frequency increments uniformly approximate local averages of powers of
volatility. In the second step, these estimates are aggregated into an empirical occupation measure. Since price increments have infinite variance in this setting, arguments based on local Gaussianity are unavailable, and the uniform theory instead rests on maximal inequalities tailored to the stable regime. We establish Glivenko–Cantelli-type uniform consistency over classes of bounded monotone, Lipschitz-in-parameter, and locally Hölder test functions. These results deliver consistent estimation of
volatility occupation times and
quantiles, together with an argmax-consistency theory for $M$-
estimators built on nonparametrically recovered latent processes. We further propose a stability-based rule for selecting the power index of the
volatility estimator, which tracks an infeasible ex ante optimal choice closely in Monte Carlo experiments. An application to high-frequency cryptocurrency
markets illustrates the framework in a jump-dominated, heavy-tailed environment.
Read the full abstract ▾
All authors Dachuan Chen, Jia Li, Tian Xie, Chengxin Yan
Statistical Methodology
Efficient model exploration with the integrated nested Laplace approximation
Héctor López-Gómez, Virgilio Gómez-Rubio and Gonzalo García-Donato
Model and variable selection are important topics in
Bayesian inference. In particular, the selection of different fixed, random effects and hyperparameters in hierarchical models can be difficult because of their complex structure. In this paper, we introduce the use of approximate
Bayesian inference for model and variable selection for hierarchical
Bayesian models. The approach is based on the application of Markov chain Monte Carlo methods on the model space. In particular, the Metropolis-Hastings
algorithm is used to sample from the set of model indices. In this way, the marginal likelihood is used to compute the acceptance probability so that it is not required to estimate the parameter models directly. For this, the integrated nested Laplace approximation (INLA) is used because it provides accurate estimates of the marginal likelihood and it can also provide estimates of the posterior marginal of the model parameters. This method can be applied not only to variable selection but to a wide range of problems subject to model uncertainty. To illustrate the potential of this approach, examples on variable selection, changepoint models and log-Gaussian Cox processes are developed.
Read the full abstract ▾
Statistics Theory
Quantiles for multivariate distribution: a short survey of population concepts
Marc Hallin
Quantiles are among the most fundamental concepts in Probability and Statistics, from descriptive to inferential. However,
quantile functions are well-defined and well-understood in the context of one-dimensional probability distributions, where their definition as the inverse of distribution functions—a definition which is intimately related to the canonical ordering of the real line. Starting with dimension $d=2$, such a canonical ordering is no longer available in~${\mathbb R}^d$; nor is it available in nonlinear
manifolds such as hyperspheres,
tori, or polyspheres (vectors of directional variables). This results in the absence of an obvious and widely accepted definition of
quantiles. The need to extend the concept of
quantile beyond the classical univariate context nevertheless has sparked a large body of literature, giving rise to a variety of more or less satisfactory
quantile concepts; the recent years have been particularly active in this respect. This short review is an attempt to complement and update Serfling's 25-year-old survey (Serfling 2008) by summarizing and unifying some of these concepts in light of recent contributions.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Dense Structural Compression of Transformers via Gauge-Correct Channel Removal
Jed A. Duersch, Naïm Es-Sebbani, Nathanaël Haas et al. (+1 more)
Inference energy per
token drives the cost and carbon footprint of deployed
transformers. It is dominated by dense
matrix products that incur fused multiply-accumulate (FMA) operations and
memory traffic. To reduce these computations while retaining dense tensors for high GPU throughput, we develop a methodology from first principles to adapt structural complexity during training to maximize inference utility per unit compute. Channel penalties drive entire tensor slices to zero to enable physical removal while preserving density and the network function. The natural approach, penalizing the norm of operator components acting through each channel, is provably destabilized by gauge freedom. We resolve this pathology with GaugeLasso: additive symmetric group-lasso penalties that recover a monotone function of product-norms when the network converges to gauge balance. Our
equilibrium analysis enables per-channel calibration to correctly suppress slices that under-perform in inference utility per unit compute. Under adaptive pressure, the network reorganizes into depth-dependent structural profiles that can be far smaller than the architecture required to learn the task. On
polynomial long division over $\mathbb{F}_{31}$, compute compresses from 148 to 255 times with perfect accuracy. On character-level
language modeling, compressed models outperform the hand-designed baseline at equal FMA. On masked autoencoding, a compression trial exposes which axes were over-provisioned and which saturated, guiding a better second design. Compaction also accelerates training monotonically as the model progresses. Post-hoc
pruning with the same utility ranking cannot reach these structures, showing that sustained pressure is central to discovery of efficient models. Retraining a discovered architecture recovers baseline quality on our statistical tasks, but fails on our exact algorithmic task.
Read the full abstract ▾
All authors Jed A. Duersch, Naïm Es-Sebbani, Nathanaël Haas, Zied Bouraoui
Statistical Methodology
Bayesian Generalized Network Autoregressive Model with Structured Shrinkage and Persistence Priors
Seongmin Kim and Kyusoon Kim
We propose a
Bayesian generalized network autoregressive (BGNAR) model for multivariate time series whose component series are associated with the nodes of a known network. The proposed framework combines the parsimonious network structure of the generalized network autoregressive (GNAR) model with structured shrinkage and persistence priors adapted from
Bayesian vector autoregressive (BVAR) modeling. We adapt Minnesota-type shrinkage to both own-lag and network-lag coefficients, with prior variances decreasing over temporal lags and, for network effects, neighborhood orders. A hierarchical prior on the own-lag coefficients allows information to be shared across nodes while retaining node-specific heterogeneity. We further adapt the sum-of-coefficients and dummy-initial-observation priors to the GNAR parameterization. Posterior inference is performed using a Gibbs sampler. Simulation studies show that BGNAR can use the same deliberately over-specified temporal and neighborhood structure across
datasets without
dataset-specific BIC order selection, while maintaining forecasting accuracy comparable to BIC-selected GNAR and outperforming the unrestricted BVAR
benchmark in the settings considered. The structured prior regularizes weakly supported coefficients toward zero within this fixed model. Posterior distributions for the dynamic coefficients and posterior predictive distributions for future observations provide direct quantification of parameter and predictive uncertainty. An application to a wind-speed network demonstrates that BGNAR achieves point-forecast performance comparable to GNAR while additionally providing posterior inference on own-lag and network-lag effects and posterior predictive uncertainty.
Read the full abstract ▾
Statistical Methodology
Adaptive Conformal Inference Under Delayed Feedback: Coverage Guarantees and a Delay-to-Memory Diagnostic
Lama El Halabi and Adam Brandt
Adaptive Conformal Inference (ACI) extends conformal prediction to non-exchangeable settings by adjusting the nominal miscoverage level online in response to recent coverage errors. When forecasts are issued with horizon $\tau$, however, the outcome needed to evaluate a prediction is observed only $\tau$ steps later, so these adaptive updates must rely on delayed feedback. We study this setting and ask how the effect of delay depends on the persistence of the residual process. First, we show that the $\tau$-delayed ACI recursion can be decomposed into $\tau$ interleaved ACI-like sequences. This representation yields a finite-sample bound on long-run empirical coverage with explicit dependence on $\tau$. We also derive an approximate marginal coverage bound that relates coverage deviation to changes in the underlying environment across the forecast horizon and to the adaptation rate $\gamma$. We then introduce the delay-to-
memory ratio $r=\tau/L$, where $L$ is the time scale over which the temporal signal driving non-exchangeability decays. Simulation results show that the usefulness of this ratio depends on the form of temporal dependence: it strongly organizes performance under AR(1) dependence, is less predictive of overall performance under GARCH(1,1) and Markov switching, but more clearly characterizes when scale normalization remains useful in those settings. Abrupt mean- and variance-shift experiments further show that the preferred adaptation rate depends on the residual dynamics. Overall, the results show that the effect of forecast delay is best understood relative to the time scale over which past residual information remains relevant.
Read the full abstract ▾
Statistics Applications
Structure-Adaptive E-Value Filter for Detecting Regional Signals in Brain Imaging
Jingcheng He, Hangjin Jiang, Wenguang Sun et al. (+1 more)
Structural MRI provides a noninvasive view of the neuroanatomical differences associated with cognitive impairment and dementia. Using data from the Alzheimer's Disease Neuroimaging Initiative, we investigate which anatomical regions exhibit widespread gray-matter differences and how these regional patterns vary across the clinical spectrum. Addressing this goal requires translating spatially dependent voxel-level evidence into regional conclusions while controlling multiplicity across anatomical regions. We propose the Structure-adaptive E-value FilTer (SEFT), which uses flexible working models to construct spatially adaptive voxel-level scores and aggregates them into regional partial-conjunction e-values. When combined with the e-value Benjamini–Hochberg (e-BH) procedure, these e-values provide finite-sample control of the set-wise false discovery rate under arbitrary interregional dependence. The ADNI analysis reveals a coherent neuroanatomical pattern: the exploratory analysis shows that differences between normal cognition and mild cognitive impairment are concentrated in medial-temporal regions, whereas the differences between mild cognitive impairment and dementia extend more broadly into temporal–limbic and posterior association regions. Both patterns largely overlap the normal-cognition–dementia
benchmark, identifying a shared anatomical core across the clinical comparisons.
Read the full abstract ▾
All authors Jingcheng He, Hangjin Jiang, Wenguang Sun, for the Alzheimer's Disease Neuroimaging Initiative
Statistical Methodology · Cross-listed from econ.EM
Filtering without recursion and some of its uses in financial economics
Simon Donker van Heel and Neil Shephard
We develop a filter for time series, defined at each time $t$ as the minimizer of a discounted convex combination of observed and expected losses. The filter can be estimated by simulation to an arbitrary level of accuracy in $O(1)$ flops at each time point $t$ and can be run for all values $t=1,...,T$ in parallel. These methods are applied to robustly compute a preaveraged price process from the more than 1.5 million trades made on a single financial asset in a single day where the noise's variance is infinite. It yields a flat "
volatility signature" plot, down to the 1 second level, so the microstructure noise no longer biases the
volatility estimate. This is not true when linear methods are employed.
Read the full abstract ▾