Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Tuesday 8 September

Statistics Applications

Bayesian palaeoclimate reconstruction from zero-inflated count-compositional pollen data: A case study of Lago Grande di Monticchio in southern Italy

Bayesian palaeoclimate reconstruction from fossil pollen counts relies on a modern pollen-climate calibration data set to infer the pollen-climate relationships used to reconstruct past climates. While geographically large calibration data sets improve coverage of climate space and reduce unreliable extrapolation, they also introduce substantial heterogeneity, structural zeros, and complex pollen-climate relationships. We propose a Bayesian modular framework for palaeoclimate reconstruction from count-compositional pollen data that addresses these challenges, and provides coherent uncertainty quantification. The framework employs the zero-and-$N$-inflated multinomial logistic-normal distribution to describe the compositional pollen counts coupled with Bayesian additive regression tree priors to model the nonlinear effects and interactions among the climate covariates. Inference is formulated through a cut posterior distribution that modularises the analysis into forward and reconstruction modules. The forward module is fitted once using a large modern calibration data set and its posterior uncertainty is subsequently propagated to reconstruct climate variables from fossil pollen counts. For the reconstruction module, we develop and compare three inverse posterior sampling schemes. Simulation studies and empirical validation on the modern data set demonstrate that a combination of sampling importance resampling with a multiple imputation technique and a continuous uniform prior over the domain of the modern climate variables achieves the best predictive performance, with well-calibrated uncertainty quantification for the climate reconstruction. In our motivating case study, we further illustrate the proposed methodology by reconstructing a three-dimensional climate vector from fossil pollen records collected at Lago Grande di Monticchio in southern Italy.
Read the full abstract

All authors André F. B. Menezes, Andrew C. Parnell, Brian Huntley, Keefe Murphy

Statistics Applications

Dynamic Latent Space Modeling of Inhomogeneous Poisson Network Processes with Applications to International Relations

We study continuous-time relational event data, where time-stamped dyadic interactions reflect both individual node propensities and evolving relational proximity. We propose a dynamic latent space model for inhomogeneous Poisson processes, where event intensities depend on node-specific activity parameters and time-varying latent distances modeled via flexible B-splines. We prove model identifiability by decoupling baseline activity from latent position, ensuring high interaction volumes do not warp the spatial map. For scalability, we develop a minibatch stochastic gradient algorithm with stable initialization and geometric anchoring, alongside an effective-degrees-of-freedom BIC for tuning model complexity. Simulations confirm accurate parameter recovery and out-of-sample prediction. Applied to cooperative diplomatic events among 60 major economies (1995–2022), the model uncovers shifting patterns of international cooperation and isolates mobile geopolitical actors from stationary institutional anchors.
Read the full abstract

Statistics Applications

Bridging Network Psychometrics and Artificial Intelligence: An Ising-Potts Model with LLM-Derived Weights

The Potts model extends the Ising model to multinomial data. We introduce a Rater Ising-Potts model that uses agreement indicators between pairs of ratings and category labels, with weights derived from LLM embeddings. The model does not presuppose ordered category thresholds or equidistant scoring; instead, it focuses on pairwise agreement among ratings and assigns category-specific positive weights, making it suited for multi-category scoring reliability. We evaluate the model on three constructed-response datasets spanning a corpus of K=14,466 short answers on a three-level rubric and two AERA essay prompts of roughly 1,200-1,400 responses on four-point rubrics. We compare three strategies for sharpening the similarity signal: top-K pruning, min-max normalization with a power transformation, and ColBERT late-interaction similarities. Top-K pruning, which replaces the dense similarity graph with a sparse local network of strongest semantic neighbors, consistently yields the highest accuracy and Cohen's kappa, and the selected neighborhoods are always a small fraction of the corpus. Power tuning consistently ranks second, while ColBERT is competitive on longer essay prompts and adds little on short answers. Across all settings, most misclassifications occur between adjacent score levels, confirming that the model preserves the ordinal structure of scoring rubrics without imposing rigid assumptions. These findings suggest that LLM-derived similarities, combined with a parsimonious Potts formulation and a sparse local graph, offer a robust and interpretable framework for reliability auditing in educational assessment. We discuss extensions to multiple raters and hierarchical rating designs.
Read the full abstract

Statistics Applications · Cross-listed from physics.comp-ph

Structure-Informed Bayesian Inference of Anomalous Transport and Hidden Molecular Trapping in Amorphous Media

Molecular diffusion in fluctuating amorphous and macromolecular media governs key transport processes across soft-matter physics, energy storage, and biological membranes. Extracting localized trapping states from single-particle tracking trajectories remains a fundamental challenge; because thermal structural breathing continuously reconfigures pore boundaries, conventional geometric algorithms suffer from severe systematic biases, erroneously merging distinct localized states during cyclic molecular returns. Here, we address this deadlock by shifting the paradigm from local geometric recurrence to a structure-informed Bayesian regularization. Leveraging discrete Morse theory, we extract the time-invariant topological skeleton of the fluctuating host matrix to construct robust, gas-specific physical priors that account for individual molecular dimensions. Trajectory steps are sequentially partitioned via a two-stage probabilistic refinement that dynamically adapts to the transport landscape. Benchmarked against a rigorous environment where synthetic particles explore the actual interconnected matrix graph, our approach eliminates systemic biases, restricting macroscopic trapping parameter deviations to just a few percent under optimal linear $O(N)$ computational scaling. Applied to hydrogen and methane transport within a type-I kerogen matrix, serving as a prototype for highly tortuous, flexible macromolecular networks, the method successfully decodes the hidden microscopic mechanisms of confined diffusion. To ensure immediate broad impact, the documented open-source code and data are made publicly available, offering an accessible strategy readily adaptable to a broad spectrum of tracking phenomena, from ion transport in battery polymers to protein trafficking within cellular environments.
Read the full abstract

All authors Andrey Ananev, Maria Potapova, Nikolay Kondratyuk, Timur Vostroknutov, Aleksey Khlyupin

Statistics Applications · Cross-listed from stat.ME

Instability in Patient Clustering: A Multiverse Analysis of Unsupervised Clustering in the CENTER-TBI cohort

Understanding patient heterogeneity is key to improving prognostic modeling in traumatic brain injury (TBI). Unsupervised clustering is widely used to explore patterns in patient characteristics that may define subgroups. However, it involves a multitude of decisions, including the choice of algorithm, the distance metric, and the method used to determine the "optimal" number of clusters. The aim of this study is to investigate how these choices influence the resulting clustering solution. We analyzed data from 4,509 patients enrolled in the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) study. K-medoids, agglomerative, and spectral clustering were applied in a complete 3 X 2 X 2 factorial design, in combination with Euclidean or Gower's distances, and silhouette score or gap statistic to choose the number of clusters. We investigated the agreement of clustering solutions with UpSet Plots and stability with the (adjusted) Rand index. Comparisons were made both across approaches using the original dataset and within approaches using bootstrap resampling. Clustering results varied substantially depending on the analysis choices. The number of suggested clusters varied widely, from one to twenty-five. Adjusted Rand indices confirmed low concordance between methods. Moreover, none of the clustering solutions demonstrated discriminatory performance comparable to a supervised logistic regression model in classifying patient recovery illustrating the limited usefulness of clustering for this purpose. The high instability in clustering results compromises interpretability and underscores that such solutions should not be blindly interpreted as underlying structure.
Read the full abstract

All authors Sean R E A Bagcik, Aneeta Merlin Chacko, Ewout W Steyerberg, Maarten van Smeden, Andrew I R Maas, Erik van Zwet, Nicole S Erler

Statistics Applications · Cross-listed from physics.ao-ph

GCMagicc v1: a fast generative emulator for multivariate climate-impact ensembles

Projecting the impacts of climate change requires large ensembles of climate variables that match historical observations, align with the warming ranges assessed by the IPCC, and can efficiently run new future emissions scenarios, including the newest generation of climate model scenarios (CMIP7) and pathways consistent with countries' Paris Agreement pledges. Generating such ensembles at the scale needed for impact studies is normally computationally prohibitive. We close this gap with GCMagicc, a hybrid model that pairs a simple physical climate model with machine learning to generate ensembles of 10 climate variables at the resolution of full-scale Earth system models, without relying on GPU resources or retraining for new scenarios. Trained on 32 CMIP6 Earth system models and observational/reanalysis data, GCMagicc complements rather than replaces Earth system models. We apply it to a range of future pathways: the canonical SSP scenarios of the latest IPCC report (1.2-6.1{\deg}C warming, min-max across scenarios of 5-95 percentile ranges), current policies (2.3-4.0{\deg}C), national pledges under the Paris Agreement (1.5-3.3{\deg}C) and the CMIP7 range from the 'VL' to 'H' scenarios (1.2-4.2{\deg}C), releasing a large public dataset. As an illustration, we perform an attribution analysis of the severe 2025 Iranian drought using GCMagicc ensembles, with three CMIP6 large ensembles for comparison, with and without anthropogenic forcings. The results suggest a strong anthropogenic signal: a median probability of drought at least as severe as observed of 29% with anthropogenic forcing, and zero under natural-forcing-only simulations. In the future, drought conditions are projected to materially worsen, amplifying the potential for agricultural and food security impacts and geopolitical conflicts that use water scarcity as a weapon. GCMagicc data is available at https://gcmagicc.org.
Read the full abstract

All authors Nicolai Meinshausen, Malte Meinshausen, Jared Lewis, Zebedee Nicholls, Sarah Schöngart, Alister Self, Xinwei Shen, Karla Spiller, Elisabeth Vogel

Monday 7 September

Statistics Applications · Cross-listed from cs.CV

Multi-View Structure-from-Motion Enables Oriented Projective Shape Analysis in Three Dimensions

Projective shape analysis provides a geometric framework for studying landmark configurations in digital images acquired by pinhole cameras. In the classical projective shape (PS) model, three-dimensional configurations ($k$-ads) are represented as points in $(\mathrm{RP}^3)^q$, $q = k - 5$. A nonparametric test is developed in Patrangenaru et al. [12], for this framework, to determine whether an object matches a design blueprint, with each configuration reconstructed from a single uncalibrated stereo pair. Such two-view reconstructions are identified only up to a 3D projective transformation, which may reverse orientation, so that the sign-blind PS summary was the only available, before oriented projective shape (OPS) was recently considered. Multi-view Structure-from-Motion (SfM) technology removes this obstruction: its bundle adjustment is identified up to an orientation-preserving projective transformation. In this paper we revisit a well-cited three-cube object, from Patrangenaru et al. [12], with $n = 8$ SfM reconstructions built in Agisoft Metashape Professional 2.3.0, and validate our implementation by reproducing the published stereo construction from the original data. This allows us to perform what is to the best of our knowledge the first three-dimensional OPS analysis, compute its extrinsic total-variance index and perform statistical inference in this novel setting. Due to the high concentration of SfM data, the OPS index is asymptotically one-half the PS index, a structural consequence of concentration rather than a property of the object. Here our blueprint hypothesis is not rejected for any of the $q = 14$ non-frame landmarks, while the SfM reconstructions are about 26 times more concentrated than the stereo ones, a substantial gain in reconstruction precision. Sample-size, photograph-count, and frame-ordering analyses support the robustness of these conclusions.
Read the full abstract

Statistics Applications

Pre-game paired-comparison modeling of professional League of Legends map outcomes

We build and evaluate a pre-game win-probability forecaster for individual maps (“games”) in professional League of Legends (LoL). The proposed model is a one-stage logistic regression fit end-to-end on the win/loss log-loss: each team's exponentially-weighted moving average of past same-side results, a ridge-shrunk stable strength that is the maximum-a-posteriori estimate of a logistic mixed model, and a first-pick draft covariate, natively calibrated out of sample (walk-forward slope $0.995$). It augments a purely dynamic Bradley–Terry specification with stable team strengths. A second, independently built two-stage composite mixed model under restricted maximum likelihood (REML) and best linear unbiased prediction (BLUP) shrinkage, with Platt calibration, serves as the strongest rival the authors could build. On $5{,}135$ games across six regional leagues and three international events (2024–2026), under paired per-game Diebold–Mariano inference, the two architectures are statistically indistinguishable on every protocol and window (global holdout $0.2230$ vs.\ $0.2257$; walk-forward $0.2207$ vs.\ $0.2215$), so the simpler model is preferred on parsimony, not accuracy; both improve on the classical dynamic benchmark ($0.2351$) by a clear margin and on the static fits ($0.2301$/$0.2268$) more modestly. Against Polymarket on $928$ matched maps, the forecasts are statistically indistinguishable from the market on its own per-game contracts, with a modest market edge concentrated on cross-region Worlds and series-decider maps.
Read the full abstract

Statistics Applications · Cross-listed from cs.AI

When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability

LLM-based digital twins promise to reduce repeated human data collection by generating person- specific responses, yet existing evaluations provide little evidence about whether they can reduce human measurement while preserving valid inference. To address this, we introduce statistical substitutability, an inferential criterion that evaluates the extent to which twin predictions can reduce human measurement for a particular estimand while preserving valid inference. We develop a framework, grounded in mixed-subject and prediction-powered inference, that evaluates statistical substitutability along four dimensions: aggregate fidelity, paired respondent-level signal, finite-sample human-label recovery, and stability across populations. Across two empirical evaluations spanning behavioral experiments, multiple models, and alternative respondent representations, we find that digital twins can reproduce average human effects while providing little information about which individuals differ from those averages. Newer models and richer respondent information improve some dimensions of performance but do not reliably translate into human-data savings. Human calibration can reduce aggregate prediction error, yet limited labeled samples often fail to produce stable precision gains. Importantly, these findings demonstrate that behavioral fidelity is neither necessary nor sufficient for statistical substitutability. More broadly, they suggest that AI-generated evidence should be evaluated based on its ability to support valid scientific inference rather than its ability to reproduce human outcomes alone. Digital twins should therefore be judged for confirmatory use by whether they reduce uncertainty about human quantities, not merely by whether they reproduce human means, distributions, or effects.
Read the full abstract

All authors Steven Wang, Kyle Hunt, Shaojie Tang, Kenneth Joseph

Statistics Applications · Cross-listed from astro-ph.IM

Neural Posterior Estimation for Tomographic Weak Lensing Mass Mapping

Weak gravitational lensing shear and convergence trace the distribution of baryonic and dark matter across space, making them a powerful probe of cosmic structure. Inferring shear and convergence from images is a challenging inverse problem. The prevailing approach to this task estimates shear from weighted averages of galaxy ellipticities, calibrates these estimates to account for systematic biases, and transforms them to reconstruct convergence, a multistage procedure that requires substantial computational resources and meticulous handling of statistical uncertainties. As an alternative, we propose a probabilistic approach to field-level weak lensing inference in which we train a deep neural network to directly map a multiband image to a variational distribution over the underlying tomographic shear and convergence fields. This neural posterior estimation (NPE) procedure implicitly marginalizes over nuisance variables in the cosmological forward model and does not require evaluating the likelihood function. It is also amortized, so it enables rapid posterior inference for astronomical surveys once the neural network is trained. When evaluated on synthetic images from the LSST-DESC DC2 Simulated Sky Survey, NPE produces well-calibrated variational distributions for shear and convergence that are consistent with the ground truth. We describe how maps sampled from these variational distributions could be used in a subsequent simulation-based inference procedure to approximate the posterior distribution over cosmological parameters.
Read the full abstract

All authors Tim White, Shreyas Chandrashekaran, Camille Avestruz, Jeffrey Regier, the LSST Dark Energy Science Collaboration

Statistics Applications

Two-resolution state modelling of intermittent online gambling activity during Sweden's temporary deposit-cap period

Intermittent online gambling records create two distinct representation problems: calendar-time analyses must retain represented days with no recorded play, whereas analyses of active behaviour must preserve variation within active episodes. We pair calendar-time and active-bout models in a two-resolution analysis of 14.2 million player-days from a single operator during 2019-2023. The calendar-time layer retains assignable represented days in five observed categories and propagates a pre-period reference. The active-episode layer fits a Bayesian mixed-emission hidden semi-Markov model to turnover, playing time, approved deposits and session counts. In the reference four-regime fit, two regimes share a rounded model-implied turnover median of 153.5 EUR, while their model-implied playing-time and approved-deposit medians differ by factors of 4.6 and 5.7, respectively, and their expected total session counts by a factor of 2.8. This separation remains under an ordinary hidden Markov model, two routing perturbations and a residual-dependence sensitivity; the tested turnover-only fits were diagnostically unstable. Over the temporary Swedish deposit-cap period, deposits and turnover per represented player-day were roughly 55% below the propagated reference and active-category occupancy was 7.8% lower. The largest negative deviations in a prespecified Sweden-assigned cohort occurred among high pre-policy deposit-intensity customers. These are operator-specific descriptive comparisons, not causal policy effects.
Read the full abstract

All authors Sam Andersson, Timo Koski, Helga Westerlind, Keenan Lyon, Per Carlbring, Olof Molander

Statistics Applications · Cross-listed from stat.ME

Modeling Brain MRI Using Persistent Homology and Multilevel Functional Data Analysis

Persistent homology provides a multiscale representation of biomedical images by capturing higher-order topological features that reflect their underlying structural organization. However, the resulting topological summaries are typically used as predictors or features for classification and group comparisons rather than treated as primary variables of interest. We construct a generalized multilevel functional framework for analyzing persistent-homology summaries as longitudinal functional responses in repeated three-dimensional structural magnetic resonance imaging (MRI). Specifically, we represent topological features using Betti curves and model these curves as count-valued functional responses. A negative-binomial distribution accommodates the discrete and potentially overdispersed nature of Betti counts, while the multilevel formulation accounts for the dependence induced by repeated measurements and separates between-subject and within-subject sources of functional variation. Functional principal component analysis further evaluates the dominant modes of variation at each level. A Bayesian approach is used for joint estimation of the functional regression and multilevel functional principal components. We apply this modeling framework to longitudinal structural MRI data from the OASIS-2 study to investigate associations between brain topology and demographic and clinical characteristics, including age, gender, follow-up time, and dementia severity. The results demonstrate that the framework can capture covariate-associated variation across the filtration continuum while evaluating distinct sources and patterns of longitudinal variation across homology dimensions.
Read the full abstract

Statistics Applications

Structural Analysis of a Dynamic Multilayer Network via Matrix Autoregressive Models: A Case Study of International Interactions between Countries

Dynamic and multilayer networks have been widely studied separately, but their joint analysis remains comparatively underdeveloped. Because the relational information of a dynamic multilayer network can be represented as a tensor at each time point $t$, each layer can be summarized through a set of structural statistics, yielding a matrix-valued observation and, consequently, a matrix-valued time series. To exploit this structure, we propose the use of matrix autoregressive (MAR) models, which simultaneously characterize temporal dependence across relational layers and structural statistics. We apply this framework to the ICEWS dataset, which records international interactions among countries under four relational domains and therefore naturally defines a dynamic multilayer network. The results indicate that negative verbal interactions (Verbal-) play a prominent role in the subsequent structural reconfiguration of the material-interaction layers, while mean strength exhibits the strongest temporal persistence and reciprocity the broadest cross-statistic influence. These findings illustrate the usefulness of MAR models for providing a parsimonious and interpretable characterization of temporal and cross-layer dependence in dynamic multilayer networks.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Translation of Black-Box Clinical Prediction Models into Standalone Transparent Nomograms: Temporal External Validation in Heart Transplantation

We convert black-box clinical prediction models for tabular data into standalone nomograms that can be audited term by term. PRiSM (Partial Responses in Structured Models) takes the shape of each effect and interaction from the source model, not merely which variables mattered, and lets the outcome select and weight them. We tested this in 50,356 heart transplant recipients, with validation in a later era than training. Nomograms from all 5 source models - a public clinical risk score, logistic regression, neural networks, random forests and extreme gradient boosting - met a prespecified noninferiority criterion for discrimination before any further simplification, and generally preserved calibration and clinical net benefit. Those from the 3 machine-learning models showed no detectable difference in discrimination from de novo generalized additive and explainable boosting models, exceeded neural additive models, and carried fewer terms than the explainable boosting model. PRiSM is released as an open-source Python package.
Read the full abstract

All authors Henry Pigot, Paulo J. G. Lisboa, Sandra Ortega-Martorell, Ivan Olier, Joseph Mahon, Johan Nilsson

Statistics Applications · Cross-listed from physics.chem-ph

Simulation-Supervised Foundation Models for Retention Time Prediction in High-Performance Liquid Chromatography beyond Experimental Data Coverage

Accurate prediction of high-performance liquid chromatography (HPLC) retention times (RTs) across diverse molecules and chromatographic methods remains challenging because experimental training data cover only a limited region of chemical and method spaces. Here, we develop FUSE-RT (Foundation model Unifying Simulation and Experimental supervision for Retention Time), a multitask foundation model that integrates RT data from 179 chromatographic methods and adapts to unseen molecules and methods using limited target-domain data. To extend transferability beyond experimental coverage, we introduce simulation-to-real (Sim2Real) transfer learning, in which molecular representations learned from large-scale computational data are transferred to experimental RT prediction. Specifically, we use PolyOmics, comprising 39 properties for approximately 21,400 molecules generated by molecular dynamics and density-functional theory calculations, as auxiliary supervision. We evaluate generalization under molecular, method, and joint molecular–method distribution shifts. Simulation-derived supervision substantially improves transfer beyond the experimental molecular domain, particularly under pronounced coverage gaps and few-shot adaptation. Moreover, RT-prediction error decreases systematically with increasing simulation-data size, following a significant power-law relationship. These results establish Sim2Real transfer as a scalable strategy for extending RT prediction beyond the finite coverage of experimental chromatographic data.
Read the full abstract

All authors Stephen Wu, Yufeng Han, Yasuhiro Mito, Yoshiyuki Watabe, Yoshihiro Hayashi, Hikaru Takaya, Takuya Kubo, Ryo Yoshida

Statistics Applications · Cross-listed from cs.LG

Statistical versus machine learning-based spatial interpolation of post-processed ensemble weather forecasts

Statistical post-processing improves ensemble weather forecasts, but generating calibrated predictions at locations without observations remains challenging. This study compares statistical and machine-learning-based methods for post-processing ECMWF 2-m temperature and 10-m wind speed forecasts at observed and unobserved stations in Germany. We consider EMOS-based approaches, distributional regression networks, Transformers, and graph neural networks under both limited and extended predictor settings. For temperature, we also investigate linear forecast combinations and propose an altitude-aware linear pool (ALP). The results show that post-processing improves upon the raw ensemble in most settings, but no single method performs best across all variables, station groups, and evaluation metrics. The proposed ALP provides a small but significant improvement over the standard linear pool at unobserved locations.
Read the full abstract

Statistics Applications · Cross-listed from stat.ML

Distributed Lag Neural Additive Models

We introduce Distributed Lag Neural Additive Models (DLNAMs), neural-additive analogues of Distributed Lag Non-linear Models (DLNMs) for learning nonlinear effects distributed over lags. DLNAMs replace a prespecified spline cross-basis with neural components that learn exposure–lag response surfaces, avoiding choices of basis family, dimension, and knot placement while preserving additive interpretability and familiar distributed-lag summaries. Exp-centered input layers, smooth activations, and learned subnetwork mixtures produce smooth, locally adaptive representations; pointwise uncertainty combines a conditional last-layer Laplace approximation with between-member ensemble variation. In simulations, DLNAMs generally outperformed DLNM comparators, including penalized and treed variants, in recovering known response functions, with lower bias, stronger boundary recovery, and better-calibrated cumulative intervals; gains were largest for more demanding functions. The architecture performed consistently across sample sizes, outcome families, lag horizons, and jointly fitted multi-exposure settings, retaining recovery performance as exposures were added; fit-specific changes were largely confined to optimization, and applications recovered established empirical patterns.
Read the full abstract

All authors Calle Helmersson, Shivang Pandey, Leonardo Olivetti, Elena Raffetti

Statistics Applications

Bayesian Emulation of Multi-fidelity Earth System Modelling Using Hierarchical Gaussian Processes

Multi-fidelity Earth system models provide simulations at different levels of complexity and computational cost, but exhaustive exploration of the parameter space at the highest fidelity is often prohibitively expensive. Multi-fidelity emulators can reduce this burden by combining abundant lower-fidelity simulations with limited high-fidelity evaluations. We compare four Gaussian-process-based multi-fidelity approaches: the Kennedy–O'Hagan autoregressive model (K&O), hierarchical kriging (HK), Bayesian hierarchical emulation for multi-level models (BayHEm), and multi-fidelity deep Gaussian processes (MF-DGP). We evaluate the methods using two contrasting applications: a three-fidelity tsunami simulator and a two-fidelity implementation of the Joint UK Land Environment Simulator (JULES). Performance is assessed using leave-one-out predictive accuracy, uncertainty representation, design requirements, and computational characteristics. In the tsunami application, BayHEm gives the lowest normalised root mean square error (NRMSE = 0.031) and highest SCORE (3.025), while MF-DGP performs worse than the single-fidelity baseline when only 10 high-fidelity simulations are available. In the JULES application, MF-DGP gives the lowest NRMSE (0.079) and highest SCORE (3.032), with all 30 held-out high-fidelity observations lying within their nominal 95% predictive intervals. These contrasting results show that no single multi-fidelity emulator is uniformly superior. Instead, method choice should reflect the complexity of the inter-fidelity relationship, the amount of high-fidelity information available, and the importance placed on predictive accuracy and uncertainty quantification.
Read the full abstract

All authors Xiaoyu Xiong, Louise Kimpton, Huiyi Yang, Mian Xu, James Salter, Peter Challenor

Statistics Applications

Structure-Adaptive E-Value Filter for Detecting Regional Signals in Brain Imaging

Structural MRI provides a noninvasive view of the neuroanatomical differences associated with cognitive impairment and dementia. Using data from the Alzheimer's Disease Neuroimaging Initiative, we investigate which anatomical regions exhibit widespread gray-matter differences and how these regional patterns vary across the clinical spectrum. Addressing this goal requires translating spatially dependent voxel-level evidence into regional conclusions while controlling multiplicity across anatomical regions. We propose the Structure-adaptive E-value FilTer (SEFT), which uses flexible working models to construct spatially adaptive voxel-level scores and aggregates them into regional partial-conjunction e-values. When combined with the e-value Benjamini–Hochberg (e-BH) procedure, these e-values provide finite-sample control of the set-wise false discovery rate under arbitrary interregional dependence. The ADNI analysis reveals a coherent neuroanatomical pattern: the exploratory analysis shows that differences between normal cognition and mild cognitive impairment are concentrated in medial-temporal regions, whereas the differences between mild cognitive impairment and dementia extend more broadly into temporal–limbic and posterior association regions. Both patterns largely overlap the normal-cognition–dementia benchmark, identifying a shared anatomical core across the clinical comparisons.
Read the full abstract

All authors Jingcheng He, Hangjin Jiang, Wenguang Sun, for the Alzheimer's Disease Neuroimaging Initiative

Statistics Applications · Cross-listed from stat.ME

Bayesian nonparametric inference for modal missing species and features

Species and feature sampling problems arise naturally whenever each observed unit is associated with one or more labels from a countable alphabet, and inference focuses on the unobserved portion of the distribution over such an alphabet. Within this framework, recent contributions have shifted attention from inference on the total probability mass over unseen labels to distribution-free confidence intervals for the largest unobserved label probability (i.e., the modal missing probability), thereby providing more refined information on whether the missing mass is concentrated on a few high-prevalence unseen labels or distributed across many negligible ones. Besides lacking a unified framework for species and features, these approaches employ a worst-case perspective that causes substantial information loss and produces overly-conservative intervals. We address these limitations through a unified model-based framework for inference on modal missing probabilities in both species and feature settings, which leverages a flexible Bayesian nonparametric formulation to localize uncertainty around models compatible with the observed data. This leads to sharper closed-form credible intervals that effectively exploit prior information and observed data, while preserving the theoretical frequentist properties and robustness of distribution-free intervals. Simulation studies confirm these improvements, while an organized crime application illustrates how our contribution has the potential to reshape law-enforcement decision-making in investigations.
Read the full abstract

All authors Alessandro Colombi, Mario Beraha, Daniele Durante, Stefano Favaro

Statistics Applications · Cross-listed from econ.EM

Choosing the Dictionary and Penalty for IV-LASSO

Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level. First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution. In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate. Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and second-order Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator, which can be consistently estimated and used to rank a prespecified list of dictionary-penalty candidates. The AMSE reveals a bias-variance trade-off: more complex first-stage fits better approximate the conditional mean of the endogenous variable but are also more correlated with the structural errors, with complexity measured by the degrees of freedom of the LASSO fit. The weight on this bias rises with the endogeneity of the regressor, a quantity that neither plug-in nor cross-validation penalty rules take into account. Despite the AMSE being derived in a Gaussian model, penalty selection by minimizing the feasible AMSE criterion delivers up to a one-third lower mean squared error compared to cross-validation and plug-in penalty rules in Gaussian and non-Gaussian simulation designs calibrated to the data of Gilchrist and Sands (2016).
Read the full abstract

Sunday 6 September

Statistics Applications · Cross-listed from stat.ME

Cluster-Based Dimensionality Reduction by Nonparametric Distributional Screening

We consider dimensionality reduction for high-dimensional observations accompanied by a supplied partition into two or more clusters. The objective is not to construct a low-rank projection, but to retain an interpretable subset of the original coordinates that preserves the distributional information distinguishing the clusters. For each coordinate, the proposed procedure compares the cluster-specific empirical distribution functions through a several-sample Kolmogorov-Smirnov separation statistic. We formalize the resulting marginal cluster support and establish simultaneous finite-sample concentration over all coordinates, explicit bounds for false inclusions and omissions, and exact support recovery when the minimum distributional separation dominates the high-dimensional stochastic error. We also quantify the dimension inflation induced by using an unadjusted testing level and give a familywise-error-controlled version. Under a conditional sufficiency condition, sure screening preserves the full-data posterior cluster probabilities, mutual information, and Bayes risk; an additional result characterizes robustness to imperfectly estimated cluster labels. The procedure is invariant to strictly increasing coordinate transformations and can retain low-variance cluster signals that principal components may discard. We further develop average dual information, a criterion combining partition agreement after transformation with structural coverage of cluster-relevant coordinates, and derive its basic properties and consistency. Simulations illustrate the theory, the interpretability of the selected coordinates, and the distinction between cluster-directed screening and variance-directed projection.
Read the full abstract

All authors Sanoja Jha, Rishikesh Muralimohan, Praveen Athauda Arachchi, Abhishek Bhattacharjee

Statistics Applications · Cross-listed from econ.EM

Beyond Aggregate VARs: A Bayesian Benchmark for HANK Models

Heterogeneous-agent New Keynesian (HANK) models characterize how entire cross-sectional distributions respond to structural shocks. Traditional representative-agent models are routinely disciplined by impulse responses from aggregate vector autoregressions (VARs). HANK models have no comparable established empirical benchmark because they make predictions not only about aggregates, but also about distributions of micro-level data. We propose a Bayesian benchmark that jointly models macroeconomic aggregates and several marginal distributions from repeated cross sections, including distributions observed in different surveys. Our approach can use both standard structural VAR identification approaches on macroeconomic aggregates and identification restrictions imposed on micro-level data. The model delivers a joint posterior of the distributional effects of shocks, without the need for household panel data or a separate first-stage density estimate.
Read the full abstract

Statistics Applications

The profit-bias identity in sports betting: bookmaker profit as the public's prediction error

Sports betting moves money continuously from a large public to a small number of firms. The most influential account of that flow, due to Levitt (2004), holds that books price away from the market-clearing point to exploit predictable public biases. It computes profit from two numbers (the probability that a side wins the proposition and the fraction of handle it attracts), treating the share on a side as independent of the outcome. Here we relax that assumption and derive a profit-bias identity: profit is affine and increasing in the expected share of handle on the losing side, with Levitt's expression as the special case of independence. The identity resolves the book's margin into exactly three channels: its hold, the product of its price shading and the public's lean, and the covariance between bet share and outcome. A lean is thus worthless without shading, and shading is worthless without a lean. Under a public-belief model, the profit driver is the public's Bayes error: the probability of a representative bettor selecting the losing side. We provide necessary and sufficient conditions for a "Goldilocks Zone": prices at which book and bettor both profit. Testing these predictions on 1,139 Major League Baseball games, we find that the apparent dependence between bet share and outcome is a Simpson's paradox: present when games are pooled, but absent once they are separated by which side the book favored. The public leans heavily toward favorites, but we detect no matching shading, and the realized margin is indistinguishable from the hold.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Shrinkage invalidates the Hosmer-Lemeshow test: goodness of fit for penalized logistic regression, with an application to glaucoma diagnosis

Clinical prediction models are increasingly fitted by penalized logistic regression, because collinearity or many candidate predictors makes maximum likelihood unstable or impossible. Calibration is then almost always assessed by a grouped goodness-of-fit test such as the Hosmer-Lemeshow test. We show that this combination is invalid. Under ridge regression the grouped standardized residuals acquire a non-centrality induced by shrinkage, so the reference distribution used in practice is wrong, and at the penalty that most improves the fitted probabilities the test rejects correctly specified models between 92 and 100 per cent of the time. We derive the corrected law and define the shrinkage-corrected Hosmer-Lemeshow test, which subtracts an estimate of that non-centrality, restoring the maximum likelihood reference exactly to first order, and is made valid by prepivoting at a power cost we measure. We also give the attenuation law governing what any such test can detect once the linear predictor must be estimated. In glaucoma diagnosis by confocal laser tomography, where the maximum likelihood estimate does not exist, the corrected test finds the evidence for misfit weaker by more than three orders of magnitude: the fitted risks are too flat rather than mis-ordered, so the model needs recalibration rather than rebuilding.
Read the full abstract

2609.06413 · 6 Sep · 18 pages, 4 figures, 5 tables; supplementary material (13 pages) included as an ancillary file. Submitted to the Journal of the Royal Statistical Society Series C. Reproduction archive: https://doi.org/10.5281/zenodo.21903202. Methods implemented in the R package ebrahim.gof (CRAN), function shrink.gof()

Saturday 5 September

Statistics Applications

From Discrete Trailing Returns to a Continuous Graphical Profile: Return-to-Present Curves

Investment performance is commonly presented either as a conventional cumulative-return chart, which fixes a historical starting date and traces performance forward, or as a trailing-return table, which fixes the current endpoint but reports only a small set of prespecified horizons. These two displays have complementary limitations: fixed-start comparisons are conditional on the selected origin, whereas trailing returns provide only discrete snapshots of the underlying fixed-endpoint return function. We present the Return-to-Present (RTP) curve as a continuous fixed-endpoint representation that brings these perspectives together by holding the evaluation date fixed while allowing the hypothetical historical purchase date to vary over the available history. Familiar 1-month, 3-month, 6-month, 1-year, and longer trailing returns therefore become selected points on a continuous curve. When multiple investments are overlaid, RTP directly displays entry-date sensitivity, persistent relative advantage, crossings, and the timing and magnitude of separation without requiring selection of a single historical origin. The same endpoint-based construction naturally accommodates investments with unequal inception dates, recurring purchases, and retrospective portfolio rotation decisions in which sale and replacement-purchase dates may differ. We illustrate these uses with real investment data and discuss its relationship to momentum. RTP does not define a new return measure; its contribution is a simple graphical organization of familiar realized returns for historical comparison and decision support rather than prediction or statistical inference.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Machine Learning for Pre-Culture ESBL Risk Stratification to Guide Empiric Antibiotic Selection: A 12-Hospital Study of Enterobacteriaceae Cultures

Empiric antibiotic therapy for suspected ESBL-producing Enterobacteriaceae must be selected 48-72 hours before culture results, forcing clinicians to choose between undertreating resistant infections and overusing carbapenems that drive further resistance. We developed a cost-sensitive XGBoost model predicting an ESBL phenotype (resistance to ceftriaxone, ceftazidime, cefepime or piperacillin-tazobactam) at culture ordering using 45 pre-culture EHR features across 132,955 cultures from 72,217 patients at 12 hospitals (14.41% with the ESBL phenotype). Cultures were partitioned at the patient level. At 90% sensitivity, the model achieved 95.8% NPV, reducing post-test ESBL probability to 4.2%, a threshold that may support safe carbapenem-sparing in non-ICU settings, while sparing 307 of every 1,000 cultures an unnecessary broad-spectrum course at the cost of 14 missed ESBL cases per 1,000. SHAP analysis identified prior ESBL colonization as the dominant predictor, ahead of prior organism burden and neighborhood deprivation; removing deprivation features caused minimal performance loss ($\Delta\text{AUROC} = -0.020$), enabling equitable bedside deployment. Discrimination was unchanged under a strict IDSA ESBL-E definition (AUROC 0.766), with specimen type added as a predictor (0.764) and without any class-imbalance correction (0.762), and ranged from 0.71 to 0.78 across organism strata.
Read the full abstract

All authors Aravind V. Kuruvikkattil, Lalitha Pranathi Pulavarthy, Rashmita Kudamala, Saptarshi Purkayastha

Statistics Applications · Cross-listed from cs.RO

Moment-Matching Probabilistic Data Association for Optimization-Based SLAM

Optimization-based simultaneous localization and mapping (SLAM) makes it possible to reduce accumulated navigation errors of sensing platforms by returning to known areas (loop closure). In this paper, we present an approach to combine probabilistic data association (PDA) with optimization-based SLAM. Instead of associating a single measurement with each landmark, we follow the PDA paradigm from the multiobject tracking community. In particular, in a processing stage performed in addition to the nonlinear least-squares solver of optimization-based SLAM, our method (i) assigns multiple measurements to landmarks probabilistically, (ii) computes the mean and covariance of landmark distributions via moment matching by taking multiple measurement-to-landmark associations into account, and (iii) establishes a virtual landmark measurement and a corresponding linear-Gaussian measurement model that leads to the mean and covariance matrix as moment-matching PDA in (ii). By converting the PDA update step into an equivalent linear-Gaussian measurement update step, PDA can be performed effectively within any optimization-based SLAM method. Our preliminary numerical evaluation in a scenario with false negatives and false positives indicates that incremental smoothing and mapping 2 (iSAM2), combined with the proposed PDA approach, can improve agent localization performance compared to conventional iSAM2.
Read the full abstract

Statistics Applications

A Network-Structured Bayesian Hierarchical Model for Sparse Mutation-Drug Response Associations: Application to Cancer Pharmacogenomics

We develop a network-structured Bayesian hierarchical model for sparse association mapping between genomic alterations and quantitative treatment-response phenotypes. The framework combines a Gaussian Markov random field prior that borrows strength across pathway-connected genes, a global-local horseshoe prior inducing sparsity, and a conjugate Gibbs sampler requiring no Metropolis-Hastings steps. Though broadly applicable to high-dimensional settings with known predictor networks, we validate it using cancer cell-line drug-sensitivity data. Applied to GDSC2 ($N=951$ cell lines, $G=219$ driver genes, $D=295$ drugs), the model identifies 126 gene-drug associations (0.195% of 64{,}605 pairs), concentrated in EZH2 (45 drugs, all sensitivity-direction, mean effect $-0.911$ $\ln$IC50) and KMT2D (36 drugs, all sensitivity-direction, mean effect $-0.496$ $\ln$IC50). These markers show external support in an independent PRISM screen (1{,}518 compounds), with KMT2D achieving complete directional replication (36/36) and EZH2 partial replication (8/12). Five-fold cross-validated predictive log-likelihood confirms each prior layer's value: the full model outperforms the no-network ablation by $+3{,}109$ log-units per fold and the no-horseshoe ablation by $+14{,}039$ log-units, consistently across folds. Simulations under three scenarios show the full model achieves the highest precision and lowest false-discovery rate throughout, while the network prior improves sensitivity recovery under network-structured signal. A tissue-stratified extension identifies coherent subgroup refinements, including lung-specific EGFR-inhibitor sensitivity and skin-specific BRAF-Dabrafenib sensitivity. These results show the framework identifies sparse, interpretable, externally supported drug-sensitivity markers while enabling principled investigation of tissue-specific departures from shared effects.
Read the full abstract

Friday 4 September

Statistics Applications · Cross-listed from stat.ME

Dynamic Spatial Bayesian Machine Learning Model: Applications to Intergenerational Economic Mobility and Geographic Income Inequality in the United States

We develop a Dynamic Spatial Panel Bayesian Additive Regression Trees model with Horseshoe shrinkage (DSP-BART-HS) for high-dimensional spatio-temporal panel data. We jointly evaluate the model against a comprehensive suite of structural spatial econometrics, non-parametric machine learning methods, and small-area estimators across nine data-generating scenarios spanning 200 replicates each, including irregular spatial topologies, dense policy effects, and non-linear individual-level interactions. DSP-BART-HS is the best or statistically indistinguishable from the best estimator in every scenario. Conventional region-time-aggregate comparators suffer severe performance degradation – trailing by a factor of three or more – whenever individual-level non-linearity drives outcome variance, a limitation this framework's tree-ensemble design directly addresses. The model also maintains strong predictive accuracy under a zero-training-region spatial holdout via its spatial diffusion mechanism. We demonstrate practical utility across two U.S. county-level panel applications – intergenerational economic mobility and geographic income inequality – under genuine unseen-region, random, and temporal holdouts, with results validated through repeated-split uncertainty quantification and paired significance testing against every comparator. One clear limitation emerges: under temporal extrapolation specifically, a lighter per-region autoregressive specification (MTS-CAR-X) achieves a robust, and consistent advantage. DSP-BART-HS nonetheless establishes strong predictive performance, automated variable selection, and flexible inference for hierarchical spatio-temporal panel data.
Read the full abstract