Peer Revenue Impact factor: volatile

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Thursday 27 August

Machine Learning (Statistics) · Cross-listed from math.FA

The role of parameter Jacobians in the stability of network outputs

In the framework of network dynamics, learning models, and neural tangent kernels (NTK), we show that the corresponding linearized dynamics leads naturally to a semigroup formulation. More precisely, in our analysis of input/output models, the time-dynamics is presented via special semigroups of linear operators on Hilbert spaces, together with an associated class of semigroup perturbations. In this context, we then present new and explicit a priori perturbation-bound results: for the fixed-kernel linearization constructions arising in the NTK setting, we prove norm-bounds on the corresponding semigroup perturbations, in the form of explicit finite-time perturbation estimates. We further present refinements on prescribed task spaces, Cesàro-averaged (ergodic) comparisons estimates, and versions in which the lower spectral edge assumption is replaced by a spectral-distribution condition. We also extend the comparison to nonautonomous NTK evolutions through piecewise-frozen approximations, record a corresponding discrete Euler specialization, and offer worked examples in order to illustrate our perturbation-bound estimates.
Read the full abstract

All authors Halyun Jeong, Palle E. T. Jorgensen, Hyun-Kyoung Kwon, Myung-Sin Song, James Tian

Statistics Applications · Cross-listed from cs.LG

Diffusion Distillation for Efficient Weather Ensembles

Diffusion models generate skillful weather ensembles but require costly iterative sampling. We introduce a supervised energy-distance distillation method that compresses a multi-step diffusion teacher into a single-step student by aligning student forecasts with teacher samples and ground-truth observations. Experiments on global forecasting and typhoon-track prediction show that our student outperforms existing distillation methods and preserves skill for extreme events. It matches or surpasses the teacher across key metrics using only one neural function evaluation per autoregressive step.
Read the full abstract

All authors Yiming Yang, Valentin Brekke, James Briant, Serge Guillas

Machine Learning (Statistics) · Cross-listed from math.OC

On the Computational and Statistical Efficiency of the Empirical Maximum Entropy on the Mean Method

The Maximum Entropy on the Mean (MEM) method provides a flexible computational framework for solving inverse problems by combining data fidelity with entropy-based regularization. In practice, however, the prior distribution is typically unknown but can be estimated from data, giving rise to the empirical MEM method. We establish a parametric convergence rate of $O(n^{-1/2})$ in expectation for empirical MEM, improving upon the previously established $O(n^{-1/4})$ guarantee by King-Roskamp et al. (2026). Our proof is based on a novel stability analysis of the primal and dual optimization problems under perturbations of the underlying probability measure, relying only on foundational tools from convex analysis and probability. We further show that the MEM dual problem admits a reformulation as an expected risk minimization problem, thereby placing MEM within the modern framework of stochastic optimization and enabling scalable stochastic gradient algorithms for large-scale inverse problems. Together, these results place empirical MEM as a statistically and computationally efficient methodology for data-driven inverse problems.
Read the full abstract

All authors Matthew King-Roskamp, Gabriel Rioux, Rustum Choksi, Tim Hoheisel

Statistics Theory

Algebraic implicitization techniques in the graphical models program

Many aspects of daily life now rely on technology driven by advanced techniques for analyzing multivariate data. Graphical models provide the general framework within which much of these analyses take place. However, different data types require different graphical models, with each requiring its own basic mathematical theory to guide sound statistical inference. Developing this theory for a family of graphical models requires solutions to fundamental questions, where emerging mathematical approaches are utilizing what can collectively be called algebraic implicitization techniques. These notes provide an introduction to the basics of graphical models and the algebraic implicitization techniques now appearing in instances of the graphical models program.
Read the full abstract

2608.27636 · 27 Aug · This is a preprint of a chapter intended for publication in a Springer Lecture Notes in Mathematics series based on the lectures given at the 43rd Finnish Summer School on Probability and Statistics in Lammi, Finland. It surveys basic results on graphical models and algebraic techniques, with the intent of being broadly accessible

Machine Learning (Statistics) · Cross-listed from cs.LG

Curvature-Aware Radius Shrinkage for Adaptive Nearest Neighbor Classification

Nearest neighbor classification relies fundamentally on how locality is defined, yet conventional $k$-NN imposes the same neighborhood cardinality throughout the feature space. This assumption can be inadequate for data whose local geometry varies substantially across the underlying manifold. We introduce Curvature-Aware Radius Shrinkage for Adaptive Nearest Neighbor Classification (CARSANN), a geometry-driven framework that adapts the spatial support of each neighborhood according to local geometric complexity. CARSANN first estimates intrinsic dimensionality using TwoNN and constructs an intrinsic representation through principal component analysis. Local mean curvature is then estimated using a shape-operator-based formulation and controls neighborhood scale: highly curved regions receive stronger radius shrinkage, whereas approximately flat regions retain broader spatial support. Unlike methods that modify only the number of neighbors or the local metric, CARSANN explicitly adapts the spatial extent of local evidence. Experiments on more than 70 real-world OpenML datasets show that CARSANN consistently improves upon standard $k$-NN and is competitive with adaptive nearest-neighbor methods. In a controlled comparison using the same base neighborhood size, CARSANN achieves higher balanced accuracy on 40 of 45 datasets, increasing mean balanced accuracy from 0.6506 to 0.7528. The advantage also persists against $k$-NN with fixed $k=5$. Friedman and Nemenyi tests confirm that the improvements are statistically significant. These results indicate that local manifold curvature can serve as an effective geometric control variable for adapting neighborhood support, providing a complementary paradigm to cardinality-based nearest-neighbor adaptation.
Read the full abstract

Statistical Methodology

Robust model-based clustering via mixtures of multivariate pseudo-Voigt distributions

We propose a multivariate extension of the pseudo-Voigt profile-a weighted convex combination of Gaussian and Cauchy distributions-within a finite mixture modeling framework for robust model-based clustering and outlier detection. To ensure parsimony and coherence within clusters, shared location and scale parameters are imposed between the Gaussian and Cauchy components. Parameter estimation is carried out via an Expectation Maximization algorithm, with latent variables facilitating efficient likelihood-based inference. The performance of the proposed model is evaluated through simulation studies and applications to real-world data. Comparisons with established robust models, including mixtures of contaminated normal distributions, are provided to illustrate the model's clustering accuracy and outlier detection capabilities. The framework is shown to be particularly effective for data characterized by heavy-tailed behavior.
Read the full abstract

Statistical Methodology

Activity-Conditioned Residual Association from Aggregated Relational Data

Aggregated relational data (ARD) record how many ties sampled respondents have to prespecified groups without revealing individual dyads. We develop a conditional test for prespecified cross-group concordance beyond an additive-activity network model. When the groups form an exhaustive partition, respondent degree is observed exactly. For arbitrary fixed activity values under an independent-Bernoulli additive-logit null, conditioning two respondents on equal degree yields a finite nonpositive sign restriction for their two disjoint group-count differences. Analyst-randomized pair thinning gives a conservative baseline. Under a stronger fixed-cell design with group-specific smooth activity profiles and positive cross-role overlap, an observable clipped full-pair statistic gives conservative one-network inference with graph-independent sampled respondents. Finite and growing matched-law constructions establish that the target contains bivariate information absent from a collapsed binary count. The full-pair procedure is uniformly consistent on a primitive relative-open neighborhood of a specified rank-one alternative. Fixed-dimensional lattice local limits make the conditioning and availability requirements explicit.
Read the full abstract

Machine Learning (Statistics)

Towards a mathematical theory of superposition

We develop a mathematical theory of superposition in neural networks using tools from frame theory and compressed sensing. In our model, a sparse binary vector \(x\) of active features is encoded through an overcomplete dictionary \(W\), and feature recovery is performed by applying \(\operatorname{ReLU}(W^\top W x+b)\) with an appropriate bias vector \(b\). We prove several recovery theorems for this model. In the random-support setting, we establish high-probability support recovery for nearly tight, low-coherence dictionaries, with guarantees when the expected sparsity is up to order \(d/\log n\). In the worst-case support setting, we give a sharp and computable criterion for which sparsity levels permit support recovery. We apply this criterion to Gaussian random matrices and equiangular tight frames. For real equiangular tight frames with \(n>d+1\), we determine the exact recovery threshold in terms of the coherence. The proof of this result for real equiangular tight frames relies on a novel characterization—which should be of independent interest to frame theorists—of the distribution of signs in the Gram matrix.
Read the full abstract

All authors Michael I. Ivanitskiy, John Jasper, Emily J. King, Dustin G. Mixon

Statistical Methodology

Bivariate geostatistical latent variable models for the analysis of antibody density data

The increasing availability of serosurveys that measure antibody responses to multiple antigens requires the development of methods that can exploit the full information content of such data, both biological and spatial. However, the non-Gaussian and potentially multimodal distributional behaviour of antibody responses makes the development of such methods inherently complex, especially in a multivariate setting. Here, we extend the latent variable framework of Giorgi and Wallin (2026), in which continuous antibody concentrations are modelled through an individual-level latent seroreactivity process that represents the level of immune activation to a given antigen. We focus primarily on the bivariate setting and set out a series of guiding principles that justify the resulting joint modelling structure. The proposed model captures distinct sources of correlation between antibody responses, arising both from shared exposure to the same environment and from biological processes occurring within the same host. Spatial dependence is introduced through a novel bivariate Matérn random field, which we use to construct a parsimonious class of cross-covariance functions between antigen-specific spatial processes. We illustrate the application of the framework to analyse data on bivariate antibody measurements from a malaria serosurvey in the Kenyan highlands. Results from the application and a simulation study show that ignoring this correlation substantially degrades inference on joint properties of the antibody distributions and on individual-level seroreactivity, but matters less when interest lies exclusively in each antibody's marginal distribution. Finally, we discuss how the framework could be extended to settings with more than two antigens, and highlight the modelling challenges that arise as the number of antigens grows.
Read the full abstract

Machine Learning (Statistics)

Optimal Transport for Network Comparison: A Unified Review with New Spectral Bounds and Machine Learning Applications

Network comparison using optimal transport is a growing area of research in network science. Unlike standard graph metrics, optimal transport computes both network dissimilarity and a transport plan that explains how one graph morphs into another. In this paper, we review how optimal transport compares undirected, unweighted simple graphs using three primary distances: the Wasserstein, Gromov-Wasserstein, and Bures-Wasserstein distances. We examine the closed form of the Wasserstein distance in one dimension via node feature probability distributions, and show how the transport plans of the Wasserstein and Gromov-Wasserstein distances visualize how mass is shifted to transform one network into another. Beyond reviewing existing transport-based approaches, we establish new spectral lower and upper bounds for the Bures-Wasserstein distance and characterize the tightness of the lower bound under eigenbasis perturbations. Finally, we evaluate these distances using a synthetic network dataset for clustering and a real-world temporal network.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Universality and sharp thresholds for ellipsoid fitting

We establish a sharp phase transition for fitting random vectors by an ellipsoid. The random vectors have independent subgaussian coordinates with mean zero, variance one, and a common fourth moment, and the number of vectors is proportional to the square of the dimension. We identify an explicit satisfiability threshold such that, with high probability, a positive definite ellipsoid passes through every data point below the threshold, whereas no positive semidefinite fit exists above it. We also determine the optimal squared fitting error throughout the unsatisfiable regime. In particular, the threshold depends on the coordinate distributions only through their common fourth moment, revealing a fourth moment universality phenomenon. For standard Gaussian data the threshold is $1/4$, resolving the ellipsoid fitting conjecture.
Read the full abstract

Statistical Computation

Deep-Control BSDE: Layerwise Brownian-Weighted Regression for High-Dimensional Semilinear PDEs

High-dimensional semilinear parabolic partial differential equations arise in stochastic control, financial engineering, and uncertainty quantification, but classical spatial discretizations suffer from the curse of dimensionality. Motivated by Gaussian perturbation and conditional regression in denoising score matching, we propose Deep-Control BSDE (\DCBSDE{}), a layerwise control-regression method for Markovian backward stochastic differential equations. From an implicit Euler scheme, we identify the discrete ideal control $z_n^\pi=\mathcal M_nu_{n+1}^\pi$ as the conditional projection coefficient of the successor value response onto the one-step Brownian increment. At each time level, the method freezes the successor value function, approximates the resulting Brownian conditional-moment target using finitely many branches, regresses the control, and then fits the value through the implicit BSDE relation. A frozen linear-response baseline and antithetic pairing reduce finite-branch fluctuations while preserving the conditional target, and a within-layer correction coordinates the value and control approximations. We establish backward stability through a contraction property of the Brownian projection and derive conditional consistency when discretization, local learning, numerical, and finite-branch errors vanish jointly. Experiments across six benchmarks demonstrate that \DCBSDE{} achieves a favorable overall balance among value accuracy, control accuracy, and dynamic consistency while exhibiting generally stable performance across random seeds. These results support the use of \DCBSDE{} in applications requiring reliable joint approximation of the value and control processes.
Read the full abstract

Statistics Applications

How exceptional was the Big Three era? Extremes and persistence in men's professional tennis

Three players won 66 of the 81 Grand Slam titles contested between 2003 and 2023, and their era is widely held to be the most dominant in the history of tennis. Assessing it means comparing players who never met, so that every comparison passes through the opponents each did face. We measure dominance by how far a player stands above the field of his own day, how many stand there at once, and how long they remain, the last through averages over fixed windows rather than through unbroken runs, which are unstable when the strengths beneath them are themselves estimated. Strengths come from a Bayesian dynamic Bradley-Terry state space model, and dominance is characterized through exceedances of a high threshold. Applied to 197,926 matches played between 1968 and 2025, only Djokovic separates from the field on peak strength, Federer and Nadal being indistinguishable from Borg, McEnroe and Lendl. What distinguishes the recent period is not how high its best players stood but how long three of them stood there together. At its height three players held a top-three position in at least four fifths of all weeks, where in the strongest comparable decade of the past those positions were shared among four. The sharper anomaly lies between: for twelve years from 1990 the upper tail was not merely unoccupied but compressed.
Read the full abstract

Statistical Methodology

Evidence, Calibration, and Stability: A Triadic Framework for Hypothesis Testing Under Model Uncertainty

Statistical tests are often asked to do too much. A single reported result is expected to describe what the observed data say, reassure readers about repeated-sampling behavior, and remain convincing when the working model is perturbed. Those tasks are connected, but they are not equivalent. Fisherian inductive inference and Neyman-Pearson decision theory clarify the first two; robust testing, sensitivity analysis, fragility measures, multiverse analysis, and distributional-stability methods speak to the third. I propose Evidence-Calibration-Stability (ECS) as a framework for keeping these roles separate while reporting them together. Evidence is post-data. Calibration belongs to the design or procedure. Stability is the post-data distance from the benchmark analysis to a conclusion-reversing perturbation within a declared model neighborhood. Full ECS support is conjunctive: a strong coordinate cannot rescue a failed one. For finite-dimensional affine perturbations, I derive an exact ellipsoidal stability radius. For smooth nonlinear margins, a uniform quadratic-remainder condition yields a certified lower bound over a declared neighborhood, showing when the affine formula is only a surrogate. I also establish coordinate invariance and a matrix extension for multiple claims, and distinguish confirmatory calibration from descriptive calibration profiles when prespecification is unavailable. Simulations for the one-sample t test and Student's historical sleep data show that the three coordinates can lead to different interpretations. ECS is a formal synthesis, not a claim that evidence, power, or robustness is itself new.
Read the full abstract

Machine Learning (Statistics)

A Finite-Sample Analysis of Quantile Temporal-Difference Learning

Quantile temporal-difference learning (QTD) is an effective method for learning return distributions through quantile approximation, yet its finite-time behavior remains poorly understood. Its update is nonlinear and nonsmooth, and the stability needed for a sharp convergence rate holds only near the target. We establish a global high-probability last-iterate guarantee for synchronous tabular QTD under general positive, nonincreasing step-size sequences and arbitrary initialization in the natural parameter range. For polynomially decaying step sizes with exponent $a\in(0,1)$, the last iterate converges to the target at rate $T^{-a/2}$ in the infinity norm, up to logarithmic and lower-order terms. A suitably tuned harmonic schedule recovers the $T^{-1/2}$ statistical rate up to logarithmic factors. For the $m$-quantile representation, its $\infty$-Wasserstein error scales as $\sqrt{m/T}$ up to logarithmic factors, matching the leading polynomial dependence on the quantile resolution and sample size of the corresponding model-based estimator. The proof uses a two-stage global-to-local argument. From arbitrary initialization, Bellman contraction and CDF monotonicity first bring the iterate close to the target, after which, a novel variance–drift matching argument sharpens the control of accumulated noise and local contraction reduces the remaining errors, yielding the sharp rate. Simulations verify the predicted polynomial decay and assess the finite-time entrance bound.
Read the full abstract

All authors Zijie Cheng, Xiang Li, Yang Peng, Zhihua Zhang

Statistics Theory

Conformal Prediction Through the Lens of Hypothesis Testing: Universality, Impossibility, and Optimality

The connections between conformal prediction and permutation tests are already widely-known in the literature. Some authors motivate conformal prediction by saying that it computes a permutation p-value for the hypothesis $H_0 : Y_{n+1} = y$, and then inverts this to form a prediction set for $Y_{n+1}$ (i.e., accepts all values $y$ into the prediction set for which the p-value is large). In this paper, we examine an alternative view, which is less well-known: we again cast conformal prediction via the inversion of a permutation test, but for the null of exchangeability of the joint distribution of the $n+1$ samples. This change in perspective, while simple, adheres more closely to traditional formalization in hypothesis testing, which offers several benefits. First, we use the duality between conformal sets and testing to show that foundational universality and impossibility results in the conformal prediction literature can be reproduced directly using classical hypothesis testing theory (due to Neyman, Lehmann, Scheffé, Kraft, Le Cam, and others). Furthermore, we show that an optimality result for conformal prediction can be derived using standard Neyman-Pearson theory: for any joint distribution of the covariates and response $X,Y$, and any sample size, the optimal method for prediction sets—delivering the most efficient set among all methods with valid coverage for exchangeable distributions—is a conformal predictor whose score is the inverse conditional density of $Y|X$.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Local Laws and Edge Universality for Noncentral Sample Covariance Matrices

We consider the real noncentral sample covariance matrices $\mathcal{W}=YY^\top$ with $Y=A+\Sigma^{1/2}X$. Here $A\in\mathbb{R}^{M\times N}$ is deterministic, $\Sigma\in \mathbb{R}^{M\times M}$ is a deterministic positive definite population covariance matrix and $X\in\mathbb{R}^{M\times N}$ has independent centered entries with variance $N^{-1}$. We prove local laws near regular right edges down to optimal spectral scales without requiring the commutativity of $AA^\top$ and $\Sigma$. As a consequence, we obtain optimal eigenvalue rigidity at the rightmost regular edge and delocalization of the corresponding left and right singular vectors. We also show that, with high probability, there are no eigenvalues in the adjacent spectral gap beyond the optimal $N^{-2/3}$ edge scale, up to an arbitrarily small $N^\varepsilon$ loss. Finally, we establish edge universality at the rightmost regular edge: after centering and scaling, the largest eigenvalue converges to the Tracy–Widom distribution. The main technical ingredient is a stability analysis of the matrix Dyson equation (MDE) associated with the linearization of $Y$, whose self-energy operator does not satisfy the flatness condition of the general MDE theory. Exploiting the special block structure, we reduce the stability analysis exactly to a two-dimensional operator. This reduction yields regularity of the spectral density and square-root behavior at regular right edges, together with sharp stability bounds near such edges.
Read the full abstract

Machine Learning (Statistics)

Recovering Expert Critic-Sourced Network Adjacency between Musical Artists from Acoustic Distributions: A Construct-Validity Approach

Music recommendation relies primarily on two signals: user-item interactions, which fail in the cold-start regime, and intrinsic musical content, available for any recording. We argue that a third, largely untapped signal is both richer and more principled: critical adjacency, the pairwise relation established when an expert critic explicitly links two artists in long-form prose. It encodes deliberate judgments about which artists belong together. Prior work established its internal validity, showing it recovers coherent, interpretable communities and can match collaborative filtering in user-satisfaction simulations, with no user data. What has been missing is external validation: whether this critic-sourced relation is grounded in the music itself versus sociological context. We test it against acoustic content, reframing the question as one of construct validity. Representing artists as empirical distributions over 80 low-level Essentia acoustic descriptors and modeling pairwise proximity via marginal optimal-transport (Wasserstein) distances, we evaluate how far critical adjacency is sonically recoverable under a cold-start, artist-disjoint split. Our ensemble recovers these edges at out-of-sample AUC of 0.767 (95% CI 0.761-0.775). Recoverability rises monotonically with critical consensus, reaching 0.865 on multi-source attested edges. Stratified evaluations align with sociological models of genre: tightly bounded, scene-based genres show higher recoverability than broad industry umbrella terms. Critical discourse is thus a rich source of information for recommendation, decomposing into a reproducible "sonic core" and a "sociological remainder" driven by narrative positioning, subcultural context, and canonical placement. The work offers both a scalable cold-start discovery mechanism and a sociologically grounded approach to MIR and MRS research.
Read the full abstract

Statistical Methodology

Combining covariate adjustment with information from secondary endpoints to improve precision in randomized trials

Background/Aims: Adjustment for prognostic baseline covariates can improve precision in randomized trials. Previous work has shown that jointly modeling primary and secondary endpoints can yield additional precision by borrowing information across endpoints. We investigated whether these approaches can be combined to achieve efficiency gains beyond those obtained through covariate adjustment alone. Methods: We extended a previously proposed one-factor structural equation modeling framework for borrowing information from secondary endpoints to incorporate baseline covariates while retaining the average treatment effect on the primary endpoint as the estimand. To mitigate sensitivity to model misspecification, we combined this estimator with a conventional covariate-adjusted estimator using cross-validated model averaging. We evaluated operating characteristics in simulations and applied the methods to a randomized trial of very low versus normal nicotine content cigarettes. Results: When the structural equation model was correctly specified, endpoint borrowing improved efficiency beyond conventional covariate adjustment across all simulated settings. Model misspecification could induce bias and undercoverage. Model averaging reduced bias and improved coverage relative to the structural equation model estimator, although coverage remained imperfect under severe misspecification. In the trial application, the model-averaged estimate was 21% more precise than the unadjusted estimate and 13% more precise than covariate adjustment alone. Conclusion: Secondary endpoints can contribute meaningful information about the average treatment effect on a primary endpoint even after baseline covariates have been incorporated. These gains require stronger assumptions than conventional covariate adjustment; model averaging provides a practical compromise between efficiency and robustness.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from q-fin.RM

On the approximation of posterior laws in compound loss models by conditional Wasserstein GANs

Bayesian inference in compound loss models must often be repeated across policies, market scenarios, and prior specifications. Outside conjugate cases, this may require repeated numerical integration or Markov chain Monte Carlo (MCMC). We formulate this problem as amortized posterior approximation and construct a conditional Wasserstein generative adversarial network conditioned on sufficient statistics, prior mean and coefficient of variation, and mixture weights of prior families. Notably, a single shared generator is able to approximate the posterior laws of both the Poisson intensity and the Pareto shape parameter under mixtures of Gamma, inverse-Gaussian, and lognormal priors. We assess the approximation by simulation-based calibration and by comparisons with analytical posteriors, deterministic quadrature, and extensive MCMC simulations. In an application to data on extreme natural catastrophe losses, we produce rolling one-year posterior predictive distributions, and examine the effects of heavy-tailed severity and prior-family uncertainty on aggregate tail risk.
Read the full abstract

Statistics Theory

Connecting Riemannian Geometry and Statistical Inference for Correlation Matrices

The quotient-affine metric gives an intrinsic Riemannian geometry to full-rank correlation matrices, but its geodesic distance has no closed form and we are not aware of an analytic asymptotic null distribution for it. We connect this geometry, introduced in 2019, with Jennrich's 1970 asymptotic test for equality of correlation matrices. The quadratic form underlying Jennrich's statistic is exactly one half of the quotient-affine metric tensor. The identity arises because eliminating marginal standard deviations from Gaussian Fisher information performs the same projection as quotienting out diagonal rescalings. Jennrich's statistic therefore evaluates the local quotient-affine quadratic form directly. Moreover, for two independent Gaussian samples with a common population correlation matrix, the squared geodesic distance, scaled by effective sample size, converges in distribution to $4\chi^2_d$, where $d = p(p-1)/2$. For $p=2$, the result reduces to the two-sample Fisher $z$ test.
Read the full abstract

Statistics Applications

A spatio-temporal block aggregation model for latent log Gaussian outcomes: application on modelling wastewater virus concentration in Wales

Wastewater-based epidemiology has emerged as a valuable tool for monitoring community-level infectious disease dynamics, providing population-wide signals that complement clinical surveillance. However, wastewater measurements are often observed as aggregated values over irregular spatial units. This work develops an approach to link an underlying spatially continuous processes and an aggregated outcome. We propose a spatio-temporal model for latent log-Gaussian outcomes that provides a coherent framework for inference and prediction, allowing the process to be integrated over arbitrary spatial configurations. This framework can also be used for subsequent analyses, such as linking wastewater signal to health outcomes at administrative areas. We use a Bayesian framework for inference via the linearised integrated nested Laplace approximation (INLA) approach. We apply the proposed methodology to model SARS-CoV-2 N1 gene copies in wastewater across 47 catchment areas in Wales from the beginning of August 2022 to the end of July 2023. The results demonstrate that the model captures spatial and temporal patterns and has good predictive performance. Results also show that estimated viral gene copies are strongly linked to positivity rates from COVID-19 PCR tests at the local authority level. Our findings highlight the importance of explicitly modelling block aggregation when analysing wastewater surveillance data. The proposed framework provides a flexible and principled approach for integrating environmental surveillance data into public health monitoring systems.
Read the full abstract

All authors Stephen Jun Villejo, Peter Diggle, Guangquan Li, Ella White, Matthew Wade, Christopher Williams, Davey L. Jones, Alisha Davies, Marta Blangiardo

Statistical Methodology

Personalized Federated Learning for Tensor Regression

The growing availability of tensor-valued data across multiple institutions creates opportunities for collaborative analysis, but also raises challenges related to data privacy, high dimensionality, and client heterogeneity. This paper introduces a personalized federated tensor regression framework that addresses all three simultaneously. Each client's coefficient tensor is decomposed into a globally shared low-Tucker-rank component and a locally sparse deviation, estimated via a two-stage privacy-preserving procedure. We establish finite-sample upper bounds and minimax lower bounds that quantify the privacy-accuracy trade-off, and prove the consistency of the supporting initialization and rank-selection steps. Simulation studies confirm that the federated approach improves estimation and prediction over purely local methods, especially when per-client data are scarce, and an MRI-based ADHD study illustrates its strong performance under real privacy constraints.
Read the full abstract

Statistical Methodology · Cross-listed from math.PR

An approximate zero bias transformation for random sums: Applications to sampling with outliers, auto insurance, and generative AI

We develop $L^1$ bounds for the difference between a test function of a random sum and a standard normal random variable, where the summands are assumed to be independent but not necessarily identically distributed. The bounds are obtained through a new version of the approximate zero bias transformation specifically developed for random sums. Although the identical distribution assumption is relaxed, the bounds are of order $1/\sqrt{n}$, matching the order of existing bounds in the literature under the same distributional assumption on the number of summands. The main results are then applied to three real-world settings: random sums obtained from simple random sampling with outliers, total insurance claims, and generative AI response times.
Read the full abstract

Statistical Methodology

Graph-based causal variance decompositions: When "variance explained" means causation

Recursive application of the law of total variance decomposes the marginal variance of an outcome into components attributed to explanatory variables and a residual component. The resulting decomposition depends on the chosen conditioning order, and its components do not in general have causal interpretations. We develop a graph-based framework for defining causal counterparts of ordered variance components and establishing their identification from observed data. Under topological orderings, identification can be assessed component by component against the full causal graph, without requiring the variables included in the decomposition to form a causally sufficient system. We also consider scientifically motivated departures from topological orderings, in which selected intermediate variables are conditioned on to obtain controlled-effect interpretations, motivated by the context of disparities in healthcare delivery. We relate the resulting estimands to causal attribution and variable-importance approaches in machine learning, and propose model-based plug-in estimators together with an approximate Bayesian procedure for uncertainty quantification. A simulation study examines finite-sample performance and sensitivity to outcome-model misspecification and flexible machine-learning estimation.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

TRACE-CRC: Trajectory-Adaptive Conformal Risk Control for Multi-Step Channel State Information Prediction

Reliable prediction of time-varying channel state information (CSI) is essential for efficient wireless communication. Each CSI frame is a matrix-valued representation of the wireless channel response, and a sequence of CSI frames forms a temporal channel trajectory. Modern deep learning-based CSI predictors, however, often provide only point predictions and lack calibrated uncertainty estimates. This limitation is particularly problematic in multi-step CSI prediction, where the target is a sequence of future CSI matrices, and downstream decisions such as beamforming or scheduling may fail if any part of the predicted trajectory is unreliable. We propose trajectory-adaptive calibration and error profiling with conformal risk control (TRACE-CRC), a method for trajectory-aware uncertainty quantification in multi-step CSI prediction. TRACE-CRC constructs Frobenius-norm uncertainty balls around predicted CSI matrices and controls the risk that at least one future frame is uncovered. Instead of calibrating each future step independently, TRACE-CRC combines future-step-dependent error profiling, trajectory difficulty stratification, and learn-then-test (LTT) risk control. Empirically, TRACE-CRC achieves reliable trajectory-level coverage with substantially smaller uncertainty balls than conservative multi-step corrections, while avoiding the trajectory undercoverage of compact stepwise and adaptive conformal baselines.
Read the full abstract

All authors Kiarash Rezaei, Mehdi Sattari, Javad Aliakbari, Tommy Svensson, Paolo Monti, Carlos Natalino

Statistics Applications

The Wasserstein Distance for Mortality Comparisons: Absolute versus Net Differences in Survival

It is well known that the gap in life expectancy at birth can be seen as the net difference between two survivorship functions. When calculating the absolute difference between the two survivorship functions instead, we derive a distributional inequality measure which is called the Wasserstein distance. The measure quantifies how far apart two probability distributions are by the minimal cost of transporting one into the other. This paper relates the Wasserstein distance to the difference in life expectancy at birth. Both measures correspond to each other, whenever the survivorship functions do not cross, i.e., the net difference equals the absolute difference in the no crossing case. Whether the curves cross is driven by the accumulated difference in age-specific death rates. Crossing occurs in situations where an accumulated survival advantage for given populations is reversed because elevated death rates at later ages outweigh the earlier survival advantage. These cases are particularly interesting because the net difference in survival (or the life expectancy gap) may be small even though the two populations show very different mortality schedules. In our empirical analysis, we search for these cases using life table data from Human Mortality Database life tables for the period 1990 to 2020. Across 69 188 population pairs, survivorship functions cross in 59.8 % of comparisons. Yet, the magnitude of the reversal in survivorship is usually very small. We therefore conclude that in most cases, the gap in life expectancy at birth is not only a comparison of means but quantifies the overall difference between the two mortality regimes
Read the full abstract

Statistics Applications

Modeling Spatially Obfuscated Street-Crime Data using Log-Gaussian Cox Processes on Metric Graphs

We develop a log-Gaussian Cox process framework for modelling street-level crime data observed on a road network when the released event locations are spatially obfuscated. Motivated by UK Police street-level crime data, where published coordinates are anonymised proxy locations rather than exact event locations, we address the resulting support mismatch by representing each observation through an aggregated support on the street network. The latent log-intensity is modelled as a Whittle–Matérn Gaussian field defined on a metric graph through an SPDE representation, allowing the crime intensity to vary continuously along streets while respecting the geometry of the road network. We compare the proposed metric-graph aggregated model with two alternatives: a planar point model that treats the released locations as exact points, and a planar aggregated model that accounts for spatial aggregation but ignores the network support. In a simulation study where data are generated on a street network, the metric-graph model provides more accurate parameter recovery and better overall fit than the planar alternative. In the City of London application, the metric-graph model also achieves the best fit across several crime types, including theft from the person, robbery, drugs, and bicycle theft. The results further suggest that the relationship between environmental amenities and crime risk varies by crime type, with supermarkets showing the most consistent positive associations. The proposed framework provides a principled approach for analysing network-constrained spatial event data with privacy-protected and imprecise locations.
Read the full abstract

Machine Learning (Statistics)

Active Diffusion-Based Inference for Ill-Posed Inverse Problems under Incomplete Priors

Many scientific and engineering applications require estimating unknown parameters from experimentally observable data – an inverse problem that is inherently challenging due to nonlinearity, noise, and ill-posedness. In this paper, we propose an active diffusion-based inverse problem solver. A DM is trained to learn the mapping between the parameter space and the observable space. By iteratively detecting and correcting model misspecification through posterior uncertainty, the method discovers and learns the correct region of parameter space, even when initial training bounds exclude the true parameters. This provides a principled, Bayesian justification for adaptive domain augmentation and ensures robust inference for inverse problems under incomplete prior knowledge. We demonstrate the effectiveness of our inverse solver for a toy inverse problem with infinite solutions, and for the parameterization of the quantum correlation functions to event observables in a Quantum Chromodynamics analysis of nucleon structure.
Read the full abstract

Statistical Methodology

An Accurate and Single-Communication Federated Inference Algorithm

Joint analyses across multiple institutions are increasingly important in biomedical and epidemiological research, particularly for rare diseases where datasets are typical small. However, privacy regulations and institutional policies often prevent the sharing of individual-level patient data. In this paper we present an accurate and single-communication federated inference algorithm. Single-communication federated inference enables statistical analyses through a single exchange of summary statistics between participating centers and a coordinating server, preserving privacy while reducing communication and computational costs compared with iterative federated learning. We extend a recently proposed single-communication federated inference strategy that is based on second-order Taylor expansions by using third-order expansions to better approximate local log-likelihood functions. The proposed method is evaluated through simulation studies based on real data and compared with existing federated inference strategies. The simulation studies assess the performance of the proposed method, with a particular focus on scenarios involving small local sample sizes, where quadratic approximations may fail to capture skewness and other higher-order characteristics of the log-likelihood function. They demonstrate that incorporating higher-order information of the log-likelihood function improves the accuracy while preserving the privacy, communication efficiency, and scalability required for collaborative biomedical and epidemiological research.
Read the full abstract