Statistical Methodology
Dynamic Spatial Bayesian Machine Learning Model: Applications to Intergenerational Economic Mobility and Geographic Income Inequality in the United States
Hammed A. Olayinka and Saheed O. Olayemi
We develop a Dynamic Spatial Panel
Bayesian Additive Regression Trees model with Horseshoe shrinkage (DSP-BART-HS) for high-dimensional spatio-temporal panel data. We jointly evaluate the model against a comprehensive suite of structural spatial econometrics, non-parametric machine learning methods, and small-area
estimators across nine data-generating scenarios spanning 200 replicates each, including irregular spatial
topologies, dense policy effects, and non-linear individual-level interactions. DSP-BART-HS is the best or statistically indistinguishable from the best
estimator in every scenario. Conventional region-time-aggregate comparators suffer severe performance degradation – trailing by a factor of three or more – whenever individual-level non-linearity drives outcome variance, a limitation this framework's tree-ensemble design directly addresses. The model also maintains strong predictive accuracy under a zero-training-region spatial holdout via its spatial
diffusion mechanism. We demonstrate practical utility across two U.S. county-level panel applications – intergenerational economic mobility and geographic income inequality – under genuine unseen-region, random, and temporal holdouts, with results validated through repeated-split uncertainty quantification and paired significance testing against every comparator. One clear limitation emerges: under temporal extrapolation specifically, a lighter per-region autoregressive specification (MTS-CAR-X) achieves a robust, and consistent advantage. DSP-BART-HS nonetheless establishes strong predictive performance, automated variable selection, and flexible inference for hierarchical spatio-temporal panel data.
Read the full abstract ▾
Statistics Applications
Kumaraswamy-Teissier Distribution: Bayesian and Non-Bayesian Inference, Regression Modeling and Climate Application
Kamana Mishra, Neeraj Poonia, Tanmay Kayal et al. (+1 more)
This study introduces the three-parameter Kumaraswamy–Teissier distribution (KTD) by applying the Kumaraswamy-G construction to the Teissier distribution. The proposed distribution provides additional shape flexibility for modelling positive-valued observations and accommodates different density and hazard-rate shapes. Several statistical properties, including moments, the moment generating function,
quantile function, and order statistics, are derived. Maximum likelihood, maximum product spacing, and
Bayesian methods are considered for parameter estimation, and simulation results demonstrate improving estimation accuracy with increasing sample size. A median-based regression model is further developed for the KTD, with its finite-sample performance examined through simulation. The practical utility of the proposed regression model is demonstrated through an application to monthly gridded rainfall data from the Northwest Himalayas, incorporating annual and semi-annual harmonic terms to capture seasonal variation. Its performance is assessed against Kumaraswamy and beta regression using residual-based goodness-of-fit diagnostics and likelihood-based model selection. The results demonstrate effectiveness of the proposed distribution for modelling heterogeneous and skewed positive-valued rainfall data.
Read the full abstract ▾
All authors Kamana Mishra, Neeraj Poonia, Tanmay Kayal, Sarita Azad
Statistics Theory · Cross-listed from cs.LG
Statistical Inference for Adversarial Training: Central Limit Theorems via Optimal Transport
Kim Jakwang and Kwon Dohyun
The purpose of this paper is to rigorously quantify the statistical and learning-theoretic properties of
adversarial training models for classification. Equivalently, we establish the statistical properties of empirical optimal partial transport. Precisely, first we provide two types of central limit
theorems (CLT): CLT centered at the expected empirical value, and CLT centered at the population one with smoothing. These results are based on the uniqueness of optimal potential for various equivalent optimal transport formulations, and the empirical process theory argument. For the binary setting, we indeed prove the uniqueness of optimal potential by leveraging the connection between optimal partial transport and the derived multi-marginal optimal transport formula. As byproducts, we also obtain the stability of a saddle point of the
adversarial training model, and the sample complexity and concentration probability of the generalization error.
Read the full abstract ▾
Statistical Methodology · Cross-listed from eess.SP
Robust Wave Origin Detection from Sensor Array Data via Randomized Hough Transform and Model Fitting
Sicheng Fan, Jiayi Lu and Xiaodan Fan
The robustness of methods for characterizing the origin of wave-like propagating signals is often challenged by the heterogeneity of transmitting media and sensor network. Detecting an individual wave under such noisy conditions remains particularly difficult. In this paper, we propose a robust framework to estimate wave origins from spatiotemporal data collected by a sensor array. First, a Butterworth filter is designed to extract relevant signals from each sensor channel. Next, a 3D randomized Hough transform is introduced to identify candidate signals originating from the same wave front. Finally, a wave propagation model is fitted to the identified signals using the least-squares method. By combining the iterative voting mechanism of the randomized Hough transform with least-squares optimization, our proposed method demonstrates superior robustness and computational efficiency in both simulation studies and real-world data analyses.
Read the full abstract ▾
Statistical Methodology
Flexible Transformations for Bayesian Score Calibration
Adam Bretherton, Joshua J. Bon, David J. Warne et al. (+2 more)
Modern statistical models are growing increasingly complex in an effort to realistically capture system dynamics. Using standard simulation-based inference, these models may be computationally prohibitive, necessitating the use of model calibration methods.
Bayesian score calibration is a computationally efficient framework for model calibration with strong theoretical guarantees. This framework learns an appropriate correction for an approximate model using a small number of simulations from the data-generating process. Currently, only a location-scale transformation has been explored, which may lack the flexibility to correct the complex error introduced by some approximate models. In this paper, we develop two flexible transformations for use in the
Bayesian score calibration framework. The first is a
polynomial extension, which can appropriately adjust approximate models with location-varying error. The second is a sequential application of
Bayesian score calibration, which can accommodate approximate models with posteriors that have low support for the true parameter values. We also discuss an additional diagnostic for use with this framework. We demonstrate the increased flexibility these two approaches provide over
Bayesian score calibration in two illustrative simulation studies.
Read the full abstract ▾
All authors Adam Bretherton, Joshua J. Bon, David J. Warne, David J. Nott, Christopher Drovandi
Statistics Applications
Estimating water levels in the High Plains Aquifer by synthesizing satellite data with groundwater well observations
Anis Pakrashi, Murali Haran and Shan Zuidema
The High Plains Aquifer (HPA) is a critical water resource in the Central United States, yet its depletion remains a major concern. While the
Gravity Recovery and
Climate Experiment (GRACE) satellite mission provides large-scale estimates of
liquid water equivalent thickness (LWET), its coarse spatial resolution (approx. 24 km) limits local inference. In contrast, well observations from the National Ground-Water Monitoring Network (NGWMN) offer valuable but spatially sparse measurements of depth-to-groundwater. In this paper, we develop a downscaling framework for the satellite data that integrates the two sources and
covariates using a
Bayesian hierarchical framework. Our model uses a latent Gaussian Markov Random Field (GMRF) that describes the groundwater storage at a high spatial resolution. To address computational issues, we use a basis representation approach specified via the Moran basis. We find that fine-scale
covariates like irrigation intensity and precipitation help refine spatial predictions. We thus provide, to our knowledge, the first statistically-rigorous approach for downscaling groundwater information based on GRACE satellite data and NGWMN groundwater measurements. Our approach yields high-resolution estimates of groundwater variations across the HPA from 2002–2022. The resulting fine-scale inference provides valuable insights into groundwater dynamics, highlighting the effects of land use and local extraction patterns.
Read the full abstract ▾
Statistics Theory
A complete characterization of sequential testability and change detectability in i.i.d. models
Aaditya Ramdas
We give a necessary and sufficient condition for the existence of power-one sequential tests in an i.i.d. composite testing problem. A level-\(\alpha\) test with power one against every alternative exists if and only if the alternatives are separated from the null by a countable family of finite-block events. We provide other equivalent conditions using randomized fixed-sample tests, bounded finite-block scores, e-processes, reduced-filtration test supermartingales, and a countable cover whose finite-block weak-$*$ closed convex hulls are positively separated in total variation. As a bonus, the constructive proof yields tests have pointwise expected sample size \(O_Q(\log(1/\alpha))\). Exactly the same conditions also characterize i.i.d.\ change detectability under optional-horizon average-run-length control: for every \(\eta>0\), they are equivalent to an alarm family \((T_\gamma)_{\gamma\ge1}\) satisfying \(\Prob_{P^\infty}(T_\gamma\le\sigma)\le \E_{P^\infty}\sigma/\gamma\) for every null law and every stopping time \(\sigma\). In fact, when these conditions hold, we can construct a single e-detector such that every null-law average run length lies between \(\gamma\) and \((1+\eta)\gamma+1\), and having robust Lorden delay \(O_Q(\log\gamma)\).
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CV
Gaussian Linear Functional Manifold Method for Massive Point Cloud Data
Hong Zhao, Tonglin Zhang, Baijian Yang et al. (+2 more)
Reconstructing continuous terrain
manifolds from massive, unstructured airborne
LiDAR point clouds remains challenging in complex Wildland-Urban Interface (WUI) environments, where deep neural networks require costly point-wise annotations and nonparametric surface reconstruction methods often lack structural interpretability. This paper introduces the Gaussian Linear Functional
Manifold (GLFM), a physics-informed statistical framework that represents continuous surface topography using deterministic linear functional bases while modeling microscale diffuse
laser backscatter as an isotropic Gaussian process. To avoid the quadratic computational cost of exact constrained maximum likelihood estimation, we develop an algebraic singular value decomposition (SVD) rank-reduction
algorithm that enables linear-time parameter estimation and closed-form quadric classification. Evaluated on 35.2 km^2 of real-world aerial
LiDAR data, GLFM automatically filters ground points and extracts morphological features, achieving an adjusted Rand index (ARI) of 0.9933 against field-verified ground truth and outperforming four leading baselines while maintaining an out-of-core
memory footprint. The framework provides a rigorous, interpretable, and scalable foundation for large-scale point cloud analytics.
Read the full abstract ▾
All authors Hong Zhao, Tonglin Zhang, Baijian Yang, Jin Wei-Kocsis, Songlin Fei
Statistical Methodology
Approximating Bayesian leave-one-group-out cross-validation
Anna Elisabeth Riha, Svenja Jedhoff, Paul-Christian Bürkner et al. (+1 more)
When data are grouped, hierarchical or multilevel models are commonly used to account for group-level variation with group-specific parameters. Leave-one-group-out cross-validation (LOGO-CV) is a suitable tool for evaluating predictive performance for new groups, providing an
estimator of the expected log predictive density (elpd). Brute-force LOGO-CV requires one model refit per held-out group, often using computationally expensive inference
algorithms such as MCMC. This is costly, particularly for large numbers of groups or complex model structures. Commonly used importance sampling approximations, intended to reduce this cost, tend to fail because the group-specific parameters of the held-out group must be integrated out. We identify two key challenges in LOGO-CV elpd estimation: approximating the LOGO posterior and computing the grouped marginal likelihood. We compare 11 strategies, including 5 newly proposed, to address them. Among others, we combine Pareto-smoothed importance sampling or adaptive importance sampling with integration techniques such as Laplace approximation, adaptive Gauss-Hermite quadrature, and bridge sampling. We evaluate these strategies in both simulation experiments and real-world case studies, which show that marginalising over the group-specific parameters substantially improves the reliability of the importance sampling approaches.
Read the full abstract ▾
All authors Anna Elisabeth Riha, Svenja Jedhoff, Paul-Christian Bürkner, Aki Vehtari
Machine Learning (Statistics) · Cross-listed from cs.LG
Connecting Score Matching, Maximum Likelihood, and Expectation-Maximization in Mixed Linear Regression
Zhankun Luo and Abolfazl Hashemi
We study variance-preserving
diffusion of the response in mixed linear regression (MLR) with unknown mixing weights. Our analysis separates the statistical guarantees of score matching from the loss geometry and optimization signal at a fixed
diffusion noise level. The KL divergence links the denoising score matching objective integrated over the
diffusion path with the likelihood and a terminal discrepancy. Under mild regularity conditions and terminal schedule, the resulting
estimator converges up to the ground truth parameters of MLR, and its scaled error converges to the Gaussian limit of the maximum-likelihood
estimator. At a fixed scale of the
diffusion noise level, we derive a decomposition linking the score matching loss to cross-
entropy and Expectation-Maximization (EM) operators. This decomposition yields an EM-related low-noise gradient expansion with additional correction terms of latent variance. In the high-noise limit, we further characterize gradient descent on this limiting loss under isotropic covariance. Along fixed high signal-to-noise ratio rays, the score matching imbalance gradient and the latent-variance term tend to zero pointwise. Numerical experiments illustrate our theoretical findings and statistical guarantees.
Read the full abstract ▾
Statistical Methodology
Nonparametric Hypothesis Testing of High-dimensional Clustering With Application to Single-cell RNA Data
Yifan Dai, Di Wu and Yufeng Liu
Single-
cell RNA sequencing studies routinely use clustering to define putative
cell types and
cell states, yet the observed separation may arise from sampling variability rather than genuine biological heterogeneity. This paper studies formal significance testing of such clustering structure in high-dimensional data. Existing SigClust methods assess clustering significance through Monte Carlo simulation under a Gaussian single-cluster null, but this assumption can be unreliable for normalized gene expression data and other non-Gaussian settings. We propose SigClust-LCP, a nonparametric extension that models a single cluster by a log-concave distribution. To make this approach computationally feasible in moderate to high dimensions, we develop a score-matching
estimator for log-concave projection inspired by recent generative modeling ideas. We establish theoretical guarantees for the
estimator and for its use in clustering significance testing. Simulations show that SigClust-LCP controls Type-I error more reliably than existing methods across a range of unimodal and mixture distributions while retaining competitive power. In a single-
cell RNA sequencing analysis of Hydra
cells, the method avoids spurious subclusters within annotated
cell populations and supports biologically meaningful separation across lineages and body-axis regions.
Read the full abstract ▾
Statistical Methodology
Adaptive Regularization via Extreme Value Distributions for Gaussian Graphical Models
Alexander P. Christensen, Jeongwon Choi, Haoyi Yang et al. (+1 more)
Edge selection in Gaussian graphical models is fundamentally a variable selection problem where pairwise relationships determine construct validity and variable importance in psychological networks. In psychology, network estimation relies predominantly on \(\ell_1\) regularization where uniform shrinkage systematically underestimates edge and centrality parameters. Alternative penalties overcome this bias but rely on fixed hyperparameters that do not adapt to the signal in the data. We develop a family of data-adaptive regularization penalties grounded in extreme value theory. Across 290 empirical psychological
datasets, we show that absolute partial correlations are well-described by the Weibull distribution. Using this empirical regularity, we derive Weibull, Gumbel, and Exponential penalties that approximate \(\ell_0\) penalization and calibrate their hyperparameters to each
dataset's noise floor. We formally prove the asymptotic properties of their static forms and conduct a large-scale simulation spanning two network
topologies and various sample sizes (\(N\) = 100–10,000), demonstrating that their adaptive forms maintain high specificity while accumulating sensitivity as sample size increases with low parameter bias and high rank-order centrality congruence relative to field standards. Empirically, method choice alone determined centrality rankings at sample sizes typical in psychology. Of the three adaptive penalties, Weibull is recommended given the interpretability of its parameters.
Read the full abstract ▾
All authors Alexander P. Christensen, Jeongwon Choi, Haoyi Yang, Lingzhou Xue
Statistical Methodology
Robust Community Detection for Noisy Networks with Covariates: Application to Functional Brain Networks
Zeyu Hu, Frederick H. Xu, Suprateek Kundu et al. (+4 more)
Community detection is fundamental to understanding the modular organization in functional
brain networks, yet noise in neuroimaging-derived networks and auxiliary node-level
covariates pose critical challenges. Existing methods typically either assume networks are noise-free or ignore
covariate information. We propose a
Bayesian framework for recovering a shared latent community structure from multiple noisy network realizations and auxiliary
covariates. The model combines a degree-corrected
stochastic block model for the latent network, a block-structured noise model linking noisy observations to latent edges, and a
covariate cluster model for node-level attributes. This specification allows anatomical or functional attributes of regions of interest to contribute information when network signals are weak or sparse. We develop an efficient Markov chain Monte Carlo
algorithm for posterior sampling and select the number of communities using the widely applicable information criterion, avoiding prior specification of this quantity. Simulation studies demonstrate improved community recovery relative to existing methods across varying noise levels,
covariate signal strengths, and numbers of noisy networks, with larger gains when network noise is moderate to high or only a small number of noisy networks is available. Applications to functional
brain networks from the Alzheimer's Disease Neuroimaging Initiative and the Human Connectome Project identify biologically interpretable structures and capture disease-related reorganization and individual-level variation.
Read the full abstract ▾
All authors Zeyu Hu, Frederick H. Xu, Suprateek Kundu, Yize Zhao, Li Shen, Jun Yan, Wenrui Li
Statistical Methodology
Bias-Corrected Subspace Intersection: Minimax-Optimal Shared Subspace Estimation in Multi-View Data
Xianwen Song, Yuepeng Yang and Cong Ma
Estimating a low-dimensional subspace shared across noisy data
matrices is a fundamental problem in multi-view
matrix estimation. We study this problem under the two-view JIVE model, where each data
matrix contains shared and view-specific low-rank components. We demonstrate that standard plug-in subspace intersection, including AJIVE, suffers from a second-order bias caused by direction-dependent leakage of the empirical singular vectors. We propose bias-corrected subspace intersection (BCSI), which removes this bias before estimating the shared subspace. We establish finite-sample risk bounds for BCSI that accommodate unequal view dimensions, signal strengths, and view-specific ranks and require no condition-number assumptions on the signal
matrices. When the shared and view-specific ranks are comparable, these bounds match our minimax lower bounds up to universal constants. The resulting minimax rate contains a new second-order term, arising from quadratic leakage perturbations relative to the shrinking spectral gap when the view-specific subspaces are nearly aligned. This term is absent from previous JIVE minimax lower bounds. Numerical experiments demonstrate the advantage of BCSI over AJIVE when the leakage bias is pronounced. Along the way, we establish a nonasymptotic concentration result for the bias-corrected leakage Gram
matrix of a rectangular spiked
matrix, which may be of independent interest.
Read the full abstract ▾
Statistics Applications · Cross-listed from physics.soc-ph
One equation allocates playing time in competitive team sports: the rulebook sets the arithmetic and the league sets the habit
Gustavo Pedro Ricou
Every team sport that lets a team replace players rations the replacements, and the ration settles who plays and for how long. Across eight codes, three in both sexes, we measure how playing time is allocated and find one equation behind it, complete in the one-way codes, its terms separating what the rulebook fixes from what a competition chooses. Under scarcity every league behaves alike: loaded starters are kept on at the same rate on four continents. Under freedom each expresses a preference, and the size of a league's departure is priced near one-for-one by what its own withdrawal minute prices (lambda = 0.87, 95% interval 0.54 to 1.21, twelve leagues), a law confirmed in four out-of-sample predictions and refused in one. The ordering of leagues by appetite persists across a rule change, and club wealth does not price it. Rules are the geometry of a competition; culture is its material.
Read the full abstract ▾
Statistics Theory
Finite-sample nonparametric mean tests: Leave-one-out duality and asymptotic optimality
Yifan Zhu and John C. Duchi
We study finite-sample valid tests of the one-sided mean hypothesis $H_0:\mu\leq 1$ against $H_1:\mu>1$ for nonnegative random variables. To do so, we develop a leave-one-out dual certificate framework, where certain pointwise inequalities imply p-value validity under the conditional mean null $\mathbb{E}[X_i\mid\mathbf{X}_{-i}]\leq 1$, and which also gives conditions that allow combining dual certificates for p-values to show that their pointwise minimum is also a valid p-value. The framework proves finite-sample validity of Wang and Zhao's nonparametric likelihood-ratio statistic $T_{\mathrm{nplr}}$, yields a new p-value $T_{\mathrm{bin}+}$ extending the Clopper–Pearson binomial test to general nonnegative random variables, and shows that the pointwise minimum $\min\{T_{\mathrm{nplr}},T_{\mathrm{bin}+}\}$ is itself a valid and more powerful p-value. We establish sharp optimality results for such testing problems in two regimes: both $T_{\mathrm{nplr}}$ and $T_{\mathrm{bin}+}$ attain a universal detectability boundary for the null $H_0$ without moment or tail assumptions, and $T_{\mathrm{bin}+}$ attains a nonparametric power lower bound under $n^{-1/2}$-local alternatives to $H_0$. Efficient
algorithms and numerical experiments demonstrate substantial finite-sample power gains over existing valid methods.
Read the full abstract ▾
Statistics Theory
Joint Distributions of Minimum and Maximum Angles on High-Dimensional Spheres
Yongcheng Qi and Lijian Yang
Consider $n$ independent random vectors sampled from uniform distribution on $(p-1)$-dimensional unit sphere. This paper investigates the limiting joint distribution for the minimum and the maximum values of their pairwise angles. It proves that the minimum and the maximum angles are asymptotically independent when both $n$ and $p$ tend to infinity, which solves an open problem raised in Cai, Fan and Jiang (2013) [Journal of Machine Learning Research 14, 1837-1864]. Cai, Fan and Jiang (2013) obtained the limiting marginal distributions for both the minimum and the maximum angles under assumption $\lim_{n\to\infty}{\ln n}/p=\beta$ according to whether $\beta=0$, $\beta\in (0,\infty)$, or $\beta=\infty$. This paper presents unified limits for both joint distributions and marginal distributions regardless of the relative divergence rate of $n$ and $p$. The paper also derives the limiting distributions for some statistics based on the minimum and the maximum angles,
Read the full abstract ▾
Statistics Theory
Subsampled Pseudo-posteriors for Scalable Bayesian Moment-condition Inference
Wenshuo Zhao and Rong Tang
Bayesian pseudo-posteriors based on moment conditions, such as
Bayesian empirical likelihood (EL) and
Bayesian exponentially tilted empirical likelihood (ETEL), provide a robust route to
Bayesian inference when the model is specified only through moment restrictions, but their computation is often prohibitive. In this work, we address two barriers to
Bayesian pseudo-inference under moment restrictions. First, scalable sampling is challenging for tall data because the pseudo-likelihoods are not log-additive across observations, so standard likelihood-subsampling methods do not apply. Second, in simulation-based and latent-variable problems, the moment function itself may be defined through an intractable expectation, making full-data pseudo-posterior inference infeasible. We develop a new family of subsampled pseudo-posteriors that replaces the full-data pseudo-posterior with an aggregate of mini-batch pseudo-posteriors. We show that naive mini-batching induces a shifted-mixture distortion, yielding aggregates that are correctly centered but overly diffuse. To remove this distortion, we introduce moment-function-level control variates that align mini-batch moment equations around a reference
estimator and recover the full-data posterior shape. For
Bayesian ETEL, we establish finite-sample total variation bounds to the full-data pseudo-posterior, allowing discontinuous and intractable moment functions. Numerical experiments show that the method applies broadly across moment-condition-based
Bayesian pseudo-inference, closely approximates full-data pseudo-posteriors, and substantially reduces computational cost.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Learning from VAE Errors to support ECG-based Differential Diagnosis of Myocardial Scar
Shayan Sharifi, Riccardo Treu, Ilaria Gandin et al. (+3 more)
Late Gadolinium Enhancement (LGE) on cardiac
magnetic resonance is a key marker of myocardial scar, but its limited accessibility motivates routine ECG-based screening. We evaluated whether $\beta$-variational autoencoder (VAE)-derived ECG representations can discriminate LGE+ from LGE- cardiomyopathic patients in a local cohort of 300 subjects. We compared 32-dimensional features from the foundation ECGx.AI model with those from a shallower $\beta$-VAE trained on normal PTB-XL ECGs, evaluating downstream classification and Dynamic Time Warping (DTW)-based reconstruction errors. ECGx.AI reached an area under ROC of 0.686 with Random Forest, while the proposed $\beta$-VAE reached 0.577 with sensitivity of 0.775 with Gradient Boosting. Notably, DTW-reconstruction errors significantly differed between classes in 10 out of 12 leads according to Mann-Whitney U test and help in classification, leading to an area under ROC of 0.643 with Logistic Regression, supporting their potential as markers of scar-related ECG alterations.
Read the full abstract ▾
All authors Shayan Sharifi, Riccardo Treu, Ilaria Gandin, Federico Garoia, Marco Merlo, Giulia Cisotto
Statistics Theory
Statistical reduction before the target is known: two boundary results
Rianne de Heide
Suppose that the eventual use of data is not known when the data are reduced or collected. This note considers two simple boundary cases. In a finite statistical experiment, a statistic preserves the Bayes risk for every finite later decision problem if and only if it is sufficient. Hence, when the minimal sufficient statistic is one-to-one, exact preservation of all later decision problems permits no nontrivial reduction. We then consider adaptive sampling from $m$ independent Gaussian streams when an external query specifies the coordinate to be classified only after sampling stops. Under coordinatewise error control, the optimal symmetric average sample size is exactly $m$ times the one-coordinate optimum. A change-of-measure argument gives the corresponding pointwise lower bound in terms of binary relative
entropy.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
An Exact Tail Condition for the Largest Error in Estimating a Rank-One Direction
Guilherme Vianna
We consider a rectangular random
matrix formed by adding a rank-one term to a
matrix with independent entries. The direction on the left side of that term is estimated by the leading left singular vector, and the error is multiplied by the square of the size of the added term. For entries with mean zero and variance one, we identify the exact tail condition for the following statement to hold for every deterministic sequence of directions: the largest error over all sizes of the added term converges to the same fixed value determined by the limiting ratio of rows to columns. This condition (weaker than the existence of fourth moments) requires that the tail probability, multiplied by the fourth power of the threshold, to tend to zero. If it fails, convergence to this value already fails when both directions are coordinate vectors, even at a size fixed before the
matrix is drawn. For regularly varying tails of order between two and four, the largest error tends to infinity.
Read the full abstract ▾
Statistics Applications
Symptom clusters in Long COVID in the UK: prospective community-based cohort study using unsupervised machine learning
Jasmine Aherne, Ines Henriques-Cadby and Thomas House
Long COVID is a condition usually defined by persisting symptoms following infection by the SARS-CoV-2 virus beyond the acute phase of infection. The condition has a significant impact on healthcare systems, the economy, and the individuals living with it. Due to the diverse and extensive symptomatology of long COVID, symptom co-occurrence tracking can be used to capture patient experiences and improve understanding, diagnosis, and management. Here, we leverage the UK Office for National Statistics (ONS) COVID-19 Infection Survey (CIS). The CIS was run between April 2020 and March 2023. On February 3, 2021, the ONS launched a new CIS question evaluating self-reported symptom persistence of 23 long COVID symptoms post self-reported COVID-19 infection. We use Jaccard symptom-by symptom distance
matrices, derived from the binary survey responses of the presence of each symptom. Three methods are then used to visualise symptom co-occurrence: heatmaps, non-metric multidimensional scaling, and agglomerative hierarchical clustering with complete linkage. We split our analysis into two parts, first looking at all long COVID survey responses (n = 207,319) and then looking at responses stratified by time-since-onset of long COVID up to 24 months (n = 30,224 at zero months-since-onset). We find higher symptom co-occurrence in prolonged long COVID. We also find clusters of neurological/systemic symptoms, gastrointestinal symptoms, and respiratory symptoms, with separability in symptom co-occurrence by organ system becoming less pronounced as time-since-onset of long COVID increases. Shortness of breath, weakness/tiredness, and muscle ache present as core symptoms long COVID. We demonstrate that the symptom experience of long COVID evolves from early stages to late stages with higher symptom burden and multi-systemic presentation. We also find three possible phenotypes of early long COVID.
Read the full abstract ▾
Machine Learning (Statistics)
PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders
Chloé Hashimoto-Cullen, Ghislain Agoua, Benjamin Guedj et al. (+1 more)
Forecasting time series accurately is critical for applications with complex data ranging from energy systems to healthcare and finance. Among current state of the art models, generative latent variable models are increasingly implemented; yet principled generalisation guarantees for modern latent variable models remain limited. In particular, while Variational AutoEncoders are widely used for sequential data, their theoretical analysis is largely restricted to i.i.d. settings. In this work, we develop a PAC-
Bayesian framework for latent variables models applied to time series. Building on reconstruction-based bounds, we extend PAC-
Bayesian guarantees to Markovian latent structures, capturing temporal dependencies through a sequential generative process. These guarantees do not grow with the length of the trajectory. Our bounds depend on assumptions which are common in the literature; we provide an example framework where they would be verified to show that they are not as restrictive as they may seem.
Read the full abstract ▾
All authors Chloé Hashimoto-Cullen, Ghislain Agoua, Benjamin Guedj, Sylvain Le Corff
Machine Learning (Statistics)
FluxDisco: Symbolic Regression for Stoichiometric Dynamical Systems via Monte Carlo Graph Search
Cassandra Durr, Alvaro Köhn-Luque, Chris Jewell et al. (+1 more)
Dynamical symbolic regression methods identify governing differential equations from noisy data, balancing interpretability and predictive accuracy. However, standard methods often produce expressions that violate known physical laws. To address this, we propose FluxDisco, a physics-informed framework tailored for flux-based, stoichiometric ODE systems. By leveraging a known stoichiometry, we reduce the expression search space and ensure physical adherence. Our framework adapts the Monte Carlo Graph Search
algorithm for the unique challenges associated with joint flux discovery of stoichiometric systems. We evaluate our method across a range of physical and biological systems, demonstrating its ability to accurately recover governing dynamics through interpretable equations.
Read the full abstract ▾
All authors Cassandra Durr, Alvaro Köhn-Luque, Chris Jewell, Lloyd A. C. Chapman