Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Friday 28 November

Machine Learning (Statistics) · Cross-listed from cs.LG

Provable Benefits of Sinusoidal Activation for Modular Addition

This paper studies the role of activation functions in learning modular addition with two-layer neural networks. We first establish a sharp expressivity gap: sine MLPs admit width-$2$ exact realizations for any fixed length $m$ and, with bias, width-$2$ exact realizations uniformly over all lengths. In contrast, the width of ReLU networks must scale linearly with $m$ to interpolate, and they cannot simultaneously fit two lengths with different residues modulo $p$. We then provide a novel Natarajan-dimension generalization bound for sine networks, yielding nearly optimal sample complexity $\widetilde{\mathcal{O}}(p)$ for ERM over constant-width sine networks. We also derive width-independent, margin-based generalization for sine networks in the overparametrized regime and validate it. Empirically, sine networks generalize consistently better than ReLU networks across regimes and exhibit strong length extrapolation.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Accelerated Execution of Bayesian Neural Networks using a Single Probabilistic Forward Pass and Code Generation

Machine learning models perform well across domains such as diagnostics, weather forecasting, NLP, and autonomous driving, but their limited uncertainty handling restricts use in safety-critical settings. Traditional neural networks often fail to detect out-of-domain (OOD) data and may output confident yet incorrect predictions. Bayesian neural networks (BNNs) address this by providing probabilistic estimates, but incur high computational cost because predictions require sampling weight distributions and multiple forward passes. The Probabilistic Forward Pass (PFP) offers a highly efficient approximation to Stochastic Variational Inference (SVI) by assuming Gaussian-distributed weights and activations, enabling fully analytic uncertainty propagation and replacing sampling with a single deterministic forward pass. We present an end-to-end pipeline for training, compiling, optimizing, and deploying PFP-based BNNs on embedded ARM CPUs. Using the TVM deep learning compiler, we implement a dedicated library of Gaussian-propagating operators for multilayer perceptrons and convolutional neural networks, combined with manual and automated tuning strategies. Ablation studies show that PFP consistently outperforms SVI in computational efficiency, achieving speedups of up to 4200x for small mini-batches. PFP-BNNs match SVI-BNNs on Dirty-MNIST in accuracy, uncertainty estimation, and OOD detection while greatly reducing compute cost. These results highlight the potential of combining Bayesian approximations with code generation to enable efficient BNN deployment on resource-constrained systems.
Read the full abstract

All authors Bernhard Klein, Falk Selker, Hendrik Borras, Sophie Steger, Franz Pernkopf, Holger Fröning

Statistical Methodology

Consensus Tree Estimation with False Discovery Control via Partially Ordered Sets

Trees are data objects that hierarchically organize categories. Collections of trees arise in a diverse variety of fields, including evolutionary biology, machine learning, social sciences and anatomy. Summarizing a collection of trees by a single representative is challenging, in part due to the dimensions of both the sample space and the parameter space. We frame consensus tree estimation as a structured feature-selection problem, where leaves and edges are the features. We introduce a partial order on trees, use it to define false discoveries for a candidate summary tree, and develop novel estimation algorithms that control the false discovery rate at a nominal level for a broad class of generative models. We also use the partial order to assess the stability of features in a selected tree. Importantly, our method accommodates unequal leaf sets and non-binary trees, which commonly arise in modern datasets. Our feature-selection perspective yields finite-sample and model-free guarantees and provides a foundation for integrating multiple testing tools into tree estimation. We apply the method to study the origins of complex life. Our estimated tree recapitulates well-known divisions but highlights that there is insufficient data to determine the most recent archaeal ancestor of eukaryotic life.
Read the full abstract

Statistical Methodology

Getting it right: Methods for risk ratios and risk differences cluster randomized trials with a small number of clusters

Most cluster randomized trials (CRTs) randomize fewer than 30-40 clusters in total. When performing inference for such “small” CRTs, it is important to use methods that appropriately account for the small sample size. When the generalized estimating equations (GEE) approach is used for analysis of “small” CRTs, the robust variance estimator from GEE is biased downward and therefore bias-corrected standard errors should be used. Moreover, in order to avoid inflated Type I error, an appropriate bias-corrected standard error should be paired with the t- rather than Z-statistic when making inference about a single-parameter intervention effect. Although several bias-correction methods (including Kauermann and Carroll (KC), Mancl and DeRouen (MD), Morel, Bokossa, and Neerchal (MBN), and the average of KC and MD (AVG)) have been evaluated for inference for odds ratios, their finite-sample behavior in “small” CRTs with few clusters has not been thoroughly investigated for risk ratios and risk differences. The current article aims to fill the gap by including analysis via binomial, Poisson and Gaussian models and for a broad spectrum of scenarios. Analysis is via binomial and Poisson models (using log and identity link for risk and differences measures, respectively). We additionally explore the use of Gaussian models with identity link for risk differences and adopt the "modified" approach for analysis with misspecified Poisson and Gaussian models. We consider a broad spectrum of scenarios including for rare outcomes, small cluster sizes, high intracluster correlations (ICCs), and high coefficients of variation (CVs) of cluster size.
Read the full abstract

All authors Shifeng Sun, Xueqi Wang, Zhuoran Hou, Elizabeth L. Turner

Machine Learning (Statistics) · Cross-listed from cs.LG

Quantized-Tinyllava: a new multimodal foundation model enables efficient split learning

Multimodal foundation models are increasingly trained on sensitive data across domains such as finance, biomedicine, and personal identifiers. However, this distributed setup raises serious privacy concerns due to the need for cross-partition data sharing. Split learning addresses these concerns by enabling collaborative model training without raw data exchange between partitions, yet it introduces a significant challenge: transmitting high-dimensional intermediate feature representations between partitions leads to substantial communication costs. To address this challenge, we propose Quantized-TinyLLaVA, a multimodal foundation model with an integrated communication-efficient split learning framework. Our approach adopts a compression module that quantizes intermediate feature into discrete representations before transmission, substantially reducing communication overhead. Besides, we derive a principled quantization strategy grounded in entropy coding theory to determine the optimal number of discrete representation levels. We deploy our framework in a two-partition setting, with one partition operating as the client and the other as the server, to realistically simulate distributed training. Under this setup, Quantized-TinyLLaVA achieves an approximate 87.5% reduction in communication overhead with 2-bit quantization, while maintaining performance of the original 16-bit model across five benchmark datasets. Furthermore, our compressed representations exhibit enhanced resilience against feature inversion attacks, validating the privacy of transmission. The code is available at https://github.com/anonymous-1742/Quantized-TinyLLaVA.
Read the full abstract

All authors Jiajun Guo, Xin Luo, Jiayin Zheng, Yiqun Wang, Kai-Wei Chang, Wei Wang, Jie Liu

Machine Learning (Statistics)

Variance-Aware Baselines and Adaptive Learning Rates for Reinforcement Learning with Verifiable Rewards

Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective paradigm for post-training large language models, yet the design of its baselines and learning-rate schedules remains largely heuristic. This limits our understanding of the statistical properties of policy-gradient estimators and their interaction with optimization dynamics. In this work, we develop a theoretical framework for variance-aware baseline design and adaptive learning-rate selection in RLVR. Under a KL-regularized policy-optimization setting, we establish the unbiasedness of the resulting gradient estimator, derive exact variance expressions including the KL cross-covariance, and obtain an optimization-loss upper bound that enables principled reasoning about learning dynamics. Building on these results, we prove convergence guarantees and derive an adaptive learning-rate schedule governed by the signal-to-noise ratio (SNR) of the policy gradient. We further show that the variance-optimal baseline is a gradient-weighted estimator of the KL-regularized reward, providing a principled alternative to commonly used reward-based baselines. These results lead to two complementary improvements: a variance-optimal baseline and an SNR-adaptive learning-rate rule. Experiments on Qwen3-4B-Base show that each component independently improves policy-optimization performance. The learning-rate rule can also be naturally integrated with existing policy optimization methods to yield further gains, while combining it with the variance-optimal baseline gives the full Optimal Baseline and Learning-Rate Policy Optimization (OBLR-PO) method and achieves the strongest overall performance.
Read the full abstract

Statistical Methodology

Conjugate Generalized Bayesian Inference for Discrete Doubly Intractable Problems

Doubly intractable problems occur when both the likelihood and the posterior are available only in unnormalized form, with computationally intractable normalization constants. Bayesian inference then typically requires direct approximation of the posterior through specialized and typically expensive MCMC methods. In this paper, we provide a computationally efficient alternative in the form of a novel generalized Bayesian posterior that allows for conjugate, closed-form or Gibbs-based MCMC inference within the class of exponential family models for discrete data. We derive theoretical guarantees to characterize the asymptotic behavior of the generalized posterior, supporting its use for inference. The method is evaluated on a range of challenging intractable exponential family models, including the Conway-Maxwell-Poisson graphical model of multivariate count data, autoregressive discrete time series models, and Markov random fields such as the Ising and Potts models. The computational gains are significant; in our experiments, the method is between 10 and 6000 times faster than state-of-the-art Bayesian computational methods.
Read the full abstract

All authors William Laplante, Matias Altamirano, Jeremias Knoblauch, Andrew Duncan, François-Xavier Briol

Machine Learning (Statistics) · Cross-listed from cs.LG

Towards Understanding Transformers in Learning Random Walks

Transformers have proven highly effective across various applications, especially in handling sequential data such as natural languages and time series. However, transformer models often lack clear interpretability, and the success of transformers has not been well understood in theory. In this paper, we study the capability and interpretability of transformers in learning a family of classic statistical models, namely random walks on circles. We theoretically demonstrate that, after training with gradient descent, a one-layer transformer model can achieve optimal accuracy in predicting random walks. Importantly, our analysis reveals that the trained model is interpretable: the trained softmax attention serves as a token selector, focusing on the direct parent state; subsequently, the value matrix executes a one-step probability transition to predict the location of the next state based on this parent state. We also show that certain edge cases not covered by our theory are indeed failure cases, demonstrating that our theoretical conditions are tight. By investigating these success and failure cases, it is revealed that gradient descent with small initialization may fail or struggle to converge to a good solution in certain simple tasks even beyond random walks. Experiments are conducted to support our theoretical findings.
Read the full abstract

Statistical Methodology

Comparing Variable Selection and Model Averaging Methods for Logistic Regression

Model uncertainty is a central challenge in statistical models for binary outcomes such as logistic regression, arising when it is unclear which predictors should be included in the model. Many methods have been proposed to address this issue for logistic regression, but their relative performance under realistic conditions remains poorly understood. We therefore conducted a preregistered, simulation-based comparison of 28 established methods for variable selection and inference under model uncertainty, using 11 empirical datasets spanning a range of sample sizes and number of predictors, in cases both with and without separation. We found that Bayesian model averaging (BMA) methods based on g-priors, particularly g = max(n, p^2), show the strongest overall performance when separation is absent. When separation occurs, penalized likelihood approaches, especially the LASSO, provide the most stable results, while BMA with the local empirical Bayes (EB-local) prior is competitive in both situations. These findings offer practical guidance for applied researchers on how to effectively address model uncertainty in logistic regression in modern empirical and machine learning research.
Read the full abstract

All authors Nikola Sekulovski, František Bartoš, Don van den Bergh, Giuseppe Arena, Henrik R. Godmann, Vipasha Goyal, Julius M. Pfadt, Maarten Marsman, Adrian E. Raftery

Machine Learning (Statistics)

Asymptotic Theory and Phase Transitions for Variable Importance in Quantile Regression Forests

Quantile Regression Forests (QRF) are widely used for non-parametric conditional quantile estimation, yet statistical inference for variable importance measures remains challenging due to the non-smoothness of the loss function and the complex bias-variance trade-off. In this paper, we develop a asymptotic theory for variable importance defined as the difference in pinball loss risks. We first establish the asymptotic normality of the QRF estimator by handling the non-differentiable pinball loss via Knight's identity. Second, we uncover a "phase transition" phenomenon governed by the subsampling rate $\beta$ (where $s \asymp n^{\beta}$). We prove that in the bias-dominated regime ($\beta \ge 1/2$), which corresponds to large subsample sizes typically favored in practice to maximize predictive accuracy, standard inference breaks down as the estimator converges to a deterministic bias constant rather than a zero-mean normal distribution. Finally, we derive the explicit analytic form of this asymptotic bias and discuss the theoretical feasibility of restoring valid inference via analytic bias correction. Our results highlight a fundamental trade-off between predictive performance and inferential validity, providing a theoretical foundation for understanding the intrinsic limitations of random forest inference in high-dimensional settings.
Read the full abstract

Statistical Methodology

A Design-Based Matching Framework for Staggered Adoption with Time-Varying Confounding

Causal inference in longitudinal datasets has long been challenging due to dynamic treatment adoption and confounding by time-varying covariates. Prior work either fails to account for heterogeneity across treatment adoption cohorts and treatment timings or relies on modeling assumptions. In this paper, we develop a novel design-based framework for inference on group- and time-specific treatment effects in panel data with staggered treatment adoption. We establish identification results for causal effects under this structure and introduce corresponding estimators, together with a block bootstrap procedure for estimating the covariance matrix and testing the homogeneity of group-time treatment effects. To implement the framework in practice, we propose the Reverse-Time Nested Matching algorithm, which constructs matched strata by pairing units from different adoption cohorts in a way that ensures comparability of covariate histories at each treatment time. Applying the algorithm to the Netflix-IPTV dataset, we find that while Netflix subscription does not significantly affect total IPTV viewing time, it does negatively affect VoD usage. We also provide statistical evidence that the causal effects of Netflix subscription may vary even within the same treatment cohort or across the same outcome and event times.
Read the full abstract

Machine Learning (Statistics)

A PLS-Integrated LASSO Method with Application in Index Tracking

In traditional multivariate data analysis, dimension reduction and regression have been treated as distinct endeavors. Established techniques such as principal component regression (PCR) and partial least squares (PLS) regression traditionally compute latent components as intermediary steps – although with different underlying criteria – before proceeding with the regression analysis. In this paper, we introduce an innovative regression methodology named PLS-integrated Lasso (PLS-Lasso) that integrates the concept of dimension reduction directly into the regression process. We present two distinct formulations for PLS-Lasso, denoted as PLS-Lasso-v1 and PLS-Lasso-v2, along with clear and effective algorithms that ensure convergence to global optima. PLS-Lasso-v1 and PLS-Lasso-v2 are compared with Lasso on the task of financial index tracking and show promising results.
Read the full abstract

Statistics Applications

Design loads for wave impacts – introducing the Probabilistic Adaptive Screening (PAS) method for predicting extreme non-linear loads on maritime structures

Wave impact loads on maritime structures can cause casualties, damage, pollution and operational delays. Consequently, their extreme values should be accounted for in the design of these structures. However, this is challenging, as wave impact events are both rare and highly complex, requiring both high-fidelity simulations and long analysis durations to reliably quantify the associated design loads. Moreover, existing extreme value prediction methods are neither specifically developed nor adequately validated for wave impact phenomena. We therefore introduce the new Probabilistic Adaptive Screening (PAS) method for predicting extreme non-linear loads on maritime structures. The method integrates copula-based statistical dependence modelling with multi-fidelity screening and adaptive sampling. This framework enables efficient extreme value prediction by statistically mapping low-fidelity indicator variables to high-fidelity impact loads. The method allows for efficient linear potential flow indicators to be used in the low-fidelity stage, even for strongly non-linear cases. Its statistical framework is validated against four non-linear test cases, including non-linear waves, ship vertical bending moments, green water impact loads, and slamming loads. It is concluded that PAS with optimal settings accurately estimates both the short-term distributions and extreme values in these test cases, with most probable maximum (MPM) values within 2-15% of the reference brute-force Monte-Carlo Simulation (MCS) results. In addition, PAS achieves this performance very efficiently, requiring in the order of 1-3% of the high-fidelity simulation time needed for conventional MCS. These results demonstrate that PAS can reliably reproduce the statistics of both weakly and strongly non-linear extreme load problems, while significantly reducing the associated computational cost compared to MCS.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

A Differentiable Measure of Algebraic Complexity: Provably Exact Discovery of Group Structures

Discovering discrete algebraic rules from data is a fundamental challenge in machine learning. We formalize this problem through Cayley-table completion – an algebraic counterpart to classical matrix completion – where the degree of associativity violation replaces linear rank as the intrinsic measure of complexity. We provide a rigorous landscape analysis of HyperCube, an operator-valued tensor factorization, on the fully observed target table $\delta$, proving that its global infimum $H_{\inf}(\delta) := \inf_{\Theta \in F_\delta} H(\Theta)$ implicitly defines an exact differentiable measure for this complexity. We show that HyperCube's native objective $H(\Theta)$ decomposes into two components: geometric alignment (collinearity) and an inverse $\ell_2$ penalty. We establish that these continuous variational pressures induce core discrete properties: collinearity enforces associativity (Collinearity–Associativity Equivalence), and the inverse $\ell_2$ penalty reduces to an exact inverse rank penalty within the collinear manifold, driving the parameters toward full-rank unitarity. Consequently, we derive an absolute lower bound $H(\Theta) \ge H_{\inf}(\delta) \ge 3 \, |\delta|$, where $|\delta|$ is the target table size. We prove this absolute floor is attained if and only if the target is isotopic to a group, and characterize the global minimizer as the regular representation of the underlying group (up to unitary gauge), resolving the central open conjecture of Huh (2025). This work serves as an existence proof that certain discrete algebraic structures can be exactly characterized by differentiable measures, enabling gradient-based discovery without the need for combinatorial search. All theoretical results are mechanically verified in Lean 4 and confirmed via small-scale experiments.
Read the full abstract

Statistical Methodology

The Bayesian optimal two-stage design for clinical phase II trials based on Bayes factors

Sequential trial design is an important statistical approach to increase the efficiency of clinical trials. Bayesian sequential trial design relies primarily on conducting a Monte Carlo simulation under the hypotheses of interest and investigating the resulting design characteristics via Monte Carlo estimates. This approach has several drawbacks, namely that replicating the calibration of a Bayesian design requires repeating a possibly complex Monte Carlo simulation. Furthermore, Monte Carlo standard errors are required to judge the reliability of the simulation. All of this is due to a lack of closed-form or numerical approaches to calibrate a Bayesian design which uses Bayes factors. In this paper, we propose the Bayesian optimal two-stage design for clinical phase II trials based on Bayes factors. The optimal two-stage Bayes factor design is a sequential clinical trial design that is built on the idea of trinomial tree branching, a method we propose to correct the resulting design characteristics for introducing a single interim analysis. We build upon this idea to invent a calibration algorithm which yields the optimal Bayesian design that minimizes the expected sample size under the null hypothesis. Examples show that our design recovers Simon's two-stage optimal design as a special case, improves upon non-sequential Bayesian design based on Bayes factors, and can be calibrated quickly, as it makes use only of standard numerical techniques instead of time-consuming Monte Carlo simulations. Furthermore, the design allows to ensure a minimum probability on compelling evidence in favour of the null hypothesis, which is not possible with other designs. As the idea of trinomial tree branching is neither dependent on the endpoint, nor on the use of Bayes factors, the design can therefore be generalized to other settings, too.
Read the full abstract

Statistics Applications

Multi-fidelity Bayesian Optimization Framework for CFD-Based Non-Premixed Burner Design

We propose a multi-fidelity Bayesian optimization (MF-BO) framework that integrates computational fluid dynamics (CFD) evaluations with Gaussian-process surrogates to efficiently navigate the accuracy-cost trade-off induced by mesh resolution. The design vector x = [h, l, s] (height, length, and mesh element size) defines a continuous fidelity index Z(h, l, s), enabling the optimizer to adaptively combine low- and high-resolution simulations. This framework is applied to a non-premixed burner configuration targeting improved thermal efficiency under hydrogen-enriched fuels. A calibrated runtime model t_hat(h, l, s) penalizes computationally expensive queries, while a constrained noisy expected improvement (qNEI) guides sampling under an emissions cap of 2e-6 for NOx. Surrogates trained on CFD data exhibit stable hyperparameters and physically consistent sensitivities: mean temperature increases with reactor length and fidelity and is weakly negative with height; NOx grows with temperature yet tends to decrease with length. The best design achieves T_bar approx 2.0e3 K while satisfying the NOx limit. Relative to a hypothetical single-fidelity campaign (Z = 1), the MF-BO achieves comparable convergence with about 57 percent lower total wall time by learning the design landscape through fast low-Z evaluations and reserving high-Z CFD for promising candidates. Overall, the methodology offers a generalizable and computationally affordable path for optimizing reacting-flow systems in which mesh-driven fidelity inherently couples accuracy, cost, and emissions. This highlights its potential to accelerate design cycles and reduce resource requirements in industrial burner development and other high-cost CFD-driven applications.
Read the full abstract

All authors Patrick Souza Lima, Paulo Roberto Santana dos Reis, Alex Álisson Bandeira Santos, Ehecatl Antonio del Río Chanona, Idelfonso Bessa dos Reis Nogueira

Statistical Methodology

Goodness-of-fit testing for the error distribution in functional linear models

We consider the error distribution in functional linear models with scalar response and functional covariate. Different asymptotic expansions of the empirical distribution function and the empirical characteristic function based on estimated residuals under different model assumptions are discussed. The results are applied for simple and composite goodness-of-fit testing for the error distribution, in particular testing for normal distribution.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from eess.SP

Strategies to Minimize Out-of-Distribution Effects in Data-Driven MRS Quantification

This study systematically compared data-driven and model-based strategies for metabolite quantification in magnetic resonance spectroscopy (MRS), focusing on resilience to out-of-distribution (OoD) effects and the balance between accuracy, robustness, and generalizability. A neural network designed for MRS quantification was trained using three distinct strategies: supervised regression, self-supervised learning, and test-time adaptation. These were compared against model-based fitting tools. Experiments combined large-scale simulated data, designed to probe metabolite concentration extrapolation and signal variability, with 1H single-voxel 7T in-vivo human brain spectra. In simulations, supervised learning achieved high accuracy for spectra similar to those in the training distribution, but showed marked degradation when extrapolated beyond the training distribution. Test-time adaptation proved more resilient to OoD effects, while self-supervised learning achieved intermediate performance. In-vivo experiments showed larger variance across the methods (data-driven and model-based) due to domain shift. Across all strategies, overlapping metabolites and baseline variability remained persistent challenges. While strong performance can be achieved by data-driven methods for MRS metabolite quantification, their reliability is contingent on careful consideration of the training distribution and potential OoD effects. When such conditions in the target distribution cannot be anticipated, test-time adaptation strategies ensure consistency between the quantification, the data, and the model, enabling reliable data-driven MRS pipelines.
Read the full abstract

All authors Julian P. Merkofer, Antonia Kaiser, Anouk Schrantee, Oliver J. Gurney-Champion, Ruud J. G. van Sloun

Statistical Methodology

Machine learning for violence prediction: a systematic review and critical appraisal

Purpose To conduct a systematic review of machine learning models for predicting violent behaviour by synthesising and appraising their validity, usefulness, and performance. Methods We systematically searched nine bibliographic databases and Google Scholar up to September 2025 for development and/or validation studies on machine learning methods for predicting all forms of violent behaviour. We synthesised the results by summarising discrimination and calibration performance statistics and evaluated study quality by examining risk of bias and clinical utility. Results We identified 38 studies reporting the development and validation of 40 models. Most studies reported Area Under the Curve (AUC) as the discrimination statistic with a range of 0.68-0.99. Only eight studies reported calibration performance, and three studies reported external validation. 31 studies had a high risk of bias, mainly in the analysis domain, and three studies had low risk of bias. The overall clinical utility of violence prediction models is poor, as indicated by risks of overfitting due to small samples, lack of transparent reporting, and low generalisability. Conclusion Although black box machine learning models currently have limited applicability in clinical settings, they may show promise for identifying high-risk individuals. We recommend five key considerations for violence prediction modelling: (i) ensuring methodological quality (e.g. following guidelines) and interdisciplinary collaborations; (ii) using black box algorithms only for highly complex data; (iii) incorporating dynamic predictions to allow for risk monitoring; (iv) developing more trustworthy algorithms using explainable methods; and (v) applying causal machine learning approaches where appropriate.
Read the full abstract

All authors Stefaniya Kozhevnikova, Denis Yukhnenko, Giulio Scola, Seena Fazel

Statistics Theory

Detecting Periodicity of a General Stationary Time Series via AR(2)-Model Fitting

Estimating the periodicity of a stationary time series via fitting a second order stationary autoregressive (AR(2)) model has been initiated by the seminal paper of Yule(1927).. We investigate properties of this procedure when applied to a general stationary processes possessing a spectral density with a dominant peak at some frequency $\lambda_0\in(0,\pi)$. We show that if the peak of the spectral density is sharp enough (in a way to be specified) then the AR(2) model, which best (in mean square sense) approximates the underlying process, correctly identifies the frequency $\lambda_0$. To investigate consistency properties of the AR(2) based estimator of $\lambda_0$, a near to pole framework is adopted. Triangular arrays of stationary stochastic processes are considered that possess a spectral density the peak of which at $\lambda_0$ becomes more pronounced as the sample size $n$ of the observed time series increases to infinity. It is shown in this set up, that the AR(2) based estimator achieves a rate of convergence which is larger than the parametric $n^{-1/2}$ rate and which can be arbitrarily close to $ n^{-2/3}$, the best rate that can be achieved by this estimator.
Read the full abstract

Statistical Methodology

Inference for quantile-parametrized families via CDF confidence bands

Quantile-based distribution families are an important subclass of parametric families, capable of exhibiting a wide range of behaviors using very few parameters. These parametric models present significant challenges for classical methods, since the CDF and density do not have a closed-form expression. Furthermore, approximate maximum likelihood estimation and related procedures may yield non-$\sqrt{n}$ and non-normal asymptotics over regions of the parameter space, making bootstrap and resampling techniques unreliable. We develop a novel inference framework that constructs confidence sets by inverting distribution-free confidence bands for the empirical CDF through the known quantile function. Our proposed inference procedure provides a principled and assumption-lean alternative in this setting, requiring no distributional assumptions beyond the parametric model specification and avoiding the computational and theoretical difficulties associated with likelihood-based methods for these complex parametric families. We demonstrate our framework on Tukey Lambda and generalized Lambda distributions, evaluate its performance through simulation studies, and illustrate its practical utility with an application to both a small-sample dataset (Twin Study) and a large-sample dataset (Spanish household incomes).
Read the full abstract

Statistical Methodology

A General Bayesian Nonparametric Approach for Estimating Population-Level and Conditional Causal Effects

We propose a Bayesian nonparametric (BNP) approach to causal inference using observational data consisting of outcome, treatment, and a set of confounders. The conditional distribution of the outcome given treatment and confounders is modeled flexibly using a dependent nonparametric mixture model, in which both the atoms and the weights vary with the confounders. The proposed BNP model is well suited for causal inference problems, as it does not rely on parametric assumptions about how the conditional distribution depends on the confounders. In particular, the model effectively adjusts for confounding and improves the modeling of treatment effect heterogeneity, leading to more accurate estimation of both the average treatment effect (ATE) and heterogeneous treatment effects (HTE). Posterior inference under the proposed model is computationally efficient due to the use of data augmentation. Extensive evaluations demonstrate that the proposed model offers competitive or superior performance compared to a wide range of recent methods spanning various statistical approaches, including Bayesian additive regression tree (BART) models, which are well known for their strong empirical performance. More importantly, the model provides fully probabilistic inference on quantities of interest that other methods cannot easily provide, using their posterior distributions.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Spectral Concentration at the Edge of Stability: Information Geometry of Kernel Associative Memory

High-capacity kernel Hopfield networks exhibit a Ridge of Optimization characterized by extreme stability. While previously linked to Spectral Concentration, its origin remains elusive. Here, we analyze the network dynamics on a statistical manifold, revealing that the Ridge corresponds to the Edge of Stability, a critical boundary where the Fisher Information Matrix becomes singular. We demonstrate that the apparent Euclidean force antagonism is a manifestation of Dual Equilibrium in the Riemannian space. This unifies learning dynamics and capacity via the Minimum Description Length principle, offering a geometric theory of self-organized criticality.
Read the full abstract

Statistics Applications · Cross-listed from eess.SP

What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals

How would the probability of scoring change if a different player took the same shot? Although expected goals models describe the difficulty of scoring opportunities, comparing players on identical chances requires accounting for uncertainty in their estimated conversion probabilities. We address this question through a hierarchical Bayesian framework that evaluates alternative shooters under the same observed circumstances. Using 35,700 eligible shots from four leagues during the 2015/16 season, we compare a contextual benchmark with a primary player-intercept model and a provisional extension that allows player differences to vary with shooting conditions. Paired posterior draws preserve shared uncertainty when individual probability differences are aggregated across opportunities. Chronological validation demonstrates improved predictive performance relative to a constant-rate baseline, but introducing player effects provides negligible additional gains. Controlled single-seed experiments show that greater information per player improves estimation accuracy and narrows uncertainty intervals, although distinguishing small differences remains difficult and confident conclusions about near-null effects can become more frequent. Applications involving Giroud–Vardy and Berardi–Sansone similarly yield estimated differences that are small relative to their posterior uncertainty. These findings highlight an important distinction between predicting the outcome of a shot and establishing whether one player would be more likely to convert it. The proposed framework makes this distinction explicit by evaluating alternative shooters on fixed recorded opportunities while accounting for their contextual exposure. The resulting comparisons concern conditional conversion rather than the causal consequences of replacing a player in a match.
Read the full abstract

All authors Mikayil Mahmudlu, Oktay Karakuş, Hasan Arkadaş, Can Eyüpoğlu

Statistics Applications

A novel method to analyze pattern shifts in rainfall using cluster analysis and probability models

: One of the prominent challenges being faced by agricultural sciences is the onset of climate change which is adversely affecting every aspect of cropping. Modelling of climate change at macro level have been carried out at large scale and there is ample amount of research publications available for that. But at micro level like at state level or district level there are lesser studies. District level studies can help in preparing specific plans for the mitigation of adverse effects of climate change at local level. An attempt has been made in this paper to model the monthly rainfall of Varanasi district of the state of Uttar Pradesh with the help of probability models. Firstly, the pattern of the climate change over 122 years has been unveiled by using exploratory analysis and using multivariate techniques like cluster analysis and then probability models have been fitted for selected months
Read the full abstract

All authors Abhishek Singh, Aaditya Jadhav, Abha Goyal, Jesma V, Vyshna I C

Statistical Methodology

Joint Bayesian Inference of Parameter and Discretization Error Uncertainties in ODE Models

We address the problem of Bayesian inference for parameters in ordinary differential equation (ODE) models based on observational data. Conventional approaches in this setting typically rely on numerical solvers such as the Euler or Runge-Kutta methods. However, these methods generally do not account for the discretization error induced by discretizing the ODE model. We propose a Bayesian inference framework for ODE models that explicitly quantifies discretization errors. Our method models discretization error as a random variable and performs Bayesian inference on both ODE parameters and variances of the randomized discretization errors, referred to as the discretization error variance. A key idea of our approach is the introduction of a Markov prior on the temporal evolution of the discretization error variances, enabling the inference problem to be formulated as a state-space model. Furthermore, we propose a specific form of the Markov prior that arises naturally from standard discretization error analysis. This prior depends on the step size in the numerical solver, and we discuss its asymptotic property in the limit as the step size approaches zero. Numerical experiments illustrate that the proposed method can simultaneously quantify uncertainties in both the ODE parameters and the discretization errors, and can produce posterior distributions over the parameters with broader support by accounting for discretization error.
Read the full abstract

Statistics Theory

Functional Gaussian Fields on Hyperspheres with their Equivalent Gaussian Measures

We develop a general framework for isotropic functional Gaussian fields on the $d$-dimensional sphere $\mathbb{S}^{d}$, where the field takes values in a separable Hilbert space $\mathcal{H}$. We establish an operator-valued extension of Schoenberg's theorem and show that the covariance structure of such fields admits a representation through a sequence of trace-class $d$-Schoenberg operators, yielding an explicit spectral decomposition of the covariance operator on $L^{2}(\mathbb{S}^{d};\mathcal{H})$. We derive a functional version of the Feldman-H'ajek criterion and prove that equivalence of the Gaussian measures induced by two Hilbert-valued spherical fields is determined by a Hilbert summability condition involving Schoenberg functional sequences, extending classical results for scalar and vector fields to the infinite-dimensional setting. We further show how equivalence of all scalar projections is contained within, and dominated by, the functional criterion. The theory is illustrated through two models: (i) a multiquadratic bivariate family on $\mathbb{S}^{d}$, where the equivalence region has a closed-form description in terms of cross-correlation and geodesic decay parameters, and (ii) an infinite-dimensional Legendre-Mat'ern construction, where operator-valued spectra yield identifiability conditions on smoothness and scale. These examples show how operator-valued Schoenberg coefficients govern both geometry and measure-theoretic behavior of functional spherical fields. Overall, the results provide a unified spectral framework for Gaussian measures on $L^{2}(\mathbb{S}^{d};\mathcal{H})$, bridging harmonic analysis, operator theory, and stochastic geometry on manifolds, and offering tools for functional data analysis, spatial statistics, and kernel methods on spherical domains.
Read the full abstract

Statistical Computation · Cross-listed from cs.CV

Optimizer Sensitivity In Vision Transformerbased Iris Recognition: Adamw Vs Sgd Vs Rmsprop

The security of biometric authentication is increasingly critical as digital identity systems expand. Iris recognition offers high reliability due to its distinctive and stable texture patterns. Recent progress in deep learning, especially Vision Transformers ViT, has improved visual recognition performance. Yet, the effect of optimizer choice on ViT-based biometric systems remains understudied. This work evaluates how different optimizers influence the accuracy and stability of ViT for iris recognition, providing insights to enhance the robustness of biometric identification models.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

A Trainable Centrality Framework for Modern Data

Measuring how central or typical a data point is underpins robust estimation, ranking, and outlier detection, but classical depth notions become expensive and unstable in high dimensions and are hard to extend beyond Euclidean data. We introduce Fused Unified centrality Score Estimation (FUSE), a neural centrality framework that operates on top of arbitrary representations. FUSE combines a global head, trained from pairwise distance-based comparisons to learn an anchor-free centrality score, with a local head, trained by denoising score matching to approximate a smoothed log-density potential. A single parameter between 0 and 1 interpolates between these calibrated signals, yielding depth-like centrality from different views via one forward pass. Across synthetic distributions, real images, time series, and text data, and standard outlier detection benchmarks, FUSE recovers meaningful classical ordering, reveals multi-scale geometric structures, and attains competitive performance with strong classical baselines while remaining simple and efficient.
Read the full abstract

Statistical Methodology

Constrained Gaussian Random Fields with Continuous Linear Boundary Restrictions for Physics-informed Modeling of States

Boundary constraints in physical, environmental and engineering models restrict smooth states such as temperature to follow known physical laws at the edges of their spatio-temporal domains. Examples include fixed-state or fixed-derivative (insulated) boundary conditions, and constraints that relate the state and the derivatives, such as in models of heat transfer. Despite their flexibility as prior models over system states, Gaussian random fields do not in general enable exact enforcement of such constraints. This work develops a new general framework for constructing linearly boundary-constrained Gaussian random fields from unconstrained Gaussian random fields over multi-dimensional domains. This new class of models provides flexible priors for modeling smooth states with known physical mechanisms acting at the domain boundaries without requiring approximation. Simulation studies illustrate how such physics-informed probability models yield improved predictive performance and more realistic uncertainty quantification in applications including probabilistic numerics, data-driven discovery of dynamical systems, and boundary-constrained state estimation, as compared to unconstrained alternatives. The framework is then applied to the problem of recovering a displacement field from experimental tensile testing data, demonstrating its ability to integrate known boundary constraints within the inferential procedure.
Read the full abstract