Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Friday 17 October

Statistics Applications · Cross-listed from econ.EM

Dynamic Spatial Treatment Effects as Continuous Functionals: Theory and Evidence from Healthcare Access

I develop a continuous functional framework for spatial treatment effects grounded in Navier-Stokes partial differential equations. Rather than discrete treatment parameters, the framework characterizes treatment intensity as continuous functions $\tau(\mathbf{x}, t)$ over space-time, enabling rigorous analysis of boundary evolution, spatial gradients, and cumulative exposure. Empirical validation using 32,520 U.S. ZIP codes demonstrates exponential spatial decay for healthcare access ($\kappa = 0.002837$ per km, $R^2 = 0.0129$) with detectable boundaries at 37.1 km. The framework successfully diagnoses when scope conditions hold: positive decay parameters validate diffusion assumptions near hospitals, while negative parameters correctly signal urban confounding effects. Heterogeneity analysis reveals 2-13 $\times$ stronger distance effects for elderly populations and substantial education gradients. Model selection strongly favors logarithmic decay over exponential ($\Delta \text{AIC} > 10,000$), representing a middle ground between exponential and power-law decay. Applications span environmental economics, banking, and healthcare policy. The continuous functional framework provides predictive capability ($d^*(t) = \xi^* \sqrt{t}$), parameter sensitivity ($\partial d^*/\partial \nu$), and diagnostic tests unavailable in traditional difference-in-differences approaches.
Read the full abstract

Statistics Applications · Cross-listed from astro-ph.IM

Neural Posterior Estimation for Cataloging Astronomical Images from the Legacy Survey of Space and Time

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will commence full-scale operations in 2026, yielding an unprecedented volume of astronomical images. Constructing an astronomical catalog, a table of imaged stars, galaxies, and their properties, is a fundamental step in most scientific workflows based on astronomical image data. Traditional deterministic cataloging methods lack statistical coherence as cataloging is an ill-posed problem, while existing probabilistic approaches suffer from computational inefficiency, inaccuracy, or the inability to perform inference with multiband coadded images, the primary output format for LSST images. In this article, we explore a recently developed Bayesian inference method called neural posterior estimation (NPE) as an approach to cataloging. NPE leverages deep learning to achieve both computational efficiency and high accuracy. When evaluated on the DC2 Simulated Sky Survey – a highly realistic synthetic dataset designed to mimic LSST data – NPE systematically outperforms the standard LSST pipeline in light source detection, flux measurement, star/galaxy classification, and galaxy shape measurement. Additionally, NPE provides well-calibrated posterior approximations. These promising results, obtained using simulated data, illustrate the potential of NPE in the absence of model misspecification. Although some degree of model misspecification is inevitable in the application of NPE to real LSST images, there are a variety of strategies to mitigate its effects.
Read the full abstract

All authors Yicun Duan, Xinyue Li, Camille Avestruz, Jeffrey Regier, LSST Dark Energy Science Collaboration

Statistics Theory · Cross-listed from cs.LG

Small Ensemble-based Data Assimilation: A Machine Learning-Enhanced Data Assimilation Method with Limited Ensemble Size

Ensemble-based data assimilation (DA) methods have become increasingly popular due to their inherent ability to address nonlinear dynamic problems. However, these methods often face a trade-off between analysis accuracy and computational efficiency, as larger ensemble sizes required for higher accuracy also lead to greater computational cost. In this study, we propose a novel machine learning-based data assimilation approach that combines the traditional ensemble Kalman filter (EnKF) with a fully connected neural network (FCNN). Specifically, our method uses a relatively small ensemble size to generate preliminary yet suboptimal analysis states via EnKF. A FCNN is then employed to learn and predict correction terms for these states, thereby mitigating the performance degradation induced by the limited ensemble size. We evaluate the performance of our proposed EnKF-FCNN method through numerical experiments involving Lorenz systems and nonlinear ocean wave field simulations. The results consistently demonstrate that the new method achieves higher accuracy than traditional EnKF with the same ensemble size, while incurring negligible additional computational cost. Moreover, the EnKF-FCNN method is adaptable to diverse applications through coupling with different models and the use of alternative ensemble-based DA methods.
Read the full abstract

All authors Zhilin Li, Zhou Yao, Xianglong Li, Zeng Liu, Zhaokuan Lu, Shanlin Xu, Seungnam Kim, Guangyao Wang

Machine Learning (Statistics)

Foresighted Online Policy Optimization with Interference

Contextual bandits, which leverage the baseline features of sequentially arriving individuals to optimize cumulative rewards while balancing exploration and exploitation, are critical for online decision-making. Existing approaches typically assume no interference, where each individual's action affects only their own reward. Yet, such an assumption can be violated in many practical scenarios, and the oversight of interference can lead to short-sighted policies that focus solely on maximizing the immediate outcomes for individuals, which further results in suboptimal decisions and potentially increased regret over time. To address this significant gap, we introduce the foresighted online policy with interference (FRONT) that innovatively considers the long-term impact of the current decision on subsequent decisions and rewards. The proposed FRONT method employs a sequence of exploratory and exploitative strategies to manage the intricacies of interference, ensuring robust parameter inference and regret minimization. Theoretically, we establish a tail bound for the online estimator and derive the asymptotic distribution of the parameters of interest under suitable conditions on the interference network. We further show that FRONT attains sublinear regret under two distinct definitions, capturing both the immediate and consequential impacts of decisions, and we establish these results with and without statistical inference. The effectiveness of FRONT is further demonstrated through extensive simulations and a real-world application to urban hotel profits.
Read the full abstract

Statistical Methodology

Bayesian Sequential Modeling of Time-to-Urination for Dynamic ED Triage

Triage tools in routine emergency care are largely static, failing to exploit simple behavioral cues clinicians notice in real time. Here, we developed a Bayesian, sequentially updating framework that integrates incoming cues to produce calibrated, time-consistent risk. Using a prospective single-center cohort of ambulance arrivals in Japan (February-August 2025; n=2,221), we evaluated time to first urination (TTU) as a proof-of-concept bedside cue for predicting hospital admission. Population-level fit to the cumulative admission curve was excellent (integrated squared error 0.002; RMSE 0.003; Kolmogorov-Smirnov 0.008; coverage 0.98). At the patient level, performance improved markedly with age/sex adjustment (AUC[t] 0.70 vs. 0.50 unadjusted), with lower Brier scores and positive calibration slopes. Platt recalibration refined probability scaling without altering discrimination, and decision-curve analysis showed small, favorable net benefit at common thresholds. This framework is readily extensible to multimodal inputs and external validation and is designed to complement, not replace, existing triage systems.
Read the full abstract

All authors Atsushi Senda, Yuki Takatsu, Ryokan Ikebe, Hiroshi Suginaka, Koji Morishita, Akira Endo

Machine Learning (Statistics) · Cross-listed from cs.LG

Robust Layerwise Scaling Rules by Proper Weight Decay Tuning

Empirical scaling laws prescribe how to allocate parameters, data, and compute, while maximal-update parameterization ($\mu$P) enables learning-rate transfer across widths by equalizing early-time update magnitudes. However, in modern scale-invariant architectures, training quickly enters an optimizer-governed steady state where normalization layers create backward scale sensitivity and the effective learning rate becomes width dependent, degrading $\mu$P transfer. We address this by introducing a weight-decay scaling rule for AdamW that preserves sublayer gain across widths. Empirically, the singular-value spectrum of each matrix parameter scales in norm as $\sqrt{\eta/\lambda}$ with an approximately invariant shape; under width scaling $d$, we observe that the top singular value scales approximately as $\sqrt{\eta/\lambda}\cdot d^{0.75}$. Combining this observation with the $\mu$P learning-rate rule $\eta_2\propto d^{-1}$ for matrix-like parameters implies an empirical weight-decay scaling rule $\lambda_2\propto \sqrt{d}$ that approximately keeps sublayer gains width invariant. Together with vector-like parameters trained at $\eta_1=\Theta_d(1)$ and $\lambda_1=0$, this yields zero-shot transfer of both learning rate and weight decay from proxy to target widths, removing per-width sweeps. We validate the rule on LLaMA-style Transformers and in a minimal synthetic setting, and we provide a simple diagnostic, matching top singular values, to check sublayer-gain invariance. Our results extend $\mu$P beyond the near-init regime by explicitly controlling steady-state scales set by the optimizer, offering a practical recipe for width-robust hyperparameter transfer under AdamW.
Read the full abstract

All authors Zhiyuan Fan, Yifeng Liu, Qingyue Zhao, Angela Yuan, Quanquan Gu

Statistics Theory · Cross-listed from cs.LG

Stress-Aware Learning under KL Drift via Trust-Decayed Mirror Descent

We study sequential decision-making under distribution drift. We propose entropy-regularized trust-decay, which injects stress-aware exponential tilting into both belief updates and mirror-descent decisions. On the simplex, a Fenchel-dual equivalence shows that belief tilt and decision tilt coincide. We formalize robustness via fragility (worst-case excess risk in a KL ball), belief bandwidth (radius sustaining a target excess), and a decision-space Fragility Index (drift tolerated at $O(\sqrt{T})$ regret). We prove high-probability sensitivity bounds and establish dynamic-regret guarantees of $\tilde{O}(\sqrt{T})$ under KL-drift path length $S_T = \sum_{t\ge2}\sqrt{{\rm KL}(D_t|D_{t-1})/2}$. In particular, trust-decay achieves $O(1)$ per-switch regret, while stress-free updates incur $\Omega(1)$ tails. A parameter-free hedge adapts the tilt to unknown drift, whereas persistent over-tilting yields an $\Omega(\lambda^2 T)$ stationary penalty. We further obtain calibrated-stress bounds and extensions to second-order updates, bandit feedback, outliers, stress variation, distributed optimization, and plug-in KL-drift estimation. The framework unifies dynamic-regret analysis, distributionally robust objectives, and KL-regularized control within a single stress-adaptive update.
Read the full abstract

Statistical Methodology

Conditional GLMMs for reaction times in choice tasks

This study connects two methods for modeling reaction times (RTs) in choice tasks: (1) the first-hitting time of a simple diffusion model with a single barrier, representing the cognitive process leading to a response, and (2) Generalized Linear Mixed Models (GLMMs). We achieve this by analyzing RT distributions conditioned on each response alternative. Because certain diffusion model variants yield Inverse Gaussian (IG) and Gamma distributions for first-hitting times, we can justify using these distributions in RT models. Conversely, employing IG and Gamma distributions within GLMMs allows us to infer the underlying cognitive processes. We demonstrate this concept through simulations and apply it to previously published real-world data. Finally, we discuss the scope and potential extensions of our approach.
Read the full abstract

Thursday 16 October

Statistical Methodology · Cross-listed from cs.LG

Measure-Theoretic Anti-Causal Representation Learning

Causal representation learning in the anti-causal setting (labels cause features rather than the reverse) presents unique challenges requiring specialized approaches. We propose Anti-Causal Invariant Abstractions (ACIA), a novel measure-theoretic framework for anti-causal representation learning. ACIA employs a two-level design, low-level representations capture how labels generate observations, while high-level representations learn stable causal patterns across environment-specific variations. ACIA addresses key limitations of existing approaches by accommodating prefect and imperfect interventions through interventional kernels, eliminating dependency on explicit causal structures, handling high-dimensional data effectively, and providing theoretical guarantees for out-of-distribution generalization. Experiments on synthetic and real-world medical datasets demonstrate that ACIA consistently outperforms state-of-the-art methods in both accuracy and invariance metrics. Furthermore, our theoretical results establish tight bounds on performance gaps between training and unseen environments, confirming the efficacy of our approach for robust anti-causal learning.
Read the full abstract

Machine Learning (Statistics)

Manifold Dimension Estimation via Local Graph Structure

Most existing manifold dimension estimators rely on the assumption that the underlying manifold is locally flat within the neighborhoods under consideration. More recently, curvature-adjusted principal component analysis (CA-PCA) has emerged as a powerful alternative by explicitly accounting for the manifold's curvature. Motivated by these ideas, we propose a manifold dimension estimation framework that captures the local graph structure of the manifold through regression on local PCA coordinates. Within this framework, we introduce two representative estimators: quadratic embedding (QE) and total least squares (TLS). Experiments on both synthetic and real-world datasets demonstrate that these methods perform competitively with, and often outperform, state-of-the-art approaches.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

A Simple Method for PMF Estimation on Large Supports

We study nonparametric estimation of a probability mass function (PMF) on a large discrete support, where the PMF is multi-modal and heavy-tailed. The core idea is to treat the empirical PMF as a signal on a line graph and apply a data-dependent low-pass filter. Concretely, we form a symmetric tri-diagonal operator, the path graph Laplacian perturbed with a diagonal matrix built from the empirical PMF, then compute the eigenvectors, corresponding to the smallest feq eigenvalues. Projecting the empirical PMF onto this low dimensional subspace produces a smooth, multi-modal estimate that preserves coarse structure while suppressing noise. A light post-processing step of clipping and re-normalizing yields a valid PMF. Because we compute the eigenpairs of a symmetric tridiagonal matrix, the computation is reliable and runs time and memory proportional to the support times the dimension of the desired low-dimensional supspace. We also provide a practical, data-driven rule for selecting the dimension based on an orthogonal-series risk estimate, so the method "just works" with minimal tuning. On synthetic and real heavy-tailed examples, the approach preserves coarse structure while suppressing sampling noise, compares favorably to logspline and Gaussian-KDE baselines in the intended regimes. However, it has known failure modes (e.g., abrupt discontinuities). The method is short to implement, robust across sample sizes, and suitable for automated pipelines and exploratory analysis at scale because of its reliability and speed.
Read the full abstract

Statistics Applications

Bayesian Additive Regression Trees (BART) in Food Authenticity: A Classification Approach to Food Fraud Detection

Feature engineering plays a critical role in handling hyperspectral data and is essential for identifying key wavelengths in food fraud detection. This study employs Bayesian Additive Regression Trees (BART), a flexible machine learning approach, to discriminate and classify samples of olive oil based on their level of purity. Leveraging its built-in variable selection mechanism, we employ BART to effectively identify the most representative spectral features and to capture the complex interactions among variables. We use network representation to illustrate our findings, highlighting the competitiveness of our proposed methodology. Results demonstrate that when principal component analysis is used for dimensionality reduction, BART outperforms state-of-the-art models, achieving a classification accuracy of 96.8% under default settings, which further improves to 97.2% after hyperparameter tuning. If we leverage a variable selection procedure within BART, the model achieves perfect classification performance on this dataset, improving upon previous optimal results both in terms of accuracy and interpretability. Our results demonstrate that three key wavelengths, 1160.71 nm, 1328.57 nm, and 1389.29 nm, play a central role in discriminating the olive oil samples, thus highlighting an application of our methodology in the context of food quality. Further analysis reveals that these variables do not function independently but rather interact synergistically to achieve accurate classification, and improved detection speed.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.NA

Fast spectral separation method for kinetic equation with anisotropic non-stationary collision operator retaining micro-model fidelity

We present a generalized, data-driven collisional operator for one-component plasmas, learned from molecular dynamics simulations, to extend the collisional kinetic model beyond the weakly coupled regime. The proposed operator features an anisotropic, non-stationary collision kernel that accounts for particle correlations typically neglected in classical Landau formulations. To enable efficient numerical evaluation, we develop a fast spectral separation method that represents the kernel as a low-rank tensor product of univariate basis functions. This formulation admits an $O(N \log N)$ algorithm via fast Fourier transforms and preserves key physical properties, including discrete conservation laws and the H-theorem, through a structure-preserving central difference discretization. Numerical experiments demonstrate that the proposed model accurately captures plasma dynamics in the moderately coupled regime beyond the standard Landau model while maintaining high computational efficiency and structure-preserving properties.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Physics-informed data-driven machine health monitoring for two-photon lithography

Two-photon lithography (TPL) is a sophisticated additive manufacturing technology for creating three-dimensional (3D) micro- and nano-structures. Maintaining the health of TPL systems is critical for ensuring consistent fabrication quality. Current maintenance practices often rely on experience rather than informed monitoring of machine health, resulting in either untimely maintenance that causes machine downtime and poor-quality fabrication, or unnecessary maintenance that leads to inefficiencies and avoidable downtime. To address this gap, this paper presents three methods for accurate and timely monitoring of TPL machine health. Through integrating physics-informed data-driven predictive models for structure dimensions with statistical approaches, the proposed methods are able to handle increasingly complex scenarios featuring different levels of generalizability. A comprehensive experimental dataset that encompasses six process parameter combinations and six structure dimensions under two machine health conditions was collected to evaluate the effectiveness of the proposed approaches. Across all test scenarios, the approaches are shown to achieve high accuracies, demonstrating excellent effectiveness, robustness, and generalizability. These results represent a significant step toward condition-based maintenance for TPL systems.
Read the full abstract

Machine Learning (Statistics)

The Minimax Lower Bound of Kernel Stein Discrepancy Estimation

Kernel Stein discrepancies (KSDs) have emerged as a powerful tool for quantifying goodness-of-fit over the last decade, featuring numerous successful applications. To the best of our knowledge, all existing KSD estimators with known rate achieve $\sqrt n$-convergence. In this work, we present two complementary results (with different proof strategies), establishing that the minimax lower bound of KSD estimation is $n^{-1/2}$ and settling the optimality of these estimators. Our first result focuses on KSD estimation on $\mathbb R^d$ with the Langevin-Stein operator; our explicit constant for the Gaussian kernel indicates that the difficulty of KSD estimation may increase exponentially with the dimensionality $d$. Our second result settles the minimax lower bound for KSD estimation on general domains.
Read the full abstract

All authors Jose Cribeiro-Ramallo, Agnideep Aich, Florian Kalinke, Ashit Baran Aich, Zoltán Szabó

Machine Learning (Statistics) · Cross-listed from cs.LG

AlignFlow: Improving Flow-based Generative Models with Semi-Discrete Optimal Transport

Flow-based Generative Models (FGMs) effectively transform noise into complex data distributions. Incorporating Optimal Transport (OT) to couple noise and data during FGM training has been shown to improve the straightness of flow trajectories, enabling more effective inference. However, existing OT-based methods estimate the OT plan using (mini-)batches of sampled noise and data points, which limits their scalability to large and high-dimensional datasets in FGMs. This paper introduces AlignFlow, a novel approach that leverages Semi-Discrete Optimal Transport (SDOT) to enhance the training of FGMs by establishing an explicit, optimal alignment between noise distribution and data points with guaranteed convergence. SDOT computes a transport map by partitioning the noise space into Laguerre cells, each mapped to a corresponding data point. During FGM training, i.i.d. noise samples are paired with data points via the SDOT map. AlignFlow scales well to large datasets and model architectures with negligible computational overhead. Experimental results show that AlignFlow improves the performance of a wide range of state-of-the-art FGM algorithms and can be integrated as a plug-and-play component. Code is available at: https://github.com/konglk1203/AlignFlow.
Read the full abstract

All authors Lingkai Kong, Molei Tao, Yang Liu, Bryan Wang, Jinmiao Fu, Chien-Chih Wang, Huidong Liu

Machine Learning (Statistics)

The Coverage Principle: How Pre-Training Enables Post-Training

Language models demonstrate remarkable abilities when pre-trained on large text corpora and fine-tuned for specific tasks, but how and why pre-training shapes the success of the final model remains poorly understood. Notably, although pre-training success is often quantified by cross-entropy loss, cross-entropy can be a poor predictor of downstream performance. Instead, we provide a theoretical perspective on this relationship through the lens of coverage, which quantifies the probability mass the pre-trained model places on high-quality responses and which is necessary and sufficient for post-training and test-time scaling methods such as Best-of-N to succeed. Our main results develop an understanding of the coverage principle, a phenomenon whereby next-token prediction (more generally, maximum likelihood) implicitly optimizes toward a model with good coverage. In particular, we uncover a mechanism that explains the power of coverage in predicting downstream performance: coverage generalizes faster than cross-entropy, avoiding spurious dependence on problem-dependent parameters such as the sequence length. We also study practical algorithmic interventions with provable benefits for improving coverage, including (i) model/checkpoint selection procedures, (ii) gradient normalization schemes, and (iii) test-time decoding strategies.
Read the full abstract

All authors Fan Chen, Audrey Huang, Noah Golowich, Sadhika Malladi, Adam Block, Jordan T. Ash, Akshay Krishnamurthy, Dylan J. Foster

Machine Learning (Statistics)

The Tree-SNE Tree Exists

The clustering and visualisation of high-dimensional data is a ubiquitous task in modern data science. Popular techniques include nonlinear dimensionality reduction methods like t-SNE or UMAP. These methods face the `scale-problem' of clustering: when dealing with the MNIST dataset, do we want to distinguish different digits or do we want to distinguish different ways of writing the digits? The answer is task dependent and depends on scale. We revisit an idea of Robinson & Pierce-Hoffman that exploits an underlying scaling symmetry in t-SNE to replace 2-dimensional with (2+1)-dimensional embeddings where the additional parameter accounts for scale. This gives rise to the t-SNE tree (short: tree-SNE). We prove that the optimal embedding depends continuously on the scaling parameter for all initial conditions outside a set of measure 0: the tree-SNE tree exists. This idea conceivably extends to other attraction-repulsion methods and is illustrated on several examples.
Read the full abstract

Machine Learning (Statistics)

Reliable data clustering with Bayesian community detection

From neuroscience and genomics to systems biology and ecology, researchers rely on clustering similarity data to uncover modular structure. Most methods work in two steps: first reduce noise, then cluster. The first step runs without regard to the second: it forces arbitrary choices, discards useful information, and can impose structure where none exists. Here we bring the two steps together. Using established Bayesian community detection methods, we let one minimum description length criterion choose both how much to sparsify the data and how to cluster it. This one-step approach keeps all the samples for inference, removes the arbitrary threshold choice, and returns a single cluster when there is no real structure. On synthetic data, our approach recovers clusters under more noise and with fewer samples than standard methods. On gene co-expression data from the human brain, it finds more coherent and more stable clusters than WGCNA, the standard tool in the field. Uniting sparsification and clustering in a single model-selection step provides a principled, noise-resistant way to uncover modular structure in similarity data across fields.
Read the full abstract

Machine Learning (Statistics)

From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons

We revisit the Universal Approximation Theorem(UAT) through the lens of the tropical geometry of neural networks and introduce a constructive, geometry-aware initialization for sigmoidal multi-layer perceptrons (MLPs). Tropical geometry shows that Rectified Linear Unit (ReLU) networks admit decision functions with a combinatorial structure often described as a tropical rational, namely a difference of tropical polynomials. Focusing on planar binary classification, we design purely sigmoidal MLPs that adhere to the finite-sum format of UAT: a finite linear combination of shifted and scaled sigmoids of affine functions. The resulting models yield decision boundaries that already align with prescribed shapes at initialization and can be refined by standard training if desired. This provides a practical bridge between the tropical perspective and smooth MLPs, enabling interpretable, shape-driven initialization without resorting to ReLU architectures. We focus on the construction and empirical demonstrations in two dimensions; theoretical analysis and higher-dimensional extensions are left for future work.
Read the full abstract

Statistics Applications

Data-driven Calibration Sample Selection and Forecast Combination in Electricity Price Forecasting: An Application of the ARHNN Method

Calibration sample selection and forecast combination are two simple yet powerful tools used in forecasting. They can be combined with a variety of models to significantly improve prediction accuracy, at the same time offering easy implementation and low computational complexity. While their effectiveness has been repeatedly confirmed in prior scientific literature, the topic is still underexplored in the field of electricity price forecasting. In this research article we apply the Autoregressive Hybrid Nearest Neighbors (ARHNN) method to three long-term time series describing the German, Spanish and New England electricity markets. We show that it outperforms popular literature benchmarks in terms of forecast accuracy by up to 10%. We also propose two simplified variants of the method, granting a vast decrease in computation time with only minor loss of prediction accuracy. Finally, we compare the forecasts' performance in a battery storage system trading case study. We find that using a forecast-driven strategy can achieve up to 80% of theoretical maximum profits while trading, demonstrating business value in practical applications.
Read the full abstract

Statistical Methodology

Asymptotic distribution of the global clustering coefficient in a random annulus graph

The global clustering coefficient is an effective measure for analyzing and comparing the structures of complex networks. The random annulus graph is a modified version of the well-known Erdős-Rényi random graph. It has been recently proposed in modeling network communities. This paper investigates the asymptotic distribution of the global clustering coefficient in a random annulus graph. It is demonstrated that the standardized global clustering coefficient converges in law to the standard normal distribution. The result is established using the asymptotic theory of degenerate U-statistics with a sample-size dependent kernel. As far as we know, this method is different from established approaches for deriving asymptotic distributions of network statistics. Moreover, we get the explicit expression of the limit of the global clustering coefficient.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores

Pairwise comparisons of large language models using total variation distance mutual information (TVD-MI) produce binary critic decisions per pair. We show that averaging TVD-MI's binary trials yields centered-probability scores with additive structure suitable for item-response theory (IRT) without nonlinear link functions. Maximum-likelihood approaches to IRT use logistic links, but we find empirically that these transformations introduce curvature that breaks additivity: across three domains, the identity link yields median curl on raw data of 0.080-0.150 (P95 = [0.474, 0.580]), whereas probit/logit introduce substantially higher violations (median [0.245, 0.588], P95 [0.825, 2.252]). We derive this clipped-linear model from Gini entropy maximization, yielding a box-constrained least-squares formulation that handles boundary saturation. At 33% coverage, we achieve holdout RMSE $0.117 \pm 0.008$ while preserving agent rankings (Spearman $\rho = 0.972 \pm 0.015$), three times fewer evaluations than full dense. Judge robustness analysis (GPT-4o-mini vs. Llama3-70b) shows strong agreement in agent rankings ($\rho = 0.872$) and consistent identity-link advantage. TVD-MI's geometry is best preserved by identity mapping for efficient LLM evaluation, applicable to other bounded-response domains.
Read the full abstract

Statistical Methodology

A formative measurement validation methodology for survey questionnaires

Model misspecification of formative indicators remains a widely documented issue across academic literature, yet scholars lack a clear consensus on pragmatic, prescriptive approaches to manage this gap. This ambiguity forces researchers to rely on psychometric frameworks primarily intended for reflective models, and thus risks misleading findings. This article introduces a Multi-Step Validation Methodology Framework specifically designed for formative constructs in survey-based research. The proposed framework is grounded in an exhaustive literature review and integrates essential pilot diagnostics through descriptive statistics and multicollinearity checks. The methodology provides researchers with the necessary theoretical and structural clarity to finally justify and adhere to appropriate validation techniques that accurately account for the causal nature of the constructs while ensuring high psychometric and statistical integrity.
Read the full abstract

Statistical Methodology

EM Approaches to Nonparametric Estimation for Mixture of Linear Regressions

In a mixture of linear regression model, the regression coefficients are treated as random vectors that may follow either a continuous or discrete distribution. We propose two Expectation-Maximization (EM) algorithms to estimate this prior distribution. The first algorithm solves a kernelized version of the nonparametric maximum likelihood estimation (NPMLE). This method not only recovers continuous prior distributions but also accurately estimates the number of clusters when the prior is discrete. The second algorithm, designed to approximate the NPMLE, targets prior distributions with a density. It also performs well for discrete priors when combined with a post-processing step. We study the convergence properties of both algorithms and demonstrate their effectiveness through simulations and applications to real datasets.
Read the full abstract

Machine Learning (Statistics)

A Geometric Approach to Optimal Experimental Design

We introduce a novel geometric framework for optimal experimental design (OED). Traditional OED approaches, such as those based on mutual information, rely explicitly on probability densities, leading to restrictive invariance properties. To address these limitations, we propose the mutual transport dependence (MTD), a measure of statistical dependence grounded in optimal transport theory which provides a geometric objective for optimizing designs. Unlike conventional approaches, the MTD can be tailored to specific downstream estimation problems by choosing appropriate geometries on the underlying spaces. We demonstrate that our framework produces high-quality designs while offering a flexible alternative to standard information-theoretic techniques.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Provable Unlearning with Gradient Ascent on Two-Layer ReLU Neural Networks

Machine Unlearning aims to remove specific data from trained models, addressing growing privacy and ethical concerns. We provide a theoretical analysis of a simple and widely used method - gradient ascent - used to reverse the influence of a specific data point without retraining from scratch. Leveraging the implicit bias of gradient descent towards solutions that satisfy the Karush-Kuhn-Tucker (KKT) conditions of a margin maximization problem, we quantify the quality of the unlearned model by evaluating how well it satisfies these conditions w.r.t. the retained data. To formalize this idea, we propose a new success criterion, termed \textbf{$(\epsilon, \delta, \tau)$-successful} unlearning, and show that, for both linear models and two-layer neural networks with high dimensional data, a properly scaled gradient-ascent step satisfies this criterion and yields a model that closely approximates the retrained solution on the retained data. We also show that gradient ascent performs successful unlearning while still preserving generalization in a synthetic Gaussian-mixture setting.
Read the full abstract

Statistics Theory

Regression Model Selection Under General Conditions

Model selection criteria are one of the most important tools in statistics. Proofs showing a model selection criterion is asymptotically optimal are tailored to the type of model (linear regression, quantile regression, penalized regression, etc.), the estimation method (linear smoothers, maximum likelihood, generalized method of moments, etc.), the type of data (i.i.d., dependent, high dimensional, etc.), and the type of model selection criterion. Moreover, assumptions are often restrictive and unrealistic making it a slow and winding process for researchers to determine if a model selection criterion is selecting an optimal model. This paper provides general proofs showing asymptotic optimality for a wide range of model selection criteria under general conditions. This paper not only asymptotically justifies model selection criteria for most situations, but it also unifies and extends a range of previously disparate results.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Causal Discovery for Linear DAGs with Dependent Latent Variables via Higher-order Cumulants

This paper addresses the problem of estimating causal directed acyclic graphs in linear non-Gaussian acyclic models with latent confounders (LvLiNGAM). Existing methods assume mutually independent latent confounders or cannot properly handle models with causal relationships among observed variables. We propose a novel algorithm that identifies causal DAGs in LvLiNGAM, allowing causal structures among latent variables, among observed variables, and between the two. The proposed method leverages higher-order cumulants of observed data to identify the causal structure. Extensive simulations and experiments with real-world data demonstrate the validity and practical utility of the proposed algorithm.
Read the full abstract

Statistics Applications

Bayes-ically fair: A Bayesian Ranking of the Olympic Medal Table

Evaluating a country's sporting success provides insight into its decision-making and infrastructure for developing athletic talent. The Olympic Games serve as a global benchmark, yet conventional medal rankings can be unduly influenced by population size. We propose a Bayesian ranking scheme to rank the performance of National Olympic Committees by their "long-run" medals-to-population ratio. The algorithm aims to mitigate the influence of large populations and reduce the stochastic fluctuations for smaller nations by applying shrinkage. These long-run rankings provide a more stable and interpretable ordering of national sporting performance across games compared to existing methods.
Read the full abstract