Peer Revenue Play credits. Real papers.

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Wednesday 1 April

Other Statistics

Debiased Estimators in High-Dimensional Regression: A Review and Replication of Javanmard and Montanari (2014)

High-dimensional statistical settings ($p \gg n$) pose fundamental challenges for classical inference, largely due to bias introduced by regularized estimators such as the LASSO. To address this, Javanmard and Montanari (2014) propose a debiased estimator that enables valid hypothesis testing and confidence interval construction. This report examines their debiased LASSO framework, which yields asymptotically normal estimators in high-dimensional settings. The key theoretical results underlying this approach are presented. Specifically, the construction of an optimized debiased estimator that restores asymptotic normality, which enables the computation of valid confidence intervals and $p$-values. To evaluate the claims of Javanmard and Montanari, a subset of the original simulation study and the real-data analysis is presented. The original empirical analysis is extended to the desparsified LASSO, which is referenced but not implemented in the original study. The results demonstrate that while the debiased LASSO achieves reliable coverage and controls Type I error, the LASSO projection estimator can offer improved power in idealized low-signal settings without compromising error rates. The results reveal a trade-off: the LASSO projection estimator performs well in low-signal settings, while Javanmard and Montanari's method is more robust to complex correlations, improving precision and signal detection in real data.
Read the full abstract

Statistics Theory · Cross-listed from math.AP

Sharp local sparsity of regularized optimal transport

In recent years, the use of entropy-regularized optimal transport with $L^p$-type entropies has become increasingly popular. In this setting, the solutions are sparse, in the sense that the support of the regularized optimal coupling, $\mathrm{supp}(\pi_\varepsilon)$, shrinks to the support of the original optimal transport problem as $\varepsilon \to 0$. The main open question concerns the rate of this convergence. In this paper, we obtain sharp local results away from the boundary. We prove that the supports $\mathrm{supp}(\pi_\varepsilon(\cdot \mid x))$ of the conditional measures, $\pi_\varepsilon(\cdot \mid x)$, behave like balls of radius $\varepsilon^\frac 1 {d(p-1)+2}$. This allows us to show that the regularized potentials are uniformly strongly convex and to derive the rate of convergence of these potentials toward their unregularized limit. Our results generalize the results of (González-Sanz and Nutz, SIAM J.~Math.~Anal.) and (Wiesel and Xu, Ibid.) to the multivariate case and beyond the case of self-transport.
Read the full abstract

Machine Learning (Statistics)

Deconfounding Scores and Representation Learning for Causal Effect Estimation with Weak Overlap

Overlap, also known as positivity, is a key condition for causal treatment effect estimation. Many popular estimators suffer from high variance and become brittle when features differ strongly across treatment groups. This is especially challenging in high dimensions: the curse of dimensionality can make overlap implausible. To address this, we propose a class of feature representations called deconfounding scores, which preserve both identification and the target of estimation; the classical propensity and prognostic scores are two special cases. We characterize the problem of finding a representation with better overlap as minimizing an overlap divergence under a deconfounding score constraint. We then derive closed-form expressions for a class of deconfounding scores under a broad family of generalized linear models with Gaussian features and show that prognostic scores are overlap-optimal within this class. We conduct extensive experiments to assess this behavior empirically.
Read the full abstract

All authors Oscar Clivio, Alexander D'Amour, Alexander Franks, David Bruns-Smith, Chris Holmes, Avi Feller

Statistics Applications

Beyond the Beta Lorenz Curve: A New Parametric Family for Poverty and Inequality Estimation

The estimation of inequality and poverty measures is frequently constrained by a lack of individual data. When only income shares are available, the Beta Lorenz curve introduced by Kakwani (Econometrica, 48, 1980) has become a standard tool for reconstructing income distributions. Together with the General Quadratic (GQ) Lorenz curve, it is used by the World Bank to produce official poverty estimates when microdata are unavailable. In this paper, we show that Kakwani's specification does not generally satisfy the formal requirements of a genuine Lorenz curve and introduce a new Lorenz curve rooted in the corrected parameter space. Using more than 1,700 datasets, we show that the proposed model yields valid Lorenz curves in all applications and consistently outperforms competing specifications in both accuracy and sampling precision. The GQ, by contrast, fails to provide genuine curves in 15 percent of datasets and underestimates extreme poverty severity in 98 percent of them.
Read the full abstract

Statistical Methodology

Non-ignorable fuzziness in granular counts: the case of RNA-seq data

RNA-seq count data are often affected by read-to-gene alignment ambiguity, especially in high-dimensional transcriptomics. This type of ambiguity can be conveniently expressed through granular counts, namely fuzzy-valued observations of latent discrete quantities. We study a class of fuzzy-reporting mechanisms and show that, when reporting exploits graded membership, ignorability fails generically, leading to a coarsening-not-at-random structure. A hierarchical model is then introduced as a tractable instance of this construction and illustrated using RNA-seq data.
Read the full abstract

All authors Antonio Calcagnì, Arianna Consiglio, Przemyslaw Grzegorzewski, Corrado Mencar

Machine Learning (Statistics)

Inverse-Free Sparse Variational Gaussian Processes

Gaussian processes (GPs) offer appealing properties but are costly to train at scale. Sparse variational GP (SVGP) approximations reduce cost yet still rely on Cholesky decompositions of kernel matrices, ill-suited to low-precision, massively parallel hardware. While one can construct valid variational bounds that rely only on matrix multiplications (matmuls) via an auxiliary matrix parameter, optimising them with off-the-shelf first-order methods is challenging. We make the inverse-free approach practical by proposing a better-conditioned bound and deriving a matmul-only natural-gradient update for the auxiliary parameter, markedly improving stability and convergence. We further provide simple heuristics, such as step-size schedules and stopping criteria, that make the overall optimisation routine fit seamlessly into existing workflows. Across regression and classification benchmarks, we demonstrate that our method 1) serves as a drop-in replacement in SVGP-based models (e.g., deep GPs), 2) recovers similar performance to traditional methods, and 3) can be faster than baselines when well tuned.
Read the full abstract

All authors Stefano Cortinovis, Laurence Aitchison, Stefanos Eleftheriadis, Mark van der Wilk

Statistical Methodology

Convergence of projected stochastic natural gradient variational inference for various step size and sample or batch size schedules

Stochastic natural gradient variational inference (NGVI) is a popular and efficient algorithm for Bayesian inference. Despite empirical success, the convergence of this method is still not fully understood. In this work, we define and study a projected stochastic NGVI when variational distributions form an exponential family. Stochasticity arises when either gradients are intractable expectations or large sums. We prove new non-asymptotic convergence results for combinations of constant or decreasing step sizes and constant or increasing sample/batch sizes. When all hyperparameters are fixed, NGVI is shown to converge geometrically to a neighborhood of the optimum, while we establish convergence to the optimum with rates of the form $\mathcal{O}\left(\frac{1}{T^{\rho}} \right)$, possibly with $\rho \geq 1$, for all other combinations of step size and sample/batch size schedules. These rates apply when the target posterior distribution is close in some sense to the considered exponential family. Our theoretical results extend existing NGVI and stochastic optimization results and provide more flexibility to adjust, in a principled way, step sizes and sample/batch sizes in order to meet speed, resources, or accuracy constraints.
Read the full abstract

Statistics Theory · Cross-listed from cs.CL

Common TF-IDF variants arise as key components in the test statistic of a penalized likelihood-ratio test for word burstiness

TF-IDF is a classical formula that is widely used for identifying important terms within documents. We show that TF-IDF-like scores arise naturally from the test statistic of a penalized likelihood-ratio test setup capturing word burstiness (also known as word over-dispersion). In our framework, the alternative hypothesis captures word burstiness by modeling a collection of documents according to a family of beta-binomial distributions with a gamma penalty term on the precision parameter. In contrast, the null hypothesis assumes that words are binomially distributed in collection documents, a modeling approach that fails to account for word burstiness. We find that a term-weighting scheme given rise to by this test statistic performs comparably to TF-IDF on document classification tasks. This paper provides insights into TF-IDF from a statistical perspective and underscores the potential of hypothesis testing frameworks for advancing term-weighting scheme development.
Read the full abstract

All authors Zeyad Ahmed, Paul Sheridan, Michael McIsaac, Aitazaz A. Farooque

Statistical Computation

Implementation and Workflows for INLA-Based Approximate Bayesian Structural Equation Modelling

Bayesian structural equation modelling (BSEM) offers many advantages such as principled uncertainty quantification, small-sample regularisation, and flexible model specification. However, the Markov chain Monte Carlo (MCMC) methods on which it relies are computationally prohibitive for the iterative cycle of specification, criticism, and refinement that careful psychometric practice demands. We present INLAvaan, an R package for fast, approximate Bayesian SEM built around the Integrated Nested Laplace Approximation (INLA) framework for structural equation models developed by Jamil & Rue (2026, arXiv:2603.25690 [stat.ME]). This paper serves as a companion manuscript that describes the architectural decisions and computational strategies underlying the package. Two substantive applications – a 256-parameter bifactor circumplex model and a multilevel mediation model with full-information missing-data handling – demonstrate the approach on specifications where MCMC would require hours of run time and careful convergence work. In constrast, INLAvaan delivers calibrated posterior summaries in seconds.
Read the full abstract

Statistics Applications

Feature Reconstruction and Monitoring of Load Test Data under Varying Environmental Conditions

System outputs in Structural Health Monitoring (SHM), such as sensor measurements or extracted features like eigenfrequencies, are influenced not only by (potential) damage but also by environmental and operational variables (EOV). Identifying these factors and removing their effects from the data is essential before proceeding with further analysis. Most existing methods for this task focus on the expected values of system outputs, e.g., using different types of response surface modeling. However, it has been shown that confounding variables can also affect the (co-)variance of and between system outputs. This is particularly important because the covariance matrix is an essential building block in many damage detection methods in SHM. Beyond standard response surface modeling, a nonparametric kernel approach can be used to estimate a conditional covariance matrix that can change depending on the identified confounding factor. This improves our understanding of how, e.g., temperature affects the system outputs. In this work, we present a new confounder-adjusted version of feature reconstruction. It uses the conditional covariance matrix as the basis for (conditional) principal component analysis. The resulting (conditional) principal component scores are then used to reconstruct system outputs with the confounding influences removed. In particular, the new approach eliminates the confounders effect on both the mean and the covariance. As will be shown on load test data from the Vahrendorfer Stadtweg bridge in Hamburg, Germany, the reconstructed features can then be employed for monitoring, e.g., using an appropriate control chart, resulting in fewer false alarms and a higher probability of detecting damage.
Read the full abstract

All authors Lizzie Neumann, Philipp Wittenberg, Alexander Mendler, Jan Gertheiss

Statistics Theory

Semiparametric Fisher Information in Models parametrized by a Normed Space

This paper studies semiparametric Fisher information in models parametrized by general normed spaces. The main contribution is to establish that positive semiparametric Fisher information is equivalent to the gradient of the parameter of interest lying in the range of the adjoint score operator. This result generalizes a key theorem Van Der Vaart (1991) and provides a unified framework linking differentiability and information, beyond Hilbert spaces. The paper develops a normed-space mean-square-differentiable models for two canonical problems: estimation of the average of a known transformation and estimation of a density at a point. In these applications, it shows that positive information holds if and only if the transformation has finite variance and if and only if the density has positive mass at the evaluation point, respectively. These findings offer a novel information-theoretic perspective on known minimax results and clarify the conditions under which root-n estimation is possible.
Read the full abstract

Statistical Methodology

Covariance Matrix Estimation for High-Dimensional Interval-Valued Data with Positive Definiteness

In the realm of high-dimensional data analysis, the estimation of covariance matrices is a fundamental task, and this holds true for interval-valued data as well. However, there is no unified definition for the covariance matrix of interval-valued data, let alone established estimation methods in high-dimensional settings. This paper presents a novel approach to estimating covariance matrices for high-dimensional interval-valued data while ensuring positive definiteness. We begin by assuming that the upper and lower bounds of interval-valued variables share the same dependency structure. Based on this assumption, we extend the classical soft-thresholding covariance matrix estimator to the interval-valued scenario, referred to as the Interval-valued Soft-Thresholding (IST) estimator. Subsequently, to ensure the positive definiteness of the estimator, we impose a positive definiteness constraint on the IST estimator. We derive an alternating direction method to solve the proposed problem and establish its convergence. Under some very mild conditions, we develop a non-asymptotic statistical theory for the proposed estimator. Simulation studies and applications to high-frequency financial data from the CSI 300 Index demonstrated the effectiveness of the proposed estimator.
Read the full abstract

All authors Wan Tian, Wenhao Cui, Rui Zhang, Bingyi Jing, Yang Liu, Yijie Peng

Statistics Theory · Cross-listed from astro-ph.CO

A Geometric Theory of Cosmological Structure via Entropic Curvature in Wasserstein Space

We construct a geometric framework for cosmological large-scale structure based on optimal transport theory and Wasserstein geometry. In this framework, Ricci curvature on the probability measure space $\mathcal{P}_2(M)$ is characterized by the geodesic convexity of entropy and is formulated as the response of probability distributions to optimal transport. We introduce effective Ricci curvatures $K_{\mathrm{eff}}^{(\infty)}$ and $K_{\mathrm{eff}}^{(N)}$ associated with Kullback–Leibler-type and Rényi-type entropies, corresponding respectively to the curvature-dimension conditions CD$(K,\infty)$ and CD$(K,N)$. By localizing these curvatures to finite scales using local and reference measures, we construct curvature indicators applicable to observational data. Under a local quadratic approximation, the effective curvature reduces to the Hessian of the log-density, showing that conventional Hessian-based structure classifications arise as a limiting case of the present framework. We further show that effective curvature depends on observational scale and formulate this dependence as a scale flow, distinct from Ricci flow because it describes a change of resolution rather than a time evolution of geometry. Treating curvature as a random field then extends the statistical description of density fields: curvature statistics are given by higher-order weighted integrals of the power spectrum and by spatial derivatives of the correlation function, emphasizing geometric rather than amplitude information. This framework provides a unified connection between optimal transport geometry and cosmological structure analysis, and offers a new perspective on multiscale structure and nonlinear statistics.
Read the full abstract

Statistics Theory

Empirical tail dependence functions in high dimensions: uniform linearizations and inference

The analysis of extremal dependence in high dimensions is a key challenge in modern extreme-value statistics. Existing methodology primarily focuses on modeling and estimation of extremal dependence structures, often supported by concentration bounds for empirical tail quantities. However, comparatively little is known about general inferential procedures in high-dimensional extremes. In this paper, we develop foundational results that enable inference for rank-based empirical tail dependence coefficients, stable tail dependence functions, and functionals derived from them. We start by establishing finite-sample probability bounds that quantify the linearization error for such estimators uniformly over collections of coordinates. Moreover, we derive high-dimensional central limit theorems and establish the validity of multiplier bootstrap procedures for collections of empirical tail dependence statistics. Within an asymptotic framework, our results allow the dimension to grow exponentially with the effective sample size. We illustrate the usefulness of the results through two applications: uniform expansions and normal approximations for M-estimators of tail dependence parameters and inference for spatial isotropy based on collections of tail dependence functions.
Read the full abstract

All authors Axel Bücher, Yeonjoon Choi, Katharina Effertz, Stanislav Volgushev

Statistics Applications · Cross-listed from astro-ph.GA

Revisiting Marked Galaxy Clustering from a Joint Point Process Perspective

Marked correlation functions, in which galaxy properties such as luminosity or stellar mass are treated as marks, are widely used to test models of galaxy formation. In astronomy, however, these statistics are typically implemented as summary measures that do not preserve the joint structure of mark pairs conditioned on separation. In this work, we formulate galaxies as points $(x,m)$ on the product space $\mathbb{R}^3\times\mathcal{M}$, where $x$ denotes position and $m$ a mark, and introduce the joint pair correlation function $g(r;m_1,m_2)$ as the fundamental quantity describing mark-dependent clustering. We further define a diagnostic quantity $\Delta_{\mathrm{ind}}(r;m_1,m_2)$ that locally quantifies deviations from the independence hypothesis relative to spatial clustering alone, thereby providing a projection-free description of which mark pairs are over- or underrepresented at a given separation scale. Within this framework, commonly used diagnostics such as the inhomogeneous cross-$J$ function are naturally interpreted as summary statistics obtained through averaging over mark sets and geometric-event-based reductions of the joint structure. This perspective clarifies that previously discussed marked effects, including assembly bias, correspond to projections of an underlying joint dependence, and that observationally accessible information is the existence of non-factorizable joint structure itself. The present formulation provides both a fundamental quantity and practical diagnostics for its characterization.
Read the full abstract

Machine Learning (Statistics)

Scenario theory for multi-criteria data-driven decision making

The scenario approach provides a powerful data-driven framework for designing solutions under uncertainty with rigorous probabilistic robustness guarantees. Existing theory, however, primarily addresses assessing robustness with respect to a single appropriateness criterion for the solution based on a dataset, whereas many practical applications - including multi-agent decision problems - require the simultaneous consideration of multiple criteria and the assessment of their robustness based on multiple datasets, one per criterion. This paper develops a general scenario theory for multi-criteria data-driven decision making. A central innovation lies in the collective treatment of the risks associated with violations of individual criteria, which yields substantially more accurate robustness certificates than those derived from a naive application of standard results. In turn, this approach enables a sharper quantification of the robustness level with which all criteria are simultaneously satisfied. The proposed framework applies broadly to multi-criteria data-driven decision problems, providing a principled, scalable, and theoretically grounded methodology for design under uncertainty.
Read the full abstract

All authors Simone Garatti, Lucrezia Manieri, Alessandro Falsone, Algo Carè, Marco C. Campi, Maria Prandini

Statistical Methodology

Estimating causal effects of continuous-time dynamic treatments with unmeasured confounders

Modern medical research demands specialized causal inference methods evaluating complex continuous-time dynamic treatment regimens using observational data. For instance, obtaining the causal effects of intravenous administration, a continuous process involving dynamic adjustments of the treatment dose, can guide clinicians on drug use. However, the existing causal inference frameworks in longitudinal studies typically assume that time advances in discrete time steps. Therefore, this paper proposes a new methodology to estimate the causal effects of continuous-time dynamic treatments in the presence of unmeasured confounding. Unmeasured confounding is incorporated into estimating continuous-time Marginal Structural Models from a Bayesian perspective. Simulation demonstrates that compared to existing methods, the proposed approach can provide approximately unbiased estimates for target causal parameters across three degrees of confounding. The proposed method is applied to analyze the causal relationship between the intravenous oxytocin administration process and postpartum hemorrhage, leading to meaningful results that may guide clinicians in using oxytocin.
Read the full abstract

Statistical Methodology

Conformal Inference for Experimental Attrition in Social Science Research

Attrition in survey and field experiments presents a challenge for social science research. Common approaches to deal with this problem – such as complete case analysis, multiple imputation, and weighting methods – rely on strong assumptions that may not hold in practice. This paper introduces a new method that combines recent advances in statistical inference with established tools for handling missing data. The approach produces prediction intervals for treatment effects that are both robust and precise. Evidence from simulation studies shows that the method achieves better coverage and produces narrower intervals than common alternatives. The reanalysis of two recently published experiment studies illustrates how this framework allows researchers to compare treatment effects across participants who remain in the study, those who drop out, and the full sample. Taken together, these results highlight how the proposed approach provides a stronger foundation for causal inference in the presence of attrition.
Read the full abstract

Statistical Methodology

Tucker Diffusion Model for High-dimensional Tensor Generation

Statistical inference on large-dimensional tensor data has been extensively studied in the literature and widely used in economics, biology, machine learning, and other fields, but how to generate a structured tensor with a target distribution is still a new problem. As profound AI generators, diffusion models have achieved remarkable success in learning complex distributions. However, their extension to generating multi-linear tensor-valued observations remains underexplored. In this work, we propose a novel Tucker diffusion model for learning high-dimensional tensor distributions. We show that the score function admits a structured decomposition under the low Tucker rank assumption, allowing it to be both accurately approximated and efficiently estimated using a carefully tailored tensor-shaped architecture named Tucker-Unet. Furthermore, the distribution of generated tensors, induced by the estimated score function, converges to the true data distribution at a rate depending on the maximum of tensor mode dimensions, thereby offering a clear theoretical advantage over the naive vectorized approach, which has a product dependence. Empirically, compared to existing approaches, the Tucker diffusion model demonstrates strong practical potential in synthetic and real-world tensor generation tasks, achieving comparable and sometimes even superior statistical performance with significantly reduced training and sampling costs.
Read the full abstract

All authors Jianhua Guo, Xinbing Kong, Zeyu Li, Junfan Mao

Machine Learning (Statistics)

Denoising distances beyond the volumetric barrier

We study the problem of reconstructing the latent geometry of a $d$-dimensional Riemannian manifold from a random geometric graph. While recent works have made significant progress in manifold recovery from random geometric graphs, and more generally from noisy distances, the precision of pairwise distance estimation has been fundamentally constrained by the volumetric barrier, namely the natural sample-spacing scale $n^{-1/d}$ coming from the fact that a generic point of the manifold typically lies at distance of order $n^{-1/d}$ from the nearest sampled point. In this paper, we introduce a novel approach, Orthogonal Ring Distance Estimation Routine (ORDER), which achieves a pointwise distance estimation precision of order $n^{-2/(d+5)}$ up to polylogarithmic factors in $n$ in polynomial time. This strictly beats the volumetric barrier for dimensions $d > 5$. As a consequence of obtaining pointwise precision better than $n^{-1/d}$, we prove that the Gromov–Wasserstein distance between the reconstructed metric measure space and the true latent manifold is of order $n^{-1/d}$. This matches the Wasserstein convergence rate of empirical measures, demonstrating that our reconstructed graph metric is asymptotically as good as having access to the full pairwise distance matrix of the sampled points. Our results are proven in a very general setting which includes general models of noisy pairwise distances, sparse random geometric graphs, and unknown connection probability functions.
Read the full abstract

Statistical Methodology

Testing for lack of fit in paired comparison data

Linear stochastic transitivity is a central assumption in paired comparison models that is rarely verified in practice. Empirical violations, however, are common and can substantially affect inference and ranking. We develop a class of tests for detecting lack of fit in cardinal paired comparison models, where lack of fit is characterized by the presence of cyclical preferences among subsets of items. We propose a suite of tests adapted to different regimes governing the growth of the comparison graph. For a fixed number of items, the proposed procedures exhibit substantially improved power relative to the classical Kendall–Smith test and its cardinal analogue. We further extend the framework to high–dimensional, sparse comparison graphs near the connectivity threshold in random graph models. The theoretical analysis characterizes the behavior of the tests under both the null and alternative, with particular emphasis on limits of detectability and consistency. Simulation studies corroborate the theoretical findings, and applications to real data uncover substantial and previously unrecognized intransitivity and structural lack of fit.
Read the full abstract

Other Statistics

Distributional regression models for meta-analysis

Meta-analyses are regarded as the highest level in the hierarchy of evidence, yet standard models traditionally concentrated on estimating the mean effect size, often under restrictive assumptions about the underlying distribution, such as homogeneous variance, symmetric shapes. We introduce a distributional regression framework for meta-analysis that generalizes these conventional models by allowing all parameters of the effect size distribution, such as location, scale, and shape, to be modelled as functions of explanatory variables. This unified framework accommodates a wide range of existing models, including random-effects, multilevel, multivariate, location-scale, and outlier-robust meta-analyses, as special cases. We provide an illustrative example, using 67,393 meta-analyses from the Cochrane Database of Systematic Reviews, employing location-scale models to investigate whether smaller studies tend to report larger effect sizes (i.e., small-study effects) and exhibit greater heterogeneity. We discuss implementation strategies using existing software, considerations for model selection and pre-registration, and the need for further methodological development. By moving beyond the mean effect size, distributional regression enables researchers to explore systematic variation in distributional structure, facilitating the joint test of new hypotheses corresponding to multiple distributional parameters.
Read the full abstract

Statistical Methodology

Differentially Private One-Shot Federated Inference for Linear Mixed Models via Lossless Likelihood Reconstruction

One-shot federated learning enables multi-site inference with minimal communication. However, sharing summary statistics can still leak sensitive individual-level information when sites have only a small number of patients. In particular, shared cross-product summaries can reveal patient-level covariate patterns under discrete covariates. Motivated by this concern, this study proposes a differentially private one-shot federated inference framework for linear mixed models with a random-intercept working covariance. The method reconstructs the pooled likelihood from site-level summary statistics and applies a Gaussian mechanism to perturb these summaries, ensuring a site-level differential privacy. Cluster-robust variance estimators are developed that are computed directly from the privatized summaries. Robust variance provides valid uncertainty quantification even under covariance mis-specification. Under a multi-site asymptotic regime, the consistency and asymptotic normality of the proposed estimator are established and the leading-order statistical cost of privacy is characterized. Simulation studies show that moderate privacy noise substantially reduces reconstruction risk while maintaining competitive estimation accuracy as the number of sites increases. However, very strong privacy settings can lead to unstable standard errors when the number of sites is limited. An application using multi-site COVID-19 testing data demonstrates that meaningful privacy protection can be achieved with a modest loss of efficiency.
Read the full abstract

Statistical Methodology

Causal Inference for Unobservable Multivariate Outcomes, with Applications to Brain Effective Connectivity

Evaluating the causal effect of an intervention on multivariate outcomes is challenging when the outcomes are interdependent and derived rather than directly observed. Effective connectivity, which summarizes the directional neural communication between brain regions, is one such derived relational outcome. Estimating how external interventions affect effective connectivity introduces two layers of causal inference problems: identifying directional relationships among brain regions from high-dimensional neuroimaging time series and estimating the causal effect of the intervention on these derived relationships. Each layer introduces distinct biases. The first arises from within-outcome dependencies unrelated to the intervention; to address this, we propose a sample-splitting method for estimating meaningful, and potentially causally informative, effective connectivity measures. The second arises from confounding between the intervention and the derived outcomes; to address this, we apply inverse probability weighting methods and incorporate multiple testing when causal effects on multiple components of the outcomes are of interest. We demonstrate, through theoretical results and simulations, that the proposed methods are asymptotically valid under certain conditions with effective type-I and familywise error control. Finally, we apply the proposed methods to examine the causal effect of amyloid on effective connectivity using the resting-state fMRI data from the Alzheimer's Disease Neuroimaging Initiative database.
Read the full abstract

Statistics Applications

Spatiotemporal Characterization of Overdose Mortality in Georgia, USA Using Spectral and Nonlinear Interaction Analysis, 2003-2021

Drug overdose mortality in the United States exhibits strong geographic heterogeneity and complex temporal evolution, yet most spatiotemporal studies focus on trends and risks without explicitly characterizing the underlying dynamical structure of overdose trajectories. We develop a nonlinear spectral-spatiotemporal framework to analyze county-level overdose mortality in the state of Georgia from 2003 to 2021. Annual mortality rates are decomposed into low- and high-frequency components to distinguish long-term epidemic pressure from short-term variability, and nonlinear cross-frequency interaction is quantified using bispectral intensity. Counties are grouped into spectral phenotypes using unsupervised clustering, and single-breakpoint change-point models are used to identify regime shifts and quantify post-break acceleration across phenotypes. We find that overdose dynamics across Georgia are dominated by persistent low-frequency growth with limited independent short-term volatility. Nonlinear amplification is spatially concentrated and co-occurs with strong long-term epidemic pressure. Despite synchronous statewide breakpoints around 2014, post-break growth accelerates most sharply in counties exhibiting high low-frequency power and elevated nonlinear interaction. Together, these results provide a mechanistically interpretable framework for identifying dynamical risk phenotypes and structural transitions in spatial overdose epidemics.
Read the full abstract

Statistics Applications · Cross-listed from q-fin.ST

Forecasting duration in high-frequency financial data using a self-exciting flexible residual point process

This paper presents a method for forecasting limit order book durations using a self-exciting flexible residual point process. High-frequency events in modern exchanges exhibit heavy-tailed interarrival times, posing a significant challenge for accurate prediction. The proposed approach incorporates the empirical distributional features of interarrival times while preserving the self-exciting and decay structure. This work also examines the stochastic stability of the process, which can be interpreted as a general state-space Markov chain. Under suitable conditions, the process is irreducible, aperiodic, positive Harris recurrent, and has a stationary distribution. An empirical study demonstrates that the model achieves strong predictive performance compared with several alternative approaches when forecasting durations in ultra-high-frequency trading data.
Read the full abstract

Statistics Applications · Cross-listed from cs.CL

Agent Q-Mix: Selecting the Right Action for LLM Multi-Agent Systems through Reinforcement Learning

Large Language Models (LLMs) have shown remarkable performance in completing various tasks. However, solving complex problems often requires the coordination of multiple agents, raising a fundamental question: how to effectively select and interconnect these agents. In this paper, we propose Agent Q-Mix, a reinforcement learning framework that reformulates topology selection as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. Our method learns decentralized communication decisions using QMIX value factorization, where each agent selects from a set of communication actions that jointly induce a round-wise communication graph. At its core, Agent Q-Mix combines a topology-aware GNN encoder, GRU memory, and per-agent Q-heads under a Centralized Training with Decentralized Execution (CTDE) paradigm. The framework optimizes a reward function that balances task accuracy with token cost. Across seven core benchmarks in coding, reasoning, and mathematics, Agent Q-Mix achieves the highest average accuracy compared to existing methods while demonstrating superior token efficiency and robustness against agent failure. Notably, on the challenging Humanity's Last Exam (HLE) using Gemini-3.1-Flash-Lite as a backbone, Agent Q-Mix achieves 20.8% accuracy, outperforming Microsoft Agent Framework (19.2%) and LangGraph (19.2%), followed by AutoGen and Lobster by OpenClaw. These results underscore the effectiveness of learned, decentralized topology optimization in pushing the boundaries of multi-agent reasoning.
Read the full abstract

All authors Eric Hanchen Jiang, Levina Li, Rui Sun, Xiao Liang, Yubei Li, Yuchen Wu, Haozheng Luo, Hengli Li, Zhi Zhang, Zhaolu Kang, Kai-Wei Chang, Ying Nian Wu

Statistics Theory

E-Values, Bayes Risk, Dual Role of Markov's Inequality

Two approaches to hypothesis testing, e-value testing and Bayes risk minimisation, both invoke Markov's inequality to control error probabilities. They differ in which distribution certifies the unit-moment condition: the null for Type I error, the alternative for Type II error. The likelihood ratio is not intrinsically an e-value; it acquires that status only relative to the experiment under which its expectation is certified. This note makes the resulting role-reversal symmetry explicit, traces its asymptotic sharpening through the information-theoretic arguments of Barron and Clarke (1994), and situates the duality within the typed evidence calculus of Polson, Sokolov, and Zantedeschi (2026).
Read the full abstract

Statistics Applications

A Rank-Based Information Fusion Framework for Comparing Clustered Multivariate Socioeconomic Outcomes

We propose a multivariate, distribution-free ranking framework for comparing clustered, correlated outcomes across groups, motivated by the evaluation of state-level policy environments using county-level socioeconomic data. Using pooled U.S. county data from 2019-2023, we study multiple dimensions of economic well-being, including poverty, income inequality, housing cost burden, medical care costs, and per capita income, observed at a finer spatial resolution than the policy itself. Rather than relying on parametric regression models, we employ a rank-based aggregation algorithm derived from the Longitudinal Rank-Sum Test (LRST), which treats clusters as independent units and aggregates information across outcomes using order statistics. This approach provides a robust, interpretable omnibus comparison that accommodates within-cluster dependence and high-dimensional outcome structure without distributional assumptions. Applied to the comparison of states with and without refundable Earned Income Tax Credit (EITC) policies, the method reveals systematic differences in the joint ranking of county-level outcomes, with results remaining stable under repeated random subsampling of counties and varying cluster sizes. While the empirical analysis is descriptive rather than causal, the study highlights the broader utility of rank-based, multi-criteria aggregation methods as computational intelligence tools for analyzing complex, clustered data in policy and social systems.
Read the full abstract

Tuesday 31 March

Statistical Methodology

Cellwise Outliers

In statistics and machine learning, the traditional meaning of the terms `outlier' and `anomaly' is a case in the dataset that behaves differently from the bulk of the data, which raises suspicion that it may belong to a different population. But nowadays increasing attention is being paid to so-called cellwise outliers, which are individual values somewhere in the data matrix. Depending on the dimension, even a relatively small proportion of outlying cells can pollute over half the cases, which is a problem for existing methods that work by downweighting or dropping outlying cases. It turns out that detecting cellwise outliers as well as constructing cellwise robust methods requires techniques that are quite different from the casewise setting. For instance, one has to let go of some intuitive equivariance properties. The problem is difficult, but the past decade has seen substantial progress. For high-dimensional data the cellwise approach is becoming dominant, and typically can deal with missing values as well. We review developments in the estimation of location and covariance matrices as well as regression methods, principal component analysis, methods for tensor data, and various other settings.
Read the full abstract