Peer Revenue Play credits. Real papers.

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Thursday 9 April

Machine Learning (Statistics) · Cross-listed from cs.LG

The ecosystem of machine learning competitions: Platforms, participants, and their impact on AI development

Machine learning competitions (MLCs) play a pivotal role in advancing artificial intelligence (AI) by fostering innovation, skill development, and practical problem-solving. This study provides a comprehensive analysis of major competition platforms such as Kaggle and Zindi, examining their workflows, evaluation methodologies, and reward structures. It further assesses competition quality, participant expertise, and global reach, with particular attention to demographic trends among top-performing competitors. By exploring the motivations of competition hosts, this paper underscores the significant role of MLCs in shaping AI development, promoting collaboration, and driving impactful technological progress. Furthermore, by combining literature synthesis with platform-level data analysis and practitioner insights a comprehensive understanding of the MLC ecosystem is provided. Moreover, the paper demonstrates that MLCs function at the intersection of academic research and industrial application, fostering the exchange of knowledge, data, and practical methodologies across domains. Their strong ties to open-source communities further promote collaboration, reproducibility, and continuous innovation within the broader ML ecosystem. By shaping research priorities, informing industry standards, and enabling large-scale crowdsourced problem-solving, these competitions play a key role in the ongoing evolution of AI. The study provides insights relevant to researchers, practitioners, and competition organizers, and includes an examination of the future trajectory and sustained influence of MLCs on AI development.
Read the full abstract

Statistics Theory

Consistency of the Bayesian Information Criterion for Model Selection in Exploratory Factor Analysis

We study model selection by the Bayesian information criterion (BIC) in fixed-dimensional exploratory factor analysis over a fixed finite family of compact covariance classes. Our main result shows that the BIC is strongly consistent for the pseudo-true factor order under misspecification, provided that all globally optimal models share a common pseudo-true covariance set, the population Gaussian criterion has a local quadratic margin away from that set, and the BIC complexity counts are order-separating at the pseudo-true order. The candidate models may have an unknown mean vector, exact-zero restrictions in the loading matrix, and either diagonal or spherical error covariance structures, and the selection target is the smallest candidate factor order that yields the best Gaussian approximation, in Kullback–Leibler divergence, to the data-generating covariance structure. The proof works directly in covariance space, so it does not require a regular loading parametrization and accommodates the familiar singularities caused by rotations and redundant factors. Under correct specification, the assumptions reduce to familiar properties of the true covariance matrix. More generally, the same argument applies to other information criteria whose penalties satisfy the same gap conditions, including several BIC-type modifications.
Read the full abstract

Statistics Applications

Socio-demographic inequalities in the maximum human lifespan: evidence from Belgium and the Netherlands

The existence of an upper limit to the human lifespan has been widely debated, with studies offering both supporting and opposing evidence. Existing studies typically treat this limit, if it exists, as a population-wide constant, overlooking potential heterogeneity across individuals. Using unique individual-level death and population records for all individuals aged 90 and older in Belgium and the Netherlands between 1995 and 2022, we use extreme value theory to estimate the upper endpoint of the lifespan distribution, conditional on survival beyond a high threshold age. We find that this endpoint is finite in both countries, but that it is not well described by a single population-wide value: differences in sex, household type, civil status, and origin translate into differences of several years in the estimated upper endpoint. We conclude that socio-demographic inequality persists even at the most advanced ages.
Read the full abstract

Statistical Methodology

Unsupervised Learning Under a General Semiparametric Clusterwise Elliptical Distribution: Efficient Estimation, Optimal Clustering, and Consistent Cluster Selection

A general semiparametric clusterwise elliptical distribution is introduced for assessing how latent cluster structure shapes continuous outcomes. Using a subjectwise representation, cluster-specific mean vectors are estimated by minimizing a weighted sum of squares criterion augmented with a separation penalty. An initialization scheme based on iterative Mahalanobis reassignment and a computational algorithm with guaranteed convergence accompany the estimation procedure. Under the localization conditions, the separation penalty criterion admits a local oracle property, and the corresponding localized partition recovers the realized latent cluster partition. The practical iterative Mahalanobis reassignment does not impose these truth-centered neighborhoods and is evaluated separately through numerical studies. The resulting first-phase partition seeds a second phase of pseudo-likelihood-based estimation and cluster reassignment. A semiparametric information criterion is also proposed for selecting the number of clusters. Monte Carlo simulations identify settings in which the proposed procedures improve on the evaluated parametric benchmarks, while additional experiments reveal sensitivity to cluster imbalance and departures from ellipticity. Applications illustrate the resulting customer and patient profiles.
Read the full abstract

All authors Jen-Chieh Teng, Sheng-Hsin Fan, Chin-Tsang Chiang, Ming-Yueh Huang, Alvin Lim

Machine Learning (Statistics) · Cross-listed from math.OC

Unified Precision-Guaranteed Stopping Rules for Contextual Learning

Contextual learning seeks to learn a decision policy that maps an individual's characteristics to an action through data collection. In operations management, such data may come from various sources, and a central question is when data collection can stop while still guaranteeing that the learned policy is sufficiently accurate. We study this question under two precision criteria: a context-wise criterion and an aggregate policy-value criterion. We develop unified stopping rules for contextual learning with unknown sampling variances in both unstructured and structured linear settings. Our approach is based on generalized likelihood ratio (GLR) statistics for pairwise action comparisons. To calibrate the corresponding sequential boundaries, we derive new time-uniform deviation inequalities that directly control the self-normalized GLR evidence and thus avoid the conservativeness caused by decoupling mean and variance uncertainty. Under the Gaussian sampling model, we establish finite-sample precision guarantees for both criteria. Numerical experiments on synthetic instances and two case studies demonstrate that the proposed stopping rules achieve the target precision with substantially fewer samples than benchmark methods. The proposed framework provides a practical way to determine when enough information has been collected in personalized decision problems. It applies across multiple data-collection environments, including historical datasets, simulation models, and real systems, enabling practitioners to reduce unnecessary sampling while maintaining a desired level of decision quality.
Read the full abstract

All authors Mingrui Ding, Qiuhong Zhao, Siyang Gao, Jing Dong

Machine Learning (Statistics)

Intensity Dot Product Graphs

Latent-position random graph models usually treat the node set as fixed once the sample size is chosen, while graphon-based and random-measure constructions allow more randomness at the cost of weaker geometric interpretability. We introduce Intensity Dot Product Graphs (IDPGs), which extend Random Dot Product Graphs by replacing a fixed collection of latent positions with a Poisson point process on a Euclidean latent space. This yields a model with random node populations, RDPG-style dot-product affinities, and a population-level intensity that links continuous latent structure to finite observed graphs. We define the heat map and the desire operator as continuous analogues of the probability matrix, prove a spectral consistency result connecting adjacency singular values to the operator spectrum, compare the construction with graphon and digraphon representations, and show how classical RDPGs arise in a concentrated limit. Because the model is parameterized by an evolving intensity, temporal extensions through partial differential equations arise naturally.
Read the full abstract

Machine Learning (Statistics)

Order-Optimal Sequential 1-Bit Mean Estimation in General Tail Regimes

In this paper, we study the problem of mean estimation under 1-bit communication constraints. We propose a novel adaptive mean estimator based solely on randomized threshold queries, where each 1-bit outcome indicates whether a given sample exceeds a sequentially chosen threshold. Our estimator is $(\epsilon, \delta)$-PAC for any distribution with a bounded mean $\mu \in [-\lambda, \lambda]$ and a bounded $k$-th central moment $\mathbb{E}[|X-\mu|^k] \le \sigma^k$ for any fixed $k > 1$. Moreover, our sample complexity is order-optimal in all such tail regimes, i.e., for every such $k$ value. For $k \neq 2$, our estimator's sample complexity matches the unquantized minimax lower bounds plus an unavoidable $O(\log(\lambda/\sigma))$ localization cost. For the finite-variance case ($k=2$), our estimator's sample complexity has an extra multiplicative $O(\log(\sigma/\epsilon))$ penalty, and we establish a novel information-theoretic lower bound showing that this penalty is a fundamental limit of 1-bit quantization. We also establish a significant adaptivity gap: for both threshold queries and more general interval queries, the sample complexity of any non-adaptive estimator must scale linearly with the search space parameter $\lambda/\sigma$, rendering it vastly less sample efficient than our adaptive approach. Finally, we present algorithmic variants that (i) handle an unknown sampling budget, (ii) adapt to an unknown scale parameter $\sigma$ given (possibly loose) bounds, (iii) require only two stages of adaptivity to achieve order-optimal sample complexity at the expense of more general 1-bit queries, and (iv) leverage multiple local samples per 1-bit query to proportionally reduce communication costs.
Read the full abstract

Statistical Methodology

Structured Semiparametric Estimation of Average Treatment Effects with Treatment-Specific Non-Gaussian Error Distributions

This paper studies average treatment effect (ATE) estimation for continuous outcomes when the treatment-covariate mean is structured but the error distributions are unknown and may differ across treatment arms in scale, skewness, or tail behavior. We introduce a semiparametric model with a finite-dimensional mean, or a prespecified basis expansion, and separate unrestricted additive error laws for treated and control outcomes. The central theoretical contribution is the ATE-efficient influence function under two treatment-specific error nuisance spaces: it combines arm-specific regression-efficient scores with variation in the marginal covariate law and is distinct from both the unrestricted AIPW gradient and a pooled location-shift score. We establish the exact nested ordering of the efficiency bounds under pooled common-error, treatment-specific-error, and unrestricted causal models, including equality conditions. A local-misspecification result further shows that the variance reduction obtained by valid pooling equals the maximal squared first-order bias incurred along unit treatment-specific directions excluded by the pooled model. We develop efficient cross-fitted plug-in and ATE-targeted implementations requiring only two one-dimensional density estimates; the targeted refinement adds a scalar update. Simulations show substantial precision gains from valid pooling under common non-Gaussian errors and protection against invalid pooling when treatment changes error shape, including under weak positivity. Applications to ACTG175 and National Supported Work data illustrate the resulting precision–robustness tradeoff for irregular continuous outcomes.
Read the full abstract

Statistical Methodology

Bayesian Tensor-on-Tensor Varying Coefficient Model for Forecasting Alzheimer's Disease Progression

We propose a novel tensor-on-tensor modeling framework that flexibly models nonlinear voxel-level relationships using Gaussian process (GP) priors, while incorporating the spatial structure of the output tensor through low-rank tensor-based coefficients. Spatial heterogeneity is captured through patch-to-voxel mappings, enabling each output voxel to depend on its spatial neighborhood. The proposed interpretable and flexible Bayesian tensor-on-tensor framework is able to capture nonlinearity, spatial information, and spatial heterogeneity. We develop an efficient Markov chain Monte Carlo (MCMC) algorithm that exploits parallel structure to sample voxel-specific GP atoms and update low-rank tensor coefficients. Extensive simulations reveal advantages of the proposed approach over existing methods in terms of coefficient estimation, inference, prediction, and scalability to high-dimensional images. Applied to longitudinal image prediction with T1-weighted MRIs from the Alzheimer's Disease Neuroimaging Initiative (ADNI), the proposed method can accurately forecast future cortical thickness. The predicted images also enable reliable prediction of brain aging, underscoring their biological relevance. Overall, the ADNI analysis highlights the model's ability to forecast future neurobiological changes that has important implications for early detection of AD.
Read the full abstract

Statistical Methodology

Fixed-Effects Models for Causal Inference in Longitudinal Cluster Randomized and Quasi-Experimental Trials

This article investigates the model-robustness of fixed-effects models for analyzing a broad class of longitudinal cluster trials (CTs) such as stepped-wedge, parallel-with-baseline and crossover designs, encompassing both randomized (CRTs) and quasi-experimental (CQTs) designs. We clarify a longstanding misconception in biostatistics, demonstrating that fixed-effects models, traditionally perceived as targeting only finite-sample conditional estimands, can effectively target super-population marginal estimands through an M-estimation framework. We comprehensively prove that linear and log-link fixed-effects models with correctly specified treatment effect structures can broadly yield consistent and asymptotically normal estimators for nonparametrically defined treatment effect estimands in longitudinal CRTs, even under arbitrary misspecification of other model components. We identify that the constant treatment effect estimator generally targets the period-average treatment effect for the overlap population (P-ATO); accordingly, some CRT designs don't even require correct specification of the treatment effect structure for model-robustness. We further characterize conditions where fixed-effects models can maintain consistency by adjusting for both cluster-level and individual-level time-invariant confounding in longitudinal CQTs. Altogether, supported by simulation and a case study re-analysis, we establish fixed-effects models as a robust and potentially preferable alternative to mixed-effects models for longitudinal CT analysis.
Read the full abstract

Machine Learning (Statistics)

Sparse $\epsilon$ insensitive zone bounded asymmetric elastic net support vector machines for pattern classification

Existing support vector machines(SVM) models are sensitive to noise and lack sparsity, which limits their performance. To address these issues, we combine the elastic net loss with a robust loss framework to construct a sparse $\varepsilon$-insensitive bounded asymmetric elastic net loss, and integrate it with SVM to build $\varepsilon$ Insensitive Zone Bounded Asymmetric Elastic Net Loss-based SVM($\varepsilon$-BAEN-SVM). $\varepsilon$-BAEN-SVM is both sparse and robust. Sparsity is proven by showing that samples inside the $\varepsilon$-insensitive band are not support vectors. Robustness is theoretically guaranteed because the influence function is bounded. To solve the non-convex optimization problem, we design a half-quadratic algorithm based on clipping dual coordinate descent. It transforms the problem into a series of weighted subproblems, improving computational efficiency via the $\varepsilon$ parameter. Experiments on simulated and real datasets show that $\varepsilon$-BAEN-SVM outperforms traditional and existing robust SVMs. It balances sparsity and robustness well in noisy environments. Statistical tests confirm its superiority. Under the Gaussian kernel, it achieves better accuracy and noise insensitivity, validating its effectiveness and practical value.
Read the full abstract

Machine Learning (Statistics)

The Condition-Number Principle for Prototype Clustering

We develop a geometric framework that links objective accuracy to structural recovery in prototype-based clustering. The analysis is algorithm-agnostic and applies to a broad class of admissible loss functions. We define a clustering condition number that compares within-cluster scale to the minimum loss increase required to move a point across a cluster boundary. When this quantity is small, any solution with a small suboptimality gap must also have a small misclassification error relative to a benchmark partition. The framework also clarifies a fundamental trade-off between robustness and sensitivity to cluster imbalance, leading to sharp phase transitions for exact recovery under different objectives. The guarantees are deterministic and non-asymptotic, and they separate the role of algorithmic accuracy from the intrinsic geometric difficulty of the instance. We further show that errors concentrate near cluster boundaries and that sufficiently deep cluster cores are recovered exactly under strengthened local margins. Together, these results provide a geometric principle for interpreting low objective values as reliable evidence of meaningful clustering structure.
Read the full abstract

Statistics Theory · Cross-listed from econ.EM

Identification in (Endogenously) Nonlinear SVARs Is Easier Than You Think

We study identification in structural vector autoregressions (SVARs) in which the endogenous variables enter nonlinearly on the left-hand side of the model, a feature we term endogenous nonlinearity, to distinguish it from the more familiar case in which nonlinearity arises only through exogenous or predetermined variables. This class of models accommodates asymmetric impact multipliers, endogenous regime switching, and occasionally binding constraints. We show that, under weak regularity conditions, the model parameters and structural shocks are (nonparametrically) identified up to an orthogonal transformation, exactly as in a linear SVAR. Our results have the powerful implication that most existing identification schemes for linear SVARs extend directly to our nonlinear setting, with the number of restrictions required to achieve exact identification remaining unchanged. We specialise our results to piecewise affine SVARs, which provide a convenient framework for the modelling of endogenous regime switching, and their smooth transition counterparts. We illustrate our methodology with an application to the nonlinear Phillips curve, providing a test for the presence of nonlinearity that is robust to the choice of identifying assumptions, and finding significant evidence for state-dependent inflation dynamics.
Read the full abstract

Statistical Computation

Vine Copulas for Analyzing Multivariate Conditional Dependencies in Electronic Health Records Data

Electronic health records (EHR) store hundreds of demographic and laboratory variables from large patient populations. Traditional statistical methods have limited capacity in processing mixed-type data (continuous, ordinal) and capturing non-linear relationships in large multivariate data when oversimplified assumptions are made about the distribution (e.g., Gaussian) of disparate variables in EHR data. This paper addresses the limitations mentioned above by repurposing the vine copula method, which is primarily used to synthesize a multivariate distribution from many bivariate cumulative distribution functions (copulas). Vine copulas produce tree structures that represent bivariate conditional dependencies at varying hierarchical levels, decomposing a multivariate distribution. The tree structure is used to rank variables by conditional dependence and to identify a subset of central variables with local dependence, thus simplifying probabilistic mining of high-dimensional EHR data. The proposed application of vine copulas is used to identify conditional dependence between co-morbid conditions and is validated for characterizing different cohorts of EHR patients. The contribution of this paper is a novel approach to probabilistic mining and exploration of healthcare data that provides data-driven explanations, visualization, and variable selection to prognosticate a healthcare outcome. The source code is shared publicly.
Read the full abstract

All authors Manar D. Samad, Yina Hou, Megan A. Witherow, Norou Diawara

Machine Learning (Statistics)

On the Unique Recovery of Transport Maps and Vector Fields from Finite Measure-Valued Data

We establish guarantees for the unique recovery of vector fields and transport maps from finite measure-valued data, yielding new insights into generative models, data-driven dynamical systems, and PDE inverse problems. In particular, we provide general conditions under which a diffeomorphism can be uniquely identified from its pushforward action on finitely many densities, i.e., when the data $\{(\rho_j,f_\#\rho_j)\}_{j=1}^m$ uniquely determines $f$. As a corollary, we introduce a new metric which compares diffeomorphisms by measuring the discrepancy between finitely many pushforward densities in the space of probability measures. We also prove analogous results in an infinitesimal setting, where derivatives of the densities along a smooth vector field are observed, i.e., when $\{(\rho_j,\text{div} (\rho_j v))\}_{j=1}^m$ uniquely determines $v$. Our analysis makes use of the Whitney and Takens embedding theorems, which provide estimates on the required number of densities $m$, depending only on the intrinsic dimension of the problem. We additionally interpret our results through the lens of Perron–Frobenius and Koopman operators and demonstrate how our techniques lead to new guarantees for the well-posedness of certain PDE inverse problems related to continuity, advection, Fokker–Planck, and advection-diffusion-reaction equations. Finally, we present illustrative numerical experiments demonstrating the unique identification of transport maps from finitely many pushforward densities, and of vector fields from finitely many weighted divergence observations.
Read the full abstract

Wednesday 8 April

Machine Learning (Statistics) · Cross-listed from cs.LG

The Attribution Impossibility: No Feature Ranking Is Faithful, Stable, and Complete Under Collinearity

No feature ranking can be simultaneously faithful, stable, and complete when features are collinear. For collinear pairs, ranking reduces to a coin flip. We prove this impossibility, quantify it for four model classes, resolve it via ensemble averaging (DASH), and machine-verify it with 305 Lean 4 theorems. We characterize the complete attribution design space: exactly two families of methods exist – faithful-complete methods (unstable, with rankings that flip up to 50% of the time) and ensemble methods like DASH (stable, reporting ties for symmetric features) – and no method lies outside this dichotomy. The impossibility is quantitative: the attribution ratio diverges as 1/(1-rho^2) for gradient boosting, is infinite for Lasso, and converges for random forests. DASH (Diversified Aggregation of SHAP) is provably Pareto-optimal among unbiased aggregations, achieving the Cramer-Rao variance bound with a tight ensemble size formula. In a survey of 77 public datasets, 68% exhibit attribution instability. Switching to conditional SHAP does not escape the impossibility when features have equal causal effects. The framework includes practical diagnostics – a Z-test workflow and single-model screening tool – and has direct consequences for fairness auditing: SHAP-based proxy discrimination audits are provably unreliable under collinearity. The design space theorem, diagnostics, and impossibility are mechanically verified in Lean 4 (305 theorems from 16 axioms, 0 sorry) – to our knowledge, the first formally verified impossibility in explainable AI.
Read the full abstract

Statistics Applications

Horizon-Aware Forecasting of Passenger Assistance Demand for Rail Station Workforce Planning

Passenger assistance services are essential for accessible rail travel, yet demand varies substantially across stations and over time, creating challenges for workforce planning and staff rostering. This paper presents a data-driven decision support framework for forecasting station-level passenger assistance demand and translating forecasts into workforce plans. The forecasting component applies a horizon-aware Prophet modelling approach using multi-source operational data, while the planning component maps demand forecasts to staffing requirements under service and operational constraints through an interpretable red-amber-green risk framework. The approach has been implemented within a production-grade system to support routine planning and staffing decisions across LNER-managed stations. Results demonstrate improved forecast accuracy relative to year-on-year baseline methods, with absolute error reduced by up to 76.9%, and show that forecast-informed staffing is associated with an approximate 50% reduction in failed passenger assistance deliveries attributable to staff availability. These findings highlight the value of integrating interpretable forecasting with operational work.
Read the full abstract

Statistical Methodology

Log-Laplace Nuggets for Fully Bayesian Fitting of Spatial Extremes Models to Threshold Exceedances

Flexible random scale-mixture models provide a framework for capturing a broad range of extremal dependence structures. However, likelihood-based inference under the peaks-over-threshold setting is often computationally infeasible, due to the censored likelihood requiring repeated evaluation of high-dimensional Gaussian distribution functions. We propose a multiplicative log-Laplace nugget that yields conditional independence in the censored likelihood, resulting in a joint likelihood function that is the product of univariate densities which are available in closed form. This eliminates multivariate Gaussian distribution function evaluations and thereby enables inference for threshold exceedances in high dimensions, which represents a major shift for spatial extremes modelling as the total computational cost is now primarily driven by standard spatial statistics operations. We further show that a broad class of scale-mixture processes augmented with the proposed nugget preserves the extremal dependence structure of the underlying smooth process. The proposed methodology is illustrated through simulation studies and an application to precipitation extremes.
Read the full abstract

Statistical Methodology

Sample-split REGression SREG: A robust estimator for high-dimensional survey data

Model-assisted regression estimation is fundamental in survey sampling for incorporating auxiliary information. However, when the auxiliary dimension grows with the sample size, the standard Generalized regression (GREG) estimator can exhibit non-negligible bias under informative sampling, even when the working model is correctly specified. This failure stems from the double use of sampled outcomes simultaneously for fitting the regression and for forming the residual correction. We propose a sample-split REGression (SREG) estimator based on K-fold cross-fitting that eliminates this bias by pairing each unit's residual with an out-of-fold prediction. The resulting estimator is first-order equivalent to the oracle difference estimator under a weak prediction-norm consistency requirement, without requiring root-n consistent estimation of regression coefficients. We establish asymptotic normality and prove consistency of a variance estimator based on cross-fitted residuals. The key conditional fluctuation assumption is verified for simple random, stratified, and rejective sampling. Simulations demonstrate that SREG effectively removes high-dimensional bias while maintaining competitive efficiency.
Read the full abstract

Machine Learning (Statistics)

Variational Approximated Restricted Maximum Likelihood Estimation for Spatial Data

This research considers a scalable inference for spatial data modeled through Gaussian intrinsic conditional autoregressive (ICAR) structures. The classical estimation method, restricted maximum likelihood (REML), requires repeated inversion and factorization of large, sparse precision matrices, which makes this computation costly. To sort this problem out, we propose a variational restricted maximum likelihood (VREML) framework that approximates the intractable marginal likelihood using a Gaussian variational distribution. By constructing an evidence lower bound (ELBO) on the restricted likelihood, we derive a computationally efficient coordinate-ascent algorithm for jointly estimating the spatial random effects and variance components. In this article, we theoretically establish the monotone convergence of ELBO and mathematically exhibit that the variational family is exact under Gaussian ICAR settings, which is an indication of nullifying approximation error at the posterior level. We empirically establish the supremacy of our VREML over MLE and INLA.
Read the full abstract

Statistics Applications · Cross-listed from physics.geo-ph

Diffusional earthquakes and their slip-distance scaling

The final size of an earthquake typically cannot be predicted from its ongoing seismic radiation. Expanding observations reveal distinct exceptions, such as slow earthquakes, injection-induced seismicity, and earthquake swarms, in which fault slip has an upper bound. A common thread among these anomalies is the diffusive migration of their active areas. Here, we report a unified scaling relation for these diffusional earthquakes. By tracking prolonged earthquake swarms in Northeast Japan, we constrained the time evolution of their active seismicity areas and cumulative seismic moments. Their moment-duration trajectories coincide with the final states documented for global swarms and induced seismicity across various scales. When plotted as seismic moment versus seismicity area, their trajectories collapse onto those of slow earthquakes, uniformly explained by a diffusional constant-slip model. This constant-slip scaling carves out a unique class of diffusional earthquakes, where the final available seismic energy is predetermined by slip distance.
Read the full abstract

Statistical Methodology · Cross-listed from econ.EM

Assessing Sensitivity to IV Exclusion and Exogeneity without First Stage Monotonicity

Exclusion and exogeneity are core assumptions in instrumental variable (IV) analyses, but their empirical validity is often debated. This paper develops new sensitivity analyses for these assumptions. Our results accommodate arbitrary heterogeneity in treatment effects and do not impose any monotonicity requirements on the first stage. Specifically, we derive identified sets for the marginal distributions of potential outcomes and their functionals, like average treatment effects, under a broad class of nonparametric relaxations of the exclusion and exogeneity assumptions. These identified sets are characterized as solutions to linear programs and have desirable theoretical properties. We explain how to estimate these solutions using computationally tractable methods even when the linear program is infinite-dimensional. We illustrate these methods with an empirical application to peer effects in movie viewership, using weather as a potentially imperfect instrument.
Read the full abstract

Statistical Methodology

From Ground Truth to Measurement: A Statistical Framework for Human Labeling

Supervised machine learning assumes that labeled data provide accurate measurements of the concepts models are meant to learn. Yet in practice, human labeling introduces systematic variation arising from ambiguous items, divergent interpretations, and simple mistakes. Machine learning research commonly treats all disagreement as noise, which obscures these distinctions and limits our understanding of what models actually learn. This paper reframes annotation as a measurement process and introduces a statistical framework for decomposing labeling outcomes into interpretable sources of variation: instance difficulty, annotator bias, situational noise, and relational alignment. The framework extends classical measurement-error models to accommodate both shared and individualized notions of truth, reflecting traditional and human label variation interpretations of error, and provides a diagnostic for assessing which regime better characterizes a given task. Applying the proposed model to a multi-annotator natural language inference dataset, we find empirical evidence for all four theorized components and demonstrate the effectiveness of our approach. We conclude with implications for data-centric machine learning and outline how this approach can guide the development of a more systematic science of labeling.
Read the full abstract

All authors Robert Chew, Stephanie Eckman, Christoph Kern, Frauke Kreuter

Statistics Theory

Data Reuse and the Long Shadow of Error: Splitting, Subsampling, and Prospectively Managing Inferential Errors

When multiple investigators analyze a common dataset, the data reuse induces dependence across testing procedures, affecting the distribution of errors. Existing techniques of managing dependent tests require either cross-study coordination or post-hoc correction. These methods do not apply to the current practice of uncoordinated groups of researchers independently evaluating hypotheses on a shared dataset. We investigate the use of subsampling techniques implemented at the level of individual investigators to remedy dependence with minimal coordination. To this end, we establish the asymptotic joint normality of test statistics for the class of asymptotically linear test statistics, decomposing the covariance matrix as the product of a data overlap term and a test statistic association term. This decomposition shows that controlling data overlap is sufficient to control dependence, which we formalize through the notion of Expected Variance Ratio. This enables the closed form derivation of the variance of the joint rejection region under the global null as a function of pairwise correlations of test statistics. We adopt mean-variance portfolio theory to measure risk, defining the Expected Variance Ratio (EVR) as the ratio of the expected variance of the Type I error count to the independent baseline. We show that data splitting is asymptotically optimal among rules that ensure exact independence. We then use concentration inequalities to establish that subsampling techniques implementable by individual investigators can ensure an EVR close to $1$. Finally, we show that such subsampling techniques are able to simultaneously perform a number of tests while ensuring sufficient power and that the bounded EVR is $O\left(\frac{1}{r^2}\right)$ compared to data splitting's $O\left(\frac{1}{r}\right)$, where $r$ is the per-statistic fraction of data required.
Read the full abstract

All authors Reid Dale, Jordan Rodu, Maria E. Currie, Mike Baiocchi

Statistics Applications · Cross-listed from q-bio.QM

Quantifying the Spatiotemporal Dynamics of Engineered Cardiac Microbundles

Brightfield time-lapse imaging is widely used in cardiac tissue engineering, yet the absence of standardized, interpretable analytical frameworks limits reproducibility and cross-platform comparison. We present an open, scalable computational pipeline for quantifying spatiotemporal contractile dynamics in microscopy videos of human induced pluripotent stem cell-derived cardiac microbundles. Building on our open-source tools "MicroBundleCompute" and "MicroBundlePillarTrack," we define a suite of 16 interpretable structural, functional, and spatiotemporal metrics that capture tissue deformation, synchrony, and heterogeneity. The framework integrates full-field displacement tracking, strain reconstruction, spatial registration, dimensionality reduction, and topology-based vector-field analysis within a unified workflow. Applied to a dataset of 670 cardiac microbundles spanning 20 experimental conditions, the pipeline reveals continuous variation in contractile phenotypes rather than discrete condition-specific clustering, with intra-condition variability often exceeding inter-condition differences. Redundancy analysis identifies a reduced core set of 10 metrics that retain most informational content while minimizing multicollinearity. Analysis of denoised displacement fields shows that contraction is dominated by a global isotropic mode, with localized saddle-type deformation patterns present in approximately half of the samples. All software and workflows are released openly to enable reproducible, scalable analysis of dynamic tissue mechanics.
Read the full abstract

All authors Hiba Kobeissi, Samuel J. DePalma, Javiera Jilberto, David Nordsletten, Brendon M. Baker, Emma Lejeune

Statistical Methodology

Marginal Persistence and Dynamic Copula Dependence in Sovereign Rating Migration Counts: A Discrete Interval-Likelihood MAGMAR Analysis

This paper develops an observed-data likelihood for applying moving-aggregate modified autoregressive (MAGMAR) copula time-series models to discrete sovereign rating-migration counts with time-varying exposure. An annual count identifies a probability-integral-transform interval rather than a unique latent point, so the likelihood integrates the latent process over the complete sequence of count intervals. A guided sequential Monte Carlo implementation incorporates the parameter-dependent stationary marginal adjustment of MAGMAR directly inside this integration. In sovereign ratings from Fitch, Moody's, and S&P over 1963–2025, the migration counts are strongly overdispersed relative to a binomial margin and, under an exposure-conditioned static beta-binomial margin, MAGMAR improves substantially on MAG. The interpretation changes once the marginal mean is allowed to depend on the lagged migration rate: residual serial correlation in the count margin disappears, a reference dynamic-margin fit gives only a small MAGMAR improvement, and information criteria favor MAG. Numerical profiles, higher-precision refits, and parameter-recovery experiments show that marginal persistence and copula persistence are only weakly separated in this short discrete series. The results therefore support interval integration as the correct observation layer while showing that conclusions about latent dynamic dependence must be conditioned on the specification of the count margin.
Read the full abstract

Statistical Methodology

Robust Mendelian Randomization Estimation using Weighted Quantile Regression

In Mendelian randomization (MR) studies, genetic variants are used as instrumental variables (IVs) to investigate causal relationships between exposures and outcomes based on observational data. However, numerous genetic studies have shown the pervasive pleiotropy of genetic variants, meaning that many, if not most, variants are associated with multiple traits, potentially violating the core assumptions of IV estimation. Uncorrelated pleiotropy occurs when genetic variants have a direct effect on the outcome that is not mediated by the exposure, while correlated pleiotropy occurs when genetic variants affect the exposure and outcome via shared heritable confounders. In this work, we propose a novel MR method, called MR-Quantile, based on weighted quantile regression (WQR) that is robust to both correlated and uncorrelated pleiotropy. We propose a procedure for selecting the optimal quantile of the ratio estimates through a likelihood-based formulation of WQR using the asymmetric Laplace distribution. Monte Carlo simulations demonstrate the empirical performance of the proposed method, especially in settings with many invalid IVs with weak pleiotropic effects. Finally, we apply our method to study the causal effect of resting heart rate on atrial fibrillation. Genetic variants associated with heart rate were identified in a genome-wide association study of 425,748 individuals from the VA Million Veteran Program, and used as instruments in a two-sample MR analysis with summary statistics from a genetic meta-analysis of 228,926 AF cases across eight studies.
Read the full abstract

All authors Julien St-Pierre, Archer Y. Yang, Mireille E. Schnitzer, Marc-André Legault

Statistical Methodology

A covariate-dependent Cholesky decomposition for high-dimensional covariance regression

Estimation of covariance matrices is a fundamental problem in multivariate statistics. Recently, growing efforts have focused on incorporating covariate effects into these matrices, facilitating subject-specific estimation. Despite these advances, guaranteeing the positive definiteness of the resulting estimators remains a challenging problem. In this paper, we present a new varying-coefficient sequential regression framework that extends the modified Cholesky decomposition to model the positive definite covariance matrix as a function of subject-level covariates. To handle high-dimensional responses and covariates, we impose a joint sparsity structure that simultaneously promotes sparsity in both the covariate effects and the entries in the Cholesky factors that are modulated by these covariates. We approach parameter estimation with a blockwise coordinate descent algorithm, and investigate the $\ell_2$ convergence rate of the estimated parameters. The efficacy of the proposed method is demonstrated through numerical experiments and an application to a gene co-expression network study with brain cancer patients.
Read the full abstract

Statistical Methodology

Langevin-Gradient Rerandomization

Rerandomization is an experimental design technique that repeatedly randomizes treatment assignments until covariates are balanced between treatment groups. Rerandomization in the design stage of an experiment can lead to many asymptotic benefits in the analysis stage, such as increased precision, increased statistical power for hypothesis testing, reduced sensitivity to model specification, and mitigation of p-hacking. However, the standard implementation of rerandomization via rejection sampling faces a severe computational bottleneck in high-dimensional settings, where the probability of finding an acceptable randomization vanishes. Although alternatives based on Metropolis-Hastings and constrained optimization techniques have been proposed, these alternatives rely on discrete procedures that lack information from the gradient of the covariate balance metric, limiting their efficiency in high-dimensional spaces. We propose Langevin-Gradient Rerandomization (LGR), a new sampling method that mitigates this dimensionality challenge by navigating a continuous relaxation of the treatment assignment space using Stochastic Gradient Langevin Dynamics. We discuss the trade-offs of this approach, specifically that LGR samples from a non-uniform distribution over the set of balanced randomizations. We demonstrate how to retain valid inference under this design using randomization tests and empirically show that LGR generates acceptable randomizations orders of magnitude faster than current rerandomization methods in high dimensions.
Read the full abstract

Statistical Methodology

Regularized estimation for highly multivariate spatial Gaussian random fields

Estimating covariance parameters for multivariate spatial Gaussian random fields is computationally challenging, as the number of parameters grows rapidly with the number of variables, and likelihood evaluation requires operations of order $\mathcal{O}((np)^3)$. In many applications, however, not all cross-dependencies between variables are relevant, suggesting that sparse covariance structures may be both statistically advantageous and practically necessary. We propose a LASSO-penalized estimation framework that induces sparsity in the Cholesky factor of the multivariate Matérn correlation matrix, enabling automatic identification of uncorrelated variable pairs while preserving positive semidefiniteness. Estimation is carried out via a projected block coordinate descent algorithm that decomposes the optimization into tractable subproblems, with constraints enforced at each iteration through appropriate projections. Regularization parameter selection is discussed for both the likelihood and composite likelihood approaches. We conduct a simulation study demonstrating the ability of the method to recover sparse correlation structures and reduce estimation error relative to unpenalized approaches. We illustrate our procedure through an application to a geochemical dataset with $p = 36$ variables and $n = 3998$ spatial locations, showing the practical impact of the method and making spatial prediction feasible in a setting where standard approaches fail entirely.
Read the full abstract