Peer Revenue Equity in science

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 20 July

Statistical Methodology

Assessing the Impact of Model Assumptions in Network Meta-Regression: A Simulation Study

Network meta-regression (NMR) extends network meta-analysis (NMA) by synthesizing evidence on multiple treatments while adjusting for potential effect modifiers. By accounting for effect modification, NMR can reduce between-study heterogeneity and improve the validity of relative treatment effects, providing insight regarding characteristics impacting treatment performance. However, choosing between available NMR models is complex, as each model addresses a similar, but unique research question, and the performance of available NMR models under varying network structures, between-study heterogeneity, and interaction assumptions remains unclear. We evaluated the consequences of model misspecification in a simulation study of 120 evidence-network scenarios designed to reflect potential complications in evidence networks introduced by trial design, heterogeneity levels, and interaction assumptions. We compared the standard interaction-free NMA model with four NMR parameterizations differing in across-comparison interaction assumptions (common vs. independent interactions) and interaction consistency assumptions (with or without consistency). Standard NMA models generally overestimated treatment effects when effect modification was present. NMR models with independent across-comparison interactions maintained appropriate confidence interval coverage in dense networks generated with their corresponding consistency assumptions. However, their coverage deteriorated in sparse networks with between-study heterogeneity. Models assuming consistent interactions are advantageous in networks with multi-arm studies. Ignoring effect modification in NMA can lead to biased treatment effect estimates. When effect modification is anticipated, thoughtful alignment between network structure and NMR assumptions can reduce bias and misleading precision, supporting more reliable medical decision making.
Read the full abstract

All authors Nana-adjoa Kwarteng, Guido Schwarzer, Adriani Nikolakopoulou, Theodoros Evrenoglou

Statistical Computation · Cross-listed from math.NA

Pathwise skew-symmetric discretisation for SDEs with superlinear drift

The skew-symmetric discretisation has recently been proposed as a new robust simulation method for weakly approximating stochastic differential equations (SDEs) with non-globally Lipschitz drift. This work develops a pathwise version of the scheme by representing the noise increment as a skew-normal distribution and coupling it with the driving Brownian increments, thereby enabling its use in the multilevel Monte Carlo (MLMC) framework. Under suitable conditions, we establish strong convergence of order 1/2 in $L^2$. Subsequently, the associated MLMC estimator is shown to have computational complexity ${O} \bigl(\varepsilon^{-2} (\log (1/\varepsilon))^2 \bigr)$ to achieve a mean-squared error $\varepsilon^2$. We then analytically compare the proposed scheme with the tamed Euler scheme, another benchmark for robust discretisation. Under a strong inward-drift regime with the current state being far from the stable region, we show that the probability of moving in the wrong direction tends to vanish in the skew-symmetric scheme, whereas the tamed Euler scheme makes such moves with a non-trivial probability. Furthermore, in the MLMC setting, employing a one-dimensional stochastic Ginzburg-Landau model, we specify the range of step sizes for which the asymptotic variance of the coupled level difference obtained via the pathwise skew-symmetric scheme is lower than that obtained via the tamed Euler scheme. Numerical experiments on several model examples support the theoretical rate of strong convergence and demonstrate the stability and effectiveness of the resulting MLMC in the superlinear drift setting.
Read the full abstract

All authors Yuga Iguchi, Samuel Livingstone, Giorgos Vasdekis, Rui-Yang Zhang

Statistics Theory

Decision-Centric Large Deviations for Data-Driven Capital Buffers in Ruin Models

We consider an insurance risk model with a random walk structure in which the underlying probability law is unknown. A decision maker observes a statistic $Q_n$ computed from $n$ historical observations and then needs to choose a capital buffer of the form $C_n=n f(c,Q_n)$, where $c=\log(1/\delta)/n$ balances the amount of data and the tolerated ruin probability $\delta$. The classical safe capital buffer is inversely proportional to the adjustment coefficient $\gamma$, the exponential rate at which the ruin probability decays; when the law is unknown, this coefficient must be inferred from the data. The profile $f$ couples the statistical cost of observing an atypical historical statistic with the future ruin exponent induced by the resulting decision. We study the joint ex ante probability (over both the historical sample and an independent future risk process) that the future maximum exceeds the data-dependent buffer. We show that the naive plug-in rule fails to achieve the prescribed logarithmic decay exponent $c$, illustrating the adverse impact of model uncertainty when making decisions under rare-event constraints. We then identify a profile $f^*=f^*(c,Q_n)$ with the following appealing properties: its lower semicontinuous majorants are safe, while regular continuous rules that fall strictly below $f^*$ are, under mild conditions, unsafe. We illustrate the potential applicability of our framework by developing three parametric examples. In nonparametric settings, we show that a single exponential envelope leads to degenerate capital buffers, whereas a two-level exponential envelope yields a nondegenerate capital buffer which we prove to be safe.
Read the full abstract

Statistics Theory

A Functional Central Limit Theorem for Localized Partial Sums of Non-Stationary Time Series

A localized functional central limit theorem is established for kernel-weighted partial sum processes of piecewise locally stationary time series under geometric decay of the physical dependence measure. The localized process is shown to converge weakly to a centered Gaussian random distribution in $D'(0,1)$, and the limit extends naturally to an isonormal Gaussian process on $L^2([0,1])$. Weak convergence is further derived for processes indexed by totally bounded subsets of $L^2([0,1])$. As an application, the localized limit theory is used to construct tests for constant mean functions against linear, polynomial, and general alternatives in non-parametric regression with locally stationary errors. Simulation results and data examples illustrate the finite sample performance and practical applicability of the proposed methodology.
Read the full abstract

Machine Learning (Statistics)

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions. The principal unconditional theorem is the residual-to-depth-flow estimate for layer controls converging in $L^1$, complemented by a finite-token-to-Volterra attention estimate that explicitly controls the first cells near the causal endpoint. We define a normalized adjoint-energy influence density and derive its exact evolution along full-batch gradient flow. The adjoint admits an exact generator-term decomposition into residual transmission, nonlocal Volterra, and local channels, including all covariance cross terms. Causal masking can amplify early-position sensitivity and residual identity paths can transmit a right-localized terminal bias, but neither mechanism alone forces a U-shaped profile. We therefore state boundary advantages under independently checkable energy, correlation, and local-channel bounds; these conditions are sufficient rather than necessary. Finite-token influence balancing, positional reweighting, and task-aligned observability are presented as diagnostics or regularizers with explicit differentiation requirements, computational costs, and limitations. Controlled simulations illustrate that each intervention controls its designated surrogate, while observability balance or outer-loop reweighting need not monotonically reduce the influence-based Lost-in-the-Middle diagnostic.
Read the full abstract

Statistical Methodology

A Globally Calibrated Bayesian Optimal Phase II Design for Adaptive Enrichment Trials

Adaptive enrichment allows development of an experimental treatment to continue when its activity is insufficient in an all-comer population but remains promising in a prespecified biomarker-positive subgroup. However, sequential application of separately calibrated phase II designs can inflate the probability of a false-positive efficacy conclusion. We develop a globally calibrated Bayesian optimal phase II (BOP2) design for branching adaptive enrichment trials. At prespecified all-comer interim analyses, the trial either continues all-comer enrollment or, after crossing the all-comer futility boundary, evaluates the accumulated biomarker-positive data. Enrichment is initiated only when a prespecified minimum number of biomarker-positive patients is available and the biomarker-positive futility boundary is not crossed; otherwise, the trial stops. The all-comer and biomarker-positive thresholds are jointly calibrated for the union of the two possible efficacy claims while accounting for the random subgroup sample size available when enrichment is considered. All decision rules are prespecified before trial initiation. For a binary endpoint, an exact finite-state recursive enumeration enables calibration and operating-characteristic evaluation without Monte Carlo error. Under the prespecified point global null, the proposed design controlled the global type I error rate over the prespecified set of biomarker-positive prevalence values while achieving higher power than the independently calibrated BOP2 comparator across the evaluated alternative scenarios. In the numerical study, the independently calibrated comparator exceeded the nominal global type I error level after its components were embedded in the branching procedure. The framework is also extended to complex categorical endpoints using a Dirichlet–multinomial formulation, with calibration and evaluation performed by simulation.
Read the full abstract

Statistical Methodology

Fitting the topology of synthetic particle systems with a novel graph representation

The shape and arrangement of particles in a material determine its macroscopic properties. The generation of synthetic data with varying particle structure, often represented as 3D voxel images, combined with simulation of macroscopic properties reveals structure-property relations. Most particle generation models focus on single-particle characteristics like shape and size. We aim at fitting the topology of the particle system using tools from persistent homology. However, the large size of the required 3D image data makes existing methods computationally infeasible. We bridge this gap by introducing a novel graph representation of particle systems and transferring the computation of persistent homology from the image domain to the graph domain. This yields a postprocessing method for synthetic images of particle systems, that is independent of the underlying generation method and improves topological and geometrical agreement with real particle systems while preserving morphological characteristics such as the particle size distribution.
Read the full abstract

Statistical Methodology

Equality, Equity, and Causality in Fairness Research: A Commentary on Cheng (2026)

This is an invited commentary on the Psychometrika focus article "Fairness Issues and Evaluation in Psychometrics and AI/ML: What Can We Learn from Each Field?" by Ying Cheng (2026, doi:10.1017/psy.2026.10110). Cheng offers a systematic comparison between long-standing test fairness and modern algorithmic fairness. Her mapping of the entire testing workflow onto the AI/ML fairness paradigm, rather than only the final selection stage, is a crucial contribution to interdisciplinary fairness research. This commentary extends her discussion by examining two conceptual issues: the distinction between equality and equity, and the role of causality in fairness research. The focus article and this commentary point to directions for future fairness research across the psychometrics and AI/ML communities.
Read the full abstract

Machine Learning (Statistics)

An efficient adaptive dimension selection algorithm for multidimensional probit graded response models

Multidimensional graded response models (MGRMs) are widely used for analyzing ordinal questionnaire data in psychological and educational assessments. A central challenge in applying these models is determining the number of latent dimensions. Conventional approaches usually fit multiple fixed-dimensional models and select among them using post-hoc criteria such as AIC, BIC, or cross-validation, which can be computationally demanding and ignore uncertainty in dimensionality during estimation. We develop an adaptive Bayesian dimension selection framework for probit MGRMs. Building on the cumulative shrinkage process, we assign a cumulative ordered spike-and-slab (COSS) prior to the column-specific variances of the item loading matrix. This prior induces increasing shrinkage across latent dimensions, allowing redundant dimensions to be shrunk toward zero while preserving flexibility for active dimensions. Albert–Chib latent response augmentation is used to handle the ordinal probit likelihood, yielding conditionally Gaussian updates for item loadings and latent traits. These updates are combined with Gibbs updates for threshold and shrinkage parameters in an efficient adaptive sampler. Simulation studies evaluate the proposed method in terms of dimension recovery, parameter estimation accuracy, and computational efficiency, with comparisons to conventional fixed-dimensional estimation and model selection procedures. The results show that the proposed approach accurately recovers the latent structure while avoiding repeated model fitting over multiple candidate dimensions. We further illustrate the method using real psychological assessment data, demonstrating its practical utility for uncovering interpretable latent structures in ordinal item responses.
Read the full abstract

All authors Yu Zhou, Yincai Tang, Bin Lv, Meng Gao

Machine Learning (Statistics) · Cross-listed from cs.LG

Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles

Stochastic nonconvex optimization is central to training deep networks and LLMs in modern machine learning. We give a black-box reduction from stochastic nonconvex optimization to ordinary static regret minimization in online convex optimization (OCO), thereby resolving the open problem posed by Chen and Hazan (2024). Our reduction maintains a predictable gradient tracker, while a black-box online learner $\mathcal{A}$ selects a preconditioner that transforms this tracker into the update direction. Given a \(\beta\)-smooth function with a range bounded by $M$ and an unbiased gradient oracle with variance bounded by $\sigma^2$, we bound the expected average squared gradient norm by $O(\sigma\sqrt{M\beta/T}+\sqrt{M\beta}\mathrm{Reg}_T(\mathcal{A})/T+\frac{M\beta}{T})$, where $\mathrm{Reg}_T(\mathcal{A})$ is the static regret of $\mathcal{A}$. Thus, any OCO oracle with $O(\sqrt{T})$ regret recovers the classical $O(T^{-1/2})$ convergence rate. We further extend the framework to nonsmooth nonconvex objectives, still relying only on ordinary static regret, and attain the optimal convergence rate for Goldstein-type stationarity. Finally, we conduct numerical experiments on nonconvex objectives to illustrate how the reduction exploits online-selected preconditioners while using the same stochastic-oracle budget as stochastic gradient descent.
Read the full abstract

Statistical Methodology

Spatial Dependence in Directed Preferential-Attachment Networks

Spatially embedded directed networks, such as airline networks, often exhibit simultaneous high activity at nearby nodes. Preferential attachment (PA) explains hub dominance. We extend it to spatial co-movement through a directed PA model whose out- and in-node weights follow temporally persistent Gaussian-process lognormal fields. Under sublinear PA, out-degree proportions converge to explicit normalized powered weights, whereas self-loop exclusion yields a coupled in-degree limit. We derive a strictly concave inverse that recovers the in-weights from terminal degree proportions. For ordered network histories, we develop a minorization-maximization (MM) weight estimator and profile likelihood for the PA exponent; temporal pre-whitening and a spatial quasi-likelihood estimate the latent covariance. Simulations verify transmission of distance-decaying dependence and show how random segment volume creates a distance-independent common mode in raw degrees. An analysis of U.S. domestic flights (2015-2019) separates network-wide volume variation from a short-range spatial component. An observed-volume reconstruction reproduces the raw-degree common mode, and the fitted field yields an exploratory co-exceedance transition scale of roughly 150 km. A per-carrier analysis of European air traffic also reveals the same decomposition.
Read the full abstract

Statistical Methodology

E-Values For Multiplicity Control In Multiverse Analysis

Multiverse analysis refers to a common situation where one wishes to assess the association between multiple possible treatment definitions and multiple possible outcome definitions, potentially within multiple sub-populations, among other possible analysis specifications. Multiverse analysis is a useful exploratory tool to assess heterogeneity across the considered specifications, but it is sometimes also used to assess statistical significance. In the latter case, it is critical to acknowledge that multiple comparisons are being performed, and to ensure a valid statistical control of false positive findings. We study the use of e-values within generalized linear models as a tool to control the false discovery rate regardless of the dependence structure of the multiple analyses being performed, while accounting for confounding covariates. We compare the performance of several approaches: universal e-values, soft-rank e-values, and p-to-e calibration. We find that, for problem characteristics typically encountered in multiverse analyses, p-to-e calibration significantly outperforms the other two approaches in terms of statistical power, but said power may be moderate unless the sample size or effect sizes are large enough. An application studying the association between teenager technology use and mental well-being reveals association between depression, low self-steem and peer problems with internet and social media usage.
Read the full abstract

Statistical Methodology

Two-stage Adaptive Testing of Large-scale Mediation Hypotheses

In testing large-scale mediation hypotheses, the exposure-mediator and mediator-outcome path-specific p-values obtained for each hypothesis from the same sample can be combined into a pair of asymptotically independent p-values, which can then be used to test a global null hypothesis and a composite mediation null hypothesis, respectively. A first-stage screening procedure targeting the more stringent global null can effectively reduce the number of mediation hypotheses to be tested in the second stage, which in turn reduces the conservativeness of multiple comparison. The framework can incorporate any data-adaptive choice of screening threshold in Stage 1, and any step-up false discovery rate control method in Stage 2. The proposed procedure controls the false discovery rate asymptotically while being consistently well powered across a wide range of scenarios.
Read the full abstract

2607.17579 · 20 Jul · 28 pages, 7 figures. Updated simulation implementations to use the authors' released M-DACT code. Results were updated accordingly and the main conclusions are unchanged. Added code and data availability information. Code: https://github.com/keyyyyyx/Two-stage-Adaptive-Mediation-Testing

Machine Learning (Statistics) · Cross-listed from cs.LG

Program Synthesis for Simulation-Based Inference: Joint Model Selection and Parameter Estimation

Neural simulation-based inference enables parameter estimation for complex models, but typically requires the user to specify a simulator encoding a fixed model structure. We present a framework for joint model selection and parameter estimation that combines large language models for program synthesis with neural simulation-based inference. Given a natural language description of the system and data under investigation, an LLM proposes candidate simulator programs which are iteratively refined via feedback-driven mutation and evaluated using neural density estimation. The approach enables simulation-based inference over a pool of models, not just parameters within a fixed model. On benchmarks spanning deterministic dynamics, stochastic epidemic models, and dark matter substructure inference from gravitational-lensing images, the method identifies plausible model families from open-ended prompts, with accuracy that reflects the information content of the data and identifiability of candidate models.
Read the full abstract

Statistics Theory

On Mixtures of Three Homoscedastic Gaussian Densities: An Unconditional Sharper Bound on the Number of Modes

It is known that a mixture of three homoscedastic multivariate Gaussian densities whose centers form an equilateral triangle can have four modes, owing to the emergence of a “ghost” mode at the center. Nevertheless, obtaining a sharp upper bound on the number of modes remains open even in this seemingly simple setting. The best previously available upper bounds applicable to the homoscedastic three-component setting were 42592 and, more recently, 72. These bounds were derived for substantially more general classes of Gaussian mixtures and are therefore not optimized for the present setting. Moreover, they are conditional on the assumption that the modal set is finite. In this paper, as a sharper bound, we prove that every mixture of three homoscedastic Gaussian densities has at most 8 modes, without imposing any finiteness or non-degeneracy assumption a priori. To the best of our knowledge, this is the first unconditional finiteness result for the modal set that holds uniformly over the full class of multivariate three-component homoscedastic Gaussian mixtures. In particular, it covers genuinely multivariate asymmetric configurations with arbitrary non-collinear centers and arbitrary positive mixture weights.
Read the full abstract

Statistical Computation

A general framework for computation and estimation using the saddlepoint approximation

The saddlepoint approximation provides highly accurate approximations to probability density and mass functions using only the corresponding moment generating functions (MGFs). Recent work has increasingly seen the saddlepoint approximation applied to likelihood functions, enabling likelihood-based inference in models where exact likelihoods are intractable. However, existing implementations have largely been developed on a model-by-model basis, and the methodology remains underutilized because of the conceptual and computational challenges of working with MGFs. We introduce a unified framework for model construction and computation using the saddlepoint approximation. The framework is based on a collection of model-building operations that preserve access to MGFs while allowing complex distributions to be constructed from simpler components. With these components, users need only provide a high-level specification of the model structure, from which the software automatically assembles the necessary generating functions, saddlepoints, and gradients, and performs the optimization of the saddlepoint likelihood. We also introduce a diagnostic that quantifies the difference between saddlepoint and exact likelihood estimates, even when the exact likelihood is unavailable. The framework is implemented in the R package saddlepoint and provides fast, convenient computation of parameter estimates, standard errors, and the discrepancy diagnostic. Numerous examples illustrate the scope and flexibility of the approach.
Read the full abstract

Statistical Methodology

Calibrating Semantic Uncertainty from Observable Language-Model Probabilities

As generative artificial intelligence enters scientific and professional work, its uncertainty must be defined on the states that matter for inference and decision-making. Language models assign probabilities to words, whereas applications require uncertainty over meaningful states such as diagnoses, hypotheses or operational conditions. We introduce a semantic map: a prespecified, testable bridge from probabilities over verbal responses to a posterior over declared finite states. The language distribution remains unrestricted; held-out calibration connects it to a reference posterior. We derive posterior-error bounds and conditions for existence, conditional uniqueness, presentation stability and stable inverse recovery. This distinction matters because language probabilities depend on prompt wording, while the target posterior should not change under information-equivalent rewording. Experiments use professional market text compiled from Federal Reserve economic and financial series, together with controlled simulations having exact posteriors. Across two fitted language models, language-derived probabilities outperform printed numerical confidence, recover held-out posteriors with valid uncertainty coverage, remain largely stable under paraphrase and respond appropriately to altered evidence. Prompt engineering optimises a wording-dependent response; robust scientific use requires validated stability of application-relevant meaning. The proposed map turns semantic uncertainty in generative systems into an identifiable and testable statistical measurement problem and, when its acceptance conditions hold, yields an auditable posterior estimate.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Calibrated Alzheimer's Conversion Risk in Mild Cognitive Impairment: Persistent Homology of Clinical Trajectories with Conformal Guarantees

Background. Predicting conversion from mild cognitive impairment (MCI) to Alzheimer's disease (AD) is central to trial enrichment and care planning, yet existing models provide no individual-level uncertainty estimates and rarely include transparent leakage audits. We introduce the first application of persistent homology to longitudinal clinical trajectory point clouds for this task, and the first split-conformal individual risk guarantee for any AD-conversion model. Methods. We analysed 741 MCI subjects (240 converters, 32.4%) from ADNI with a uniform 4-year follow-up cap. Five leakage sources were corrected; without them a naive pipeline achieved AUC=0.934, inflated by +0.075. Vietoris-Rips persistent homology and sublevel-set proxies were combined with trajectory slopes and engineered features (76 total) in a stacking ensemble evaluated by 5-fold cross-validation. Results. Cox and Random Survival Forest models with TDA features achieved concordance C=0.799 and C=0.826 versus C=0.753 and C=0.812 without (+0.045 and +0.014). The primary nested AUC is 0.840 (same-fold bound 0.866); external AUC was 0.879 on a zero-overlap ADNI-2/GO/3 cohort. H0 persistence entropy was the top SHAP feature and significantly associated with APOE4 dosage (Spearman r=-0.191, p<0.0001, Bonferroni-corrected). Cross-conformal coverage was 90.4%+-2.2% (target 90%); empirical external coverage 96.9%. Maximum fairness gap in false-negative rate across seven subgroups was 0.092. Conclusions. We propose H0 persistence entropy as a topological biomarker of cognitive decline and demonstrate that a leakage-audited, conformally calibrated pipeline reaches competitive accuracy with individual-level uncertainty quantification not previously available for this task.
Read the full abstract

Sunday 19 July

Statistical Computation

Fast inversion of the generalized Fisher transformation of correlation matrices

The generalized Fisher transformation maps a non-singular correlation matrix to an unconstrained real vector through the off-diagonal elements of its matrix logarithm. Evaluating its inverse is a computational bottleneck in dynamic correlation and multivariate volatility models. We develop a fast inversion algorithm by characterizing the unknown diagonal as the minimizer of a smooth, strictly convex, and coercive objective. An explicit Hessian and global spectral bounds identify the standard fixed-point iteration as a quasi-Newton method and explain why it can converge slowly near singularity. Every fixed-point step decreases the objective, and the iteration converges from every starting point. These results motivate GFT-FP+N, a hybrid of fixed-point and matrix-free Newton steps that never forms the Jacobian. In benchmarks with up to 1,000 replications per design and dimensions up to 800, GFT-FP+N reduces computation time by up to a factor of forty-five relative to the fixed-point iteration and converged in every replication, including on designs where Broyden's method almost always fails. Julia and R packages are provided.
Read the full abstract

Statistical Methodology

Do stable performance metrics guarantee stable model predictions?: An empirical investigation using the GUSTO-1 dataset

Model validation is routinely used to quantify and correct for overfitting in performance estimates, but it does not fully address the stability of models developed from slightly different training samples. We investigated whether stable discrimination implies stable individual predictions and compared logistic regression (LR) with artificial neural networks (ANNs). Using data from 40,830 participants in the GUSTO-I trial, with 30-day mortality as the outcome, we created six sample-size scenarios with varying events per variable (EPV). In each scenario, LR and ANN models were developed using eight predictors, and 200 bootstrap resamples were used to assess stability in discrimination, mean absolute prediction error (MAPE), and the classification instability index (CII). For LR, discrimination stabilized at an EPV of 44.50, whereas MAPE and CII did not stabilize until an EPV of 89.12. ANN discrimination was more volatile, with optimism stabilizing only at an EPV of 178.25, and ANN predictions were severely miscalibrated at an EPV of 4.50. Overall, discrimination stabilized earlier than individual predictions for both approaches, with greater instability and larger sample-size requirements for ANN. Model developers should therefore assess and report prediction-level stability, because discrimination metrics alone may not reflect the reliability of individual predictions.
Read the full abstract

All authors Natthanaphop Isaradech, Wachiranun Sirikul, Wuttipat Kiratipaisarl, Pakpoom Wongyikul, Noraworn Jirattikanwong, Phichayut Phinyo

Machine Learning (Statistics) · Cross-listed from cs.LG

Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

Linear attention promises constant-time recurrent inference but degrades sharply on associative recall. We formulate attention recall as a spherical-packing problem and introduce Kernelized Linear Attention Activations (KATA), a framework whose feature maps are derived from first principles by certifying nonnegative attention weights through a self-dual homogeneous cone. Building on this observation, we show that rank-one positive semi-definite (PSD) features offer a favorable capacity–interference tradeoff. KATA recovers a parameter-free convex output gate and characterizes associative capacity through the Welch interference floor. For tolerances above this floor, KATA enlarges the state without adding parameters and admits spherical codes with exponentially many keys in the projection dimension. We implement KATA as fused Triton kernels at two operating points: a flash-attention-style forward up to ${\sim}1.6\times$ FlashAttention-2 throughput, and an exact $O(T)$ chunked-state form that reaches ${\sim}11\times$ FlashAttention-2 forward throughput at $131$k tokens. An associative scan of the first-order feature lowers the inter-chunk recurrence depth to $O(\log(T/C))$ for chunk size $C$ and averages ${\sim}2.4\times$ the throughput of a matched sequential linear-attention baseline. On long-range MQAR and repeated-key overwrite, several KATA variants outperform Gated DeltaNet, with parameter counts and state sizes reported alongside accuracy. Induction preserves near-perfect recall, while kernel benchmarks show that the maps can be implemented efficiently. KATA retains $0.985$ MQAR at a $16\times$ out-of-distribution length, approaching the softmax with roughly one quarter of the KV-cache entries. Experiments on 340M-parameter LLMs reveal a feature-dependent fluency trade-off and clarify how positional embeddings, delta rules, and decay gates interact with feature geometry.
Read the full abstract

Machine Learning (Statistics)

Efficient Sequential Evaluation of Large Language Models

We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs. Our goal is to construct a confidence sequence (CS) for the model's capability on this question set and to design active querying rules that shrink the CS width as quickly as possible. For CS construction, we invert a family of test supermartingales and focus on two representative approaches: a reverse information projection (RIPr)-based approach and a testing-by-betting-based approach. We first study these approaches under an oracle setting, and demonstrate the oracle optimality of the RIPr-based construction. We then propose a growth-oriented querying rule that aims to maximize the worst-case one-step expected log-increment over the endpoints of the current CS. In practice, we build these test supermartingales and the querying rule on predictions of question-level correctness learned from historical data. We then analyze the shrinkage behavior of the resulting CSs and identify two key factors that slow the shrinkage rate of CSs: accumulated prediction mismatch and the spikiness of the querying distribution. Finally, motivated by this analysis, we propose several mixture querying rules that combine growth-oriented querying, prediction refinement, and uniform exploration, trying to mitigate the effects that slow the shrinkage rate. We provide experiments comparing different querying rules for the RIPr-based and testing-by-betting-based CSs across several synthetic testing datasets. Interestingly, we observe that the simplest querying rule, uniform sampling, can sometimes outperform more adaptive querying rules for both methods.
Read the full abstract

Statistics Theory

EM-based iterations for multiple instance learning on a query-value model

In multiple instance regression (MIR) data are organized into bags (collections of instances in feature space) and the goal is to learn a mapping that assigns labels to bags. A typical assumption is that there is a so-called concept point in feature space, the proximity to which dictates the bag label. Motivated by modern MIR architectures which are based on attention, we study a softmax model that decouples the concept point and the labeling scheme. The two are respectively determined by a query direction and a value direction value in feature space. This problem isolates a basic challenge of learning both the query and value vectors from bag-level supervision. From this model we derive a parametric family of iterations in the noiseless limit, which generalizes a method known as the EM-DD algorithm. We then derive concentration results for the MLE estimators of the query and value vectors obtained from a random selection of instances. Our result for the value vector shows that a single random initialization of the value vector already points in the correct direction on average, so that a polynomial (in the number of instances per bag and the feature dimension) number of bags is enough for the EM algorithm to converge in $O(1)$ steps with high probability. A key aspect of this analysis is the interplay between concentration of empirical covariance matrices and extremal statistics arising from the selection rule.
Read the full abstract

Machine Learning (Statistics)

Kernel Regression with Tensor Trains and Hadamard Overparameterization

Kernel regression with tensor trains and Hadamard overparameterization (KReTTaH) is introduced as a training-data-free, interpretable, and nonparametric framework for multi-way data imputation. The imputation problem is reformulated as regression in reproducing kernel Hilbert spaces (RKHS), where the tensor regression coefficients are explicitly constrained to lie on fixed-rank tensor-train (TT) manifolds and structured via Hadamard overparameterization to promote sparsity and high representational efficiency. Rather than relying on costly cross-validation, KReTTaH jointly optimizes the TT coefficient tensors and the kernel covariance matrices within a Riemannian product-manifold framework – the former on fixed-rank TT manifolds, the latter on the manifold of positive-definite matrices – thereby enabling automated kernel-hyperparameter selection. Numerical tests on two challenging applications – imputation of high-dimensional functional magnetic resonance imaging (fMRI) data and recovery of missing edge flows in dynamic graphs – demonstrate that KReTTaH consistently outperforms state-of-the-art tensor-, Bayesian-, and neural-network-based baselines in terms of modeling accuracy.
Read the full abstract

All authors Duc Thien Nguyen, Konstantinos Slavakis, Eleftherios Kofidis, Dimitris Pados

Statistics Theory · Cross-listed from econ.EM

Econometrics with Pre-Trained Embeddings for Unstructured Data

Unstructured data, such as images and text, are increasingly used in empirical economics. Since training machine-learning models on unstructured data is costly, economists often use off-the-shelf pre-trained deep learning models developed by computer scientists to extract embeddings, which are then used as covariates in target economic analyses. Despite the popularity of this practice, its theoretical foundations remain limited. There are two main difficulties. First, pre-trained models are typically trained on different datasets and for different tasks, making it unclear when they can be used reliably for the target task. Second, the embedding function is subject to an identification problem, complicating the analysis of its estimation error and the effect of that error on the target task. We provide sufficient conditions to overcome these difficulties. A key condition, which we call transferability, governs the convergence rate we derive. To assess transferability, we develop a computationally feasible bootstrap test that does not require re-estimating the embeddings and nuisance functions. Our theory applies to a wide range of double machine learning applications, including partially linear regression with unstructured controls, price elasticity estimation in demand models accounting for product quality measured by images and text, missing-data imputation using unstructured data, and average treatment effect estimation with unstructured confounders. As an empirical application, we estimate the labor supply elasticity on Amazon Mechanical Turk, an online labor market platform, using job-description embeddings as controls.
Read the full abstract

Statistics Theory

Mixture and separation of log-concave measures

For a two-component log-concave mixture model, we investigate the extent to which the weight, mean, and covariance of each component distribution can be accurately recovered when given sufficient samples of the mixture distributions. One fundamental obstruction is that the mixture distribution itself could sometimes be log-concave, and in this case, accurate recovery is impossible. In this paper, we identify two regimes, one where the mixture distribution itself could be log-concave and another one where the mixture distribution is never log-concave, and one can always separate the two component distributions using a quadratic classifier.
Read the full abstract

Statistical Methodology

The Resolution of Causal Heterogeneity

Causal subgroup analyses often report a small number of groups summarizing treatment effect heterogeneity, as if that number were a well-defined estimand. Outside genuinely latent class populations, however, a “true” subgroup count is model dependent rather than a population functional. We replace it with a new population estimand, the resolution profile, a functional of the causal feature law giving the fewest groups explaining a prescribed fraction of causal heterogeneity, defined for every population without latent structure. Inference is organized around one cross-fitted Bayesian-bootstrap posterior for a single structured moment process, its scores corrected with influence functions, so that paths, profiles, fixed-resolution summaries, and subgroup effects follow by composition. A uniform conditional Bernstein–von Mises theorem over a loss class containing the nonsmooth quantization losses shows this posterior merges with the efficient Gaussian limit under stated nuisance-rate and margin conditions. Subgroup-number uncertainty is not model selection but threshold nonregularity, the profile being an integer-valued threshold of a continuous path, discontinuous in the law at each knot. At these knots no single-valued selector is locally uniformly consistent over root-$n$ neighborhoods, and the set-valued report obtained by inverting a simultaneous band retains locally uniform validity over exactly the same perturbations. Simulations support the approximations, and an analysis of the MineThatData e-mail experiment illustrates the resolution-indexed report, in which two to three groups summarize the visit response while finer structure falls below a noise-floor diagnostic.
Read the full abstract

Statistical Methodology · Cross-listed from econ.EM

Two Gaussians, Too Many: A bootstrap-based approach to assess identifiability in non-Gaussian structural vector autoregressions

Normality pre-tests on reduced-form innovations cannot detect whether two or more underlying structural shocks are Gaussian, and hence cannot detect a failure of identification in non-Gaussian SVARs. We propose a bootstrap-based approach that reveals such a failure. Under the null of valid identification and certain regularity conditions, the conditional bootstrap distribution of the maximum likelihood estimator of the impact matrix is asymptotically normal, so the diagnostic reduces to a normality test on the bootstrap replications. The impact-matrix estimator remains consistent and asymptotically normal when one shock is Gaussian and its shape parameter lies on the boundary of the parameter space, so the diagnostic is valid across the entire null hypothesis. When the number of bootstrap replications and the sample size diverge jointly at an appropriate rate, the test statistic, conditional on the data, is asymptotically pivotal. This implies that conditioning on this diagnostic does not induce pre-testing bias asymptotically. In simulations with Normal-inverse Gaussian shocks, the multivariate diagnostic is close to its nominal size under the null, its power against an additional Gaussian shock increases with the sample size, and its univariate version localizes the failure in large samples. Based on the estimates of a SVAR in the macroeconomic and financial uncertainty literature, we demonstrate its potential as a practical, robust tool for assessing non-Gaussian identification.
Read the full abstract

Statistics Applications

National Versus Domain: Coverage Properties of HB Credible Intervals Under Survey Redesign

A companion paper to Tam (2026) reports an extended Monte Carlo (MC) study examining the frequentist coverage of 95% hierarchical Bayes (HB) credible intervals at the national and domain levels under four stress-test scenarios. The extended study covers 140 strata and 13 estimation domains separately, adds a classical direct-estimator benchmark, and tests two strategies for restoring domain coverage: prior sensitivity and Prasad and Rao (PR) MSE correction. At the national level, HB credible intervals achieve near-nominal coverage across all four scenarios and all three labour force variables (Employment 93 to 96%, Unemployment 87 to 97%, Hours Worked 99.5 to 100%). At the domain level, Hours Worked coverage is nearnominal (94 to 98%) in all scenarios; Employment and Unemployment coverage is below nominal for scenarios with low between-domain heterogeneity, a direct consequence of HB shrinkage toward the national mean. A key operational finding emerges from the Rare Event scenario (D): the classical direct estimator collapses to 0% national coverage for Employment and Hours Worked because five unsampled strata introduce a systematic bias; the HB estimator achieves 96 to 100% national coverage at roughly 15% of the classical sample cost. Neither a weaker prior nor PR MSE correction reliably restores domain coverage, confirming that the failure is bias-driven and cannot be remedied by variance inflation alone.
Read the full abstract

Statistical Methodology

Bayesian Seasonal Adjustment for Survey Time Series

Seasonal adjustment procedures used by national statistical offices – X-11 and X-12-ARIMA – treat each survey estimate as an exact observation, discarding the accompanying standard errors that survey methodologists routinely compute. This paper closes that gap by embedding time-varying sampling error variances into a Basic Structural Model (BSM), extending a recently proposed Dynamic Mini-Max (DMM) Bayesian framework for survey estimation. Via the Harvey-Todd equivalence, BSM with zero measurement error variance reduces to X-11-style seasonal adjustment, so DMM-BSM is a principled Bayesian generalisation of existing practice rather than a departure from it. A two-block Gibbs sampler delivers the full joint smoothing posterior of the latent state trajectory. Exact credible intervals for the trend level, k-step trend movements (k=1,2,...), and seasonally adjusted estimates follow directly, together with posterior probabilities of directional change – outputs that X-11 cannot provide. Simulation studies with within-replication parameter estimation confirm substantially higher credible interval coverage than the X-11-equivalent model across both large and small survey domains. Applied to 120 months of Australian Bureau of Statistics Labour Force Survey data, once sampling variance is modelled the maximum likelihood estimate of stochastic seasonal variance collapses to zero – evidence that apparent seasonal fluctuations in the published series are largely attributable to measurement noise rather than genuine seasonal drift, a distinction X-11 cannot make.
Read the full abstract