Peer Revenue Impact factor: volatile

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 10 August

Statistical Methodology

Local conformal prediction for individual causal effects

Standard CATE estimators become inadequate under strong treatment-effect heterogeneity: confidence intervals for conditional means need not cover individual counterfactual effects. We propose an Individualized Causal Prediction (ICP) framework that constructs finite-sample valid conformal prediction intervals for the individual causal effect of a specific query unit. The method localizes calibration to a causally relevant neighborhood using cosine similarity weighted by Causal Forest variable importance, augments small local samples synthetically, and calibrates intervals with doubly robust AIPW conformity scores satisfying Neyman orthogonality. Under standard identifying assumptions (SUTVA and strong ignorability) and an outcome-independent calibration-set selection condition, the resulting intervals attain marginal coverage at the nominal level. The local design also supports approximately conditional coverage by making calibration scores more representative of the query unit. Experiments on a high-heterogeneity synthetic dataset and the IHDP benchmark demonstrate that local strategies improve point accuracy over global baselines while maintaining nominal or above-nominal coverage.
Read the full abstract

Statistics Theory

Estimating conditional expectation

In this paper, we consider the problem of estimating conditional expectations as an ill-posed inverse problem. We propose a solution based on a generalization of Vapnik's theorem \cite[Theorem 7.2]{Vapnik1998} for solving stochastic ill-posed problems in Hilbert spaces. As an application, we derive a new upper bound for sample errors of conditional expectation estimation.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

Training-Free Universal Approximation by Prompting Random Transformers

How expressive is prompting a transformer? Answering this question is important for separating the roles of prompting, architecture, and pretraining in transformer models, and for determining whether task-specific behavior must be stored in model weights or can instead be induced at inference time through the prompt. We show, in an approximation-theoretic sense, that pretraining is optional: a single-layer softmax attention network with random, untrained weights can approximate any Hölder function on a compact manifold when steered by an appropriate soft prompt. Guided by the connection between softmax attention and kernel methods, we construct explicit soft prompts (a prompt per target function, independent of the query) as solutions to linear systems matching attention logits to Gaussian kernel exponents, under which the frozen transformer emulates the classical Nadaraya-Watson kernel estimator. The construction requires only a mild rank condition on the weights, which we show holds almost surely under Gaussian initialization. The prompted network inherits the theoretical guarantees of kernel regression, leading to universal approximation theorems with minimax-optimal rates that depend on the intrinsic dimension. We further quantify the cost of prompting, exposing a tradeoff between the norm of the constructed soft prompt tokens, prompt length, and hidden dimension. Numerical experiments corroborate the constructions and predicted rates.
Read the full abstract

Statistical Methodology

Conditional Regime Analog Forecasting with Trajectories: A Nonparametric Framework for Multivariate Probabilistic

We propose Conditional Regime Analog Forecasting with Trajectories (CRAFT), a nonparametric framework for multivariate probabilistic time-series prediction. The method constructs paired backward and forward trajectory profiles from cumulative multi-horizon returns, learns recurrent low-dimensional regime labels in both spaces using singular value decomposition, change-point segmentation, and segment clustering, and estimates a backward-to-forward conditional regime correspondence. Forecast distributions are obtained by sampling historically realized future trajectory profiles according to a composite compatibility score that combines future-regime correspondence with similarity to the current backward trajectory profile. Unlike parametric vector autoregressions or Gaussian state-space models, CRAFT preserves empirical cross-sectional and multi-horizon dependence by resampling complete future paths. We describe the estimator, its diagnostics, and a reproducible simulation benchmark comparing CRAFT with direct analog resampling, SVD analogs, unconditional bootstrap, OLS VAR bootstrap, and random-forest forecasts.
Read the full abstract

Statistical Methodology

Relative Variability Estimation for the Power Lindley Distribution with Progressive Type-I Interval Censored Data

The measures of relative variability, such as the coefficient of variation, are highly utilized tools of inference in several interdisciplinary fields such as reliability, finance, quality control, and biological sciences. Inferential studies of these measures are highly limited for lifetime models, such as the flexible power Lindley distribution, particularly under progressive type-I interval censoring. In this article, frequentist approaches including the midpoint approximation, maximum likelihood estimation, method of moments, bootstrap, and nonlinear least squares methods, and Bayesian inference using slice sampling are applied for estimation of these measures of relative variability for the power Lindley distribution under progressive type-I interval censoring. Confidence intervals are also constructed based on asymptotic theory, bootstrap and Bayesian paradigm. A discussion on choosing optimal monitoring intervals is also highlighted. A comprehensive simulation study is conducted to evaluate the performance of the proposed estimators across various censoring plans and sample sizes. A real data application illustrates the practical utility of the proposed methodologies. The results indicate that the Bayesian framework generally exhibits superior performance in both point and interval estimation.
Read the full abstract

Statistics Applications

Beyond Zipf's Law: Equifinality and Mechanistic Inference from Scaling Laws

Scaling laws summarize complex systems through low-dimensional regularities, but the same marginal law can arise from different stochastic dynamics. We examine this ambiguity for Zipf rank–frequency scaling. An i.i.d. finite-Zipf process, a persistent Markov chain, and canonical sample-space reduction (SSR) are constructed to have exactly the same stationary marginal, $p_j=(jH_V)^{-1}$, while a latent-scale mixture produces a similar marginal through aggregation. The first three models therefore hold the population rank distribution fixed while changing temporal organization. Markov dependence shifts the finite-sample distribution of fitted exponents; at moderate persistence, a block-adjusted effective sample size reproduces most of this shift. Excess lag-1 mutual information detects serial dependence relative to a shuffle null, whereas transition direction separates reversible Markov persistence from the directional contraction of SSR. Conditioning on latent scale reveals the aggregation mechanism. Fit-window, alphabet, and sequence-boundary analyses quantify sensitivity to the observation design. These examples separate the population marginal, the finite-sample behavior of a fitted exponent, and temporal structure. Matching a scaling law is therefore a compatibility condition, not a mechanism identifier: discrimination requires observables on which candidate processes make different predictions.
Read the full abstract

Statistics Applications

From Risk Prediction to Risk Mechanisms: A Multi-Resolution Causal Representation for Road Safety and Motor Insurance

Operational risk models can estimate event frequencies precisely while leaving the underlying mechanisms weakly resolved. We study this resolution mismatch using motor insurance and road safety. A revisable DAG represents trip-level crash generation; a separate predictive layer links annual rating information to latent driving states; and an observation process maps crashes into recorded liability claims. Compatibility sets collect the structural and crash-to-claim laws that reproduce an observed annual contrast under stated restrictions. External studies enter only through explicit bridge assumptions and sensitivity bounds. The framework therefore distinguishes sampling uncertainty from uncertainty about structure, observation, and study-to-target correspondence. Two limited examples illustrate the gap. A sublinear mileage relation constrains an aggregate accident rate per unit distance but not its mechanism. In French motor-liability data, the 18-20 versus 40-49 claim-frequency relativity is 3.388 in a model including vehicle and geographic variables and 1.235 when the same model also conditions on a medium-resolution bonus-malus score. These are different predictive functionals, not successive causal adjustments. A Spanish culpability estimate is used only to show how a strong cross-study bridge would restrict a toy bookkeeping region. The framework does not estimate the full DAG; it makes explicit which assumptions are needed before an annual predictive contrast can support a mechanism-specific risk statement.
Read the full abstract

Other Statistics

leaspy: LEArning Spatiotemporal Patterns in PYthon

Longitudinal data are fundamental across scientific disciplines for modeling how complex systems evolve over time. A core challenge in these settings is handling temporal misalignment: different subjects undergo a similar underlying process but at varying speeds and starting times. This difficulty is further compounded when tracking multivariate dynamics, where features interact dynamically rather than following simple, independent pathways. To address these challenges, we present leaspy (LEArning Spatiotemporal patterns in PYthon), an open-source Python library. Built on a mixed effects model, leaspy enables the estimation of population-level trajectories while accounting for subject-specific variability. The library supports multivariate formulation across diverse data types, including continuous, time-to-event (joint), and mixture models-and has been successfully applied to characterize disease heterogeneity, and generate individual predictions We demonstrate its practical utility through an application in neurodegenerative disease progression. Developed following modern software engineering practices, including systematic testing and continuous integration, leaspy facilitates the integration of new models and provides a robust user-friendly library for longitudinal progression modeling.
Read the full abstract

All authors Juliette Ortholand, Sofia Kaisaridi, Nicolas Gensollen, Etienne Maheux, Caglayan Tuna, Raphael Couronne, Arnaud Valladier, Pierre-Emmanuel Poulet, Nemo Fournier, Léa Aguilhon, Maylis Tran, Gabrielle Casimiro, Jean-Vincent Martini, Sebastian Mendez, Igor Koval, Stanley Durrleman, Sophie Tezenas Du Montcel

Statistics Applications · Cross-listed from cs.CV

Deep Learning based Detection of Fishing Vessels and Fishing Monitoring using Nightlight Images

The demand for maritime surveillance has given rise to the need for monitoring fishing vessel activities, particularly in addressing the challenge of "dark vessels" that operate without Automatic Identification System (AIS) transmission. This study presents a novel approach for detecting small-scale fishing vessels using nighttime light (NTL) imagery from the SDGSAT-1 satellite, combined with deep learning techniques to enhance fishing monitoring awareness along the western coast of India. A dual-branch YOLO11 architecture was developed to exploit both the 10-meter panchromatic and 40-meter RGB imagery from SDGSAT-1. The custom model architecture was specifically optimized for small object detection in NTL imagery, featuring parallel convolutional backbones that process both modalities before concatenation for enhanced feature extraction. The dual-branch YOLO11 model demonstrated optimal performance with a precision of 0.99, recall of 0.93, F1-score of 0.96, and mAP@50 of 0.96, significantly outperforming single-branch implementations of YOLOv5s, YOLOv8s, and standard YOLO11s architectures. When applied to the western coast of India, the model detected 31525 vessel instances across the temporal dataset spanning 2022-23. Cross-matching analysis with AIS data revealed that only 7146 (22.7%) of detected vessels had corresponding AIS transmissions, while 24379 (77.3%) were identified as potential dark vessels. Spatio-temporal analysis showed peak fishing activity during January-April, with a primary activity corridor parallel to the coastline within 50-100 km, corresponding to productive continental shelf areas. This research contributes to maritime surveillance capabilities by highlighting the effectiveness of nighttime lights satellite imagery for fishing vessel detection and provides valuable insights into fishing patterns and potential regulatory compliance issues in Indian waters.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment. Self-consistency is one of the established approaches, which spends this budget entirely on the output side by sampling repeated reasoning paths. We study Test-Time Augmentation (TTA), which extends self-consistency by also perturbing the input, aggregating predictions across transformed versions of the input, and ask whether input-side diversity converts compute into accuracy more efficiently than output-side diversity. We perform a systematic, matched-compute comparison: we evaluate three simple input-side strategies (semantic rephrasing, lexical perturbations, and visual transformations) across six datasets covering general and multilingual knowledge, mathematical reasoning, multi-modal question answering, and sentiment classification, against chain-of-thought prompting and self-consistency. Semantic rephrasing delivers consistent and statistically significant accuracy gains while Pareto-dominating self-consistency on cost-effectiveness, delivering roughly 1.8X more accuracy per dollar and outperforming it on five of six tasks. We further analyze the number of augmentations, multi-modal strategies, and base model scaling, finding that TTA is most cost-effective for mid-tier models where a stronger model is unavailable or too expensive. Our findings indicate that for current mid-tier LLMs, varying the input converts inference compute into accuracy more efficiently than varying the reasoning path alone. The TTA implementation is available at https://github.com/aws-samples/sample-genai-reflection-for-bedrock.
Read the full abstract

Statistical Methodology

Estimating the average treatment effect under limited overlap via Polynomial Approximation and Extrapolation

Estimating the average treatment effect (ATE) remains a fundamental challenge in observational studies in the presence of poor or limited covariate overlap. Although the inverse probability weighting (IPW) estimator is a widely used approach for estimating the ATE, its performance can deteriorate substantially when overlap is limited, often resulting in increased finite sample bias and unreliable confidence intervals. One common strategy is to shift attention from the original target estimand, the ATE, to alternative estimands such as a class of weighted ATEs that are less sensitive to extreme propensity scores; however, doing so changes the scientific question of interest. In this manuscript, we propose a novel ATE estimator that preserves the original target estimand, the ATE, while improving robustness to limited overlap. A key idea is that this class of estimands can be represented by a polynomial function of a hyperparameter characterizing the estimands. Exploiting this structure, the proposed method computes IPW estimators for a sequence of such estimands, models these estimates using a polynomial regression, and extrapolates to recover the ATE. We show that the estimator has consistency and asymptotic normality under weaker overlap conditions than required for the standard IPW estimator. Simulation studies demonstrate that the proposed method improves estimation accuracy and interval performance in settings with limited overlap. In addition to its theoretical and empirical advantages, the proposed approach has a clear interpretation and is easy to implement using standard statistical software.
Read the full abstract

Statistical Methodology

Sparse Variable Sharpening in High-Dimensional Kernel Density Estimation

High-dimensional kernel density estimation suffers from the curse of dimensionality. This study proposes a hybrid density estimator defined as the product of a joint density over a pre-specified subset of dimensions and marginal densities for the remaining variables, instead of estimating the full high-dimensional density directly. Under this framework, given the observed data, we select a subset of variables for joint density construction to improve estimation accuracy, while modeling the remaining variables through their respective marginal densities. This construction involves a trade-off between the approximation error induced by simplifying the dependence structure and the variance reduction achieved by lowering the dimension of the joint density. We employ a genetic algorithm to efficiently identify such variable subsets. Theoretical and numerical results demonstrate that the proposed estimator can outperform conventional full-dimensional kernel density estimation when this trade-off is appropriately balanced.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.OC

Entropic Partial Optimal Transport and Partial Gromov–Wasserstein Distance between Gaussian Mixtures

Optimal transport and Gromov–Wasserstein distances are useful tools for comparing probability measures and metric measure spaces, but their balanced formulations force all mass to be matched. This constraint is often too strong for data with outliers, missing parts, or only partial overlap. In this paper, we develop entropic partial optimal transport for Gaussian mixture models and define a partial mixture Gromov–Wasserstein distance. For the finite entropic partial optimal transport problem, we prove the existence and uniqueness of the minimizer and establish quantitative large-penalty estimates. Moreover, the resulting entropic partial component couplings induce continuous partial transport plans through Gaussian optimal maps. We analyze their large-penalty and subsequent zero-entropy limits and construct the associated displacement interpolations and barycentric projection maps. In addition, by identifying each Gaussian mixture with a finite metric measure space of Gaussian components, we establish the metric property and large-penalty limit of the partial mixture Gromov–Wasserstein distance. Finally, numerical experiments on synthetic Gaussian mixtures and point clouds illustrate the effects of the penalty and entropic regularization and the robustness of partial matching to outliers.
Read the full abstract

Statistical Computation

Distributed Stochastic Smoothing ADMM for Penalized Quantile Regression

Quantile regression is well suited to heterogeneous and heavy-tailed data, but computation becomes challenging for large, distributed data sets because the check loss is nonsmooth. We propose a distributed stochastic smoothing alternating direction method of multipliers (DSS-ADMM) for horizontally partitioned penalized quantile regression. Each worker computes a mini-batch gradient of a Huber-smoothed check loss, and a coordinator performs a single proximal aggregation step for the regularizer. Raw observations remain local, worker updates run in parallel, and the method requires no matrix inversion. For proper, closed, and convex penalties, stacking the local coefficient vectors yields a standard two-block stochastic ADMM formulation. With fixed smoothing, we establish an expected $O(\log K/\sqrt K)$ joint objective-feasibility bound and an explicit $\eps/4$ approximation term for the original check-loss objective; when the smooth block is strongly convex, the bound improves to $O(\log K/K)$. We also characterize the scope of an extension to the minimax concave penalty and the smoothly clipped absolute deviation penalty. Reproducible simulations consider both homogeneous worker partitions, in which observations are independently and identically distributed across workers, and heterogeneous partitions, in which worker-specific covariate distributions differ. Sensitivity studies and analyses of the diabetes and Engel data illustrate the trade-offs among per-observation gradient evaluations, communication, consensus, sparsity, and prediction.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

Online Learning of Scale Parameters in Score-Driven Filters

A score-driven filter multiplies its scaled log-likelihood score by a scale parameter. We call this coefficient the gain and learn it online. Given the current state and realised scaled score, each admissible gain selects a reachable next state and predictive density. A scalar gain moves along a line; diagonal gains control coordinatewise transmission and may change direction. We evaluate gain selection using a one-step predictive Kullback-Leibler objective. In the scalar unscaled case, the negative consecutive-score product is a stochastic gradient; the positive product used in accelerated recursions is a descent direction. Positive scalar score scaling changes only the effective learning rate. Strictly increasing, continuously differentiable gain links with positive derivative induce mirror-descent geometry, while persistence adds a Bregman pull towards a reference gain. Under convexity, compactness, integrability, and schedule conditions, projected and discounted mirror updates satisfy dynamic-regret bounds relative to time-varying, current-information comparators. Simulations isolate score scaling, link geometry, persistence, and coordinatewise gains. Across twelve equity indices, the bounded discounted-logistic gain records a lower out-of-sample mean negative log score than the constant gain in eleven markets, although market-level evidence is mixed. It also avoids the extreme transients of the numerically capped exponential-link benchmark.
Read the full abstract

Statistical Methodology · Cross-listed from hep-ex

Silent coverage failures in rare-event searches and a degeneracy index that predicts them

Searches for new physics in low-background experiments infer a non-negative signal strength from few events and often report an upper limit. Nominal frequentist coverage requires both a valid interval construction and an adequate data model. We study how controlled model departures affect lower- and upper-endpoint coverage for six interval procedures in an exact Poisson counting experiment, dark-matter recoil spectra, and a neutrinoless-double-beta-decay peak search. We introduce the Poisson–Fisher degeneracy index $\mathcal I_{\mathrm{PF}}(\delta\nu;\vartheta_0)=(\beta,\gamma)$, which maps a specified expected-count deformation, after projection onto the complete fitted tangent space, to $\beta$, the signed fitted signal shift in profiled standard-error units, and $\gamma$, the Poisson–Fisher norm of the unabsorbed residual. Locally, the sign of $\beta$ identifies the threatened endpoint, while larger $\gamma$ implies greater detectability by the saturated-Poisson goodness-of-fit test used here at a fixed $5\%$ type-I error rate. Across the studied deformations, positive signal-like bias degrades discovery-side coverage while making upper limits conservative; negative signal bias from overestimated signal efficiency can make the upper endpoint undercover. Calibration under the nominal simulator does not protect against misspecification of that simulator relative to the data-generating process. A plausible deformation with large $|\beta|$ and small $\gamma$ may therefore evade diagnosis and should be represented by a nuisance constrained with auxiliary information or included in a defensible envelope. In the exactly collinear constrained-nuisance benchmark, modelling the deformation restores coverage at the evaluated grid points, at a quantifiable cost in interval sensitivity.
Read the full abstract

Machine Learning (Statistics)

CPDA: Class-Conditional Path Distribution Alignment for Unsupervised Time-Series Domain Adaptation

Unsupervised time-series domain adaptation (DA) addresses the challenge of transferring a classifier from a labeled source domain to an unlabeled target domain under distribution shifts induced by different users, sensors, devices, acquisition conditions, or temporal dynamics. Existing methods typically mitigate this shift by aligning marginal feature distributions through adversarial training, optimal transport, or moment-based discrepancies. In this paper, we propose Class-Conditional Path Distribution Alignment (CPDA), a non-adversarial discrepancy-based framework that aligns source and target class-conditional latent path distributions rather than only global feature marginals. CPDA introduces a composite signature-spectral kernel that jointly captures pooled semantic features, temporal path structure, frequency-domain information, and low-rank path-signature dynamics, while using source labels and target soft pseudo-labels to perform class-preserving alignment. We further provide a theoretical analysis showing that CPDA defines a valid kernel discrepancy, admits existing moment-matching methods as restricted cases, and yields a class-conditional target-risk bound. Extensive experiments with CNN, ResNet18, and TCN backbones on 13 different time-series DA benchmarks demonstrate the effectiveness of CPDA against 30 discrepancy, adversarial, and pseudo-labeling baselines.
Read the full abstract

Statistical Methodology · Cross-listed from cs.RO

Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spaces

Reliable uncertainty representation is essential for deploying autonomous systems that interact with their environment, as robots must reason about how uncertainty arising from both stochasticity and model mismatch is impacted by contacts with obstacles (e.g., when navigating through a cluttered environment or inserting a part into an assembly). We propose Calibrated Particle-sets for Trans-dimensional Uncertainty Representation (CaPTURe), a geometry-aware, conformal prediction-based algorithm that generates probabilistically valid prediction regions of the unknown future system configuration using particle-based models of arbitrary fidelity. While calibrated uncertainty predictions are essential for safe and efficient planning, analytical or learned motion models are often inaccurate - due to limited data, simplifying assumptions, unmodeled effects, etc. - which can lead to unsafe executions or task failure. Additionally, when a robot contacts an obstacle, the distribution of its future configurations can become multimodal or disjoint, or lie along manifolds of lower intrinsic dimension than the space of possible robot configurations. Our method uses a calibration dataset of system transitions to locally calibrate motion uncertainty estimates, constructing regions guaranteed to contain the future robot configuration at a user-set probability. Our calibration procedure captures how motion uncertainty varies between contact-rich and contactless motions, leading to sufficient coverage in both cases. We evaluate our method on two simulated planning tasks: controlling a marble around a labyrinth and performing tight-tolerance peg-in-hole insertion with a manipulator. Compared to relevant baselines, CaPTURe achieves the user-specified coverage requirement both in and out of contact and achieves up to a 30% absolute improvement in task success rate over the best baseline.
Read the full abstract

Statistics Theory

Limiting eigen-structure of spiked sample covariance matrices under missing observations

High-dimensional Principal Component Analysis (PCA) has become an essential tool in modern data analysis, offering dimensionality reduction and feature extraction. However, the presence of missing data introduces significant challenges, distorting the performance of PCA and complicating statistical inference. In this paper, we study the asymptotic behavior of PCA under a spiked population model with missing observations, leveraging recent advances in random matrix theory. We demonstrate that while the spiked sample eigenvalues exhibit asymptotic normality, the limiting parameters differ substantially from those in the complete data case, reflecting the non?trivial influence of the missing data mechanism. As an application of our results, we propose a test to evaluate the independent structure of a spiked population.
Read the full abstract

All authors Haotian Cheng, Huiqin Li, Yanqing Yin, Zhixiang Zhang

Statistical Methodology

Bayesian ACCESS for Understanding Latent Epidemic Trajectories from Publicly Released Suppressed Data: Application to U.S. Opioid-related Overdose Mortality

Publicly released health statistics play a central role in characterizing temporal trends and identifying structural changes in population health. However, disclosure limitation through suppression of small cell counts, as implemented in systems such as the Centers for Disease Control and Prevention Wide-ranging ONline Data for Epidemiologic Research (CDC WONDER), produces partially observed count data that complicate statistical inference. These challenges are particularly acute for rare outcomes and subgroup analyses, where suppression is widespread and varies across geographic regions, demographic populations, and time. We propose Bayesian ACCESS (Autoregressive Change-point and Clustering Estimation for Suppressed Count Series), a Bayesian hierarchical framework for inference on latent epidemic trajectories and their structural changes from disclosure-limited health statistics. The proposed model directly represents suppressed count data through a suppression-aware observation model, jointly infers multiple temporal change points and latent trajectories, and borrows information across related geographic and demographic populations through Bayesian nonparametric clustering while preserving meaningful heterogeneity. We apply Bayesian ACCESS to opioid-related overdose mortality data from CDC WONDER for U.S. states from 1999 to 2024. The analysis identifies distinct subgroup-specific epidemic trajectories and structural changes that would be difficult to characterize using publicly released health statistics without explicitly accounting for data suppression.
Read the full abstract

Statistics Applications

Extreme Value Alpha and Crash Risk: Separating Structural Tails from Lottery Tails with LLM-Extracted Disclosure Networks

A heavy upper tail in a stock's returns is ambiguous: it can be a lottery tail, transient jump risk that investors overpay for (the MAX discount), or a structural tail, the statistical shadow of an economic reconfiguration that precedes extreme winners. Returns alone cannot separate them, so tail heat alone is not an alpha signal. Our discriminator is the firm's disclosure-measured network: a directed, span-grounded graph from 10-K filings via an auditable LLM pipeline, whose rewiring decomposes into edge birth, death, and drift. The central sign pattern: tail heat with network death is the crash side; tail heat with an intact or forming network is where structural tails and historical winners live. Pilot evidence from 24 technology firms (2014-2025) supports the crash side: upper-tail heat interacted with death mass predicts negative forward abnormal returns (monthly t = -2.9; firm-vintage t = -3.9; wild-cluster p = 0.04; robust to two-way clustering and controls), and the same configuration preceded NVIDIA's 2018 and 2022 drawdowns. This death-side signal is immediately useful as a risk-monitoring danger flag. The alpha side is directionally positive but not significant in the pilot and awaits the confirmatory test. A pre-registered replication on 50 random S&P 500 firms failed, defining the boundary: outside coherent ecosystems the disclosure graph nearly vanishes (83% of firm-vintages have zero death mass), so the discriminator exists only where firms densely document counterparties. The confirmatory design is fully pre-specified, with a gatekept primary pair, archived power simulations, positive-only winner labels, selection-corrected benchmarks, and a frozen ecosystem-coherent universe with a density gate; it activates only if the gate passes. If confirmed, tail heat becomes a conditional signal separating crash risk from structural winners.
Read the full abstract

Machine Learning (Statistics)

Personalized Federated Learning via Variance-Aware Nonparametric Empirical Bayes

We develop a new approach to Personalized Federated Learning across heterogeneous clients using Nonparametric Empirical Bayes (NPEB). Leveraging the asymptotic normality of local parameter estimates obtained from Empirical Risk Minimization or M-estimation, our method formulates these estimates as noisy observations to estimate an unknown shared prior via Nonparametric Maximum Likelihood. A key challenge in applying NPEB in this setting is that existing approaches assume known fixed variances, which is not true in practice. To address this, we introduce a Variance-Aware Nonparametric Empirical Bayes (VANEB) framework that leverages the parameter-dependent asymptotic variance of local M-estimators. A key technical contribution is a generalized Tweedie's formula for this heteroskedastic setting. We then establish non-asymptotic error rates for density estimation in the average squared Hellinger distance and derive an oracle denoising inequality that provides error bounds for our estimator. While our theoretical guarantees are rooted in the asymptotic regime of M-estimators, we empirically explore heuristic extensions of VANEB to modern federated learning settings involving Deep Neural Networks (DNNs). For DNNs, we propose VANEB-head and VANEB-FT, which personalize the last fully connected layer via an NPEB step using an approximate diagonal variance estimator. We show that our method has strong performance on popular vision datasets MNIST and CIFAR-10, using a convolutional neural network architecture.
Read the full abstract

Statistics Applications

When Can Text Embeddings Replace Item Calibration? A Geometric Diagnostic for Semantic Loadings in Multidimensional Adaptive Testing

Multidimensional item response theory relies on calibrated item parameters, such as discrimination and category threshold values, which are usually estimated from large samples of human test responses. This study investigates whether the directional loadings of these parameters can be recovered directly from item text using pre-trained sentence embeddings, avoiding the need for initial item calibration. Using the open-source IPIP Big-Five dataset ($n=19{,}719$; 50 items), we built a multidimensional computerized adaptive testing (CAT) simulation using D-optimal item selection. We compared three item loading sources: fitted graded response model parameters, semantic text embeddings, and a lexical baseline. In simulation, semantic embeddings recovered latent trait profiles almost as accurately as fitted parameters (correlation $0.825$ vs. $0.857$), performing noticeably better than simple word overlap ($0.752$). However, the embedding-based model produced inflated posterior variance, showing nearly four times higher measurement uncertainty despite accurate point estimates. We attribute this to collinearity across dimensions, as embedding-derived loadings pointed in similar directions across traits (condition number $137$ vs. $1.0$; mean trait cosine $0.90$). This outcome reflects the shared vocabulary common in personality items. We propose a simple diagnostic metric based on the loading matrix condition number to evaluate whether an item bank is suitable for text-derived loadings prior to testing.
Read the full abstract

Statistics Applications

Forecasting in the Fog: Real-Time versus Revised-Data Evidence on Machine Learning's Edge over the Phillips Curve

ML inflation forecasts are almost universally trained on fully revised data, even though real-time forecasters never have such data, and reported feature importances are typically computed in-sample, conflating predictive relevance with retrospective fit. This paper asks whether the ML advantage over the Phillips curve documented in Agyekum (2026) survives when models are trained and evaluated on real-time (ALFRED) vintages rather than revised series, and whether SHAP feature-importance rankings are an artifact of in-sample estimation. Using 2000-2026 U.S. data on unemployment, CPI and PCE inflation, payrolls, real GDP, and the 10-year-2-year Treasury spread, vintage-consistent panels are built for four traditional models (random walk, AR(1), Phillips curve, ADL-OLS) and four ML models (Random Forest, Gradient Boosting, Elastic Net, SVR), re-estimated recursively at 3-, 6-, and 12-month horizons (208, 206, 204 forecasts). Real-time/revised accuracy differences are small and, apart from one exception at 6 months (Gradient Boosting vs. Phillips curve, DM = -1.671, p = 0.097), indistinguishable under Diebold-Mariano tests; Gradient Boosting alone shows consistent positive skill at longer horizons. The random walk remains a strong short-horizon benchmark, consistent with the puzzle in Agyekum et al. (2026) for exchange rates. Using walk-forward, out-of-sample SHAP, a Random Forest on revised data assigns dominant importance to PCE inflation (mean |SHAP| = 0.778, rank 1 of 9), while on real-time data it assigns PCE negligible importance (0.039, rank 6), relying instead on current CPI (0.834 vs. 0.223). This twenty-fold swing, larger than the in-sample estimate, is invisible to point-forecast metrics and shows the model's PCE reliance is substantially a hindsight artifact. An RSI summarizes the accuracy gap by model and horizon, with implications for auditing ML inflation forecasts.
Read the full abstract

Statistical Methodology

Selecting among Missingness Models for Sequential Outcomes with Nonignorable Nonresponse

Sequential outcomes in longitudinal studies and multi-wave surveys may be missing not at random at both earlier and later occasions. We study graphical models in which at least one outcome is self-censoring and the response indicator for a later outcome may depend on either the earlier response indicator or the realized earlier outcome. These restrictions define two candidate families under which the relevant full-data distributions are identifiable and whose observed-data models overlap; graphs containing both dependencies form a broader class outside the prespecified comparison. For each candidate family, we establish identification of the full-data distribution under rank or completeness conditions and develop likelihood-based estimation. We then propose a two-stage Vuong-type procedure. The first stage determines whether the candidate models are observationally distinguishable; only after distinguishability is established does the second stage compare their Kullback–Leibler divergences from the true observed-data distribution. We also show that ordinary Wald inference remains asymptotically valid for the selected model-specific functional when the selected model has a fixed positive expected log-likelihood advantage. Simulations evaluate the two-stage procedure across graph classes. We finally apply the procedure to compare candidate missingness models in the Job Corps data and perform downstream functional estimation under the selected model.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.AI

Context Is Not Authority: Structured Runtime Governance for Financial Market Agents

Financial agents can turn correct context into an unauthorized effect: a customer-facing commitment, trade, or deployed policy. We present SAGE-Fin, a finance-specific authority-handoff contract that makes the proposed effect, not merely its text, the object of runtime control. SAGE-Fin compiles proposals into typed, adapter-bound candidates; records missing or stale institutional obligations as coverage debt; contracts authority under current market, account, policy, and dialogue state; and requires an exact-artifact receipt whose nominal type matches the consuming response, execution, or policy adapter. Evidence and workflow progress cannot substitute for effect authority, and prior authorization is rechecked after state changes. Across an authored 616-case catalog, five deterministic specifications yield 3,080 outputs; a label-isolated harness obtains 616/616 binary reference-prototype parity, including 3/3 named response-gate fixtures, while 22 tests cover selected paths. These results establish executable conformance, not independent safety accuracy. Separately, SAGE-Fin's response gate processed real customer-facing production requests at a confidential digital-asset platform. An operational team independent of the implementation team reached a strongly positive post-deployment conclusion on practical usefulness and workflow fit, and end-user feedback was also strongly positive. Disclosure permits only the review's independence, stakeholder classes, assessed dimensions, and directional conclusion, so this is qualitative field corroboration rather than an aggregate effect estimate. Three distinct de-identified predecessor failures, with independently confirmed 0/3 interception, ground repeated-emission drift, stale account evidence, and missing escalation state without estimating prevalence or treatment effect.
Read the full abstract

All authors Rui Tang, Qiangqiang Liu, Yichi Zhang, Youwei Yang, Xi Chen, Chen Dong

Machine Learning (Statistics) · Cross-listed from math.OC

Provable Parameter-Free Fixed-Point Algorithms with Linear Convergence Rates

In this paper, we develop provable parameter-free fixed-point algorithms for contractive mappings, with an emphasis on automatically exploiting hidden contractivity without requiring prior knowledge of the contraction factor. Our first method is a parameter-free variant of the Halpern fixed-point iteration with no line-search, bisection, or prior estimate of the contraction factor, while retaining essentially the same per-iteration computational cost as classical fixed-point schemes. We establish explicit linear convergence rates for both the fixed-point residual and the distance to the unique fixed point. The second algorithm is also a parameter-free Halpern method, but it requires a line-search procedure to find a local estimate for the contraction factor, which may need a few additional operator evaluations. This method also enjoys explicit linear convergence guarantees. We further extend these ideas in two directions. First, using the relation between Halpern iterations and Nesterov-type accelerated fixed-point schemes, we derive parameter-free Nesterov-type (three-term) variants that inherit linear convergence in the contractive setting. Second, by combining the proposed fixed-point schemes with Tikhonov regularization, we obtain new methods for solving cocoercive equations and establish an iteration complexity of $\mathcal{O}({\epsilon^{-1}\log(\epsilon^{-1})})$ for computing an $\epsilon$-solution, provided that a valid lower bound on the cocoercivity constant is available. Numerical experiments on several examples demonstrate that the proposed algorithms are competitive with, and often outperform, existing adaptive fixed-point methods. In particular, the methods successfully exploit contractive behavior when it is present on the selected nonexpansive test problems.
Read the full abstract

Sunday 9 August

Statistics Applications · Cross-listed from physics.ao-ph

Do AI Forecast Ensembles Sample the Correct Conditional Distribution?

Ensemble forecasting aims to sample the conditional distribution of outcomes; whether AI forecast ensembles do this correctly in a joint sense remains largely untested. We train a diffusion model for probabilistic subseasonal coastal sea level forecasts at eight US East Coast tide gauge stations, with sea level derived from reanalysis, and find that marginal and joint forecast quality decouple: positive skill at every station and lead time marginally, while joint spatial structure is worse than climatological draws. A shuffle-based permutation decomposition reveals this failure is invisible to the energy score but detected by the variogram score. Lorenz-96 experiments across 0.7-170 equivalent years show the gap persists regardless of training volume and is reproduced by a linear baseline, indicating structural inadequacy of the learned distribution. A dynamical ensemble does not replicate the failure while a deterministic emulator does, suggesting it is specific to learned emulators rather than ensemble forecasting generally.
Read the full abstract

Machine Learning (Statistics)

Physics-Informed Learning for Robust Acoustic Localization with Calibrated Uncertainty

Recent advances in Passive Acoustic Monitoring (PAM) offer an opportunity to obtain ecological spatial point-process data at unprecedented scale. However, realizing this opportunity necessitates the development of accurate and scalable localization methods. In real-world outdoor soundscapes, however, the assumptions underlying classical localization methods such as hyperbolic and score-based localization are routinely violated by multipath dominance, near-field effects, and complex propagation. Under these conditions, classical localization methods become brittle, with extreme errors possible even in small detection arrays. Rather than statistically replacing the underlying physics, we propose a method to refine it and increase robustness outside of ideal operating conditions: a learned model operating on physics-informed acoustic features corrects a fast hyperbolic solver where it produces implausible solutions, substantially reducing catastrophic worst-case errors while matching its median accuracy on field data. We further provide calibrated, geometry-aware uncertainty estimates suitable for propagation into downstream spatial models. Evaluating on distributed microphone arrays in real and simulated outdoor environments, we demonstrate that the proposed method yields robust, uncertainty-aware localization, providing a step toward scalable automated wildlife monitoring in complex acoustic environments.
Read the full abstract

All authors Jennifer N. Kampe, Changwoo J. Lee, Xin Shen, Ari Lehtiö, Sandro von Brandenburg, Ossi Nokelainen, David B. Dunson, Otso Ovaskainen

Statistical Methodology

Win Time In Favor of Treatment (WINFT) for Hierarchical Endpoints

Standard win statistics methods determine a win, loss, or tie for a pair of subjects based on their worst outcomes (up to the end of study) that may not fully utilize all patients' conditions or disease experience throughout the follow-up period. While the newly developed win-time statistics fully utilize all patients' longitudinal information, these statistics have been limited to time-to-event endpoints and require monotonic pattern of the events. As such, they are not applicable to any type nor number of hierarchical longitudinal endpoints. We propose the win time in favor of treatment (WINFT), a general measure for any hierarchical longitudinal endpoints, that summarizes the total time a subject in the treatment group spends in a more favorable health state than a subject in the control group. Unlike existing win time methods, the WINFT does not require the component outcomes to be monotone and does not rely on modeling assumptions for estimating state probabilities. This flexibility allows analysis of a complex and diverse set of endpoints, and includes existing win time methods as special cases. Moreover, the WINFT is estimated based on U-statistics, which provide direct framework for variance estimation and confidence interval derivation, under independent censoring and missing at random assumptions, without expensive bootstrapping. We examine the performance of the proposed WINFT estimation method through simulation studies, and illustrate the method using data from the ACTT-1 COVID-19 and HF-ACTION trials. Overall, the WINFT offers a flexible and interpretable estimand for assessing treatment in clinical trial data with complex longitudinal outcomes.
Read the full abstract

All authors Sahil S. Patel, Huiman Barnhart, Lu Mao, Roland A. Matsouaka, Yuliya Lokhnygina