Machine Learning (Statistics) · Cross-listed from cs.LG
PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes
Yang Ba, Michelle V. Mancenido and Rong Pan
Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As
datasets grow, they often add semantically redundant samples rather than expanding semantic coverage, which deteriorates the quality of the whole
dataset. Existing diversity metrics primarily summarize lexical variation,
entropy, or pairwise similarity, but provide limited characterization of the shape of the underlying data distribution. In this paper, we propose PLDiv, a diversity metric derived from persistence landscapes that quantifies the structural organization of a
dataset from its
geometric representation. PLDiv summarizes how data neighborhoods evolve as nearby samples progressively connect, producing an interpretable scalar measure of structural diversity without requiring task-specific supervision. Whether measuring intrinsic diversity in text generations or image
embeddings, or detecting semantic collapse in instruction
datasets and
reasoning trajectories, PLDiv performs competitively against evaluated baselines and aligns strongly with diversity labels during semantic collapse. Consequently, it effectively supports applications in synthetic data monitoring,
dataset construction, data augmentation, and generative model evaluation.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Rotate the Loop: Blockwise Conformality for Looped Transformers
Alexey Zemtsov, Albina Klepach, Vadim Lomshakov et al. (+5 more)
Looped
transformers reuse one shared block across many recurrent steps, so depth becomes a quantity that can be spent at runtime rather than a stack of distinct layers. Those extra steps help only if a write made on one recurrent step is still readable on later ones, and that depends on the linear map applied to the residual between recurrent steps. We show that whether the loop learns to track state is a
geometric question about the form of this map, and we propose blockwise conformal injections: the residual splits into coordinate-aligned groups and rotates inside each one, so every group has its own timescale while the geometry stays exact within it. Our experiments show a performance increase on a demanding form of state tracking - last-
token accuracy when the rest of the prefix is already correct - and on
language modelling, where a signal written earlier has to be retrieved at the end of the sequence.
Read the full abstract ▾
All authors Alexey Zemtsov, Albina Klepach, Vadim Lomshakov, Andrei Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Vladislav Kurenkov, Alexander Nikulin
Machine Learning (Statistics) · Cross-listed from cs.LG
Can We Trust This Evaluation? Instance-Level Reliability of LLM-as-a-Judge
Yang Ba, Michelle V. Mancenido and Rong Pan
LLM-as-a-judge evaluation commonly aggregates multiple judges' decisions into a majority vote, yet this panel decision process does not guarantee that the evaluation of an individual item is reliable. We propose Structured Cross-fitted Uncertainty Risk (SCUR), a label-free approach for estimating instance-level evaluation reliability based on the judge panel's behavior. A latent measurement model serves as the backbone for measurement system analysis, from which SCUR derives three complementary signals capturing panel uncertainty, measurement cross-
entropy, and perturbation sensitivity. These three sources of evidence are combined into a risk score. We evaluate the SCUR approach on five real-world
benchmarks using majority-vote error ranking, selective trust, and risk stratification. Furthermore, we show our approach to be more effective at detecting high-consensus risk items, which are the blind spots for many other LLM-as-a-judge evaluation methods.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Text
Xinyi Yi, Moy Yuan and Ioannis Lestas
Electricity price forecasting (EPF) supports scheduling, bidding, and risk management in electricity
markets, yet existing
benchmarks focus mainly on numerical inputs, leaving the forecasting value of forecast-time textual context insufficiently evaluated. We introduce TRACE, a reproducible
benchmark of 7,300 zone–day instances pairing prices from five zones in a major U.S.
market with official operational text available at the forecast cutoff. TRACE reconstructs official operational text at each cutoff, preventing post-cutoff information leakage. We evaluate TRACE for semantic
alignment and forecasting value. Semantic assessments align with central movement and both tail risks in ground-truth prices, most consistently for upper-tail price risk. Forecasting value is reflected in a median 7.4% reduction in upper-tail pinball loss across time-series foundation models. A controlled cross-day text-mismatch ablation reverses the gains, falling below the no-text baseline.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Counterfactual Predictions in Scientific Emulators Without Controlled Experiments
Dingling Yao, Kahaan Gandhi, Valentin Duruisseaux et al. (+3 more)
Many scientific questions require
reasoning about what was never observed: What if the conditions, interventions, or history had been different? Models can predict accurately on observed data yet fail on such what-if queries when correlated inputs are varied independently. A common remedy is to add controlled simulation data in which these factors are explicitly disentangled, but this requires access to a simulator, can be computationally expensive, and inherits the simulator's modeling assumptions. We introduce ReRoute, a framework for targeted scientific what-if prediction that combines factual data with partial mechanistic knowledge, without requiring controlled intervention data for adaptation. ReRoute fixes the queried input of a pretrained backbone to a reference value, reintroduces its variation through a known mechanistic pathway, and fine-tunes on the original factual data, while leaving downstream effects to the learned dynamics. We provide a causal identification result for this construction under explicit structural assumptions, with the core argument machine-checked in Lean. After showing that ReRoute achieves highly accurate counterfactual predictions in a controlled advection-
diffusion system where exact responses are available, we turn to state-of-the-art
climate emulation. On held-out coupled-
climate interventions, ReRoute reduces aggregate
climate error by 18.2-31.8% under severe CO$_2$ distribution shifts while preserving skill under standard conditions, at a small fraction of the cost of retraining on additional controlled simulations, without even accounting for the substantial expense of generating such data. Finally, on an emulator trained from historical ERA5 reanalysis, where no counterfactual reference exists, ReRoute preserves substantially more of the surface warming implied by the observed boundary conditions under a fixed-CO$_2$ counterfactual.
Read the full abstract ▾
All authors Dingling Yao, Kahaan Gandhi, Valentin Duruisseaux, Boris Bonev, Francesco Locatello, Anima Anandkumar
Statistical Computation · Cross-listed from cs.LG
Rank-Aware Speculative Sampling for Diffusion Draft Trees
Marcello Bullo, Yanxiao Liu, Öykü S{\i}la Güner et al. (+2 more)
Speculative sampling accelerates
diffusion generation by verifying inexpensive draft states in parallel while preserving the target law. Recent tree-based methods allocate the parallel compute budget more effectively than single-chain drafts, as demonstrated by
Diffusion Greedy Rejection Sampling (D-GRS). D-GRS generates $K$ conditionally independent candidates per node, and sequentially tests them in their generation order. Yet the sampled candidates admit an informative ranking without additional target-model evaluations. To exploit this, we introduce Rank-Aware Speculative Sampling (RASS), a verification rule for speculative draft trees based on rank-aware list coupling. RASS orders draft candidates along the proposal-target mean displacement and samples a rank with weights optimized to minimize total variation between the selected-proposal and target laws. Finally, the selected candidate is maximally coupled with the target, with residual correction ensuring exact sampling for any choice of rank weights. We evaluate RASS on a Gaussian-mixture target, unconditional pixel-space generation on FFHQ, conditional generation on CIFAR-10, and latent
diffusion with Stable
Diffusion 3.5 using COCO2014
prompts. Measured by the ratio of standard to speculative sampling's target-model evaluation counts, RASS improves on D-GRS across the evaluated settings, with gains reaching approximately 20% on CIFAR-10 at matched compute budgets.
Read the full abstract ▾
All authors Marcello Bullo, Yanxiao Liu, Öykü S{\i}la Güner, Arpan Mukherjee, Deniz Gündüz
Machine Learning (Statistics) · Cross-listed from cs.LG
Nearest-neighbour baselines for fingerprint prediction from MS/MS spectra under different assumptions
Ling Min Serena Khoo
It has recently been shown that nearest-neighbour
retrieval provides a strong baseline for molecular fingerprint prediction from MS/MS spectra, with several variants matching or outperforming current deep learning models (Khoo and Barzilay, 2026; Liu et al., 2026; Gupta et al., 2026). Importantly, "nearest neighbour" encompasses a family of
retrieval methods that differ in the information assumed to be available at inference. In this report, we systematically compare several nearest-neighbour variants and show how these differing assumptions affect performance. Our goal is to establish stricter baselines that enable more rigorous benchmarking and better measure progress in this area.
Read the full abstract ▾
Machine Learning (Statistics)
Inference for stochastic differential equations driven by weighted sub-fractional Brownian motion using neural networks and the Euler approximation
J. H. Ramirez-Gonzalez
We consider the estimation of drift,
diffusion, and noise covariance from discrete observations of
stochastic differential equations driven by Gaussian processes. For a fixed observation horizon $T>0$ and a known initial state $x_0\in\mathbb R$, we study \begin{equation*} dX_t=a(X_t)\,dt+\sigma(X_t)\,dZ_t^{\beta,f}, \qquad X_0=x_0,\quad 0\leq t\leq T. \end{equation*} \smallskip\noindent Here $a:\mathbb R\to\mathbb R$ is the drift coefficient, $\sigma:\mathbb R\to(0,\infty)$ is the
diffusion coefficient, and $Z^{\beta,f}$ is a centered Gaussian process from the weighted sub-fractional Brownian family, with covariance \begin{equation*} \operatorname{Cov}(Z_s^{\beta,f},Z_t^{\beta,f}) =\int_0^{s\wedge t} f(r)q_\beta(s-r,t-r)\,dr, \qquad 0\leq s,t\leq T. \end{equation*} \smallskip\noindent Here $s\wedge t=\min\{s,t\}$. The temporal weight $f:[0,T]\to[0,\infty)$ is measurable, bounded, and positive almost everywhere, and $\beta\in(0,2)$ is the covariance exponent. For $u,v\geq0$, the kernel is $q_\beta(u,v)=[u^\beta+v^\beta-(u+v)^\beta]/(1-\beta)$ when $\beta\ne1$. Its continuous extension at $\beta=1$ is $q_1(u,v)=(u+v)\log(u+v)-u\log u-v\log v$, with $0\log0=0$. Using the Euler approximation, we reconstruct the Gaussian driving increments from observed transitions and use their joint density to obtain a trajectory likelihood. Neural and radial-basis representations model the drift,
diffusion, and normalized temporal weight, while a likelihood profile estimates the covariance exponent and
diffusion scale. We compare the method with two neural alternatives on the same simulated trajectories in twenty coefficient settings.
Read the full abstract ▾
Statistics Theory
Integration of external predictions for efficient estimation of the risk ratio
Victoria Mezger, Nils Krüger and Georg Hahn
We consider the augmentation of randomized experiments or trials with data from observational studies for the purpose of improving statistical precision. In particular, we focus on the Hybrid Augmented Inverse Probability Weighting
estimator, designed to integrate predictions from several foundation models while preserving valid statistical inference. In this article, we extend the Augmented Inverse Probability Weighting framework to the estimation of the risk ratio, the quotient of the absolute risk of an exposed to an unexposed group. Our approach allows one to use information from black-box foundation models trained on external and possibly unstructured data, yielding an
estimator of the risk ratio whose asymptotic variance is never larger than the one of the default
estimator based on experimental data alone.
Read the full abstract ▾
Statistical Methodology
Factor Model Estimation for High-Dimensional Time Series with Heteroskedastic Noise
Da Zhao and Jing Chen
Factor modeling provides a framework for extracting common components from high-dimensional time series. We develop an
estimator of the factor loading space that combines contemporaneous covariance with lagged autocovariance information. Under cross-sectionally heteroskedastic white noise, the contemporaneous covariance contains heterogeneous diagonal noise contributions. We apply the existing HeteroPCA
algorithm to the combined estimation
matrix to correct this contamination while retaining information from serial dependence. The framework also accommodates noise contamination on a known set of entries subject to sparsity and incoherence conditions. Under regularity conditions for dependent observations, we establish consistency and derive convergence rates, identifying regimes in which incorporating contemporaneous covariance improves the rate relative to the lagged-autocovariance
estimator. Simulations evaluate loading-space estimation error and the effects of contemporaneous covariance inclusion and diagonal correction. An application to S&P 500 stock returns illustrates the method.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Scalable Multi-Task Inverse Reinforcement Learning
Allen Tran, Jia Wan, Nathan Kallus et al. (+1 more)
By learning transferable rewards, inverse reinforcement learning (IRL) enables counterfactual evaluation of
agents under modified environments. Such transfer places strict requirements on coverage since target environments affect
agents' state occupancy. We propose a multi-task IRL method that pools data across multiple
agents with different rewards in the same environment under a low-rank assumption. In addition to alleviating coverage requirements, so each task need not visit every state as long as others do, the method offers scalable evaluation of multiple tasks under new environments as computationally intensive planning scales with rank rather than the number of tasks. We provide finite sample guarantees on reward recovery and on policy learning in new environments. Experiments show our method is robust to limited coverage, recovers rewards on and off of each task's support, transfers to target environments at lower regret than baselines, with its computational advantage over per-task methods widening as tasks grow.
Read the full abstract ▾
All authors Allen Tran, Jia Wan, Nathan Kallus, Aurélien Bibaut
Machine Learning (Statistics)
Learning to Price Electricity for Optimal Demand Response
Jing Shang, Mohammad Mehrabi, Xinyang Zhou et al. (+3 more)
There is considerable interest in using time-varying electricity prices to shape consumer demand response, and better align energy demand with renewable production. However, optimal prices generally vary over time in response to complex signals such as weather forecasts, sunrise/sunset times, and day-of-week patterns; and existing methods are not able to make efficient use of such rich contextual information. Here, we propose a neural-network-based
algorithm for contextual energy pricing, modeling pricing as a Stackelberg game and leveraging a mean-field solution representation from Mehrabi et al.(2024). The approach learns constrained mappings from contextual features to feasible price signals. We validate our approach by simulating the energy grid in several US cities, and show that incorporating contextual information can considerably increase the value of the demand response programs.
Read the full abstract ▾
All authors Jing Shang, Mohammad Mehrabi, Xinyang Zhou, Mahmoud Saleh, Andrey Bernstein, Stefan Wager
Machine Learning (Statistics) · Cross-listed from cs.LG
Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces
Sakin Kirti and Joel Zylberberg
Recent theoretical work identified fundamental properties of representation geometry that shape inference ability of deep neural networks. These include signal-noise factorization (SNF), the ability to segregate signal from noise, and signal-signal factorization (SSF), the ability to segregate task-specific and task-irrelevant signals. Here, we built regularizers that reinforce these two properties during training. We compared networks trained with these regularizers to $L_2$-regularized baseline networks on the CIFAR-100 classification task to understand how our regularizers shape representation geometry and impact performance on a well-known computer
vision baseline. Enhancing SNF via regularization improved model performance but enhancing SSF did not. Motivated by biomedical applications, we investigated how our regularizers affected performance on the BloodMNIST
dataset treated with MedMNIST-C corruptions at five severity levels, and found even larger performance gains using the SNF regularizer. To understand the mechanism by which SNF-regularization produces improved performance, we analyzed the nuisance subspaces across regularization regimes, finding that the SNF-regularized models represent noise in distinct subspaces, separate from class-relevant signal. Because this geometry is explicit, the dominant corruption-induced directions can be estimated on held-out data and projected out of the representations. This manipulation led to a substantial gain in accuracy. These results show that regularizers that enforce signal-noise factorization can produce substantial improvements on computer
vision tasks that contain out-of-distribution image distortions at inference time. They also highlight how shaping representations affects model performance: isolating nuisance variables from categorical ones is more important than maintaining factorized representations of categorical variables.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CL
Sequential Functional Structured Tucker Compression for Large Language Model Attentions
Jiangfeng Chen, Xinyu Wang, Tianshuo Yan et al. (+4 more)
Post-training compression of LLM
attention is often formulated as independent
matrix approximation, ignoring both the shared structure among
attention projections and the representation shift introduced by earlier compression. We propose FTC, a sequential structured compression framework that adapts the approximation to the current compressed model while jointly exploiting the native Q/K/V head structure under a fixed storage budget. The output projection is handled separately to account for the changed post-
attention representation. FTC requires neither fine-tuning nor gradient-based recovery. Across seven decoder-only LLMs from 6B to 32B parameters, FTC achieves the lowest WikiText-2 perplexity among the compared methods at every tested keep ratio on five modern GQA models, with the largest gains under aggressive compression. The improvements transfer to downstream tasks and remain substantial at the 32B scale.
Read the full abstract ▾
All authors Jiangfeng Chen, Xinyu Wang, Tianshuo Yan, Hanwei Wu, Xiao-Wen Chang, Yang Zhang, Lei Ding
Statistical Methodology
Discussion on "Principled type I error rate inflation in two-arm clinical trial designs with external control information borrowing" by Silvia Calderazzo, Manuel Wiesenfarth, Vivienn Weru, and Annette Kopp-Schneider
Tim Friede and Christian Röver
Consideration of frequentist type-I error or power based on certain fixed conditions may not fully embrace the aims of a
Bayesian analysis; this is particularly true in settings where informative priors are involved. Calderazzo et al. consider adaptations of a testing procedure involving
Bayesian borrowing from external control data. Particular caution may be necessary here in order not to condition on parameters and data simultaneously. We consider their example case and argue that (a) the conditioning scheme may have limited practical implications, and (b) if the main concern are assumption violations, these might be better addressed by adapting the likelihood function.
Read the full abstract ▾
Statistical Computation
Multifidelity Formulations for Triangular Transport
Owen Davis, Daniel Sharp, Youssef Marzouk et al. (+1 more)
We develop multifidelity methods for constructing triangular transport maps from samples, when high-fidelity data are scarce but lower-fidelity data are more abundant. Using this set of multifidelity data, we approximate a triangular transport map that bijectively maps between a tractable reference density and the high-fidelity target distribution. We introduce two strategies to leverage low-fidelity data: a hierarchical approach that composes maps between adjacent fidelity levels, and a non-hierarchical method that incorporates low-fidelity information through monotonicity-preserving corrections to the map parameterization. Numerical experiments compare these strategies with single-fidelity transport and demonstrate how the proposed multifidelity approaches can improve map estimation from limited high-fidelity data. To illustrate the broader utility of the learned maps, we also deploy them in a downstream amortized simulation-based inference task. This example shows that multifidelity improvements in map estimation can translate to improved conditional sampling and uncertainty quantification when high-fidelity data are scarce.
Read the full abstract ▾
All authors Owen Davis, Daniel Sharp, Youssef Marzouk, Gianluca Geraci
Machine Learning (Statistics) · Cross-listed from cs.CL
How Divergence Becomes Decision Flips in Compressed Language Models
Beatriz Almeida Felicio
Compression reports summarize how far a compressed
language model moved from the dense one, usually by a KL divergence; a deployment that relies on the dense model's outputs needs to know how many of its decisions changed. We show that total variation, not KL, answers this directly. Across 802 compressed and perturbed copies of 19 open models on five corpora and nine mechanically unrelated perturbation families, the rate at which the arg-max
token changes (the flip rate) tracks total variation at a ratio with median $1.05$, with no fitted constant. KL converts into flips only through its square root and a factor that varies fourfold across models and corpora, because KL averages over
tokens before the root is taken; first-order statistics averaged per
token, such as Hellinger distance, avoid this, but reports rarely give them. As a result, of two compressors reported on different models and corpora whose flip rates differ by at least $10%$, KL assigns the smaller divergence to the one that changes more decisions in $11%$ of cases, total variation in $1%$. Two pre-registered tests mark the limits: on a held-out code corpus the ratio held for all eight models while three predictions about KL each failed for half of them or more, and on three new models with real kernels it stayed in its band for 37 of 38 checkpoints but fell below one on code for two models. In vLLM speculative decoding, total variation measured under teacher forcing predicts greedy draft acceptance with a mean relative error of $1.1$–$2.4%$, without the task-specific calibration that KL needs.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Grand Canonical Generators
Andreas Burger, Malte Franke, Luka Mucko et al. (+2 more)
We introduce Grand Canonical Generators (GCG), a generative framework that extends Boltzmann generators to the grand canonical ensemble. We present two designs. The first conditions a variable-size generative model on the chemical potential, sampling particle number and configuration jointly. The second factorizes the grand canonical distribution into a particle-number distribution and the corresponding canonical Boltzmann density. This factorized formulation can use any existing Boltzmann generator for the canonical component, encodes the known linear chemical-potential dependence analytically, and yields a tractable likelihood that supports self-normalized importance sampling (SNIS). Empirically, GCG accurately reproduces grand canonical observables on a Lennard–Jones fluid and methane adsorption in a zeolite, demonstrating generalization across chemical potentials and correction via SNIS and grand canonical Monte Carlo.
Read the full abstract ▾
All authors Andreas Burger, Malte Franke, Luka Mucko, Kjell Jorner, Alan Aspuru-Guzik
Statistics Applications · Cross-listed from cs.AI
Agent Evaluation Reliability: More Tasks Won't (Always) Fix An Agent Leaderboard
Michael Hardy, Ruhana Azam, Anka Reuel et al. (+2 more)
Agent evaluations are increasingly used to compare LLMs and inform deployment decisions, yet ranks can reflect not only the model but also the effects of the evaluation conditions such as the scaffolds or tasks. This makes reliability claim-dependent: an evaluation that reliably ranks deployed systems may not reliably rank underlying models. We ask which conclusions current
agent evaluations reliably support and what additional evaluation would improve them. We develop a
Bayesian variance-decomposition framework for sparse, imbalanced
agent leaderboards and apply it to 22
benchmarks from the Holistic
Agent Leaderboard and Harbor Index. The framework separates signal, performance differences relevant to the intended claim, from noise, irrelevant variation that can still change rankings. We find: (1) Reliability depends on the measurement goal. Fixed model-scaffold systems are ranked reliably (0.935-0.994), while underlying-model reliability is substantially lower (0.148-0.841). (2) Scaffold choice can change conclusions. Inter-scaffold reliability measures whether scaffolds preserve model rankings, showing that scaffold effects vary substantially across evaluations. (3) More tasks cannot resolve all uncertainty. Even infinitely many similarly constructed tasks improve model-ranking reliability of a
benchmark by at most 0.097 when uncertainty is dominated by limited scaffold coverage. (4) Pooling diverse
benchmarks can improve cross-task rankings at lower cost. For rankings across diverse agentic tasks, pooling
benchmarks raises projected reliability from 0.44 to 0.75 at the same task budget and can reduce projected cost by up to 83%. Evaluation design should follow the intended claim: identify what a score or ranking should mean, diagnose what limits its reliability, and spend evaluation budget on the sources of uncertainty that matter.
Read the full abstract ▾
All authors Michael Hardy, Ruhana Azam, Anka Reuel, Mykel Kochenderfer, Sanmi Koyejo
Statistical Methodology
Standardized Dynamic Borrowing: Uniform Control of Information Borrowing
Huan Wang, Fei Wu and Yeh-Fong Chen
Dynamic borrowing can improve efficiency in clinical trials using external information, but existing methods may not offer transparent, prespecified control over borrowing. We introduce Standardized Dynamic Borrowing (SDB), which combines a source-informed component with a no-borrowing component and updates their relative odds using a Bayes factor standardized by its supremum over admissible target data. The posterior density under the no-borrowing component is obtained by normalizing the target-data likelihood. The prespecified source-component initial weight is a sharp uniform upper bound on its posterior weight, providing an interpretable borrowing constraint. When source and target models belong to the same one-parameter canonical exponential family, posterior expected local-information-ratio effective sample size equals target sample size plus borrowed source-prior effective sample size minus a nonnegative heterogeneity penalty. For normally distributed outcomes, borrowing depends on a standardized source-target conflict $Z$-score and decreases according to a standard normal kernel. We also jointly select the initial weight and target sample size to meet prespecified power and Type I error requirements. Normal and binomial illustrations demonstrate how SDB translates source-target compatibility into data-adaptive borrowing and how the calibration procedure prespecifies the source-component initial weight and target sample size under operating-characteristic constraints. These results provide a transparent framework for prespecifying, controlling, and interpreting dynamic borrowing.
Read the full abstract ▾
Statistical Methodology
Foundation or Formula? A Simulation-Based Comparison of Google's TimesFM-3 and Classical Time-Series Models
Ebrahim Khaled Ebrahim, Somaia Mohamed Ali and Ahmed El-Kotory
Time-series foundation models such as Google's TimesFM-3 forecast series they were never trained on, but whether they should replace classical methods such as ARIMA and exponential smoothing is hard to settle on public
benchmarks, because few
benchmark datasets are absent from every model's pre-training corpus. This exploratory study compares TimesFM-3 with classical methods on data generated from nine known processes, which the model cannot have seen. Across 7200 series and a pre-registered protocol with a 12-step horizon, neither family dominates. TimesFM-3's error is never more than 1.31 times that of the best method in a scenario, whereas every classical method's is at least 2.2 times somewhere; the largest classical failures occur with only two seasonal cycles, where the automatic methods fall back to non-seasonal models. With four or more cycles automatic ARIMA is 21-24% more accurate than TimesFM-3 on seasonal ARIMA data. Among five foundation models this robustness is specific to TimesFM-3, and it holds only at short horizons: at steps 25 to 48 the three foundation models tested there forecast a decline on a saturating process whose level stays flat, and automatic ARIMA has the smaller worst case elsewhere. On intermittent demand TimesFM-3 shows no advantage over simple
benchmarks built from the history. Randomised parameters, heavy-tailed noise and outliers leave these patterns qualitatively unchanged; on the official test period of 1000 M4 monthly series TimesFM-3 is level with the fifth- to seventh-ranked M4 entries, behind the four best, and on 101 macroeconomic series observed after the models' documented training data it is statistically level with the classical methods. The study characterises the behaviour of a black-box model, not the reasons for it, and its findings are conditional on the processes and horizons examined.
Read the full abstract ▾
Statistical Methodology
Nonparametric Identification of Latent Dimension under Heavy-Tailed and Mixed-Signals
Chetkar Jha
Principal component analysis and factor analysis are foundational to the study of high-dimensional data, yet their efficacy depends entirely on correctly identifying the latent dimension $r$. While parallel analysis (PA) is widely regarded as the gold standard for this task, its performance suffers in high-dimensional regimes consisting of mixed signals and/or heavy-tailed error distributions. This arises because under the preceding conditions, column-wise permutation mechanism (or any modern variations) applied to the data
matrix $\textbf{X}$ fails to destroy the signal, leading to an inflated null spectrum. To address this, we propose a nonparametric solution. We show that the latent dimension is uniquely preserved in the degree of order within the rank-transformed data
matrix, which restores spectral exchangeability under permutation. We prove the consistency of Rank PA under relaxed conditions, requiring neither the pervasiveness of factors nor restrictive fourth moment assumptions. We establish that Rank PA can consistently estimate the number of factors up to the maximum of $O(n^{1/3 - \tau})$ where $\tau >0$. The robust performance of our remedy is demonstrated through numerical simulations and an analysis of the FRED-MD
dataset.
Read the full abstract ▾
Statistical Methodology
Probabilistic linkage with graphs: leveraging cluster structure to improve the accuracy of entity resolution at scale
Evelyn Lauren, Laura White, Trevor Graham Bell et al. (+1 more)
Probabilistic record linkage identifies records belonging to the same entity even when identifiers are incomplete or imperfect, and is increasingly used to conduct population-level inferences from administrative data at scale. Linkage produces noisy pairwise links that must then be resolved into distinct individuals. The standard approach, transitive closure, applies a fixed threshold to each link independently and assigns clusters from the resulting connected components. However, the risk of merging distinct individuals grows with population size, motivating approaches that use graph structure to improve linkage accuracy. Clustering
algorithms from adjacent fields solve the same graph-clustering task in other domains, yet have never been systematically evaluated for entity resolution. We evaluated eleven clustering
algorithms – five established in entity resolution and six from adjacent fields – across three simulated
datasets and one real-world
dataset, and propose an adaptation tailored to linkage graphs. Cross-field
algorithms – Leiden, Louvain, label propagation – were among the most accurate, reducing linkage error by up to 66% relative to transitive closure. The three
algorithms also ran up to 40 times faster than Markov clustering (a leading approach in the entity resolution literature) and integrate directly into existing pipelines at scale.
Read the full abstract ▾
All authors Evelyn Lauren, Laura White, Trevor Graham Bell, Jacob Bor
Statistical Methodology
Three-group variance-ratio tests for heteroscedasticity in linear regression: an exact null distribution and an outlier-resistant version
Ahmed El-Kotory and Ebrahim Khaled Ebrahim
Tests for heteroscedasticity in linear regression lose their level or power in three situations: when the data contain outliers, when the variance is not monotone, and when it changes along a variable outside the regressors, such as time. The proposed tests sort the observations by any chosen variable, split them into three equal parts, fit the regression in each part and compare the largest with the smallest error scale. With least squares fits and normal errors, the ratio follows Hartley's maximum F distribution with three groups exactly. With least trimmed squares fits, the ratio resists outliers spread along the ordering. Its square is approximately a maximum F ratio with effective degrees of freedom, whose limit we derive in closed form from the influence function of the trimmed variance. A factorial simulation covered 72 settings of variance shape, ordering variable, number of regressors, contamination and sample size. The robust test had the highest mean size-adjusted power, 67%, against 45% for the next test; when the variance changed along a regressor, which all tests were given, it led White's test after an outlier screen, 67% against 56%. With heavy-tailed t3 errors its lead was similar, 64% against 35%. Unlike the tests built on the regressors, it can follow a variance changing along time, and it identifies the part of the data where the variance changes. Two published robust versions of the Goldfeld-Quandt test, as implemented from their descriptions, did not hold their nominal level. The R package KOTORY implements the methods.
Read the full abstract ▾
Machine Learning (Statistics)
Adaptive Conformal Prediction for Image Regression Models with Application to an Inertial Confinement Fusion Emulator
Carrie J. Lei-Cramer, Michael S. Jones and Laura J. Wendelberger
Uncertainty quantification is critical in scientific machine learning, where black-box, image-based models are increasingly deployed in high-stakes settings. In many such applications, model outputs inform costly decisions, yet most methods provide only point estimates without quantifying predictive uncertainty. This challenge is compounded by the limited accessibility and interpretability of model internals, making it difficult to assess reliability across different regions of the input space. As a result, there is a growing need for methods that can provide input-dependent uncertainty estimates to guide both model development and downstream experimentation. To address this need, we propose Adaptive Conformal Prediction using Nearest Neighbors (ACPNN), an input-adaptive conformal framework for image regression. ACPNN leverages information from neighboring samples to produce locally adaptive uncertainty estimates while maintaining low computational cost. The neighborhood structure is defined using a scaled distance metric learned via a Gaussian Process with an automatic relevance determination (ARD) kernel. We demonstrate the effectiveness of ACPNN on a
diffusion model for emulating inertial confinement fusion (ICF) simulations, showing that it achieves reliable and adaptive uncertainty quantification.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
SimplexUQ: An Evaluation Framework and Benchmark for Conformal Uncertainty on Simplex-Valued Predictions
Liang You, Hengyu Shi and Dongwen Ou
Conformal prediction guarantees marginal coverage, but a single calibration threshold can still spread that coverage unevenly, over-covering easy regions and under-covering hard ones. SimplexUQ is, to our knowledge, the first
benchmark and reproducible protocol for measuring this allocation problem on simplex-valued predictions; it compares existing conformal wrappers rather than proposing a new one. Its task suite, SimplexTasks-12, combines six controlled synthetic regimes with six frozen-predictor real tasks spanning class probabilities, topic mixtures, spectral abundances,
cell-type fractions, age distributions, and emotion mixtures. Each comparison fixes the predictor, score, and response-free stratification map, varies only the wrapper, and reports marginal coverage, worst-stratum coverage, max disparity, and within-task radius and compute. Global calibration can look valid while failing badly: on CIFAR-10 it attains 0.900 marginal coverage but only 0.542 in the worst
entropy stratum, and Mondrian calibration raises that stratum to 0.886 while reducing max disparity from 0.358 to 0.022. No wrapper dominates, however. Under smooth synthetic heterogeneity, several repairs are competitive; fixed-map analyses show that rankings depend on the evaluation groups and protocol; and in a 12-task comparison, Mondrian has lower disparity on its single target partition for all 12 tasks, whereas BatchMVP has lower disparity over overlapping groups on five. These are empirical comparisons, not new coverage guarantees. A controlled predictor-bias sweep shows that removing predictor bias only partly reduces global-threshold disparity. We release task cards, result provenance, permitted derived arrays, and rebuild instructions, and treat wrapper selection as a diagnostic comparison rather than a universal ranking.
Read the full abstract ▾
Machine Learning (Statistics)
Fractional Laplace Neural Operators: Exact Architectures, an Expressivity Frontier at Criticality, and Certified Stability for Memory-Driven Network Dynamics
Mauricio Herrera-Marín
Neural operators learn maps between function spaces, while hereditary network dynamics are described by Volterra resolvents with non-rational Laplace symbols. We introduce a fractional Laplace neural operator (fLNO) that embeds this structure in the learned map. For commuting excitation–
Laplacian pairs, one block graph-spectral layer represents the full linear Volterra solution operator exactly. We establish an expressivity frontier for finite rational realizations: they approximate fractional
memory geometrically on compact frequency windows, but cannot reproduce the non-integer critical asymptotics generated by a branch point, and on the half-line the best rational rate is root-exponential. The same theory yields trainable parametrizations that enforce a prescribed stability margin by construction, and a graphon-transfer
theorem separates genuine operator consistency from parameter sharing. In a common-data
benchmark, positive rational operators can match or exceed fLNO accuracy on finite horizons, whereas in controlled near-critical experiments fLNO recovers the branching coordinate more faithfully with far fewer parameters; unconstrained rational fits can cross the stability boundary, while certified parametrizations cannot. A four-parameter spectral law transfers without retraining from graphs of size 48 to 192 with 0.51–0.62% relative error. Applications to Chilean aftershock sequences and to renewal models for Chile and 21 Italian regions illustrate structured inference with explicit uncertainty. The contribution is an operator-learning architecture in which exact
memory structure, physical coordinates and stability guarantees coexist with competitive accuracy.
Read the full abstract ▾
Statistics Theory
Weighted Power Fréchet Means in Metric Spaces with Curvature Bounded Above
Christof Schötz
We establish non-asymptotic risk bounds for power Fréchet means in geodesic metric spaces with
curvature bounded above. The observations form weighted, possibly infinite sequences of independent random variables whose laws and means may differ. We treat three settings. For $2$-Fréchet means in Hadamard spaces, we obtain a sharp mean squared error bound that becomes an identity in
Hilbert spaces. For $2$-Fréchet means in CAT($\kappa$) spaces with $\kappa>0$, we establish variance and Wasserstein contraction inequalities with optimal constants depending on the circumradius $\varrho$ of the closed convex data domain, and obtain mean squared error bounds throughout the maximal range $\varrho<\pi/(2\sqrt{\kappa})$. For $\alpha$-Fréchet means in Hadamard spaces, $1<\alpha<2$, we derive finite $L^\alpha$ risk bounds under weighted $\alpha$-moment conditions, allowing even an infinite $\alpha$-moment of the population mixture. The proofs combine variance, quadruple, and contraction inequalities with a leave-one-out stability technique. Applications give prior and posterior bounds for Dirichlet-process Fréchet means and finite-sample guarantees for local constant Fréchet regression. The regression results require no response-space
entropy condition and replace density smoothness assumptions common in earlier work with transport smoothness; for $\alpha<2$, the bounds remain finite even when the responses have infinite variance.
Read the full abstract ▾
Statistical Methodology
Heteroskedastic Canonical Polyadic Tensor Decomposition
Kyle Ritscher and Carlos Llosa-Vite
When minimizing the squared-error loss, the popular CP decomposition can be interpreted as parameter inference in a Gaussian model with a low-rank mean tensor and constant variance across the tensor entries. We introduce heteroskedastic-CP (HCP), which models entrywise variability with a non-constant, low-rank precision tensor, and develop an alternating block-coordinate ascent method to recover both the low-rank mean and precision tensors from noisy observations. Our procedure is computationally competitive, with the same leading-order factor-update complexity as CP-ALS. We demonstrate HCP on synthetic experiments and an
EEG application.
Read the full abstract ▾
Statistics Theory · Cross-listed from quant-ph
Learning Local Fermionic Lindbladians under Parity Superselection
Tim Möbus, Daniel Stilck França and Cambyse Rouzé
Parity superselection forbids direct measurement of odd Majorana observables. We learn time-independent, parity-covariant, $k$-mode-local Lindbladians on $m$ modes using even preparations and measurements with uninterrupted short-time
evolution. Internal markers turn odd probes into even observables, and pair measurements among three separated markers calibrate their dynamical contributions. Signed Fierz inversion recovers canonical coefficients, while semidefinite fitting yields a valid generator. For finite-range models on known bounded-degree graphs with suitable marker access and a supplied weighted-strength bound $\bar\alpha$, entrywise error $\varepsilon$ is achieved using $\widetilde{\mathcal{O}}(\bar\alpha^2\varepsilon^{-2}\log(m/\delta))$ samples, without external modes or known nonzero coefficient locations. Recovery to diamond-norm error $\varepsilon$ costs an additional factor $m^2$, matching lower bounds for short-time experiments on fresh systems up to logarithmic and fixed
geometric factors. Without a supplied graph, one idle ancillary mode per system mode and potentially nonlocal pair operations give total absolute coefficient error at most $\varepsilon$ per mode using $\widetilde{\mathcal{O}}_k(\bar\alpha^2\mathsf d^2m^{\lfloor k/2\rfloor}\varepsilon^{-2}\log(m/\delta))$ samples, under a supplied approximate coefficient-degree bound $\mathsf d$ and controlled weak-coefficient tails. All guarantees hold with probability at least $1-\delta$. For
geometric models, fixed-support even observables can be predicted with logarithmic system-size sample complexity at fixed time and accuracy. We also give finite-volume and exponential-tail extensions.
Read the full abstract ▾