Statistical Computation · Cross-listed from math.PR
A Randomized Approximation Algorithm for Doubly Free-Boundary Macdonald Processes
Zhongyang Li
We construct an approximate sampler for finite doubly free-boundary Macdonald rail-yard processes with Macdonald, boundary, and edge parameters in $(0,1)$. The construction allows arbitrary finite sequences of additions and removals of horizontal or vertical strips and all four boundary types. It reverses a finite commutation–reflection history from a partition sequence whose entries are equal, with the common partition weighted by both boundaries. We derive local reconstruction kernels from the skew Cauchy, Littlewood, and branching identities. Bulk and boundary updates combine integer increments with conditional shape sampling at a prescribed size; same-sign swaps use finite conditional resampling. For $K$ reflection periods and an initial partition-size cutoff $M$, the
algorithm terminates almost surely and samples the resulting approximating law exactly. We prove total variation convergence to the target process as $K,M\to\infty$ and give explicit bounds for both truncation errors, decaying geometrically in the cutoffs for fixed input parameters. Under fixed parameter bounds, we also bound the expected operation count of direct enumeration in a unit-cost model. We further construct an exact rejection sampler for the initial partition law without a size cutoff or evaluation of its normalizing constant. Using this sampler removes the size-truncation error, leaving only the error from the finite reflection history.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
BridgeTwist: Twisting Schrödinger Bridges for Training-Free Conditional Sampling
Mengyu Li, Qianqian Qu and Jun S. Liu
Generative models are often required to serve many downstream conditional tasks. In the training-free setting, a pretrained unconditional model is reused as a prior and combined with a task-specific likelihood to sample from the corresponding posterior. While such training-free samplers have been developed for
diffusion and flow models, their counterpart for Schrödinger bridge (SB) models remains unexplored, despite the growing appeal of SBs as flexible
stochastic transport models between general source and target distributions. We propose BridgeTwist, a training-free conditional sampler for pretrained SB generative models, formulated as a tempered twisted sequential Monte Carlo scheme. Its twist is a closed-form plug-in surrogate, obtained by combining the velocity and score fields exposed by SB pretraining into an endpoint predictor and evaluating the likelihood there. Since this surrogate is less reliable near the source, we temper the twist with a time-varying exponent, downweighting its influence at early steps. We further introduce a bridge-shaped inference-time
diffusion schedule that concentrates the proposal noise in the interior of the path. Telescoping importance weights then cancel all intermediate twist factors, yielding an asymptotically exact sampler for the posterior. Empirically, BridgeTwist outperforms training-free baselines on class-conditional and text-prompted image generation, and is competitive with specialized inverse problem solvers on inpainting, deblurring, and super-resolution.
Read the full abstract ▾
Statistics Theory
Inference for Fréchet Mean Sets
Ryunosuke Miyazaki
We construct confidence sets for the entire Fréchet mean set by inverting studentized pairwise risk inequalities. Gaussian multiplier calibration localized around empirical near minimizers gives asymptotic coverage at least the nominal level, uniformly over distribution classes satisfying empirical-process conditions. The construction requires neither unique ness nor a
manifold structure. We characterize the limiting binding directions and give conditions for exact asymptotic coverage. On the intrinsic circle, exactness holds for every fixed non-point-mass law and uniformly over a family in which two means merge at a quar tic minimum. With ideal multiplier
quantiles, the construction reproduces the Euclidean Wald set under both nonsingular and singular covariance. Under regular local conditions, it is first-order equivalent to intrinsic Wald inference.
Read the full abstract ▾
Statistics Theory
Minimax rates for pointwise CATE estimation: the role of design smoothness
Yuxuan Hou and Zhenghan Li
We study the role of design smoothness in the minimax estimation of conditional average treatment effects at a fixed
covariate point. For design densities bounded above and away from zero, without further regularity, we derive upper and lower risk bounds with an arbitrarily small gap in the convergence-rate exponent. Under uniform design, we obtain matching upper and lower bounds at interior points in the regime of low nuisance smoothness. These results show that rough
covariate designs can lead to slower rates than uniform designs, even under the same smoothness assumptions on the treatment effect and nuisance functions, highlighting the role of the
covariate distribution in pointwise causal estimation.
Read the full abstract ▾
Statistics Applications
Women and the Nobel Prize in the sciences: an update to 2026
Per Lunnemann and Liselotte Jauffred
Every October, the Nobel Prizes in Physics, Chemistry, Physiology or Medicine and Economic Sciences are announced, with one to three prizes in each discipline. Of the 769 Laureates (since 1901) in these four disciplines only 4% are women (30 Laureates). In 2026, there are eight Laureates within these disciplines and they are all men. Based on historical data across four fields and a
Bayesian hierarchical model, we update the 8 year old model on bias in Nobel Prizes and confirm the gender gap, with exceedingly large confidence.
Read the full abstract ▾
Machine Learning (Statistics)
Understanding Circulant Permutation to Extend the CMinHash Estimator
Keegan Kang, Avery Hood and Benedict H.W. Wong
CMinHash promises a new direction for designing MinHash
algorithms with reduced variance and storage space via circulant permutations. However, the original variance analysis is difficult to extend to other MinHash
estimators. We present a new approach that gives explicit and easily comparable approximate variance expressions, and use it to analyze the Minner
estimator. Numerical experiments show that our variance expression matches the empirical mean square error (MSE) of estimates. Our work thereby demonstrates how to apply circulant permutation to other MinHash
estimators and compute their approximate variance.
Read the full abstract ▾
Statistical Methodology
Simultaneous Inference for Centre-Outward Rankings of Random Objects
Lingfeng Lyu and Doudou Zhou
In many applications one object is chosen to represent a sample and the least typical objects are set aside, usually without any measure of uncertainty. We measure how central an object is by its reference depth, the probability that a random reference point is closer to it than to a random opponent. Using only pairwise dissimilarities, we construct simultaneous intervals for all ranks, a centre set that contains the most central observation under stated conditions, and outlier sets that contain only objects among the least central ones, each with asymptotic probability at least a prescribed level. These guarantees hold even though the ranked objects are themselves random. For symmetric distributions on the real line, the centre set has the smallest possible order of size up to a logarithmic factor, and in dimension three or more it reduces to the most central observation with probability tending to one. For typical meteorological years the centre set usually contains several candidate years, so the data do not determine the year on which building-energy simulations rest; for three collections of gene trees it contains one tree, whose features can be reported as those of the most central tree.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.CO
High-Arity Rademacher Complexity and Uniform Laws of Large Numbers
Aaron Anderson and Henry Towsner
We characterize several properties of higher-arity function classes, specifically finite $\VC_k$-dimension, slicewise finite $\VC$-dimension, and finite slicewise Littlestone dimension, in terms of generalizations of Rademacher complexity. These characterizations sets up simple new proofs of several results on finite $\VC_k$-dimension and slicewise finite $\VC$-dimension. These include the preservation of finite $\VC_k$-dimension under taking expectations and uniform law of large numbers statements for both properties. All of these results apply naturally to real-valued function classes as well as boolean-valued classes. We also prove a new
adversarial uniform law of large numbers for classes with finite slicewise Littlestone dimension.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Mehdi Makni, Natalia Ponomareva and Rahul Mazumder
Post-training
pruning enables efficient deployment, but conventional methods do not explicitly preserve internal-gradient
alignment between sparse and dense models. We study novel connections between downstream performance of compressed models and the ability to retain gradient information via influence-function preservation. Central to this connection is a
curvature-aware objective that accounts for both layer activations and backpropagated gradients. Despite its promise, this objective remains underexplored in
pruning due to a lack of efficient solvers. We introduce $\texttt{CACTUS}$, a novel ADMM-based optimizer supporting both
pruning and sparse-plus-low-rank decompositions of pre-trained weights. Compared to LLM-Surgeon which targets the same objective, $\texttt{CACTUS}$ can achieve over an order-of-magnitude speedup. At $50\%$
pruning of Qwen3-4B, $\texttt{CACTUS}$ reduces the C4 perplexity gap to the dense model by $45\%$, improves compressed-to-dense per-sample gradient cosine similarity on WikiText-2 from $0.670$ to $0.755$, and runs over $28\times$ faster than LLM-Surgeon on substantially weaker hardware. By preserving influence functions, $\texttt{CACTUS}$ supports effective data selection. On Llama-3.2-3B, fine-tuning a fresh dense model on the $5\%$ of data selected by $\texttt{CACTUS}$ achieves $49.44\%$ accuracy on BIG-Bench Hard, compared with $47.22\%$ using ALPS and $49.54\%$ using dense-model selection. Finally, integrating $\texttt{CACTUS}$ with LoGRA, a recent gradient-projection strategy, on Qwen3-4B with MLP layers pruned to 2:4 sparsity and a rank-64 correction retains a $0.711$ cosine
alignment with dense LoGRA-projected C4 gradients.
Read the full abstract ▾
Statistical Computation
Rapid Mixing of Stereographic MCMC for Heavy-Tailed Sampling
Federica Milinanni, Tyler Farghly and Jun Yang
Sampling from high-dimensional, heavy-tailed distributions is a fundamental challenge in computational statistics, as many standard Markov chain Monte Carlo (MCMC) methods mix poorly in such settings. Recently, Stereographic MCMC [Yang et al., 2024] and the Sub-Cauchy Projection Sampler [Grazzi et al., 2026] have been shown to perform successfully on such tasks. However, establishing their non-asymptotic convergence properties remains an important open problem. In this work, we fill this gap by establishing non-asymptotic upper bounds on the mixing time of the stereographic projection and sub-Cauchy projection samplers. Our results demonstrate that, under certain conditions on the target and initial distributions, the mixing time is
polynomial in dimension for a broad class of distributions, including light- and heavy-tailed cases. Motivated by the theoretical analysis, we further establish a new weighted isoperimetric inequality that extends the classical version for log-concave distributions to the heavy-tailed setting with optimal dimension dependence. The proof techniques provide new insights into the
geometric properties of heavy-tailed distributions that govern rapid mixing in high dimensions.
Read the full abstract ▾
Machine Learning (Statistics)
How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
Aditya Kulkarni, Davide Sartor, Carlos Soto et al. (+1 more)
Recently, Morris et al. introduced TinyLoRA, a strategy for LLM fine-tuning which introduces arbitrarily few parameters, and demonstrated accuracy improvements using less than $10$ parameters rather than the ${\gtrsim 10,000}$ typically demanded by LoRA-based methods. However, that study continued to use gradient-based local search methods of the kind deployed for training multi-billion parameter LLMs in the first place. This article is predicated on the observation that, in the low dimensional setting, global optimization methods become tractable, unlocking Large
language Model
Bayesian Optimization (LlMBO). LlMBO is sample-efficient, gradient-free,
memory-light, parallelizes across devices without synchronization, and, unlike policy-gradient methods, can optimize greedy-decoding accuracy directly. We describe numerical experiments comparing LlMBO against GRPO on fine-tuning tasks for a variety of models on a math
benchmark suite which show that LlMBO can be a suitable alternative in this regime, matching or exceeding GRPO's fine tuning gains, while using a fraction of the compute and
memory.
Read the full abstract ▾
All authors Aditya Kulkarni, Davide Sartor, Carlos Soto, Nathan Wycoff
Machine Learning (Statistics) · Cross-listed from cs.LG
Recalibrating Many Questions with Few Labels: Minimax Rates and Identification Limits
Xiaoyu Li and Chiwun Yang
Many related prediction problems, which we call questions, come with few labels each. What information can be shared across them, and what is actually necessary for good decisions? We answer both for recalibration, motivated by decision models such as Jev (TypeSafe AI, 2026) that return a probability for any yes/no question a developer writes. A predictor reports a probability $P$ with output law $\mu$ on $m$ questions, each with $n$ labels and a nondecreasing map $g_j(p)=\mathbb{P}(Y=1\mid P=p)$; the compound risk is the excess Brier score over the true maps, averaged over questions. Over maps within sup-distance $\tau\le 1$ and total variation $V\le 2$ of one nondecreasing shape (fixed effects), the minimax compound risk is $(mn)^{-2/3}+\min\{\tau^2,1/n\}+\min\{\tau^2,V^2,(V/n)^{2/3}\}$ up to universal constants, for every non-atomic $\mu$. A pooled isotonic fit with a per-question correction sized once, on held-out labels of all questions, attains it without knowing $\tau$, $V$ or $\mu$ ($n\ge 2$, $m\ge 3$); tuning each question on its own labels loses a factor of order $\min\{n,\log(m^2/n)\}$ when the maps differ by constants and $m^2/n$ is large. When the maps are drawn from an unknown prior (random effects), other questions need not identify the Bayes rule. The loss, the identification regret, is at most $2^{-n}$ once the law of a question's data is known, extending a bound of Skibinsky (1968) for coins, and in the worst case at least $\beta_\star^{-n}/(72\sqrt{n})$ at every finite $m$, where $\beta_\star\approx 3.583$. Beyond a few labels per question, what cannot be learned is nearly unnecessary. For constant maps (an uninformative predictor) and constant-map priors, the exponential base is exactly $\beta_\star$, not the 4 suggested by the moment range; for monotone maps it lies in $[2,\beta_\star]$.
Read the full abstract ▾
Statistics Theory
A pedestrian's approach to Bayesian methods for inverse problems
Heiko Gimperlein and Michael Oberguggenberger
In the past two decades, the
Bayesian approach to inverse problems in infinite dimensional spaces has seen significant developments in theory, Markov chain Monte Carlo (MCMC) methods, and in applications. The
Bayesian approach in finite dimension produces posterior distributions of finite dimensional random variables. In infinite dimensions, the desired objects are random fields and their laws as measures on
Hilbert or Banach spaces. There are many textbooks and tutorials available that explain
Bayesian methods and MCMC in finite dimensions. To enter the field of
Bayesian inverse problems in infinite dimensions, a newcomer has to engage with a quite diverse and extensive literature. The purpose of this article is to provide a basic path to the subject that is easily accessible to a reader with general knowledge in probability and mathematical analysis. We start with a swift introduction to the finite dimensional case. The infinite dimensional case is introduced for Gaussian priors on
Hilbert spaces and Gaussian-type likelihood functions. In this situation, mere continuity of the forward map suffices to prove that the posterior measure is well-defined. We also give some new and simple stability results. Two application examples are elaborated, one finite dimensional, one infinite dimensional.
Read the full abstract ▾
Machine Learning (Statistics)
Principled Design of Expected Improvement for Continuous Simulation Optimization
Songhao Wang, Jianglin Xia, Haowei Wang et al. (+1 more)
Expected improvement based on
stochastic kriging (SK) surrogate is widely used in continuous simulation optimization, but unknown heteroscedastic noise makes three basic design choices consequential: the incumbent to define improvement, replication allocation, and final recommendation. Existing methods typically either assume the noise levels are known or choose these components separately through heuristic rules or empirical calibration. We instead derive a high-probability upper bound on the simple regret and systematically choose the three components within general rule classes to tighten this bound. We first establish a finite-sample confidence bound for the SK predictor as a foundation for the three design choices. Within general posterior-
quantile rule classes, tightening the regret certificate selects a lower-
quantile incumbent and an upper-
quantile recommendation among sampled designs. The opposing
quantiles reflect the different roles of exploratory experiments and the final implemented decision. Under mild conditions satisfied by common kernels, minimizing a tractable kernel-specific bound yields a logarithmic replication order. Based on these design choices, the resulting simple regret matches the existing results under known homoscedastic noise up to logarithmic factors. To our knowledge, this is the first simple-regret guarantee of this form for a continuous-domain EI-type simulation optimization method that learns unknown heteroscedastic variances through replications.
Read the full abstract ▾
All authors Songhao Wang, Jianglin Xia, Haowei Wang, Szu Hui Ng
Statistical Methodology · Cross-listed from q-fin.RM
Risk Measures for the Climate Transition Age: Why Symmetric Risk Budgets are unlikely to mitigate two-dimensionally asymmetric Climate Change risks
Lars Dijkstra, Andreas G. F. Hoepner, Fabiola I. Schneider et al. (+1 more)
The most widely used investment risk measures - variance, standard deviation, and tracking error - treat upside deviations as equally undesirable as downside losses. In stable
markets with assumed symmetric return distributions, this simplification is potentially harmless. In the
climate transition age, it is not.
Climate risk is two-dimensionally asymmetric, across assets and over time. Both dimensions are negatively skewed against investors. Hence,
climate transition investment strategies deliberately deviate from
benchmarks that reflect an insufficiently decarbonising global economy. When fiduciaries enforce tracking error budgets that penalise outperformance just as heavily as underperformance, they inadvertently constrain exactly the portfolio return opportunities that
climate-aware investing seeks. We argue that replacing symmetric deviation-based risk measures with their downside-only counterparts - semi-variance, standard risk and trailing error - resolves this conflict without sacrificing any fiduciary discipline. The distinction is not merely semantic: as the gap between
climate transition trajectories and
market benchmarks widens over the coming decades, symmetric risk budgets will become binding constraints on transition strategies well before actual investment risk warrants it. We propose specific changes to mandate design that would allow institutional investors to pursue
climate transition opportunities within genuinely risk-aware (rather than deviation-averse) frameworks.
Read the full abstract ▾
All authors Lars Dijkstra, Andreas G. F. Hoepner, Fabiola I. Schneider, Agnieszka Słomka-Gołębiowska
Statistical Methodology · Cross-listed from cs.MA
Strategic Multi-Agent Resource Allocation via Statistical Contracts
Ben Chugg, Michael I. Jordan and Aaditya Ramdas
We study a class of strategic hypothesis testing problems in which a principal aims to allocate resources among competing
agents who possess data or data-generating mechanisms that may be relevant to the principal's goals. Each
agent can pay a participation cost to opt into a statistical contract, in which case they generate data from a private distribution and receive an allocation based on a predetermined (and publicly known) mechanism. Each
agent is either "null" or "non-null" and the principal's goal is to design the contract to limit the allocation to the former while maximizing transactions with the latter. Using e-values, we define a "continuous-closure," a set of allocations which prevents null
agents from profiting in expectation and provides post-hoc risk control for general loss functions under arbitrary dependence. When the loss is the fraction of resources allocated to null
agents, the region admits
polynomial-time linear optimization. We also give explicit, interpretable closed-form approximations, and we analyze participation from a non-null
agent's perspective, giving conditions under which entry into the contract is profitable. Finally, we extend the framework to support online arrival of the
agents, in which case the principal has to make irrevocable on-the-fly decisions, and provide a method which gives time-uniform error-control in this setting.
Read the full abstract ▾
Statistical Methodology
Multilevel Main Effects Matrix Factor Model
Kaixin Liu, Clifford Lam and Zetai Cen
We propose a multilevel main effects
matrix factor model (MMEFM) to extract meaningful row and column effects from groups of
matrix data. As a generalization of a multilevel
matrix factor model, we provide rigorous definitions of MMEFM and identifications, together with iterative
algorithms for the
estimators of all global and local components in the main effects and common component, including global/local main effects and core ranks, with rates of convergence spelt out. An important feature of the model is that all main effects can be nonstationary and still be consistently estimated at each time point. At the same time, our estimated global common factors are
alignment independent, and hence are truly global, and can be presented and given interpretations more easily. We propose a Bootstrap procedure for detecting the existence and identifying the groups of data that are driven by the global common factors. The power of the test, together with the accuracy of various
estimators, are demonstrated in extensive simulation settings. Estimation performance is also compared to other state-of-the-art methods for
matrix time series by simulations and a set of U.S. equity data. All numerical works are done using the R package MMEFM, which is available at CRAN.
Read the full abstract ▾
Machine Learning (Statistics)
On the Optimality of Random Features under General Conditions: Misspecified Cases and Robust Learning
Caixing Wang
Random features provide a standard approach to scaling kernel methods to large
datasets by approximating the kernel with a finite-dimensional feature map. Existing works of random feature methods primarily focus on well-specified models and light-tailed noise conditions, leaving a gap in understanding their performance under more general conditions, such as misspecified models and heavy-tailed noise. In this paper, we provide a comprehensive and improved analysis of kernel ridge regression with random features (KRR-RF) under general conditions. In the light-tailed noise setting, we improve upon the worst-case uniform bounds of the feature map used in the previous works and lead to the minimax optimal learning rates with $2r+\gamma>\alpha_0$, where $(r,\gamma,\alpha_0)$ denote the regularity of the target function, the effective dimension of the kernel, and the
embedding index of the RKHS, respectively. In the heavy-tailed noise setting, we leverage a recently developed
Hilbert-valued Fuk-Nagaev inequality and extend the same optimal results under finite weak moment conditions. The additional cost to achieve robustness is a mildly stronger explicit sample-size requirement than in the light-tailed case. For both settings, our results recover the conditions random features budgets (up to log factors) required in the previous works. In the most benign case when $\alpha_0=\gamma$, KRR-RF are minimax optimal for all $r \in (0,1]$. To the best of our knowledge, this is the first work to establish optimal learning rates for KRR-RF under weak moment conditions in the misspecified setting. Numerical experiments and a real data analysis validate the derived learning rates and demonstrate their practical relevance in the presence of mis-specification and heavy-tailed noise.
Read the full abstract ▾
Statistical Methodology
Residual Model Criticism for Fitted SDE Implementations
Shengda Zheng
Stochastic differential equation (SDE) models can reproduce trajectories while missing temporal dependence. We compare residual-based scores from a held-out path with scores from paths simulated by a fitted implementation, keeping fitted parameters, simulation rules and the scoring rule fixed. We give conditions for Monte Carlo calibration and bound conditional rejection error by the gap between conditional score CDFs. Residuals use zero-noise centres and covariance approximations that track noise propagation over observation intervals; report residuals also include reporting variation. Residual weights affect sensitivity and the influence of fitting error. For a quadratic energy score in a linear example with singular true covariance, whitening with nonsingular fitted covariance
matrices leaves a nonvanishing score-CDF gap as nonzero coefficient estimates approach the truth. A fixed positive-definite weighting
matrix makes the gap vanish. In a controlled SIR experiment using reference paths simulated at fitted parameters, adding susceptible-state endpoints to training or downweighting a low-variance residual reduces null rejection. A separate comparison using reference paths simulated at the true parameter shows reduced sensitivity to added susceptible-coordinate noise after downweighting. An exploratory weekly-report application incorporates reporting and reconstruction into reference simulations, revealing excess correlation between squared residuals in Czechia and persistently negative residuals in Finland.
Read the full abstract ▾
Statistical Methodology
Sparse-group Convoluted Smoothed Quantile Regression in High Dimension
Jing Lv and Anxi Wang
In many applications,
covariates are naturally organized into groups, yet conventional group-selection methods often retain all variables within selected groups, failing to achieve within-group sparsity. To address this limitation, we propose an adaptive two-stage sparse-group selection procedure for high-dimensional
quantile regression. The proposed method integrates convolution-smoothed
quantile loss with $\ell_{2,0}$ and $\ell_0$ penalties to identify important groups and relevant variables within them, achieving sparsity at both levels. We establish theoretical guarantees for sure screening and exact support recovery at both stages, as well as consistent model-size selection using information criteria, with explicit bounds on failure probabilities. The proposed procedure is computationally efficient by restricting optimization to selected groups and active variables. Simulation studies and a breast cancer gene-expression analysis demonstrate its competitive performance in variable selection, estimation accuracy, and computational efficiency, while identifying biologically relevant genes and delivering accurate
quantile predictions.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Operator Surrogates
Christoph Schwab, Josephine Westermann and Jakob Zech
This chapter provides a mathematical introduction to operator surrogates, which are finite-parametric maps approximating complex (typically nonlinear) operators between infinite-dimensional (function) spaces. A typical approach to constructing operator surrogates is the encoder-approximator-decoder paradigm, whereby infinite-dimensional inputs are mapped to a finite-dimensional latent space, processed, and then mapped back to the infinite-dimensional output space. We discuss various methods for creating efficient representation systems, including stable linear frameworks like Banach frames and data-driven techniques such as Principal Component Analysis (PCA), Active Subspaces, and nonlinear autoencoders. Subsequently, we survey key surrogate architectures, contrasting spectral methods, based on
polynomial chaos expansions and related technologies, with more recent neural network approaches. We further discuss architectures that do not rely on representation systems, such as
Fourier Neural Operators and
transformers. We elaborate on their relationship and how they relate to other neural network operator surrogates, particularly those obtained by so-called "
algorithm-unrolling". Additionally, we provide an introduction to the theoretical analysis of approximation and generalization errors. We explain how underlying low-dimensional structures, provided for example by holomorphy, are essential to overcome the so-called "curse of dimensionality": this refers to the fact that the complexity of numerical
algorithms tends to increase exponentially in the problem dimensionality. We present convergence rate bounds for holomorphic operators that depend on the number of trainable parameters and the regularity of the input and output data. Throughout, we prioritize intuition over detailed technical proofs and provide precise pointers to the literature for detailed arguments.
Read the full abstract ▾
Statistics Theory
Interpolation and coherence in multiple testing
Ninh Tran, Dennis Leung and Jesse Hemerik
In the literature on multiple testing methods that control the familywise error rate, a fundamental concept is "coherence". A procedure is coherent if it is "logically consistent" (Hochberg and Tamhane, 1987) in the sense that if a hypothesis is rejected, then all hypotheses that imply it are also rejected. If a method is not coherent, then it can be made coherent without increasing the familywise error rate. Recently, the concept of coherence was generalized to a much broader class of multiple testing procedures, namely all methods that provide simultaneous confidence bounds for false discovery proportions. Such procedures were defined to be coherent if they cannot be further improved through a process called "interpolation". However, we show that such a coherent procedure can sometimes be further improved by exploiting remaining logical inconsistencies. This leads to what we call a "perfectly coherent" method. Thus, interpolation does not necessarily lead to perfect coherence. However, we provide conditions under which it does. Fortunately, many methods based on interpolation turn out to be perfectly coherent.
Read the full abstract ▾
Machine Learning (Statistics)
Hierarchical Time Series Forecasting with Operator-Valued Kernels
Junjie Yang, Adeline Fermanian, Claire Boyer et al. (+1 more)
Hierarchical time series forecasting requires generating accurate predictions that are coherent across multiple levels of aggregation. Existing methods either rely on two-stage pipelines with post-hoc reconciliation or on end-to-end models that often lack statistical guarantees. We introduce a unified framework that formulates hierarchical forecasting as a multivariate regression problem in a vector-valued reproducing kernel
Hilbert space. By designing structured operator-valued kernels, our approach enforces coherence directly within the hypothesis space, ensuring that all predictions satisfy hierarchical constraints by construction. We further propose tailored kernel constructions for hierarchical data and derive excess risk bounds that characterize generalization performance. Empirical results on synthetic and real-world
datasets demonstrate the relevance of the proposed method.
Read the full abstract ▾
All authors Junjie Yang, Adeline Fermanian, Claire Boyer, Gérard Biau
Machine Learning (Statistics)
Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence
Luis Wiederhold, Nil Ayday and Debarghya Ghoshdastidar
Randomly initialised neural networks of width $m$ converge to Gaussian processes as $m \to \infty$, with two types of quantitative bounds characterising this convergence. Finite-sample bounds compare the network with its Gaussian limit at a fixed finite sample set, on an arbitrary domain, in the quadratic Wasserstein distance $\mathcal W_2$, achieving a convergence rate of $m^{-1/2}$ with explicit constants. However, these constants grow with the number of samples, so the bound becomes uninformative as the sample set is refined. Functional bounds remove that dependence by comparing the two as random functions, at the price of spherical input, metrics weaker than $\mathcal W_2$, or slower rates. In this paper, we therefore lift a finite-sample bound to a functional one. The lifting refines the sample set until it is dense in the domain, so the bound must first be made independent of the number of samples. We achieve this by measuring the error per sample, which makes the constant uniform. For a
polynomial activation and any compact input set, this gives a rate of $m^{-1/2}$ in the supremum norm, with an explicit constant.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Learning a Group Element from Its Action: Galois Correspondences, Optimal ERM, and an Abel-Ruffini Threshold for the Natarajan Dimension
Xiaoyu Li
We study learning an unknown element $g$ of a group acting with finite orbits on a countable set, from examples $(x,gx)$. After a sample on $A$, the consistent hypotheses are the restrictions of a coset of the pointwise stabilizer of $A$, and the sample forces exactly the labels of points this stabilizer fixes; for a Galois group, they form the field generated by the data. Through this correspondence, multiclass dimensions become group invariants: the DS and graph dimensions equal the height, the Littlestone dimension the longest irredundant sequence length, and the teaching dimension the minimal base size. Among deterministic learners,
symmetry makes empirical risk minimization optimal: on nondegenerate classes every consistent learner, whatever its tie-breaking, attains the optimal realizable sample complexity up to constant factors, and every consistent online learner the optimal mistake bound. Agnostically, every ERM's sample complexity exceeds the known lower bound by at most the DS-to-Natarajan ratio, up to constants. For finite Galois extensions of countable fields, the Natarajan dimension equals a group invariant, the coset-cube number, which is at least the number of complemented factors of any chief series of any subgroup. Hence the dimensions coincide for every solvable Galois group, while for the groups $A_5$ and $S_5$ of the quintic they are $2<3$ and $3<4$, and $A_5$ is the smallest group with a gap. For the Galois group $S_n$ of the general equation of degree $n$, they coincide exactly when $n\le 4$, i.e., in characteristic zero, exactly when the equation is solvable by radicals; the implication is one-way, as the simple group $\mathrm{PSL}(2,8)$ has no gap. For linear, orthogonal and
unitary maps of $\mathbb R^d$ or $\mathbb C^d$ both dimensions equal $d$, and a two-fibered pseudo-cube is a disjoint union of Natarajan cubes exactly when its monodromy group is abelian.
Read the full abstract ▾
Statistical Methodology
Rational Quadratic Kriging
Difan Song and V. Roshan Joseph
This article proposes rational quadratic kriging, a kriging
estimator that addresses the limitations of existing approaches: the mean reversion issue of ordinary kriging and the diverging variance issue of rational kriging. Through the rational quadratic kernel, we establish a connection between rational quadratic kriging and kernel regression. The connection allows us to prove desirable theoretical properties of the new
estimator and further propose a novel estimation procedure that utilizes factor weights from kernel regression. We also derive expressions to enable efficient leave-one-out estimation and conformal prediction. Extensive simulations across both deterministic functions and noisy
datasets show significant improvements in both prediction and uncertainty quantification. A fast implementation is available in the open-source R package RQK.
Read the full abstract ▾
Statistics Theory
A new approach for estimating the number of spikes in high-dimensional Gaussian models
Hugo Jeannin, Pascal Vallet, Florent Bouchard et al. (+2 more)
In this paper, we study the estimation of the signal subspace dimension $k$ in a $p$-variate signal-plus-noise complex Gaussian model, when the noise covariance
matrix is assumed to be diagonal or Toeplitz. We consider the high-dimensional asymptotic regime in which both the data dimension $p$ and the available sample size $n$ diverge to infinity at the same rate, while we also allow $k$ to diverge at a slower speed. In this context, the classical estimates of the literature, consistent in the high-dimensional regime, often rely on the a priori knowledge of an upper bound $k_*$ of $k$, which can be misspecified in practice. Our proposed approach relies on building estimates of the tracial moments of the noise covariance
matrix using the $p-k_*$ smallest
eigenvalues of the sample covariance
matrix (SCM), which are in turn used to build a threshold above which the $k$ largest
eigenvalues are isolated. The resulting estimate of $k$ is proved to be consistent in the high-dimensional regime, even for the misspecified situations where $k_* < k$. Numerical experiments in different challenging scenarios illustrate the improvement of the proposed approach over the aforementioned methods.
Read the full abstract ▾
All authors Hugo Jeannin, Pascal Vallet, Florent Bouchard, Guillaume Ginolhac, Audrey Giremus
Machine Learning (Statistics)
Reliable Scale-Free Bandits via Empirical Likelihood
Pratham Gupta, Alhad Sethi and Shubhada Agrawal
We study
stochastic bandits without knowledge of reward scales or tail parameters. Almost-sure regret guarantees describe eventual learning but do not quantify expected performance or the risk of large regret in a finite run. We introduce $\texttt{EL-F}$, an anytime empirical-likelihood (EL) extension of the $g$-Forcing policy of Cowan and Katehakis (2020), and characterize its performance at three levels. First, for integrable rewards and non-decreasing, diverging exploration schedules $g$ with vanishing increments, $\texttt{EL-F}$ preserves the $g$-Forcing asymptotics for pseudo-regret $R(T)$: $R(T)/g(T)\to\sum_{i:\Delta_i>0}\Delta_i$ almost surely as $T\to\infty$, where $\Delta_i$ is the suboptimality gap of arm $i$. Second, on every sub-exponential instance, for non-decreasing $g(t)=\omega(\log t)$ satisfying a mild feasibility condition, expected pseudo-regret is at most $g(T)\sum_{i:\Delta_i>0}\Delta_i+C_{\underline{P}}$, where $C_{\underline{P}}$ is an instance-dependent constant independent of $T$. Moreover, we also establish finite-time tail bounds with super-
polynomial decay in regret beyond the forced-exploration budget. For slowly varying $g$, we determine the exact exponent of linear regret deviations on two-armed Gaussian instances with common variance and establish a matching lower bound among policies satisfying the same expected-regret budget on every such Gaussian instance. These quantitative guarantees also cover the greedy variant obtained at zero EL radius. A fixed positive radius introduces local optimism while preserving the leading asymptotic regret. We establish coupled pathwise dominance over the greedy variant in two-armed bandits with a deterministic suboptimal arm. Experiments demonstrate reductions in prolonged suboptimal allocation compared with $g$-Forcing on challenging instances.
Read the full abstract ▾
Statistical Methodology
ndscape: a structural map of the nested dichotomy space
Maxwell Dix-Matthews, Melinda Hodkiewicz and Adriano Polpo
Nested dichotomies (NDs) reduce multiclass classification to a tree of binary classifiers. Tree structure can change accuracy, bias, and variance, but the full space of structures has not been mapped in aggregate. We introduce ndscape, a two-dimensional mapping of ND structures based on depth-weighted structural
tokens. We use this
embedding to conduct what is, to our knowledge, the first bias-variance analysis across the ND structure space. In simulations with 6 to 10 classes, NDs with similar tree structures exhibit similar squared bias, variance, and squared probability loss, regardless of whether the binary classifier is logistic regression or a decision tree. Variance, however, follows mean root-to-leaf depth when logistic regression is used at each node, so flatter, more balanced NDs are more stable, whereas with decision-tree node models, variance is not associated with depth and differs little across ND structures. On six
datasets with 6 to 19 classes, held-out accuracy, log loss, and prediction variance remain spatially organised on the same
topology-only map. None of the six standard construction heuristics tested selects a tree from the best-performing cluster on any
dataset, although some select high-performing individual trees. The ndscape
embedding turns ND selection from one opaque construction step into a readable comparison of class groupings, exposing the upper-level splits that distinguish high-performing and stable regions.
Read the full abstract ▾
Statistical Computation
How to Train Your Intercept: A Comparison of Intercept Update Strategies in Coordinate Descent
Johan Larsson and Frederik Fabricius-Bjerre
Coordinate descent solvers for regularized generalized linear models (GLMs) differ in how they update the intercept. This seemingly routine choice can substantially affect convergence when the response is imbalanced. For direct coordinate descent on the original GLM loss, we compare a conservative step based on worst-case
curvature, a safeguarded Newton step based on current
curvature, and repeated Newton steps that solve the intercept subproblem. The conservative update can leave much of the interaction between the intercept and the coefficients unresolved, producing a slowdown when that interaction controls convergence. A single Newton step avoids this attenuation and, in our experiments, follows nearly the same outer trajectory as exact minimization while requiring fewer inner steps. Simulated and real-data experiments, including comparisons across six production solvers and a controlled intervention within skglm, support this explanation. We also show how the same mechanism appears in quadratic-approximation methods and extends to multinomial models with rare classes. We recommend a safeguarded Newton update for direct coordinate descent and full intercept optimization within frozen local quadratic surrogates.
Read the full abstract ▾