Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Mathematics Statistics Theory

math.ST Watch Statistics Theory (sign in to use)

Thursday 8 October

Statistics Theory

Adaptive minimax multivariate \(L^1\)-deconvolution: harmonic-mean rates under noise filtering, and a \(1\)-Wasserstein equivalence

We study the multivariate convolution model with additive independent noise, multidimensional signal and noise, aiming to recover the signal's cumulative distribution function from contaminated observations. The noise is ordinary smooth with anisotropic regularity, and the signal belongs to an anisotropic Nikol'skii density class. We extend to the multivariate setting an approximate minimum \(L^1\)-distance estimator based on integrated kernel density estimation, and establish matching upper and lower bounds for the \(L^1\)-risk over anisotropic Nikol'skii classes. To our knowledge, this is the first minimax rate, not a sharpening of existing bounds. Although the lower-bound scheme is classical, the test-function construction is novel: it distinguishes no active component from at least one active component. The rate is governed by a censoring mechanism through the positive part, a noise filter in a harmonic-mean representation of the exponent. We further propose a fully data-driven, rate-adaptive procedure selecting an optimal bandwidth vector over the full Nikol'skii scale. On Nikol'skii product density classes, the \(L^1\)-distance between cumulative distribution functions and the \(1\)-Wasserstein distance share the same minimax rate, though not equivalent in general, via coordinate-wise decoupling of the \(1\)-Wasserstein cost on product measures. These findings reveal a link between the two distances, whose full understanding remains an open question.
Read the full abstract

Statistics Theory

Single-graph inference for fractal Gaussian networks

We study inference on the strength of Gaussian multiplicative chaos from one geometric graph with unobserved vertex positions. In the planar model, the edge-count statistic has a piecewise deterministic limit with a transition at $\gamma=1$, whereas degree quantiles consistently estimate $\nu=\gamma^2/2$ throughout the subcritical range. Fractional moments give uniform finite-sample risk bounds. At finite resolution, common graph envelopes calibrate tests over continuous parameter boxes with unknown Poisson intensity. They give simultaneous coverage under adaptive refinement for a fixed statistic, and conditional coverage for an independently piloted size–degree residual on a fixed partition. For the exact periodic FFT–pixel approximation, we prove explicit total-variation rates for the annealed spatial Cox law and the observed graph law, uniformly on every compact subcritical range $0\le\gamma\le\Gamma<2$. The rates allow intensity to grow with resolution and imply uniform asymptotic coverage after recalibration at each resolution. Numerical studies examine parameter refinement, residual calibration, and multiresolution stability.
Read the full abstract

Statistics Theory

Sequential Conditional Independence Testing with Machine Learning Models

Conditional independence testing is a ubiquitous problem in scientific discovery. The widely employed model-X assumption shifts the modelling burden from the dependence of the output on the inputs to the dependencies within the inputs. Log-optimal e-variables have been studied in this setting, but it remains unclear how to incorporate machine learning models into their design. Other approaches test exchangeability directly, yielding an e-variable with lower power in theory but, surprisingly, higher power in practice. We explain this phenomenon by decomposing the error into null enlargement, approximation, and estimation error. The decomposition shows that GRO e-variable estimates can be beaten because of their worse approximation and estimation errors, and we explore intermediate null hypotheses between model-X conditional independence and exchangeability to reduce these errors. Moreover, the model-X assumption often only holds up to an estimation error, invalidating exact type-I error guarantees. We provide estimation error bounds that accommodate triple robustness results, achieving fast convergence rates.
Read the full abstract

All authors Angel Reyero-Lobo, Michele Meziu, Sebastian Uriel Arias, Peter Grünwald

Statistics Theory · Cross-listed from stat.ME

Anchored multiple testing: a transparent use of e-closure to improve FDR procedures

The recent e-closure method can recover every procedure that controls FDR (and other expectation losses). But the recovered e-collection is “self-referential” and gives no insight on how to improve the procedure (if improvable), and some recent improvements have been somewhat opaque. We introduce an elementary new technique called anchoring that exploits looseness in existing FDR proofs to enlarge a baseline multiple-testing procedure's self-referential local e-value. The resulting e-closure thus transparently retains every baseline discovery (and usually adding more) and controlling the false discovery rate under the same conditions as the baseline. To show that this principle is broadly applicable, we use it to improve a large suite of multiple testing procedures: (i) Anchored-BH dominates the Benjamini-Hochberg (BH) procedure under PRDS while being incomparable to Goeman's recent closed-BH, (ii) Anchored-BY dominates the Benjamini-Yekutieli (BY) procedure under arbitrary dependence while being incomparable to closed-BY, (iii) For two-sided Gaussian p-values (under appropriate covariance conditions), Anchored-2BH dominates running BH twice at half the level on two one-sided p-values, (iv) Anchored-dBH dominates dependence-adjusted BH, (v) Anchored e-BH dominates e-BH and is incomparable to closed e-BH, (vi) Anchored SeqStep+ improves the original (including selective and adpative variants) while preserving ordered rejection structures. All of these are accomplished in sorting or quadratic time. The appendix shows how to dominate Shifted-BH (for two-sided arbitrarily correlated Gaussians) and NDBH (for negatively dependent p-values), IndBH (for graph-dependent p-values), some online FDR procedures and the support line procedure for boundary FDR control.
Read the full abstract

Statistics Theory

Decision-Sufficient Posterior Approximation

We investigate the consequences of requiring a posterior approximation to preserve a specified downstream decision problem. A target posterior $P$ and loss determine a regret geometry on actions, a baseline approximation $Q_0$ determines the forward-Kullback-Leibler information required to induce action changes, and a restricted approximation family $\mathcal{Q}$ determines which such changes are available. Contracting KL divergence over Bayes-action fibers gives exact distances to decision adequacy and decision failure together with the least-informative posterior deformations that reach either side of the decision boundary. In regular finite-dimensional problems, the target and baseline constructions have quadratic local limits: a target regret Hessian $G$ and a baseline information metric $J_I$ . Their generalized eigenproblem $Gv = \gamma J_Iv$ orders local decision directions by regret consequence per unit information cost and induces a tolerance-dependent effective dimension. For restricted approximation families, the tangent image separates decision coverage from information efficiency: a family may miss consequential decision directions, or it may realize reachable directions only at excess Fisher cost. The resulting framework provides decision-relative criteria for comparing and designing posterior approximation families.
Read the full abstract

Wednesday 7 October

Statistics Theory

When Does Inexact Matching Ensure Balance and Inference without Adjustment?

One-to-one matching without replacement is a classical approach to constructing comparable treated and control samples in the design of observational studies. It pairs each treated unit with a distinct control while minimizing a covariate distance objective. With continuous covariates, the matched pairs generally remain inexact, which contributes to bias in downstream analysis. Its key theoretical properties, such as the resulting imbalance between matched pairs and when it is negligible to support valid inference, remain unclear. In this paper, we analyze one-to-one matching based on $d$-dimensional, continuous covariates with a quadratic covariate-distance objective. First, we find that when $d\leq 3$, under standard conditions on the propensity score ensuring abundant control samples near each treated sample, the imbalance (difference between within-group averages) is root-$n$ negligible uniformly over the family of smooth functions with a common first- and second-order derivative bound. However, such balance is subject to a dimension restriction, as we construct examples in which the imbalance is root-$n$ non-negligible when $d=4$ and dominates root-$n$ rate when $d>4$. Second, we show that when $d\leq 3$, the matched design allows valid Wald-type and bootstrap inference for the average treatment effect on the treated, distributional treatment effects, and quantile treatment effects. Thus, the same outcome-blind matched design supports various downstream inferences without having to tailor the design to the targets. Finally, paired randomization inference based on the matched design is asymptotically valid in the super-population sense for $d\leq 3$ but can fail when $d=4$. We corroborate the theoretical results with numerical experiments.
Read the full abstract

Statistics Theory

On the Impossibility of Estimating the Average Mixing Time

We establish nearly-matching converse bounds on the sample complexity of estimating the average mixing time of a Markov chain from a single stationary trajectory. Unlike the worst-case mixing time, which maximises the total-variation distance to stationarity over all initial states, the average mixing time averages this distance under the stationary distribution and can therefore be substantially smaller. Our converse bounds establish when estimation is impossible, and match the key dependencies in existing achievability bounds, up to logarithmic factors. Together, these bounds provide nearly-tight PAC guarantees for estimating the average mixing time.
Read the full abstract

Statistics Theory

Statistical Guarantees for Covariance Estimation with Adversarial Corruption under Constraints

We obtain a near-minimax rate on covariance estimation of adversarially corrupted Gaussian data in Frobenius norm when the covariance estimation is assumed to lie in a star-shaped set $K \subseteq \mathbb{R}^{n \times n}$. Assuming a known upper bound of $\varepsilon < 1/32$ on the fraction of the $N$ observations of data $X$ are arbitrarily corrupted by an omniscient adversary, we obtain the near-minimax rate (up to constants) of \begin{align*} \left(\varepsilon^2 \vee \delta^{\star2} \right)\wedge d^2 \lesssim \inf_{\hat\Sigma} \sup_{\Sigma \in K} \sup_{\mathrm{C}} \mathbb{E} \| \hat\Sigma(\mathrm{C}(X)) - \Sigma \|_F^2 \lesssim \left(\varepsilon^2\log^2(1/\varepsilon) \vee \delta^{\star2} \right)\wedge d^2 \end{align*} where $d$ is the diameter of the set $K$, $N \gtrsim \sup_{\delta > 0} \log \mathrm{M}^{\operatorname{loc}}(\delta, 2c)$, and \begin{align*} \delta^*=\sup\left\{ \delta\geq 0: \frac{N}{\Upsilon^2} \delta^2\leq \log \mathrm{M}_K^{\operatorname{loc}}(\delta,c) \right\} \end{align*} for an arbitrary large constant $c$ and the local metric entropy of $K$, $\mathrm{M}^{\operatorname{loc}}_K(\delta,c)$. We believe this is the first such general result on structured robust covariance estimation in the Frobenius norm.
Read the full abstract

Statistics Theory

How Many Directions Must a Truncated Diffusion Sampler Retain? Matching Bounds Under Power-Law Spectra

Diffusion samplers can reduce computation by generating selected spectral coordinates and filling the remaining directions with noise. How many directions must they retain? We study this question for data with power-law covariance spectra. For Gaussian data compared to a smoothed target, we prove matching bounds on the required number of retained directions, provided that the ambient dimension is sufficiently large. The truncation error depends on the combined Wiener gains of the omitted directions, regardless of the accuracy of the sampler on the retained coordinates. Keeping only directions whose signal exceeds the output noise level can therefore leave a non-vanishing error: many individually weak directions remain significant in aggregate. Combining this characterization with a diffusion convergence bound yields sufficient sampling-step complexity under exact scores. The upper bounds also extend to estimated principal components and, componentwise, to Gaussian mixtures. The practical prescription is to select the retained subspace using an aggregate spectral-tail error budget, then to choose the diffusion noise level accordingly.
Read the full abstract

Statistics Theory · Cross-listed from cs.IT

Settling the Sample Complexity of Rényi Entropy Estimation

Rényi entropy estimation has been comprehensively investigated by Acharya, Orlitsky, Suresh and Tyagi (SODA 2015; IEEE Trans. Inf. Theory 2017) and consequent works, whereas only the sample complexity of Rényi entropy estimation of integer order has been settled. In this paper, we settle the sample complexity of Rényi entropy estimation of noninteger order, thereby completing the complexity picture of Rényi entropy estimation. Specifically, we show that for any noninteger $\alpha > 0$, it is sufficient and necessary to use \[ \Theta\!\left(\frac{d^{\max\{1/\alpha,1\}}}{\varepsilon^{1/\alpha}\log(d)} + \frac{d^{|1-1/\alpha|}}{\varepsilon^2}\right) \] samples to estimate the Rényi entropy of order $\alpha$ of an unknown discrete distribution over an alphabet of size $d$ to within additive error $\varepsilon$. For the upper bound, we reduce the bias using a refined polynomial approximation estimator for large probabilities. For the lower bound, we employ a different hard instance equipped with a new moment matching construction. The constructive moment matching has constant bounded high-order moments, while attaining a fixed ratio between the $\alpha$-th moments, which is of independent interest.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Exact bounds on the distribution function of isotropic log-concave distributions

For each real $b$, exact upper and lower bounds on the probability $\mathsf P(X\ge b)$ over all random variables $X$ with log-concave p.d.f.'s such that $\mathsf E X=0$ and $\mathsf E X^2=1$ are obtained, as well as the best constant factor $C$ in the inequality $\mathsf P(X\ge b)\le C e^{-b}$ for all real $b\ge0$. Exact upper and lower bounds on the corresponding exponential moments (Proposition 9.4) and explicit exponentially decreasing upper bounds on the mentioned p.d.f.'s (Propositions 9.2 and 9.3) are given as well. Some general results concerning log-concave p.d.f.'s are also obtained.
Read the full abstract

2610.10289 · 7 Oct · Comments to Version 2: Added a direct link to Supplement B, described on p. 49; mathematical content unchanged. Comments to Version 3: Added exact upper and lower bounds on the exponential moments (Proposition 9.4); mentioned p. 49 has become p. 50

Statistics Theory · Cross-listed from math.PR

Wasserstein de-initialization for Markov chains

This article generalizes the de-initialization framework, proposed by (Roberts and Rosenthal, 2001) for total variation distance, to Wasserstein distances. In essence, de-initialization captures the phenomenon that in the presence of certain structural features, the convergence behaviour of a Markov chain can be reduced to that of a "simpler" stochastic process. Our results allow to measure this in terms of a general Wasserstein distance. We apply them to analyse the Wasserstein convergence behaviour of Gibbs sampling, linchpin variable sampling and slice sampling.
Read the full abstract

Statistics Theory · Cross-listed from stat.ML

RoBART: Bayesian Additive Regression Trees with Tree-Specific Rotations

Bayesian additive regression trees (BART) can require many splits to approximate boundaries misaligned with the predictor axes. RoBART assigns each tree a rotation shared by all internal nodes, retaining axis-aligned splits in rotated coordinates and constant leaves. We jointly propose a Givens rotation sequence and cutpoints on the resulting grid by Metropolis-Hastings and establish reversibility with respect to the conditional posterior with leaf means integrated out. For additive functions with component-specific rotations and anisotropic Hölder smoothness, we prove posterior contraction in empirical $L_2$ distance and for the noise standard deviation. Under the stated prior, design, and grid conditions, with fixed numbers of predictors, trees, and components and no more components than trees, the rate is a sum of componentwise rates determined by smoothness and the number of rotated coordinates used. We also establish a posterior contraction lower bound showing that there exist functions for which RoBART adapts to the intrinsic dimension but axis-aligned BART does not.
Read the full abstract

Statistics Theory · Cross-listed from stat.ME

Policy Learning with Weak Signals

Policy learning in digital experimentation faces three challenges: weak signal-to-noise ratios, rich covariate spaces, and massive data volumes. We formalize this regime by modeling treatment-effect estimates from increasingly fine covariate partitions as Gaussian observations with bounded signal-to-noise ratios. We establish that, in general, the optimal treatment policy is not learnable in this setting. Even learning the optimal policy value suffers from impractically slow rates. However, when treatment effects vary smoothly, we derive minimax-adaptive policies based on linear smoothers that achieve vanishing welfare regret. We demonstrate the practical value of our framework by applying it to large-scale real-world experiments at Netflix, showing that personalized linear-smoothing policies can dominate unpersonalized policies even in this challenging empirical setting.
Read the full abstract

All authors Benedikt Koch, Winston Chou, Aurélien Bibaut, Nathan Kallus

Statistics Theory

A polyhedral framework for hierarchical Archimedean copulas: estimation, hierarchy recovery and singular asymptotics

Hierarchical Archimedean copulas (HACs) admit redundant tree representations: when adjacent internal nodes have equal parameters, contracting the connecting edge leaves the copula unchanged. This creates a structural overparametrization. For homogeneous one-parameter families, we remove this redundancy and identify the resulting reduced model class with the ultrametric fan. We treat the Clayton, Frank, Ali-Mikhail-Haq, Gumbel and Joe families. Using the negative logit of normalized Kendall's tau as a common node-height coordinate, reduced hierarchies form polyhedral strata, binary hierarchies index maximal cones, and multifurcating hierarchies lie on common contraction faces. The induced cophenetic metric generates the same topology as uniform convergence of copulas. We also show that informative pairwise dependence summaries act componentwise on cophenetic coordinates and hence provide global coordinates for the reduced model. This representation yields a global estimator obtained by inverting estimated pairwise summaries and projecting onto the fan, without preselecting a hierarchy. We prove consistency and convergence-rate transfer. At binary truths, the unthresholded estimator recovers the hierarchy with probability tending to one; at multifurcating truths, it asymptotically selects a binary refinement, while vanishing thresholding consistently recovers the reduced hierarchy. Under a joint central limit theorem for the pairwise summaries, the projected estimator has a root-$n$ limit given by Euclidean projection of a Gaussian vector onto the tangent fan. The limit is Gaussian at binary hierarchies and generally non-Gaussian at multifurcating ones. For empirical Kendall's tau, we verify the required regularity conditions and obtain the corresponding singular limit.
Read the full abstract

Statistics Theory

Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel

Gaussian processes (GPs) are widely used across machine learning, spatial statistics, time-series analysis, optimization, Bayesian statistics, and scientific applications. A central component of a GP model is its kernel, which is typically specified through a parametric family. Among the most widely used choices is the radial basis function (RBF), also known as the squared exponential or Gaussian kernel, owing to its simple form, smoothness, and flexibility. In practice, the kernel parameters are routinely estimated by the maximum likelihood estimators (MLEs), as implemented by standard GP software. Despite this widespread use, the asymptotic behavior of the MLEs remains poorly understood under fixed-domain asymptotics, even for the RBF kernel. The main difficulty arises from the increasingly strong dependence among densely sampled observations and the nonlinear dependence of the covariance matrix on the kernel parameters. In this paper, we address this gap by providing, to the best of our knowledge, the first complete asymptotic characterization of the joint MLE of the spatial variance, lengthscale, and nugget variance under fixed-domain asymptotics. We establish consistency, derive convergence rates for all three parameters, prove joint asymptotic normality, and show that these rates are minimax optimal.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Improved Berry-Esseen bounds for multivariate nonlinear statistics in convex distance

In this paper, we establish two nonasymptotic Berry–Esseen bounds over convex sets for the Gaussian approximation of multivariate nonlinear statistics. The statistics of interest can be written as a sum of independent centered random vectors plus a remainder that may depend on all observations. The first bound retains the classical factor $d^{1/4}$ in the contribution of the independent sum, where $d$ is the dimension, while controlling the remainder through its size and its sensitivity to replacing a single observation. The second bound expresses the contribution of the independent sum in terms of fourth moments and can allow the dimension to grow faster with the sample size. For sums of independent random vectors, it removes the logarithmic factor from an existing fourth moment bound without imposing additional moment assumptions. As applications, we apply these results to Polyak–Ruppert averaging for nonsmooth stochastic approximation, temporal difference learning with linear function approximation, and multivariate $U$-statistics. The resulting bounds provide explicit Gaussian approximation errors and sufficient conditions under which these errors converge to zero as the dimension grows with the sample size.
Read the full abstract

Statistics Theory · Cross-listed from stat.AP

Multivariate LSTM Modeling for Wastewater Quality Prediction

The proposed approach relies on high-frequency multivariate time series collected at the Seine aval (SIAAP 1 ) treatment plant, including pH, temperature, conductivity, and total suspended solids, as well as exogenous variables related to precipitation and measurements from an upstream plant. The objective is to forecast wastewater quality over a 24-hour horizon. A multivariate Long Short-Term Memory (LSTM) recurrent neural network is implemented to capture complex temporal dependencies and nonlinear patterns in the data. The model is trained on one year of data and validated on four months of data (one per season), and is compared with a persistence model and SARIMA models. The evaluation shows an overall superiority of the LSTM model, particularly for variables exhibiting high levels of noise and nonlinearity.
Read the full abstract

Statistics Theory · Cross-listed from stat.ME

Anchored Reversible Jump Sequential Monte Carlo

Reversible Jump Markov Chain Monte Carlo (RJMCMC) is a principled framework for Bayesian model comparison, but its practical use is often limited due to the difficulty of designing between-model proposals that reach regions of high posterior probability. In addition, its inherently sequential nature limits efficient use of modern GPU architectures. We address both challenges by introducing Anchored Reversible Jump Sequential Monte Carlo. Our approach combines RJMCMC with Sequential Monte Carlo (SMC) to explore the model space in parallel. Importantly, the particle population enables effective between-model proposals without problem-specific knowledge: particles are used to construct kernel density approximations of the target distributions within each model, which are then used to generate trans-dimensional proposals. Numerical experiments show that the proposed method is computationally efficient and competitive with state-of-the-art RJMCMC approaches, demonstrating its potential for scalable Bayesian model comparison.
Read the full abstract

Statistics Theory · Cross-listed from cs.AI

Outperformance Inverse Optimization: Learning Objective Functions that Outperform Agent Decisions

Inverse optimization estimates the weights of an objective function that explain observed decisions as optimal solutions, and is used in a variety of fields. For mixed-integer linear programs (MILPs), existing methods aim to reproduce the observations as optimal solutions, and thus learn compromise weights when the observations are suboptimal. We propose outperformance inverse optimization, which instead seeks weights that induce, at each state, an optimal solution outperforming the observed action in every component. We give a loss function that can be evaluated with forward-problem oracles alone and is thus applicable to MILPs, together with gradient-based and DC optimization algorithms for minimizing it. For weights inducing a unique outperforming optimal solution at all observations, we prove that the probability of failing to induce such a solution at a new state (the generalization error) is bounded by a quantity inversely proportional to the number of observations, and that this bound is tight in the number of observations up to logarithmic factors. In experiments on synthetic and real data, the proposed methods improve the prediction of solutions outperforming the actions over existing methods.
Read the full abstract

Statistics Theory

Drift Estimation for a Multi-Dimensional Lévy-Driven Stochastic Differential Equation Using Deep Neural Networks

We develop a non-asymptotic theory for nonparametric drift estimation in discretely observed multi-dimensional Lévy-driven stochastic differential equations using sparse ReLU neural networks. Under exponential \(\beta\)-mixing and an exponential-tail condition on the Lévy measure, we establish an oracle inequality for the least-squares estimator. The jump component changes both the martingale structure and the concentration regime relative to diffusion models: continuous martingales are replaced by discontinuous martingales, while the relevant empirical-process fluctuations are sub-exponential rather than sub-Gaussian. We handle these difficulties without truncating the observed increments, using an exponential-supermartingale argument for compensated Poisson integrals and \(\psi_1\)-chaining. Despite the weaker concentration, the resulting statistical bound is, up to constants, of the same order as the corresponding diffusion bound. For drift functions with hierarchical compositional structure, this yields intrinsic-dimensional convergence rates. The framework allows infinite jump activity and, in some cases, infinite variation. Finally, we establish a minimax lower bound on a fixed non-trivial compound-Poisson submodel. Together with the upper bound, this shows that the estimator is minimax optimal up to logarithmic factors.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

Fluctuations of Nonlinear Observables in Mean Field Neural Network Training

Mean field limits describe the training dynamics of wide neural networks through the evolution of the empirical distribution of their parameters. Although functional central limit theorems characterize the asymptotic fluctuations of this distribution, quantities of practical interest are typically nonlinear observables of the parameter distribution rather than the distribution itself. In this work, we show how these mean field fluctuations propagate to finite dimensional nonlinear observables for shallow neural networks trained by stochastic gradient descent. Working in the weighted Sobolev space in which the limiting fluctuation process is constructed, we apply a functional Delta method under ordinary Fréchet differentiability, without requiring Lions derivatives with respect to the measure variable. We obtain a central limit theorem for the observables and, under a suitable representation of their differentials, an explicit covariance formula inherited from the underlying mean field fluctuation theory. We also study whether prescribed quantities of interest can be recovered from the selected observations. Under a constant rank assumption, we prove that a quantity of interest factors locally through the observation functional if and only if, throughout a neighborhood, the kernel of the differential of the observation is contained in that of the quantity of interest. Thus, a differential condition expressed directly in the ambient Sobolev space yields an exact nonlinear local factorization. These results provide a framework both for quantifying finite-width uncertainty on observable, statistically or physically meaningful quantities and for assessing whether the chosen observations contain the information required to identify them.
Read the full abstract

Statistics Theory

Unbounded Characteristic and Universal Kernels

Kernel methods are among the most powerful tools in machine learning and statistics, with a large number of successful applications. Their immense success stems from the flexible function class associated to each kernel—its reproducing kernel Hilbert space (RKHS)—which facilitates statistical analysis, as well as from their computational tractability and applicability to many domains. Multiple notions (such as characteristic, $L_p$-universal, and integrally strictly positive definite) capture the expressivity of kernels and their RKHSs and play a key role in understanding the statistical properties of kernel methods; these concepts and their relations are well-understood for bounded kernels. Even though unbounded kernels have received significant attention over the past decade (for instance, in the construction of kernel-based discrepancy and dependence measures such as the maximum mean discrepancy, the Hilbert-Schmidt independence criterion, and the kernel Stein discrepancy), surprisingly little is known about the relations of these notions in the unbounded case. In the present paper we tackle this severe bottleneck, establishing their relations under mild assumptions.
Read the full abstract

Statistics Theory

The Silhouette Operator: Identifiability of Low-Rank Measures from One-Dimensional Projections

Structured recovery phenomena, such as restricted isometry properties in compressed sensing, have shown that high-dimensional objects can often be reconstructed from remarkably low-dimensional linear measurements. This work develops an analogous recovery framework for low-rank signed measures on $\mathbb{R}^2$, defined here as measures that can be expressed as finite sums of product measures with one-dimensional factors. The framework is based on linear operators, termed "silhouette operators," that map a measure to a fixed finite collection of one-dimensional linear pushforwards. The main results show that a suitably chosen collection of $2k$ projected marginals suffices to identify every compactly supported rank-$\le k$ signed measure, that this number is optimal, and that the projection directions cannot be chosen arbitrarily. The framework is also extended to higher-dimensional sums of product measures by establishing sufficient conditions under which collections of pairwise marginals identify the full model. Building on this framework, a computationally efficient estimator, termed "silhouette mixture estimation" (SME), is introduced for constructing a low-rank empirical measure from data by matching its one-dimensional projected marginals to the corresponding empirical marginals in Wasserstein distance. When combined with one-dimensional density estimators, SME yields an efficient nonparametric density estimator that performs strongly relative to a range of parametric, nonparametric, and deep-learning baselines in settings of moderate dimension and sample size.
Read the full abstract

Statistics Theory · Cross-listed from stat.ME

Finite-Rank Logistic Gaussian Processes with Exact Likelihood for Conditional Density Estimation

Conditional density estimation describes how the entire distribution of a response changes with covariates, and in imaging studies also with location. Logistic Gaussian processes (LGP) give a flexible prior for such densities. However, the normalizing constant of an LGP has no closed form, so existing methods approximate or replace the likelihood. We propose the exact likelihood finite-rank LGP (ExFR-LGP), which writes the log density as the sum of two bivariate functions, one of the response and a location-varying linear index of the covariates, and one of the response and the location. Each function is assigned a finite-rank Gaussian process prior that is piecewise linear on a regular grid, enabling the normalizing constant, the conditional mean and the quantiles to enjoy closed form representations. Posterior samples are drawn from the exact posterior under this prior via a Gibbs sampler that updates the Gaussian components by elliptical slice sampling. When the true log density is the sum of two such bivariate functions, we show that the posterior contracts at the minimax rate of an $\alpha$-smooth bivariate density up to a logarithmic factor. Numerical experiments using synthetic data demonstrate the effectiveness of ExFR-LGP in conditional density estimation. Furthermore, application to fractional anisotropy responses along the corpus callosum in the Alzheimer's Disease Neuroimaging Initiative data yields covariate-adjusted percentile bands with uncertainty, elucidating how diagnosis changes the distribution along the tract.
Read the full abstract

All authors Jaehoan Kim, Indrajit Ghosh, Debdeep Pati, Dipankar Bandyopadhyay

Statistics Theory · Cross-listed from stat.ME

Sequential resetting procedures and false discovery rate

Data arrive sequentially, each associated with a null hypothesis. We develop testing procedures to locate intervals in which some null hypotheses fail, with false discovery rate (FDR) control. The new procedures are called sequential resetting procedures, and they are based on e-values and test supermartingales. We also develop a refined version of the procedures by dropping less informative data points before the block minimum of the test supermartingale in each rejection block. These procedures have explicit FDR bounds under two settings: a classic setting of independence and the more general setting of possible dependence across null data and non-null data. These FDR bounds are independent of the testing horizon, and the general one has anytime validity, but it has an extra logarithm factor compared with the standard FDR level. We present simulation studies and data experiments with applications of sequential resetting procedures to LLM watermark detection and financial backtesting.
Read the full abstract

Statistics Theory · Cross-listed from math.OC

The Geometry of Anisotropic Dilation for Optimal Regularization

A central question in data-driven inverse problems is how to construct a regularizer that adapts to the geometry of the data distribution. Recent work in optimal regularization shows that, within a broad Gibbs class, the regularizer best matched to a distribution $P$ is determined by a single, direction-dependent radial summary statistic $\rho_P$. This suggests a way to control regularizer geometry by transforming the data. In particular, which transformations act on $\rho_P$ in a simple, explicit way? Can they improve the optimization properties of the resulting variational problems or adapt a fixed base regularizer to data? To address these questions, we introduce anisotropic dilation, a direction-preserving map that rescales each point along its Euclidean ray by a positive profile on the sphere. Despite its simplicity, this family acts transitively on the space of radial summary statistics: any target regularizer in the Gibbs class can be reached from any source distribution by a single explicit profile. We characterize the resulting orbit structure on distributions, derive in closed form the profile that optimally adapts a fixed base regularizer to the data, and show that its improvement over isotropic rescaling is governed by a Jensen gap that is provably positive for several natural families. We also establish finite-sample generalization bounds for jointly learning the base regularizer and anisotropic profile, with explicit rates when the logarithm of the profile is parameterized by linear feature models or sparse ReLU networks. Building on this theory, we parameterize the base regularizer and anisotropic profile and learn them jointly from samples, yielding a data-adaptive regularizer for variational inverse problems. Experiments on a controlled two-dimensional family and MNIST denoising show that learning the profile further improves performance.
Read the full abstract

Statistics Theory

Tightness and Error Exponents of SDP with Logarithmically Many Communities

We study a semidefinite programming (SDP) relaxation for community recovery when the number of communities grows logarithmically. In the balanced stochastic block model with $n=km$ vertices, we consider the regime $k/\log m\to\gamma>0$, with edge probabilities $\alpha\log m/m$ within communities and $\beta\log m/m$ across them, for fixed $\alpha>\beta>0$. We derive the sharp asymptotic tightness boundary away from critical cases. When rare vertices cause tightness to fail while the bulk remains spectrally stable, the normalized matrix error of every near-optimal solution still vanishes. We prove matching high-probability exponents for this error and the normalized optimal objective gain, governed by the same local correction that determines tightness. Throughout this spectrally stable region, a single SDP solve followed by explicit rounding and refinement achieves exact community recovery above the information-theoretic threshold. This guarantee holds even when the planted community matrix is not an optimal solution to the SDP.
Read the full abstract

Statistics Theory · Cross-listed from quant-ph

Witnessing Quantum Bayesian Inference beyond Classical Learning

Can an agent's predictions reveal whether its reasoning is based on a classical or a quantum model? Using known results on matrix factorizations, here we show that quantum Bayesian retrodiction can produce one-step-ahead forecasts incompatible with every classical Bayesian explanation based on learning about a fixed but unknown sampling law from conditionally independent observations. The separation has a sharp threshold in the number of outcomes: for any measurement with at most four outcomes, the quantum predictions admit a classical realization, irrespective of the prior state and the finite Hilbert-space dimension, whereas a five-outcome measurement on a single qubit already violates an explicit classical bound. This bound holds for arbitrary classical latent states, prior probabilities, and sampling laws, and involves only the agent's initial and updated predictions. Thus, in this setting, quantum Bayesian retrodiction is strictly more expressive than classical Bayesian learning, and this difference can be witnessed without observing or specifying the agent's internal description.
Read the full abstract

Tuesday 6 October

Statistics Theory · Cross-listed from stat.ML

The optimal information complexity of VC learning

Steinke and Zakynthinou(2020) introduces the Conditional Mutual Information (CMI) framework of analyzing the information complexity of learning algorithms based on algorithm-dependent information-theoretic quantities. We study one of these quantities, the evaluated Conditional Mutual Information (eCMI). It has been an interesting question whether the optimal PAC guarantee for VC classes can be recovered from the algorithm-dependent analyses via CMI. And we show that it is possible to recover this guarantee by constructing a learning algorithm whose eCMI is of order O(d) in the realizable case, where d is the VC-dimension of the concept class. Specially, our algorithm is a randomized Majority-of-5 base learners with optimal in-expectation generalization guarantee.
Read the full abstract