Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Wednesday 16 September

Statistics Theory

On the continuity of the Tukey depth function for fuzzy data

The practical use of statistical depth for fuzzy data requires regularity, inferential stability, and computational feasibility. This paper studies these aspects for the first time for the fuzzy depth, in particular for the fuzzy Tukey depth. In particular, we investigate the continuity of this depth with respect to each of its arguments. As a function of the elements of the underlying space, we prove its upper semicontinuity under the main metrics used in fuzzy spaces. As a function of the distribution with respect to which the depth is computed, we establish the almost sure uniform consistency of its empirical version for continuous fuzzy random variables. These results ensure closed depth regions and support the use of sample depth values to approximate their population counterparts. We also derive consistency of empirical deepest-point estimators and study a finite-grid approximation of the depth, supported by theoretical and simulation results.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Beyond Quadratic Loss: The Stability Phase Diagram of Adam

Loss spikes are recurrent instabilities in neural-network training and can arise from multiple mechanisms. For Adam in particular, macroscopic loss spikes have been linked to optimizer dynamics, yet how its two momentum timescales govern them remains unclear. We investigate this dependence by mapping training dynamics across the $(\beta_1,\beta_2)$ plane. Across a range of model–task settings, an approximately linear boundary, $1-\beta_2=C(1-\beta_1)$, separates spiky from non-spiky dynamics, whereas a one-dimensional quadratic loss produces approximately cubic slope. A one-dimensional superquadratic loss $L(x)\propto|x|^n$ recovers the near-linear scaling and links the boundary coefficient to the effective loss exponent $n$. We further show that confident cross-entropy losses develop a core–wall landscape comprising a narrow quadratic core followed by a steep wall, which produces effective superquadratic behavior at the scale of an optimizer update. Together, these results connect Adam loss spikes to both the mismatch between momentum timescales and finite-scale superquadratic loss geometry beyond the Hessian.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

RDT based upper bounds on the largest average submatrix values

We study statistical variants of the classical largest average submatrix problem. For small submatrices (where the dimension is less than linearly proportional to the original matrix), the problem is typically well understood and believed to exhibit the statistical-computational gap (SCG). However, analytical treatment of both the information-theoretic and algorithmic aspects of the linear regime remains challenging, and no mathematically rigorous results have yet arrived anywhere close to proving or disproving SCG existence in this setting. Focusing on the linear regime, we make strong progress in several key directions: 1) We develop a generic Random Duality Theory (RDT) framework to characterize largest average submatrix values. 2) Using the plain RDT variant, we obtain closed-form upper bounds as explicit functions of dimensionality proportionality parameters. 3) We demonstrate that a lifted RDT variant strictly improves upon the plain RDT within a certain linear range of submatrix dimensions. 4) For small submatrices where the dimensional proportionality constants approach zero, we prove that our results match both the replica results from [34] (obtained via one-step replica symmetry breaking) and the sublinear results from [18,45].
Read the full abstract

Statistical Methodology

Pseudo-Incrementality Testing: Measuring Advertising Lift from Naturally Occurring Interventions

We develop a method for measuring the incremental effect of advertising when randomized exper- iments are unavailable. Firms generate abrupt interventions in their own marketing as a byproduct of operations: budgets are cut, channels launch, programs pause. We propose a two-stage proce- dure that treats these events as quasi-experiments. The first stage discovers and dates interventions by exact Bayesian run-length inference on marketing activity series. The second stage runs a causal impact analysis against a variance-constrained structural time series counterfactual, with simulation-based inference, five qualification conditions, and a closed-form power bound. We validate the procedure on the two experimental benchmarks that Meta released with its GeoLift framework. The method identifies the day of intervention exactly in both cases. On the experiment where advertising was removed, it recovers the effect within 6.2 percent of the experimental esti- mate (implied return of 1.50 versus 1.60). On the experiment where advertising was added, its 90 percent interval covers the experimental estimate and excludes zero. Of six estimators evaluated on the same released data, ours is the only one that requires no geographic panel and produces intervals that are both correct on both experiments and narrow enough to act on.
Read the full abstract

Machine Learning (Statistics)

Preservation of Log-Concavity and Convergence of Wasserstein-Fisher-Rao Gradient Flows

We study the convergence of Wasserstein-Fisher-Rao (WFR) gradient flows for sampling from probability distributions known up to a normalisation constant. By combining Wasserstein transport with Fisher-Rao birth-death dynamics, WFR flows balance exploration and selection. These flows have been recognised as a promising mechanism to accelerate convergence beyond Langevin dynamics. We show that for a class of strongly log-concave target distributions satisfying additional curvature conditions, WFR flows preserve strong log-concavity, in contrast to Wasserstein flows which enjoy this property only in the Gaussian setting. Exploiting this result, we derive explicit non-asymptotic convergence rates for the symmetrised Kullback-Leibler divergence, without requiring a warm-start as required in current estimates. In particular, we show that the convergence rate decomposes additively into Wasserstein and Fisher-Rao contributions, thereby confirming a recent conjecture within this setting. These results provide refined convergence guarantees and further develop the theoretical foundations of WFR gradient flows for sampling and Bayesian inference.
Read the full abstract

Statistics Theory · Cross-listed from math.DG

Generalized Einstein Laurent polynomials, Toric Kähler-Einstein Rigidity, and Finite Exponential Families

We study when the logarithm $\psi$ of a positive finite exponential sum on $\mathbb{R}^d$ satisfies $\det\nabla^2\psi=C\exp(\langle b,\theta\rangle-\lambda\psi)$. This is the Kähler–Einstein equation for metrics induced by exponential maps into projective space and, for natural exponential families, the condition that the Jeffreys prior be a Diaconis–Ylvisaker conjugate prior. First, we classify the bivariate Laurent polynomials with unimodular support satisfying the generalized Einstein condition of Di Scala and Sombra: up to units and monomial changes of coordinates, they are powers of an affine trinomial or products of powers of two independent binomials. Second, we show in every dimension that a smooth compact toric manifold with a projectively induced Kähler–Einstein metric is a product of projective spaces with matched multiples of the Fubini–Study metrics, immersed by a complete Veronese–Segre system up to automorphisms. This proves the compact toric case of the homogeneity conjecture for such metrics and the fixed-point germ and univalent forms of a conjecture of Manno and Salis. Third, without lattice or rationality assumptions, the finite-support exponential families satisfying the equation are, up to affine changes of statistic, exactly the products of multinomial families with a common ratio of categories to trials; this settles the finite-support case of a question of Casalis.
Read the full abstract

Statistical Methodology

Drift Inference for Unit-Root Galton-Watson Processes with Immigration

We study inference on the drift of a critical Galton–Watson process with immigration, a count time series with a unit root. Climate change motivates such nonstationary models for weather-related disaster counts. The drift is the expected increase in the count per period. Ordinary least squares is inconsistent for the drift, so we study state-weighted least squares, giving greater weight to observations following small counts, where conditional variance is lower. Under strict recurrence, we apply null-recurrent regenerative limit theory to obtain a polynomial convergence rate and a standard normal studentized limit. Our main contribution is the recurrence boundary, where we establish a logarithmic convergence rate and a parameter-free non-Gaussian studentized limit. Estimating the optimal weights yields the same first-order limiting distribution as knowing them. Simulations show lower root mean squared error than time-weighted least squares with weights $1/t$, and improved state-weighted confidence-interval coverage when the unit root is imposed.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.OC

Matching Multi-Loop Complexities with a Single Loop: Optimal Optimization Stationarity and Best-Known Game Stationarity in Nonconvex–Concave Minimax Optimization

We introduce a new single-loop algorithmic framework for smooth nonconvex–concave minimax optimization. The resulting projected damped extragradient method combines projected extragradient updates, dual momentum, and a moving proximal center. Under both the optimization-stationarity and game-stationarity criteria, our method achieves the best-known complexity among single-loop first-order methods. For optimization stationarity, our method achieves a gradient complexity of $O(L^2D_Y\bar\Delta_0\varepsilon^{-3})$, where $L$ is the gradient Lipschitz constant, $D_Y$ bounds the diameter of the dual feasible set, and $\bar\Delta_0$ is an initialization quantity involving the value-function gap and the initial gradients. Moreover, by incorporating a fixed-center warm-up phase, the complexity can be improved to $O(L^2D_Y\Delta_\phi\varepsilon^{-3})$, up to an additive lower-order cost, where $\Delta_\phi:=\phi(x_0)-\inf_x\phi(x)$. We further establish a lower bound of $\Omega(L^2D_Y\Delta_\phi\varepsilon^{-3})$ for optimization stationarity over projected zero-respecting first-order methods. This lower bound proves that the warm-started version of our algorithm is optimal up to a constant factor for optimization stationarity within this oracle class. For game stationarity, our method achieves $\mathcal{O}\!(L^{3/2}D_Y^{1/2}\Delta_\phi\varepsilon^{-5/2})$ gradient complexity. This matches the best-known complexity of multi-loop first-order methods, thereby establishing the same complexity with a single-loop algorithmic structure. Under dual strong concavity, the proposed framework achieves $O\!(\sqrt{\kappa}\,L\Delta_\phi\varepsilon^{-2})$ leading complexity for both stationarity criteria, where $\kappa=L/\mu$ is the dual condition number, up to an additive initialization cost. The $\varepsilon^{-2}$ accuracy dependence is optimal under fixed regularity and initialization bounds.
Read the full abstract

Statistical Methodology

Variable-Projection Sparse Functional Principal Component Analysis: Interpretable Functional Dimensionality Reduction with Applications to Raman Spectral Data

Functional principal component analysis (FPCA) provides low-rank representations of functional data but generally produces dense components, making it difficult to identify the localised regions contributing to dominant modes of variation. This limitation is particularly relevant in Raman spectroscopy, where spectra are observed over an ordered domain and interpretation often focuses on chemically meaningful spectral regions. This study proposes variable-projection sparse FPCA (VP-SFPCA) for interpretable functional dimensionality reduction through sparse weight functions that promote localisation. The method formulates sparse FPCA as a regularised matrix-factorisation problem that incorporates the functional inner-product geometry and distinguishes sparse weight functions used to generate component scores from orthonormal loading functions used for reconstruction. Variable projection conditionally minimises over the loading functions, reducing the optimisation problem to the sparse weights. Performance was evaluated through simulation studies and empirical analyses of surface-enhanced Raman scattering (SERS) spectra, with conventional FPCA and SCAD-SFPCA serving as dense and sparse functional benchmarks, respectively. In the simulations, VP-SFPCA recovered localised functional structure while requiring substantially less computation than SCAD-SFPCA. In the empirical analysis, VP-SFPCA retained this computational advantage while yielding held-out reconstruction error close to that of conventional FPCA. Several prominent features of the estimated weight functions also coincided with established adenine SERS bands. Overall, VP-SFPCA provides a computationally practical approach to improving the interpretability of dominant functional modes through localisation.
Read the full abstract

All authors Han Ying Lim, Dharini Pathmanathan, Philipp Otto, Sophie Dabo-Niang

Statistical Methodology

Taylor Diagram and Wasserstein Distance for Model Evaluation

The Taylor diagram is used to evaluate and compare predictive models with observed data and has many applications in climate and environmental sciences. Three statistics of interest–the centered root-mean-squared error, standard deviation, and the product-moment correlation coefficient–are related by the law of cosines; Taylor diagrams leverage this fact to allow visualization of all three statistics in a two-dimensional plot without any loss of information. In this work, we present a novel model evaluation tool–the Wasserstein-Taylor diagram–formed from integrating the Wasserstein distance from optimal transport into a Taylor diagram framework. This tool is built upon the fact that the law of cosines still holds in the Taylor diagram if the centered root-mean-squared error and product-moment correlation coefficient are replaced by the centered 2-Wasserstein distance and quantile correlation coefficient, respectively. The advantage of this Wasserstein-Taylor diagram is that one can use the distribution perspective afforded by the Wasserstein distance to compare observed and predicted data sets of different sizes and even to compare statistical models with only observed data. We further show that the quantile correlation coefficient on empirical measures converges to the quantile correlation coefficient on the sampled measures.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

On the Identifiability of Mixed Ordinal and Exponential Family Causal DAGs under Linear Parametric Models

The problem of identifiability in linear parametric models (LPMs) whose nodes follow either an ordered logit model or a regular one-parameter exponential family is evaluated. The results go beyond classical structural equation models as well as results for nodes with observations from a homogeneous family of distributions. The main result establishes that the orientation of every edge joining an ordinal node to an exponential-family node is identifiable from the joint distribution alone at every parameter value, provided the ordinal node has at least three categories and the exponential-family node at least three points of support, with no restriction on the sufficient statistic. Converses show that both requirements are necessary: the three-category requirement is binding only for affine sufficient statistics, and the three-point requirement is binding under the canonical link. The guarantee extends to orienting every such mixed ordinal-exponential family edge of a given $d$-node undirected skeleton. Numerical experiments illustrate the theoretical results by successfully separating orientations within a Markov equivalence class, which are indistinguishable by conditional independence alone.
Read the full abstract

Tuesday 15 September

Other Statistics

When Can World Models Recover Physical Laws?

Accurate prediction does not establish that a world model has recovered a physical law: distinct dynamics can generate identical records under the same observation protocol. We formulate law recovery on a fixed physical domain under an explicit catalog of experiments, sensor uncertainty, and an acquisition budget. A rate–distortion converse separates the information needed to describe a law from the information the apparatus can reveal. Its constructive counterpart gives a finite response codebook and an explicit decoding budget. On compact world classes, uniform recovery is possible exactly when every pair of different laws is experimentally distinguishable; equivalently, the apparatus can recover all the entropy of every finite law source. An inverse response modulus quantifies stability. For Lipschitz fields on a $d$-dimensional state–action domain, noisy full-state readouts after resets require minimax budget $\Theta(\varepsilon^{-(d+4)/2})$ for squared law error $\varepsilon$, compared with $\Theta(\varepsilon^{-(d+2)/2})$ for direct field observations. Exact crossing-time symmetries establish the lower bound under adaptive experiment selection and arbitrary durations with constant inputs. Reproducible synthetic cases illustrate the separate roles of intervention, calibration, and repeated measurement. Together, the results identify which evidence supports a claim of physical-law recovery and the cost of acquiring it.
Read the full abstract

Statistics Theory

Recursive-Head Geometry and Order-Free Efficient Inference in Finite-State Nested Markov Models

Exploiting the equality restrictions that nested Markov models encode requires their tangent-space geometry. For strictly positive finite-state models on arbitrary acyclic directed mixed graphs (ADMGs), we differentiate the recursive-head chart and prove that the range of its score map is the full tangent space. Intrinsic-set coordinate blocks form an algebraic direct sum, blocks of distinct districts are orthogonal, and the resulting Gram projection needs neither mb-shieldedness nor a district order. Exact counterexamples show that observational centering and kernel normalization alone do not certify tangency. For the node, complete-source edge, and compatible path-specific intervention targets considered here, boundary substitution yields a normalized configured active law and an order-free canonical-gradient formula, extended to finite mixtures by independent source redraws. A coherent one-step estimator with an exact remainder identity and a model-valid chart-flow targeted maximum likelihood estimator are efficient under stated local conditions. On a narrower source-isolated fixed-node subclass, a sequential estimator has an exact transition-factorized drift and up to $2^m$ nuisance-correctness regimes over $m$ active-district transitions; its all-correct influence function projects onto the canonical gradient, and an exact rational law exhibits a strict variance gap. These results separate order-free efficiency on arbitrary finite-state ADMGs from multiple robustness of a narrower construction.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Symmetry without a manifold: intrinsic dimension on orbits

The standard geometric derivation of neural scaling exponents takes the intrinsic dimension of a data manifold as its input. On modular addition in $\mathbb{Z}_p$ that derivation has no input. The exact algebraic solution is an orbit of $\mathbb{Z}_p$ acting by isometries. Transitivity alone makes the ratio statistic underlying the standard dimension estimator a point mass, so the estimator is undefined, and here the two nearest neighbour distances coincide exactly. Breaking the symmetry at scale $\epsilon$ returns a number, but one that tracks $1/\epsilon$ with no scale free plateau. We show that the failure is general, since on any finite orbit of a group acting by isometries the estimator reports the resolution at which the set is probed rather than a dimension. What replaces the power law is exponential in hidden width, $L(h)=L_\infty+A\exp(-c\,h^{\alpha})$, with $R^2$ between 0.982 and 0.995 against 0.857 to 0.906 for a power law admitting the same floor and fitted under the same protocol. Where the data supply is sufficient the rate belongs to the regulariser rather than to the group, since weight decay moves $c$ by a factor of 47 while group order moves it by 1.10, a residual below seed to seed resolution, for every fixed $\alpha$ between 0.75 and 2. The critical width falls with group order rather than rising, against capacity counting that assigns a fixed number of neurons to each irreducible representation.
Read the full abstract

Statistical Methodology

Coherent Hierarchical Forecasting for Proportion and Discrete Time Series

Hierarchical and grouped time series arise when a multivariate time series is forced to satisfy a set of aggregation constraints, motivating forecast reconciliation methods that ensure coherent forecasts of such hierarchical structures. Many real-world applications involve discrete or bounded supports, introducing additional challenges that are not addressed Gaussian-based reconciliation methods. We develop a post-hoc hierarchical forecasting approach to construct coherent forecast hierarchies for discrete and bounded time series. The method constructs coherent forecasts by convolution and exponential tilting, preserving the distributional properties and the underlying support throughout the hierarchy. We evaluate the proposed approach against state-of-the-art reconciliation methods for both discrete and continuous settings, demonstrating strong performance across a range of experiments. Through simulation studies and empirical applications in epidemiological and demographic data, we show that the method provides reliable and coherent distributional forecasts in challenging scenarios.
Read the full abstract

Statistical Methodology

Cross Validation for the log Gaussian Cox Process

The log Gaussian Cox Process (LGCP) is one of the most widely used models for the analysis of spatial point patterns. Although Bayesian methods and software for fitting LGCPs are now well established, practical tools for model criticism, predictive assessment, and model comparison remain comparatively underdeveloped. This paper develops a practical Bayesian cross-validation framework for LGCPs while placing it within a broader framework for Bayesian model assessment. Our approach is motivated by the conceptual and computational challenges that arise when defining predictive validation for point processes, including the choice of holdout data, prediction task, scoring rule, and computational strategy. We propose a leave-region-out cross-validation framework in which the fundamental unit of prediction is a bounded spatial region rather than an individual event. Predictive performance is assessed through the logarithmic scoring rule for joint probabilistic forecasts, while computational efficiency is achieved by combining a new likelihood approximation with a Bayesian unlearning strategy and Laplace approximations to obtain leave-region-out predictive distributions without repeated model refitting. We validate the proposed approximations against Monte Carlo estimates obtained via brute-force refitting, demonstrate the methodology on simulated and real spatial point pattern datasets defined on Euclidean domains and networks, and provide implementations in the R-INLA, inlabru, and MetricGraph packages.
Read the full abstract

All authors Hans Montcho, Håvard Rue, Finn Lindgren, David Bolin, Elias T Krainski, Daniela Castro-Camilo, Sara Martino

Machine Learning (Statistics)

Generalized DCCQ: From Binary Quotients to Multinomial Simplex Geometry and Critical-Strip Coordinates

We extend the discrete complex complement quotient (DCCQ) framework from binary Bernoulli counts to multinomial count compositions. For m+1 categories, m is the number of independent probability degrees of freedom. Integer count vectors modulo common scaling determine rational points of the m-dimensional probability simplex. Building on standard simplex and log-ratio coordinate geometry, for m >= 2 we define the full multinomial DCCQ coordinate map and show that it is a real-analytic diffeomorphism The previously established binary baseline m=1 gives a critical-line coordinate, while the ternary case m=2 gives the full open critical strip; higher multinomial models retain m-2 additional real contrasts. We also give a one-versus-rest specialization, an exact integer-lattice realization of the ternary coordinate, and a hyperbolic representation of its log-ratio. No zero-location theorem or proof of the Riemann Hypothesis is claimed.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

TabPFN-3.5: Technical Report

We introduce TabPFN-3.5, our new flagship Tabular Foundation Model. It significantly outperforms its predecessor, TabPFN-3, and all existing baselines across a broad range of tabular problems. TabPFN-3.5 sets a new state of the art on standard tabular prediction in TabArena, and extends it to the data practitioners encounter in practice: non-i.i.d. data with temporal or grouped splits, tables with strings, text and images, high-cardinality categorical features, and wide tables with many features. These gains carry over to our task-specific harnesses: state of the art on relational data and stronger time-series forecasting. For faster inference, our variant TabPFN-3.5-Fast runs up to 3x faster than TabPFN-3 while keeping most of the accuracy gains. In addition, we upgrade TabPFN-3.5-Plus, expanding our multimodal capabilities with advanced text and date handling alongside proprietary inference optimizations. Finally, we release a new version of our Thinking mode, TabPFN-3.5-Thinking, which scales inference-time computation to push the state of the art further. It benefits from our stronger base model and from inference-time improvements that make it up to 12x faster than TabPFN-3-Thinking.
Read the full abstract

All authors Benjamin Jäger, Nick Erickson, Léo Grinsztajn, Felix Birkel, Klemens Flöge, Oscar Key, Kürşat Kaya, Jonas Kübler, Adèle Frankel, Tobias Schröder, Anurag Garg, Jan Hendrik Metzen, David Salinas, Simon Bing, Kristina Collins, Tuana Çelik, Vahid Balazadeh, Lydia Sidhoum, Tomás Pereda, Brendan Roof, Andrej Tschalzev, Siyuan Guo, Philipp Singer, Lennart Purucker, Jake Robertson, Marie Salmon, Philipp Jund, Jerry Chen, Diana Kriuchkova, Arthur Cahu, Eliott Kalfon, Adrian Hayler, Georg Grab, Vitor Monteiro, Lilly Wehrhahn, Dominik Safaric, Clara Cornu, Alan Arazi, Rylee Grace, Simone Alessi, Mihir Manium, Bernhard Schölkopf, Yann LeCun, Madelon Hulsebos, Sauraj Gambhir, Noah Hollmann, Frank Hutter

Statistical Methodology

Spectral Dynamics of DeepWalk Embeddings for Dynamic Network Change-Point Detection

Dynamic networks describe evolving relational systems in which abrupt structural changes may signal anomalous events or important transitions. Detecting such changes requires distinguishing genuine structural signals from fluctuations in network observations and learned representations. We propose a DeepWalk-based framework for detecting and localizing structural changes in dynamic networks. For each snapshot, we learn low-dimensional node embeddings, align them to a fixed reference using orthogonal Procrustes transformations, and aggregate them by mean pooling into comparable graph-level vectors. We then construct a multivariate cumulative sum (CUSUM) scan statistic to identify changes in the embedding mean. Under stated regularity conditions, we derive bounds on stochastic fluctuations under the null hypothesis and sufficient conditions for reliable detection and consistent localization under the alternative. The analysis connects detection performance with network sparsity, spectral separation, embedding dimension, and change magnitude, clarifying the balance between structural signals and representation variability. Simulation studies demonstrate the effectiveness of the proposed method in detecting and localizing structural changes across a range of dynamic network settings. An application to an international trade network further illustrates its practical usefulness in identifying shifts in trade allocation.
Read the full abstract

All authors Houlin Zhou, Yejin Wang, Xufei Tang, Dan Zhuang

Machine Learning (Statistics)

Bracketing Uncertainty in Clustering Under the Manifold Hypothesis

The manifold hypothesis suggests a natural criterion for clustering: partition data according to the manifold component from which each point is drawn. Whether two components are separable depends on a geometric tradeoff: the ambient separation between components versus the largest gap in sampling. In practice, this tradeoff is rarely assessed explicitly, leading standard methods to over-commit to a single clustering assignment even when the data do not support a unique answer. We formalize this tradeoff by combining intrinsic manifold geometry (volume growth and reach) with sample-level quantities (fill distance and density), yielding a threshold phenomenon for mutual-$k$-nearest-neighbor graphs: when the offset-to-fill ratio exceeds a conservative upper threshold, component separation is preserved; below a lower threshold, components fuse. The gap between these thresholds defines a geometric uncertainty zone in which the number of clusters is not identifiable from the data. Nevertheless, conventional approaches still seek one: sweeping parameters (an engineering approach) or fitting a generative mixture model (a model-based approach). Rather than forcing a single estimate of the number of clusters, we propose Manifold-Based Clustering (MBC), which returns an explicit bracket interval to quantify the underlying data uncertainty. This bracket acts as an empirically calibrated diagnostic: it narrows when a single resolution is supported, widens when multiple resolutions coexist, and collapses to one when no separated structure is detectable. Empirically, we find that many real datasets lie within the uncertainty zone rather than admitting one clear answer. Our results suggest that ambiguity in cluster number is often intrinsic, and should be quantified rather than resolved.
Read the full abstract

Machine Learning (Statistics)

Sharp margin-based generalization bounds for realizable SVM

Let the exact homogeneous hard-margin support vector machine be trained on \(m\) independent observations from a Borel probability law on a real Hilbert space. We prove that, with score zero counted as an error, there is a universal numerical constant \(C\) such that \[ \Pp\left( \gamma_m>0,\quad \Risk(u_m)> \frac{C}{m} \left( K_m+\log\frac1\delta \right) \right) \le \delta . \] Here \(\gamma_m\) is the empirical homogeneous margin, \(u_m\) is the exact minimum-norm unit-margin separator, \(r_m\) is the largest training radius, and \(K_m:=r_m^2\norm{u_m}^2=r_m^2/\gamma_m^2\) on \(\{\gamma_m>0\}\). The proof is driven by a deterministic deletion problem. Given vectors \(x_1,\ldots,x_n\) in the unit ball, delete a set \(B\) of constraints and let \(u_B\) be the closest point to the origin that satisfies every retained unit-margin constraint. Suppose that \(\norm{u_B}^2\le k\) and that every deleted vector has nonpositive score under \(u_B\). We prove that a family of such deletion sets of cardinality \(q\) has size at most \(\exp(8k+2q)\). The conceptual step is an exact identity obtained from the KKT representation of \(u_B\). For a random deletion set, the identity converts the mean squared spread of the separators into a weighted sum of score deficits. It therefore forces a coordinate whose deletion status separates the two conditional means by a quantitatively large amount. Revealing that coordinate decreases the conditional separator variance enough to control the binary entropy of the split. An entropy induction gives the deletion count, and an exact factorial ghost-sample identity converts that count into the stated high-probability SVM bound.
Read the full abstract

Statistical Methodology

Robust Tests for Step-Stress Models under Exponential Lifetimes

Highly reliable products with extended lifetimes present a challenge in reliability analysis: obtaining enough failure data under normal operating conditions is often incompatible with reasonable time and cost constraints. Step-Stress Accelerated Life Tests (SSALTs) offer a practical solution by progressively increasing stress levels to accelerate product degradation, allowing results to be extrapolated to normal conditions. However, the small sample sizes and Type-I censoring inherent to these experiments render classical maximum likelihood-based inference vulnerable to data contamination. Robust point estimation has been studied in the literature. However, robust test statistics have not yet been developed in this context. In this paper, we propose robust test statistics for SSALTs under exponential lifetime distributions, based on the minimum density power divergence estimator (MDPDE). This paper directly works with the exact failure times recorded before the end of the experiment. Once this limit time is reached, all surviving components are censored. This creates a mixed discrete-continuous distribution, which is a major analytical breakthrough in the context of SSALT. We introduce the restricted version of the MDPDE, establish its asymptotic properties, and construct Z-type and Rao-type test statistics for linear hypotheses on the model parameters. The proposed tests are shown to maintain their nominal significance levels and statistical power under data contamination, where classical MLE-based procedures fail. An extensive simulation study confirms the robustness gains of the proposed methods, and a real data application illustrates their practical value.
Read the full abstract

Machine Learning (Statistics)

METALICA: METAdynamics and repLICA exchange for enhanced diffusion sampling

Many proteins function through transitions between conformational states, yet rare states are rarely sampled by diffusion models trained on an equilibrium ensemble, demanding better sampling methods. We introduce METALICA, which implements Metadynamics on a pretrained diffusion model via Replica Exchange. It accumulates a bias potential along a Collective Variable, repels new samples from previous ones through biased sampling, and reweights samples onto the unbiased distribution. METALICA holds one replica per diffusion level, forming a Markov Chain that evolves through inter-replica communication and is refined in place as the bias grows. METALICA is the dual of sequential control, in which Sequential Monte Carlo parallelizes the sampler over a batch of particles. Parallelism over the levels of the diffusion-time schedule instead allows METALICA to generate samples from long chains, essential for the discovery of rare events, with accuracy set by run length rather than by the memory available. We validate on a bimodal target with known free energies, then apply METALICA to the unfolding of a protein. At a budget for which sequential control yields no unfolded structure, METALICA populates the basin and resolves a second free energy minimum.
Read the full abstract

All authors Alireza Omidi, Jiajun He, Jörg Gsponer, Saifuddin Syed

Machine Learning (Statistics) · Cross-listed from math.PR

Approximating Measures on Function Spaces: Transport and Truncation

Measures on function spaces arise throughout Bayesian inverse problems and generative modeling, often with low-dimensional structure relative to a tractable reference measure. We introduce the class $\mathcal{P}_\psi(\mu)$ of measures that differ from a reference measure $\mu$ only through a finite-dimensional map $\psi$ while preserving the reference conditionals on its fibers. Class members are determined by their $d$-dimensional pushforwards under $\psi$ and admit convenient block-triangular transport map representations. Draws are taken from this $d$-dimensional distribution and then completed to function space through sampling of the reference conditionals. For Gaussian references, these transport maps are finite rank perturbations of the identity. In contrast, optimal transport maps do not preserve this low-dimensional structure. For covariance perturbations that are trace class in the Cameron-Martin geometry of the reference, we show the optimal map is a trace class perturbation of the identity. Even finite rank perturbations yield corrections whose rank, governed by an invariant subspace, is typically infinite. We develop approximation theory for $\mathcal{P}_\psi(\mu)$ and error analysis for fitting within it, splitting the total error into an irreducible class error and a finite-dimensional marginal term set by the dimension of $\psi$ rather than the ambient discretization. We present numerical results including inference from low-dimensional nonlinear observation maps, deconvolution under a jump process prior, and state estimation for Navier-Stokes flows.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.OC

Random tilts to find stationary points in stochastic convex optimization

We consider the problem of finding stationary points of stochastic convex functions and related variational inequalities. For each, we show that regularized empirical risk minimization, coupled with a random tilting perturbation, obtains stationarity residual order $\sqrt{d/n}$ for $d$-dimensional problems given $n$ observations. We present a few complementary results that show that some dimension dependence is necessary, in distinction from standard stochastic optimization and empirical risk minimization, by providing minimax lower bounds scaling as $\sqrt{\log d / n}$.
Read the full abstract

Statistical Methodology

Online Bayesian Model Averaging with Joint Uncertainty Quantification for Models and Regression Coefficients in Binary Regression

Streaming binary response data arise in many applications, including virtual learning platforms where student responses are collected sequentially and estimated probabilities of correct responses may inform future question assignment. Logistic regression provides an interpretable framework for such analysis, but the data may support multiple plausible predictor subsets. Bayesian model averaging (BMA) accounts for such model uncertainty by averaging inference across competing models. Since repeatedly applying BMA to accumulating data can be computationally burdensome, we develop an online implementation based on renewable estimation. At each update, retained summaries and new observations are used to approximate the joint posterior of models and model-specific coefficients without revisiting historical data, enabling point and interval estimates that incorporate model uncertainty. Simulations show that online BMA closely approximates its offline counterpart in coefficient estimation, model and variable importance, prediction, and interval estimation of predictive probabilities, while substantially reducing computational cost. In a virtual learning application, we find substantial uncertainty across competing predictor subsets and large variability in credible interval widths across questions. Our method distinguishes questions with precise estimates from those with substantial uncertainty, providing information beyond point estimates for adaptive question assignment.
Read the full abstract

Statistical Methodology

Efficient estimation and the cost of complete-case coarsening under monotone sequential MAR

Complete-case coarsening discards observed confounder values from partially complete records. We study its consequences for average treatment effect estimation with two ordered, partially observed confounders under monotone sequential missing at random. We specialize the standard coarsening-at-random transformation to the causal influence function, establish the canonical gradient, and give an exact drift identity for a cross-fitted estimator with sequential multiple robustness. In the submodel where both sequential and complete-case missing-at-random assumptions hold, we express the efficiency loss from coarsening as a nonnegative expectation involving two iterated projections. The gain is strict when the intermediate confounder supplies residual information where second-stage missingness occurs. Oracle simulations and deterministic quadrature illustrate this efficiency comparison. When second-stage response depends on the intermediate confounder, the comparison instead concerns identification: coarsening can introduce persistent bias. Estimated-nuisance simulations include a bounded-propensity design and a Gaussian stress design. The latter exhibits substantial interval undercoverage and can reverse the finite-sample precision ordering, qualifying the practical interpretation of the efficiency bound.
Read the full abstract

Statistical Methodology

Information Set Emulation: Causal Certificates for AI Derived EHR Features

AI and large language models can recover clinically meaningful features from electronic health records (EHRs), but predictive usefulness does not establish admissibility for causal inference. We introduce information set emulation: an AI typed lift attaches source evidence, clinical and recording times, decision-time availability, representation version, proposed causal roles, and unresolved ambiguity to extracted features under a locked target trial. Causal certificates record auditable evidence for those roles. Features with unresolved downstream roles are routed to compatible reporting or separate analyses. Typed evidence defines an observational fiber of causal worlds consistent with the observed law. The locked scalar estimand maps this fiber to a compatible image whose squared Chebyshev radius equals the residual minimax mean squared error when the image is nonempty and compact. This classical identity provides a target-specific measure of information ambiguity. The contribution is its integration with a joint EHR observation map and an auditable certificate architecture. Under explicit exchangeability, positivity, and nuisance-consistency conditions, we give identification and cross-fitted augmented inverse probability weighted estimation, distinguishing empirical and population targets. An EHR compression-drift identity separates the roles of frame presence, treatment assignment, and outcome observation. Artificial simulations and a common-law finite-world example illustrate estimation failures and information-radius reduction. Synthetic Phase 0 notes demonstrate audit diagnostics; a separate role-specific analysis spread illustrates routing and is not an exact fiber radius. All experiments are synthetic. The framework specifies when reconstructed information can support a point claim and when compatible reporting is required.
Read the full abstract

Statistical Methodology

When AI Generates Covariates: Causal Typing and Estimand Drift in Sequential Experiments

AI-generated covariates from notes, conversations, images, and wearable streams can change the causal question when their roles are left unspecified. A generated feature may represent a treatment version, pre-action state, history, design variable, mediator, outcome proxy, observation process, or intercurrent event; these roles are not interchangeable. We formulate a causal type discipline for sequential experiments: a versioned representation map, a causal role classifier, a claim-status filter, and an estimand lock. The lock fixes a standardized proximal effect before generated covariates enter the analysis. Under audit correctness and standard identification assumptions, admissible role assignments preserve this estimand. We apply the established conditional-covariance characterization of compression bias to substitution of generated representations for design-relevant states. A standardized decomposition separates compression, conditional-law, and standardization drift. Further results cover mediator adjustment, post-action leakage, marker-intervention conflation, outcome-guided discovery, and state-measurement error. Cluster-level orthogonal estimators distinguish empirical and superpopulation targets under repeated sessions and missing outcomes. Simulations show that refinement helps when it retains design-relevant information, whereas design erasure, leakage, and same-data marker selection can produce bias or undercoverage. The framework places causal semantics and claim status before confirmatory inference with generated representations.
Read the full abstract

Statistical Computation · Cross-listed from math.NA

On the computational cost of Stochastic Gradient Langevin Dynamics

Stochastic Gradient Langevin Dynamics (SGLD) reduces the cost of Langevin-based sampling by replacing full-dataset drift evaluations with mini-batch approximations, but the resulting subsampling error may offset this computational saving. We study this trade-off for stochastic differential equations with finite-sum drifts and compare the computational cost of SGLD with that of the Euler-Maruyama (EM) method. For a prescribed mean-square accuracy $\varepsilon^2$, we derive complexity estimates that explicitly track the dependence on the dataset size $m$, mini-batch size $s$, and accuracy parameter $\varepsilon$. The resulting comparison reveals distinct parameter regimes in which either method is preferable. In particular, EM can have lower leading-order cost only in a small-data, aggressive-subsampling regime, whereas SGLD is favoured over most of the remaining parameter space. In the practically relevant regime $s \ll m$, the transition between the two methods occurs at the scale $m \asymp \varepsilon^{-1}$. We complement the theoretical analysis with numerical experiments based on a Gaussian Bayesian inference model, which examine the predicted cost regimes together with the underlying discretisation error and variance estimates.
Read the full abstract

All authors Mateusz B. Majka, Tigran Nagapetyan, Łukasz Szpruch, Yue Wu, Danqi Zhuang