Machine Learning (Statistics)
Empirical Auditing of Edge-Private Graph Generators
Anum Fatima, Stratis Limnios, James Adams et al. (+4 more)
We empirically audit
privacy leakage by testing whether outputs from edge-neighbouring inputs remain distinguishable, using statistically valid lower bounds on the
privacy loss witnessed by our attacks. Our framework compares direct-edge, local-structural, and GNN-based attacks through the geometry surrounding a target edge. Experiments across two generators and two networks show that
privacy leakage is both mechanism- and network-dependent, with learned representations revealing information not captured by conventional local statistics.
Read the full abstract ▾
All authors Anum Fatima, Stratis Limnios, James Adams, Lukasz Szpruch, Carsten Maple, Gesine Reinert, Andrew Elliott
Machine Learning (Statistics)
Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning
Abhishek Srivastava, Arijit Hazra and Rajesh Dubbaku
Variational autoencoders (VAEs) offer an efficient approach to amortized
Bayesian inference for inverse problems, but posterior accuracy can depend strongly on the choice of variational regularization, particularly when the inverse problem contains weakly identified parameter directions. This study investigates three objectives: a reverse Kullback–Leibler formulation (VAE-KL), an asymmetric Jensen–Shannon formulation (VAE-JS), and a Jensen–Shannon–Wasserstein formulation (VAE-JSWA), which replaces the reverse Kullback–Leibler regularizer with the squared 2-Wasserstein distance while retaining forward-Kullback–Leibler posterior supervision. A full-covariance Gaussian encoder and a pre-trained physics-based surrogate are used for amortized posterior inference. A local linear–Gaussian analysis in the generalized Fisher basis is developed to characterize the variance-dependent gradients of the three objectives. The formulations are first evaluated using linear–Gaussian
benchmarks with known posterior solutions and subsequently tested on nonlinear physics-based inverse problems, including an inverse problem governed by a linear ODE and two PDE-constrained problems. VAE-KL performs slightly better than the other formulations in the well-conditioned
benchmark, where all three approaches yield comparable posterior approximations, whereas VAE-JSWA provides substantially lower posterior errors in the strongly ill-conditioned
benchmark. The nonlinear physics-based problems exhibit a similar conditioning-dependent trend, with JS-based formulations providing greater benefit as posterior ill-conditioning increases. These results indicate that posterior conditioning is an important factor in selecting variational objectives and motivate geometry-adaptive variational inference for
Bayesian inverse problems.
Read the full abstract ▾
Statistical Methodology
The Informational Content in Lepto-Variance and Its Relation to Higher Moments
Vassilis Polimenis
Lepto-regression is defined as the machine learning process of constructing a Regression Tree of a target feature on itself. It is a novel, model-free method potentially revealing information on important sample structure properties. But it is yet not clear what the informational content of lepto-variance is and how it is related to other well-known statistics of a sample. One significant finding is that 58% of the historical US stock return variability is 1-bit lepto-variance that can not be explained by any financial factor. The central question investigated in this paper is to use small normal N(0, 1) drawn samples to explore how the 1-bit sample lepto-variance and lepto-ratio relate to sample variance, skewness and excess kurtosis. Using a large sample simulation, the lepto ratio of a normal is found to converge to 36.3%. For smaller normally distributed simulated N(0, 1) samples, while lepto-variance itself is highly correlated to sample variance, lepto-variance as a fraction of total variance is highly correlated to excess kurtosis. Both lepto-variance and lepto-ratio are orthogonal to sample skew. Another finding is that while lepto-ratio is strongly correlated to lepto-variance it remains orthogonal to sample variance.
Read the full abstract ▾
Statistical Methodology
Calibration Count Reuse: Validity Does Not Determine Efficiency
Rudra Chopra
Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity scores: transferring one count from another class to the scored class must not improve its conformity. A leave-self-out full conformal reference proves the criterion without requiring normalization or preservation of same-class score order. For a common separable transformation, universal exchangeable validity is equivalent to being nondecreasing in the count, provided $K\alpha \geq 1$; normalized multiplicative weights obey the complementary nonincreasing condition. Additive penalties are covered under the stated information restrictions. Efficiency has no parallel ordering: two iid constructions make the same valid rule improve or worsen expected size at unchanged coverage. An expanded 55-rule study finds no resolved advantage from selected live-count rules over uniform weights. Image studies identify undercoverage under iid resampling, including at numerical convergence. Separately, execution of the released Conf-OT pipeline on its DTD and Aircraft
benchmark subsets produces near-nominal median coverage under fixed stratified counts. The native results are reported separately from the iid analyses, without treating a
benchmark observation as a universal guarantee. The findings separate validity, classifier confidence, numerical convergence, and population-specific efficiency.
Read the full abstract ▾
Statistical Methodology
Bounding the Risk Difference Bias Given Incomplete Parameters for Externally Adjusting Uncontrolled Confounding
Onyebuchi A. Arah
Bias due to uncontrolled confounding can be analyzed using several external adjustment formulas. Like other bias expressions, the risk difference bias formula involving an uncontrolled polytomous confounder requires only three externally obtainable parameters: confounder prevalences among the exposed and unexposed and the confounder-outcome risk difference. This paper presents two-parameter bounds on the risk difference bias expression when only one of the three required bias parameters is unavailable. Each bound is sharp over the full range of the unknown parameter. Some numerical examples with graphs are provided.
Read the full abstract ▾
Machine Learning (Statistics)
JAREX: An Acquisition Function for Multi-Objective Algorithmic Process Characterization
Xinyang Li, Kevin Stone and Ajit Vikram
Pharmaceutical process characterization is central to Quality by Design because it defines how variations in process parameters affect the ability to meet product quality specifications, thereby supporting proven acceptable ranges and robust manufacturing. In practice, however, characterization still relies largely on factorial design of experiments (DOE) approaches, which are inefficient for resolving multivariate pass/fail boundaries in higher-dimensional spaces. While
Bayesian optimization has transformed process optimization, adaptive methods for multi-objective process characterization remain lacking. Here, we introduce JAREX (Joint Acceptable Region EXploration), a
Bayesian active-learning acquisition function for multi-objective process characterization. JAREX formulates characterization as a joint boundary-learning problem and adaptively selects experiments to recover the joint pass region defined by simultaneous satisfaction of threshold criteria across multiple objectives. JAREX combines an optimistic joint-feasibility mask with a multi-objective extension of randomized straddle, focusing sampling on the joint edge of failure. Our
benchmark study suggests that JAREX provides more accurate and sample-efficient recovery of the joint pass region than factorial DOE, space-filling designs, and greedy objective-wise strategies over the full experimental budget range. For batched experimentation, it reduces the number of iterative process characterization experiments by more than half while preserving high accuracy for the boundary-identification task. Implemented in the open-source obsidian package, JAREX provides a modular framework for adaptive, data-efficient multi-objective algorithmic process characterization, supporting sample-efficient range finding in high-dimensional spaces.
Read the full abstract ▾
Statistical Methodology
Selective Inference for CART with Binary Outcomes
Tomoshige Nakamura
Binary classification trees select subgroups using the same outcomes later used to assess their differences. We develop finite-sample conditional tests of a common success probability within a parent selected by deterministic Gini CART. The construction retains all eligible cutpoints and conditions on the selected split, its ancestor path, the parent success total, and outside outcomes. The resulting uniform label fiber gives an exact count distribution, while a reversible parallel Monte Carlo construction yields super-uniform inclusive and exactly uniform tie-randomized p-values for any prespecified finite run budget. Within a two-child constant-risk model, the selected count law is an exponential family with information equal to its conditional count variance. We separate information loss from computational limitations and exhibit a selected fiber disconnected under single-label swaps. Simulations with 200 or 400 observations, ten independent or correlated predictors, and trees of depth three show conservative inclusive tests and nontrivial power for large risk differences. At 400 observations and a generating risk difference of 0.4, randomized rejection conditional on reaching the prespecified third-level target is 51–71%, while selection followed by rejection occurs in 11–21% of
datasets. Smaller signals remain difficult to detect, and a representative fivefold increase in computation gives little power improvement. The guarantee concerns parent homogeneity, or equality of two constant child risks, and does not cover equality of heterogeneous regional averages.
Read the full abstract ▾
Statistical Methodology
Structured Screen-and-Select for Ultra-High-Dimensional Variable Selection
Nilotpal Sanyal and Padmore N. Prempeh
Ultra-high-dimensional data, with p far exceeding n, are common in genomics and biomedical research. Marginal screening can miss active predictors with weak marginal signals, especially under strong predictor correlation. We develop Structured Screen-and-Select Variable Selection (S3VS), an iterative framework that combines outcome-based screening with correlation-based local predictor sets. At each iteration, S3VS identifies leading variables, forms local sets through predictor associations, applies a model-specific selector, aggregates selected and nonselected variables, and updates the candidate set and, when appropriate, the outcome representation. The framework allows flexible leading-variable, local-set, and aggregation rules, with implementations for linear, generalized linear, accelerated failure-time, and Cox models. For a specified one-step linear configuration, we establish sure screening under conditions on proxy coverage, correlation separation, within-set retention, and active-preserving aggregation. Simulations compare full and first-iteration S3VS with one-pass SIS procedures in linear, logistic, and Cox settings. S3VS can improve variable recovery or prediction when correlated predictors provide useful proxy information, although gains depend on predictor structure and selector choice. In ovarian-cancer data, full S3VS with clinical variables showed the strongest internal discrimination and early prediction, whereas SIS–Cox–LASSO with clinical variables showed the strongest external discrimination. Neither molecular approach consistently reduced prediction error, and no gene was selected in all five outer folds. S3VS provides a flexible framework for exploiting predictor dependence before model-specific selection. The method is implemented in the CRAN R package S3VS.
Read the full abstract ▾
Statistics Theory
Conformalized Quantile Regression and Minimax Limits of Fixed-Score Calibration under Known Covariate Shift
Rustam Isaev, Anton Conrad, Denis Belomestny et al. (+2 more)
In this paper, we study nonasymptotic $L^p$ error bounds for interval length and conditional coverage in split conformalized
quantile regression (CQR). Our bounds rely on local regularity conditions and accuracy guarantees for the estimated
quantiles. We further instantiate our bounds for
quantile regression with sparse ReLU neural networks. We also consider
covariate shift, where the calibration and test
covariates have different distributions, and derive nonasymptotic bounds for this setting. We obtain matching minimax upper and lower bounds in expectation for two constructed fixed-score calibration
benchmarks under known
covariate shift. The bounds match for every $p\in[1,\infty]$ in the scalar problem and for finite $p$ in the $K$-threshold problem; for the latter, a high-probability minimax lower bound holds for every $p\in[1,\infty]$.
Read the full abstract ▾
All authors Rustam Isaev, Anton Conrad, Denis Belomestny, Eric Moulines, Sergey Samsonov
Statistics Theory · Cross-listed from q-bio.PE
Phylogenetic Inference and the Stickiness of Fréchet Means, via Precise Asymptotics of an Embedded Random Walk
Adam Quinn Jaffe
A well-known phenomenon in statistical analyses of populations of phylogenetic trees in the Billera-Holmes-Vogtmann space is that the
topology of the Fréchet mean tree can contain multifurcations (i.e., internal nodes with more than two children), which raises the practical question of whether this reflects a population-level branching structure (hard polytomy) or merely sampling variability in the data (soft polytomy). This is an instance of the more general phenomenon of "stickiness" in non-Euclidean statistics, whereby the sample Fréchet mean in certain non-positively curved stratified spaces becomes permanently trapped in a lower-dimensional stratum. In this work, we identify a particular multidimensional random walk embedded within the Fréchet mean process, and we show that the time at which stickiness occurs is determined by the largest last-passage time above zero of the coordinates of this random walk. Using this representation, we develop a fully nonparametric procedure for estimating the probability that trifurcations in a sample Fréchet mean tree will bifurcate at some future time if more observations are collected. Lastly, we apply our methodology to a problem in phylogenetics where we consider whether an observed trifurcation in the species tree of primates, glires, and tree shrews is genuinely trifurcated at the population level.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.DS
Improved polynomial-time algorithms for detecting and recovering planted $\Theta(\sqrt{n})$-cliques
Dmitriy Kunisky and Songtao Mao
In the planted clique problem, one observes either an Erdős–Rényi graph on $n$ vertices or such a graph with a clique added to $k = k(n)$ vertices, and seeks to detect or recover the clique. It is widely believed that $k = \Theta(\sqrt{n})$ is the smallest clique size for which
polynomial-time
algorithms exist for these tasks. We develop new
algorithms in this regime using color-coding to estimate signed subgraph counts, further accelerated with fast
matrix multiplication. We first show that, for each $t \geq 1$, for $c(t)$ a constant associated to the order of growth of the number of connected graphs of treewidth at most $t$, cliques of size $k = \lambda\sqrt{n}$ planted in a random location with $\lambda > 1 / \sqrt{c(t)}$ can be detected and recovered in time $n^{t + 1 + o(1)}$. For instance, since $c(1) = e$, this recovers by counting signed trees the performance of the $\widetilde{O}(n^2)$-time message-passing
algorithm of Deshpande–Montanari (2015) that succeeds when $\lambda > 1 / \sqrt{e} \approx 0.6066$. For $t \geq 3$, the exact value of $c(t)$ is not known, but lower bounds on it give a hierarchy of slower
polynomial-time
algorithms that succeed for smaller $\lambda$. We further show that the above
algorithm for $t = 2$ can be implemented in time $n^{\omega + o(1)}$ for $\omega$ the constant of square
matrix multiplication and succeeds when $\lambda > 0.3320$; under the folklore
conjecture that $\omega = 2$, this runs in the nearly-linear time of the
algorithm of Deshpande–Montanari while finding smaller cliques. Second, we show that the above
algorithm for $t = 1$ can be combined with the boosting scheme of Alon–Krivelevich–Sudakov (1998) using rectangular
matrix multiplication, giving improved runtimes for smaller $\lambda$. Taken together, our results achieve the best known tradeoff between runtime and signal strength $\lambda$.
Read the full abstract ▾
Statistical Methodology
On the use of meaningful score regions to interpret treatment effects on continuous clinical outcome assessments
Andrew Trigg and Fraser D. Bocell
Clinical outcome assessments (COAs), such as patient-reported outcomes (PROs), are faced with an interpretability issue due to a variety of score metrics being employed. Therefore, evidence linking COA scores to interpretable patient experiences is necessary to judge the extent to which the results of COA-based clinical trial endpoints are meaningful. The fourth Patient-Focused Drug Development guidance, published in draft form by the FDA in 2023, proposes the use of meaningful score regions (MSRs) to divide COA scores into more easily interpretable categories (e.g. severity-based MSRs of 'none', 'mild', 'moderate' and 'severe'). A recommendation is to compare the magnitude of estimated treatment effects for continuous COA scores (e.g. the difference in mean change from baseline between two treatment groups) to the maximum MSR width. In this article we explore this recommendation further, through historical evidence and simulations, and conclude that using the maximum MSR width as a
benchmark may fail to detect meaningful treatment effects in practice. We suggest alternative approaches to evaluate treatment effects against MSRs, describing the expected outcome within each arm in terms of the probabilities of MSR membership.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
A Kernel Approach to Multivariate Generalized Gammas Convolutions
Léo Gonin, Esterina Masiello and Véronique Maume-Deschamps
We propose a kernel-based estimation framework for Multivariate Generalized Gamma convolutions (MGGC), a class of probability distributions on R d + . MGGC are useful for risk modeling e.g. Their densities are in general not explicit. The construction relies on a new Stein operator for the MGGC class, derived from a partial differential equation satisfied by the characteristic function and expressed as an integral against the underlying Thorin measure. This leads to construct a new kernel and an estimation procedure in the derived reproducing kernel
Hilbert space (RKHS); with consistency results. Numerical experiments illustrate the practical behaviour of the proposed
estimator.
Read the full abstract ▾
Statistical Computation
Bayesian Calibration with Functional Outputs Using Elastic Partial Matching
Paul Casteras, Julien Bect, Josselin Garnier et al. (+1 more)
Calibrating a simulation model involves estimating its parameters by comparing model outputs with experimental data, so that simulation results faithfully reproduce the experimental observations. When the outputs are functions of time, there are multiple ways to quantify the discrepancy between experimental and simulated curves. A recent approach based on elastic functional data analysis decomposes a functional output into two components: a function temporally aligned to a template, and the corresponding warping function. This decomposition splits the problem into two independent calibration tasks, thereby addressing functional misalignment. However, it assumes that experimental and simulated curves share the same temporal support, an assumption often violated in practice when initial or end times are themselves uncertain or depend on the calibration parameters. In this work, we reinterpret the decomposition step as an approximation to a more general
Bayesian calibration problem that incorporates an error term on the time axis. This perspective allows us to naturally extend the framework to a broader family of time warpings with varying initial or end times, using partial elastic
alignment. We illustrate the method on a synthetic test case, comparing it with existing
Bayesian calibration methods and demonstrating improved surrogate performance and error modeling. We then apply the proposed approach to the calibration of an equation of state (a thermodynamic equation relating the state variables of a material).
Read the full abstract ▾
All authors Paul Casteras, Julien Bect, Josselin Garnier, Gwenaël Salin
Statistical Methodology
Statistical Depth for Interval Data: A Closeness-Based Approach and Its Applications
Xiaozheng Chen and Wenlin Dai
With the rapid development of technology, interval data have been increasingly widely used in various fields, as they can effectively carry information on measurement uncertainty and original variability. Reasonable ranking and outlier screening of interval data are crucial for exploring their value, and statistical depth theory is an important tool to achieve this goal. Existing statistical depth methods are mostly defined for ordinary point-valued data, which are difficult to adapt to the characteristics of interval data and cannot meet the needs of their ranking and outlier detection. Therefore, this paper proposes a statistical depth suitable for interval data and conducts relevant application verification.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
Augmented Hypothesis Testing with Persona-Based LLM Simulations
Ziyad Benomar, Aymen Al Marjani, Paul Missault et al. (+1 more)
A/B testing requires large sample sizes, long timelines, and significant costs. When auxiliary predictions of experimental outcomes are available from machine learning models, uncertain prediction quality precludes replacing human experiments entirely, yet these predictions may still contain useful signal. We propose a principled framework for learning-augmented hypothesis testing that leverages predictions of unknown quality to reduce sample sizes while maintaining statistical validity. Predictions naturally vary in granularity, from coarse aggregate signals to fine-grained individual-level estimates, and our framework addresses both ends of this spectrum: (1) for population-level directional predictions, where only a binary signal on the treatment effect sign is available, we use an asymmetric test and prove consistency and robustness bounds within the learning-augmented
algorithms paradigm; (2) for individual-level predictions, we introduce Generalized PPI++ (GPPI), extending Prediction-Powered Inference to handle nonlinear prediction errors through higher-dimensional transformations. Both methods benefit from accurate predictions while remaining robust to inaccurate or
adversarial ones. We validate our framework using persona-based LLM simulations, where AI
agents equipped with user personas predict individual behavior, as a natural prediction source spanning both granularity levels. Experiments on four real-world
datasets demonstrate that our methods, combined with persona-based predictions, substantially reduce experimental costs while preserving rigorous statistical validity.
Read the full abstract ▾
All authors Ziyad Benomar, Aymen Al Marjani, Paul Missault, Saab Mansour
Statistics Applications
Overcoming Model Misspecification in Bayesian Inference of Molecular Signalling Networks
Yinghan Li, Heishiro Kanagawa, Francesca Romana Crucinio et al. (+2 more)
Bayesian inference of molecular signalling networks usually relies on tractability of the marginal likelihood, enabling the set of possible networks to be efficiently explored. As such, linear models with independent errors and conjugate priors are routinely used. However, the dynamics of molecular signalling are nonlinear, and relevant confounders are often unobserved; failure to account for these complexities will almost certainly lead to over-confident inferences in the standard
Bayesian framework. To confront this reality, we develop a post-
Bayesian approach to inference of molecular signalling networks, guided by the principle that uncertainty should not vanish when the statistical model is misspecified, even in the infinite-data limit. Technically, we extend the predictively-oriented (PrO) posterior of McLatchie et al. (2025) to the setting of latent variable models, empirically investigating the properties of PrO posteriors in the challenging network inference context.
Read the full abstract ▾
All authors Yinghan Li, Heishiro Kanagawa, Francesca Romana Crucinio, Matthew A. Fisher, Chris. J. Oates
Statistical Methodology
Filtering and Smoothing with Score-Driven Models
Giuseppe Buccheri, Giacomo Bormetti, Fulvio Corsi et al. (+1 more)
Score-driven models are, by construction, purely predictive filters. When such models are read as filters for an underlying latent process, rather than as data generating processes, this is a restriction rather than a feature, because the contemporaneous and the future observations also carry information on the current state. Starting from the observation that the Kalman filter and smoother recursions for linear Gaussian models can be written in terms of the score of the conditional log-likelihood, and that the predictive step then takes the form of a score-driven recursion, we generalize score-driven models along this direction by deriving an update filter and a smoother, which exploit the contemporaneous observation and the whole sample respectively, together with the corresponding conditional variances. In extensive Monte Carlo analyses the update filter lowers the mean square error of the predictive filter by 3.5% to 8.8%, and the smoother by 32% to 44%, with the ordering holding in every replication; confidence bands built on the associated conditional variances attain their nominal coverage, whereas bands that ignore filtering uncertainty capture less than a third of it. Empirically, we demonstrate the benefits of employing score-driven models as filters rather than as purely predictive processes, showing that the resulting smoothed estimates align more closely with realized quantities than their predictive counterparts.
Read the full abstract ▾
All authors Giuseppe Buccheri, Giacomo Bormetti, Fulvio Corsi, Fabrizio Lillo
Machine Learning (Statistics)
Identifying Representational Biases in Datasets Using PCA: A Max-Disparity Partition Framework
Arjun KM and Shashi Jain
Principal Component Analysis (PCA) minimises aggregate reconstruction error, which can inadvertently represent majority subgroups with substantially higher fidelity than minority subgroups.
Fairness-aware extensions of PCA correct this disparity but require group labels as input. We address the logically prior question: given only a data
matrix, which binary partition of the data suffers the greatest representational disparity under a shared PCA projection? We formalise this as the max-disparity partition problem and propose a greedy local-search
algorithm, grounded in the Fiduccia-Mattheyses bipartitioning framework, that discovers the disparity-maximising partition without any predefined group labels. Two
benchmark algorithms, a fixed-projection sorting baseline and a simulated-annealing variant, confirm that the greedy solution is empirically near-optimal. Having identified the partition, we attribute the disparity to specific features via PCA loading scores and association rule mining, enabling a practitioner to assess whether the disadvantaged group corresponds to a human-meaningful minority. On the Predict Students' Dropout and Academic Success
dataset, representational disparity is driven predominantly by institutional and programmatic proxies for socioeconomic disadvantage, with gender emerging as a secondary but consistent contributor within the disadvantaged group. The discovered partition is then passed directly to Fair PCA, completing a detect-explain-mitigate pipeline.
Read the full abstract ▾
Machine Learning (Statistics)
Beyond Point Prediction: Artificial Representative Trees with Uncertainty
Lea L. Mairhöfer, Silke Szymczak, Björn-Hergen Laabs et al. (+1 more)
Random forests (RFs) predict well but are opaque, whereas single decision trees are interpretable but unstable. Artificial representative trees (ARTs) were developed as interpretable surrogate models for RFs, but their use as standalone prediction models with uncertainty quantification has not been systematically investigated. We combine ARTs with leaf-wise Mondrian conformal predictive systems (CPS), enabling a single tree to provide continuous predictions, prediction intervals, and probabilities of exceeding arbitrary thresholds. We compared ARTs with CPS against decision trees with CPS and separate regression and probability trees across five simulation scenarios, 21
benchmark datasets, and a cross-sectional NHANES example data set. Repeated cross-validation assessed predictive performance, interpretability, and stability. ARTs with CPS yield compact, structurally stable trees with substantially more reproducible split-variable selection than decision trees across
benchmark datasets and NHANES. Decision trees showed slightly better predictive performance and narrower prediction intervals, while coverage was broadly comparable. CPS-based trees generally achieved lower and less variable Brier scores than multi-model approaches. Combining ARTs with CPS therefore provides a single, interpretable, and stable model for continuous predictions and calibrated probabilities, balancing predictive performance with reproducibility and transparency in settings where stability and interpretability are essential.
Read the full abstract ▾
All authors Lea L. Mairhöfer, Silke Szymczak, Björn-Hergen Laabs, Tuwe Löfström-Cavallin
Statistical Methodology
Tensor Completion using Subspace Information
Jingyang Li and Michael K. Ng
Tensor completion has attracted significant
attention in both applications and theoretical research. Under standard uniform sampling, existing
polynomial-time guarantees generally require more observations than the number of degree of freedom, motivating the study of a possible statistical-to-computational gap in highly missing regimes. Fortunately, in many practical scenarios, side information is available, which can provide valuable insights to mitigate these challenges. In this paper, we introduce an
algorithm called Tensor Completion using Subspace Information (TCSI) that incorporates side information through an estimated subspace. Our approach first extracts the subspace from the available side information and then reformulates tensor completion as a
matrix regression problem. We provide a theoretical analysis showing that, when accurate subspace information is available, the required sample complexity is reduced to nearly linear order in the uncoupled ambient dimensions, removing the coupled-mode dimension from the leading term. Leveraging the estimated subspace information, we obtain a less stringent sufficient signal-to-noise ratio requirement than those in several existing passive-uniform-sampling guarantees. Under additional mild conditions, we obtain a sharper statistical error bound. Our theoretical findings are supported by numerical simulations. We apply TCSI to the reconstruction of global Total Electron Content (TEC) maps and observe lower reconstruction errors than the compared methods in our experiments.
Read the full abstract ▾
Statistics Theory · Cross-listed from astro-ph.IM
Natural coordinates for constrained correlation functions: Partial autocorrelations and the geometry of positive power spectra
Thomas Erben
Two-point correlation functions are a standard summary statistic in
cosmic shear and large-scale-structure analyses. Their values are, however, constrained: non-negativity of the underlying power spectrum restricts any admissible sequence of correlation coefficients $(r_1,\ldots,r_N)$ to a bounded convex region, described in the one-dimensional case by the recursive interval geometry of Schneider & Hartlap (2009). Their formalism introduces an affine variable $x_n$ that maps the admissible interval for $r_n$, given $r_1,\ldots,r_{n-1}$, to $[-1,+1]$. We identify $x_n$ with the partial autocorrelation coefficient $\alpha_n$ of the associated positive Toeplitz correlation
matrix, a classical quantity in time-series analysis. The partial autocorrelations thus provide natural coordinates on the admissible region, each $\alpha_n$ varying independently in $[-1,+1]$. This identification makes the standard partial-autocorrelation toolbox directly applicable. The admissible intervals for $r_n$, cumbersome to obtain at higher order in the original determinant-based formalism, now follow from an $\mathcal{O}(N^2)$ recursion at arbitrary order. The inverse
hyperbolic tangent used in the quasi-Gaussian likelihood construction of Wilking & Schneider (2013) becomes Fisher's $z$-transformation of partial autocorrelations, providing a classical statistical interpretation of its empirical Gaussianising effect. Numerical experiments illustrate the practical use of these natural coordinates in the tested one-dimensional settings. Higher-dimensional isotropic constraints require additional
geometric input and lie outside the direct scope of this identification.
Read the full abstract ▾
Statistical Methodology
Conformalized Safe Feasible Sets in Uncertain Decision Systems
Yajie Bao, Yinjie Min, Haojie Ren et al. (+1 more)
Safety-critical decision systems often require a downstream optimizer to choose from an unknown feasible set determined by an unobserved label $Y$. Given a context $X$, the goal is to construct a safe subset $D(X)$ contained in the oracle feasible set $A(X,Y)$ with probability at least $1-\alpha$. Existing conformal approaches typically construct a prediction set of the unobserved label $Y$ and retain decisions that are safe for every value in this set. Although valid, this requires a stronger intermediate event than set inclusion. We propose Directed Inclusion Safety Calibration (DISC), a conformal framework that directly controls the probability of this inclusion event by reducing its verification to a scalar critical-inclusion score. Given a pretrained nested family of candidate feasible sets, DISC assigns each labeled observation the smallest nestedness level at which the corresponding subset is contained in $A(X,Y)$, and constructs the safe feasible set using an empirical
quantile at test-time. With data exchangeability, this yields a finite-sample, distribution-free inclusion guarantee. Under two practical set families, we show that DISC produces a safe feasible set containing that obtained by the corresponding calibration baseline. We further develop optimization-based score computation and decision-aware procedures for learning subset families. Experiments across continuous and structured decision problems show that DISC achieves the target inclusion guarantee while producing larger feasible regions.
Read the full abstract ▾
All authors Yajie Bao, Yinjie Min, Haojie Ren, Changliang Zou
Machine Learning (Statistics) · Cross-listed from cs.LG
Prior-Amortized In-Context Bayesian Inference for Generalized Linear Mixed-Effects Models
Alex Kipnis, Marcel Binz and Eric Schulz
Hierarchical data is ubiquitous in the empirical sciences and is most commonly analyzed with generalized linear mixed-effects models (GLMMs).
Bayesian inference for GLMMs yields calibrated uncertainty but requires MCMC; the No-U-Turn Sampler (NUTS) is the gold standard but is slow and must restart from scratch for every new
dataset, model and prior. We introduce metabeta, a pretrained neural network for prior-amortized in-context
Bayesian inference over GLMMs. Unlike previous neural posterior
estimators that fix the prior at training time, metabeta accepts prior families and hyperparameters as inputs at test time, enabling zero-shot generalization. Two set
transformers and conditional normalizing flows mirror the posterior's two-level structure (global parameters shared across groups, local parameters per group). The model is trained on millions of realistic simulated
datasets spanning continuous, binary, and count outcomes. By default, the flow posterior is refined by Independence Metropolis-Hastings against the unnormalized posterior, so its correctness rests on the sampler rather than the network; this yields tuning-free inference two to three orders of magnitude faster than NUTS. Alternatively, the flow can warm-start NUTS, giving nearly identical inference with substantially increased speed and stability. On controlled
benchmarks with ground-truth parameters, metabeta matches NUTS in parameter recovery, calibration and out-of-sample prediction. On out-of-distribution real
datasets, its posteriors closely match those of NUTS across all parameter types, and they remain faithful under misspecified likelihoods and priors, out-of-distribution predictors, collinear designs, and data-poor regimes. The model is open-source and open-weights and thus immediately deployable.
Read the full abstract ▾
Statistical Methodology
A Three-Way Testing Framework for Quantifying Epistemic Calibration Uncertainty in SBI
Luben M. C. Cabezas, Pedro L. C. Rodrigues and Rafael Izbicki
Current experimental scientists increasingly rely on simulation-based inference (SBI) to invert complex models with intractable likelihoods. A primary goal in these settings is to obtain credible regions with valid coverage. While recent model-agnostic conformal calibration methods have succeeded in constructing credible sets with prescribed local
Bayesian coverage, their approximate nature introduces inherent epistemic uncertainty in the calibration process. In this work, we propose a novel tool for diagnosing calibration uncertainty. Our approach is based on a simple three-way hypothesis testing procedure. We demonstrate how this tool can be used to effectively assess necessary simulation budgets for calibration sets and analyze the epistemic uncertainty associated with cutoff estimation.
Read the full abstract ▾
Statistics Theory
Nonparametric Estimation of the Diffusion-Interaction Function in Particle Systems
Chiara Amorino, Francisco Pina and Mark Podolskij
We study the problem of nonparametric estimation of the interaction function in particle systems whose interaction enters through the
diffusion coefficient. The particle system is observed discretely over a fixed time interval, and the interaction function is assumed to belong to a suitable nonparametric function class. We propose an estimation procedure based on empirical risk minimization over finite-dimensional sieve spaces, introducing a computable contrast function that exploits the local
volatility of the observed trajectories. We derive a general non-asymptotic error bound under the intrinsic norm associated with the statistical problem and, focusing the analysis on multiresolution wavelet approximation spaces, we obtain explicit
polynomial convergence rates in two distinct regimes, depending on whether the particle or discretization error dominates. Finally, we establish convergence rates for the classical $L^2$-norm.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
A Distributional Optimisation Perspective on Combining Models in Deep Learning
Congye Wang, Yan Lin, Zheyang Shen et al. (+2 more)
Combining predictions from different models can improve performance at machine learning tasks, but the training of the individual models and the rule used to combine them are typically chosen separately, and by ad hoc means. Recent advances in distributional optimisation (i.e. where the optimisation occurs over the set of probability distributions) offer an opportunity for principled joint training, viewing the collection of models as a discrete distribution whose support points are to be optimised, but the potential of these methods is not well-understood. In this paper we (1) cast two standard combination strategies - ensembles and low-rank adapter averaging - as
entropy-regularised distributional optimisation, observing that the resulting objective is convex in the ensemble case but not in the adapter-averaging case, so that existing convergence guarantees for mean field Langevin dynamics transfer only to the former; (2) assess existing and novel
algorithms for this task, including a functional variant of variational gradient descent; and (3) report an empirical study spanning synthetic classification tasks and fine-tuning of large
language models on a commonsense
reasoning benchmark.
Read the full abstract ▾
All authors Congye Wang, Yan Lin, Zheyang Shen, Matthew A. Fisher, Chris. J. Oates
Statistics Applications
Transporting summary measures of relative effects from randomised trials to the treated patient population: an application to breast cancer endocrine therapy
Bonnie E. Shook-Sa, Paul N. Zivich, Carolyn Taylor et al. (+4 more)
Randomised trials often report relative treatment effects, such as risk ratios and hazard ratios, for trial populations. Clinical decision-making, however, often benefits from estimates of absolute treatment effects in the population eligible for treatment. Trial participants may not represent this target population well, and restrictions on access to individual participant trial data can further complicate absolute effect estimation. Routine care data are often representative of the target population but may be subject to uncontrolled confounding. We consider estimation of the average treatment effect on the treated (ATT), an absolute measure, by combining a representative sample of treated routine care patients with summary measures (i.e., estimated risk or hazard ratios) from either a randomised trial or a meta-analysis of trials. Under marginal or conditional transportability assumptions, the ATT is shown to be identifiable. The implications of collapsibility of the effect measure on transportability are discussed, and plug-in
estimators of the ATT are presented. Simulation studies are used to assess finite sample performance of the
estimators in a range of settings. The proposed methods are applied to estimate the ATT of endocrine therapy on 15-year breast cancer mortality using results from a meta-analysis of randomised trials and England's National Disease Registration Service.
Read the full abstract ▾
All authors Bonnie E. Shook-Sa, Paul N. Zivich, Carolyn Taylor, David Dodwell, Jake Probert, Sarah C Darby, Paul McGale
Machine Learning (Statistics)
Adversarially Robust PAC Learning with Optimal VC Rates
Steve Hanneke and Amirreza Shaeiri
We study the problem of adversarially robust PAC learning. In this framework, the learner observes independent samples from an unknown distribution over $\mathcal{X} \times \{0,1\}$, as in classical PAC learning. However, given a perturbation map $\mathcal{U} : \mathcal{X} \to 2^{\mathcal{X}}$ known to the learner, the goal is to output, with high probability, a predictor that correctly classifies every perturbation $z \in \mathcal{U}(x)$ of most future examples $(x,y)$ drawn from the same underlying distribution. We determine the optimal $\mathcal{U}$-independent sample complexity of this problem in both the realizable and agnostic settings. More specifically, for every concept class $\mathcal{H}$ of $\operatorname{VC}$ dimension $d$, we prove upper bounds of $\mathcal{O} \big( d/\epsilon + \log(1/\delta)/\epsilon \big)$ in the realizable setting and $\mathcal{O} \big( d/\epsilon^2 + \log(1/\delta)/\epsilon^2 \big)$ in the agnostic setting, together with an optimal first-order refinement of the latter. These bounds match the corresponding lower bounds for classical PAC learning. Consequently, and perhaps surprisingly,
adversarial robustness incurs no additional distribution-free statistical cost, uniformly over all perturbation maps. Our bounds improve exponentially on those of [Montasser, Hanneke, and Srebro; COLT '19]. On the technical side, we present short and elementary proofs based on a new algorithmic principle that we call binomial-bagging. We believe that binomial-bagging and its analysis may be of independent interest.
Read the full abstract ▾
Statistics Theory
Conditional Independence Is Not (Quite) Pointwise Testable
Danica J. Sutherland
Shah and Peters showed that a conditional independence test with finite-sample or uniformly-controlled level has only trivial power against any alternative. Most practical tests, however, only claim pointwise asymptotic level. There have been incorrect claims in the literature of conditional independence tests with pointwise asymptotic level and consistency against any alternative; whether such a test actually exists has remained open. We resolve this question. Even restricting to hypotheses with a bounded density on compact subsets of Euclidean spaces, for any sequence of (possibly randomized) tests with pointwise asymptotic level $\alpha$, for every $\varepsilon > 0$ there is a conditionally dependent distribution where the test's limsup power is at most $\alpha + \varepsilon$. This remains true when the level control is required only over conditionally independent distributions with a continuous density and a uniformly continuous conditional law, and the conditionally dependent distributions have smooth densities. On the other hand, the extreme limitation of trivial power against any alternative for uniform-level tests does not apply to tests with pointwise level. We exhibit a test that, without regularity assumptions, has pointwise asymptotic level $\alpha$, strictly higher power for all alternatives, and power tending to one for all distributions whose dependence exceeds a chosen scalar threshold.
Read the full abstract ▾