Machine Learning (Statistics)
Multi-Sensitive Distributional Fairness: Joint Wasserstein Audits and Shapley Allocation
Agathe Fernandes Machado, Charlotte Jamotton and Arthur Charpentier
Fairness audits with several sensitive attributes are often performed one coordinate at a time. This can miss purely intersectional disparities: a score may be independent of each sensitive coordinate while depending on their joint profile. We study distributional demographic parity for a real-valued score using one-dimensional Wasserstein geometry. We distinguish two complementary constructions. Conditional and sequential audits characterize joint parity through their zero sets, but their positive values do not generally decompose a fixed joint Wasserstein disparity. By contrast, full-
cell conditional
quantile functions define a
Hilbert-valued random variable whose variance is the joint disparity. Conditional expectations of this
quantile process induce a monotone information game: every reveal order yields an exact martingale decomposition of the same joint disparity, while Shapley values provide an order-free attribution across sensitive attributes. Under independent sensitive attributes, the game reduces to a Hoeffding-ANOVA decomposition and Shapley splits each interaction energy equally among its members; under dependence, orthogonality is lost and explicit formulas reveal suppression and redundancy effects that marginal audits can hide. We give fixed-grid
estimators and permutation calibration, and illustrate the framework on synthetic examples and ACSIncome. The resulting framework separates three questions that are often conflated: whether joint distributional parity holds, how a fixed joint disparity decomposes along an information path, and how that disparity can be attributed across sensitive coordinates without privileging an ordering.
Read the full abstract ▾
Machine Learning (Statistics)
Generalizing Identifiability Results in Causal Discovery from Complete Data to Missing Data Settings
Matteo Ceriscioli, Shohei Shimizu, Mallika Gupta et al. (+1 more)
While causal discovery methods have matured for complete data, such
algorithms for handling missing values remain limited. We establish general sufficient conditions under which causal graphs remain identifiable (i.e., uniquely determined from the available data distribution) under various missingness categories, including Missing Not At Random, and provide a framework to extend correctness guarantees from causal discovery
algorithms for complete data to missing data settings. Our results show that identifiability of causal graphs with no unobserved confounders is preserved when (1) missingness mechanisms do not causally interact, (2) no variable directly causes its own missingness, and (3) either faithfulness holds for missingness mechanisms, or they follow generalized linear models. Guided by these theoretical results, we introduce two asymptotically correct
algorithms for Additive Noise Models (ANMs) that recover the underlying causal graph and provide consistent parameter estimates.
Read the full abstract ▾
All authors Matteo Ceriscioli, Shohei Shimizu, Mallika Gupta, Karthika Mohan
Statistics Theory
Pointwise minimax lower bounds under a multi-index constraint: the price of an unknown orientation
Nora Serdyukova
We consider nonparametric regression with random design and non-Gaussian noise, where the regression function has the multi-index form $F(x)=f(\Theta x)$ with an unknown link $f:\RR^m\to\RR$ in an anisotropic Hölder class and an unknown
matrix $\Theta$ with $m\le d$ orthonormal rows. For estimation of $F$ at a fixed point we prove that the minimax risk is bounded from below by a multiple of $L^{1/(2\gamma+1)}(n^{-1}\ln n)^{\gamma/(2\gamma+1)}$, where $\gamma$ is the harmonic mean of the smoothness indices of the link divided by $m$. For $m<d$ this extends a Gaussian white noise result to regression. The main new case is $m=d$, where no dimension reduction takes place: the logarithmic factor still appears as soon as the link is anisotropic and the orientation of its axes is unknown. In contrast, when the link is isotropic with smoothness index at most one, the class is rotation invariant up to a constant and the classical rate without logarithm is attained. The logarithmic loss is therefore caused by the unknown orientation of the directions in which the function is least smooth
Read the full abstract ▾
Statistics Theory
Finite-sample certification of coverage regret for budgeted document selection under adaptive stratified auditing
Nora Serdyukova
Selecting a budget-constrained set of documentary fragments, for instance the knowledge base of a
retrieval-augmented generation system, raises a statistical question: how much expected query coverage is lost relative to the best feasible selection when the query distribution is unknown? We construct finite-sample, distribution-free upper confidence bounds for this coverage regret that remain valid after data-dependent selection. A pilot stage may select the document set, screen competitors, build comparison networks and learn an annotation policy. Then an independent audit acquires relevance labels with probabilities that are constant within observable strata. The main result is a simultaneous bound for all retained coverage contrasts. The bound is made observable by empirical-Bernstein variance calibration or by a one-stage construction without calibration, and it accounts for estimated stratum proportions. For maximum-type coverage we prove variance contractions for swaps of redundant fragments, give a condition under which multiscale radii improve on a direct union bound, and show that such redundancy cannot be certified from samples alone. A simulation study confirms validity and quantifies the gains from variance calibration and certified redundancy.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.AI
Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization
Leonardo Torti, Emanuele Rimoldi, Tomer Galanti et al. (+2 more)
With the growing number of LLMs available, selecting which one to use for a
prompt has become a key question. A large part of the routing literature relies on semantically
embedding prompts into a vector space and then applying a lightweight classifier to the embedded representations. These methods perform robustly in-distribution but they struggle to generalize to unseen tasks. By studying the geometry of semantic
embeddings, we trace a clear failure mode: features that determine the neighbors of unseen
prompts are an unreliable proxy for LLM selection. We therefore replace the semantic
embedding with the following map: we run a partial forward pass of Qwen2.5-7B-Instruct on the
prompt and use the hidden state of its last
token at a deep layer as the representation. When applying kNN and other simple classifiers to this new representation, we obtain a clear result in generalization: on average across cost penalties, their performance improves. Moreover, kNN and another simple classifier that we propose outperform every evaluated semantic-
embedding baseline. Our two simple methods also beat a recent SoTA hidden-states based router on RouterBench.
Read the full abstract ▾
All authors Leonardo Torti, Emanuele Rimoldi, Tomer Galanti, Tomaso Poggio, Pierfrancesco Beneventano
Machine Learning (Statistics)
Structure-aware Bayesian full waveform inversion via diffusion-regularized Stein variational gradient descent
Chao Li, Mrinal K. Sen, Sergey Fomel et al. (+1 more)
Full waveform inversion (FWI) is a powerful tool for high-resolution subsurface model reconstruction, but it remains highly nonlinear and ill-posed, especially when the initial model is inaccurate or the acquired data are noisy and incomplete. Annealed Stein variational gradient descent (ASVGD) provides a particle-based
Bayesian framework for FWI by approximating the posterior distribution with an ensemble of velocity models, thereby enabling uncertainty quantification beyond a single deterministic solution. However, conventional ASVGD-FWI commonly relies on simple handcrafted priors, such as Gaussian or smoothing assumptions, which are often insufficient to represent complex geological structures. To address this limitation, we introduce a learned
diffusion model (DM) as an implicit regularization mechanism within the ASVGD-FWI framework. Instead of explicitly evaluating the
diffusion prior density, the trained
diffusion model acts as a learned geological projection operator that guides updated particles toward the distribution of plausible velocity models. In this way, the proposed method combines data consistency from FWI, posterior diversity from ASVGD, and structural realism from the
diffusion prior. Numerical experiments demonstrate that the
diffusion-regularized ASVGD-FWI produces more geologically consistent mean models, suppresses nonphysical artifacts, and yields uncertainty estimates that are more concentrated around structurally complex and poorly constrained regions. These results suggest that learned
diffusion priors can effectively enhance
Bayesian FWI by improving both inversion stability and the interpretability of posterior uncertainty.
Read the full abstract ▾
All authors Chao Li, Mrinal K. Sen, Sergey Fomel, Yangkang Chen
Statistics Applications · Cross-listed from physics.soc-ph
A Sociophysics Approach to Measuring Democracy in the Decision Making of a Group: Three Indices
Sudipto Roy
A group does not usually consult all its members before every decision. This paper develops a quantitative framework for measuring how widely members are consulted in decision making within a group. Two kinds of records are considered: the number of members consulted for each decision and the number of decisions in which each member is consulted. From these records, two participation indices are defined, and their equality is proved analytically. A third index measures the fraction of decisions for which at least a specified percentage of members are consulted. The distribution of consultation levels is described using a discrete maximum-
entropy model from statistical physics. Tail probabilities are used to obtain bounds for the main participation index. This study examines aggregate survey data from two groups. Their observed characteristics are broadly consistent with the patterns found across the full survey. Using binomial and Poisson calculations, the future probability that all members will be consulted for a decision is examined. Inequality in consultation among members is examined through Lorenz curves, the Gini coefficient and a normalised Shannon participation measure. The member-decision structure is represented as a bipartite network. Controlled simulations show that the same overall level of consultation can have different distributions among the members. The framework measures both the overall breadth of consultation and how consultation is distributed among the members. It therefore provides a quantitative sociophysical description of one measurable aspect of democracy in decision making within a group.
Read the full abstract ▾
Machine Learning (Statistics)
Student Copula Markov Networks
Salha Mamane and Cheikh Tidiane Seck
This paper extends the applicability of probabilistic graphical models to non Gaussian data. It establishes a characterization of the pairwise Markov property in undirected probabilistic graphical models in terms of copula densities. Building on this characterization, we derive a necessary condition for the pairwise Markov property in Student copula Markov networks. An Expectation-Conditional Maximization
algorithm (ECM) is provided to facilitate the penalized maximum likelihood estimation and graph selection. The performance of the proposed
algorithm is evaluated through simulation studies, which demonstrate its effectiveness in estimating model parameters and recovering the underlying graph structure.
Read the full abstract ▾
Machine Learning (Statistics)
Neural Supervised Clustering for Learning Individualized Treatment Rules with Many Treatments
Daixi Huang, Yixin Yuan, Yifan Dai et al. (+1 more)
Learning an individualized treatment rule (ITR) with many treatments requires estimating treatment-specific effects from limited observations per arm. Treatments with similar conditional responses can share information, but linear contrast models may miss nonlinear heterogeneity. We propose nonlinear supervised clustering fusion (NSCF), a neural Q-learning framework with a shared nonlinear
covariate representation and low-dimensional treatment
embeddings. Pairwise fusion groups treatments with similar learned contrasts, yielding an ITR and an interpretable partition. An alternating
algorithm combines neural training,
embedding fusion, validation-based partition selection, and unpenalized grouped refitting. NSCF lowers regret relative to a strong neural reference in nonlinear exact-sharing simulations; perturbations characterize the limits of hard sharing. In a melanoma xenograft application, NSCF improves average policy value over the neural reference. These results support nonlinear representation learning with supervised treatment fusion when treatment effects exhibit shared structure.
Read the full abstract ▾
All authors Daixi Huang, Yixin Yuan, Yifan Dai, Yufeng Liu
Statistics Applications · Cross-listed from eess.SY
Global Parameter Identification and Uncertainty Quantification of reduced-order ADM1 using Maximum Likelihood Estimation and Bayesian Inference
Simon Hellmann, Leander Lerch, Terrance Wilms et al. (+6 more)
Anaerobic digestion (AD) models can be challenging to calibrate against real data due to limited practical identifiability, data scarcity, and model complexity. Weighted least squares objective functions and gradient-based solvers using finite differences are often used in literature, but result in suboptimal parameter estimates and large uncertainty bands. This paper identifies the parameters of a reduced-order ADM1 by comparing the frameworks of maximum likelihood estimation (MLE) and
Bayesian inference (BI), and quantifies uncertainties of parameters and outputs. Multirate experimental data are taken from a lab-scale AD reactor fed with agricultural substrates and operated at organic loading rates of 3$\,$g VS $\mathrm{m^{-3}d^{-1}}$. Global optima are found via global and gradient-based local search
algorithms using forward sensitivities. Enhanced scatter search provides the best global search results. These are refined by the Gauss-Newton
algorithm as the best local search, which delivers $R^2$ values of 0.95 for acetic acid and 0.77 for biogas production. BI yields substantially narrower uncertainty bands than MLE and avoids the need for a global search by incorporating prior knowledge about parameter distributions. The search space is captured efficiently by log10 scaling of parameters. Algorithmic details on its implications are provided. Despite a high-resolution lab-scale
dataset, the ADM1-specific inhibition parameter $K_\mathrm{I,nh3}$ is practically not identifiable with MLE, and shows a low output sensitivity, whereas a parameter for the effective ion concentration shows superb practical identifiabilty at high confidence with both MLE and BI. The present work lays a scalable base for calibration of nonlinear AD models by providing reliable simulations. It thus advances model-based monitoring and control applications of multirate processes such as agricultural AD plants.
Read the full abstract ▾
All authors Simon Hellmann, Leander Lerch, Terrance Wilms, Moritz Reumschüssel, Alberto Meola, Félix Delory, Steffi Knorn, Stefan Streif, Sören Weinrich
Statistical Computation · Cross-listed from cs.LG
Theoretical and Practical Advances in Hierarchical Bayesian Deep Learning Methodology
Neil K. Chada, Simon Cotter, Kody J. H. Law et al. (+1 more)
Reliable uncertainty quantification remains an important challenge in AI. Hierarchical
Bayesian methods provide a principled framework for this by representing unknown quantities through posterior distributions. Joint inference of neural-network parameters and prior hyperparameters is challenging for conventional Markov chain Monte Carlo (MCMC) methods due to strong dependencies. In this work, we introduce the multilevel particle marginal Metropolis-Hastings (MLPMMH) method, which combines pseudo-marginal inference for the hyperparameters with coupled sequential Monte Carlo (SMC) across hierarchy of network-width truncations for conditional parameter inference. The hierarchy is induced by a trace-class neural network (TNN) prior with width-dependent decay, motivated empirically by the compressible structure observed in trained neural-network parameters. Under standard multilevel coupling assumptions, we establish that MLPMMH achieves mean-square error (MSE) $\mathcal{O}(\varepsilon^2)$ at expected computational cost $\mathcal{O}(\varepsilon^{-2})$, and verify the predicted rate on a
Bayesian regression problem. To address larger-scale applications, we further introduce S-MLPMMH, a practical approximation that replaces the exact sampling components with a PCA-based hierarchy and scalable
Bayesian Monte Carlo. Experiments on MNIST demonstrate competitive predictive performance and uncertainty quantification while exploiting the parallel structure of the approximation. Software has been made publicly available at https://github.com/submission-code-anon/mlpmmh-anon.
Read the full abstract ▾
All authors Neil K. Chada, Simon Cotter, Kody J. H. Law, Xinzhu Liang
Machine Learning (Statistics)
Conformalized Physics-Informed Weak Regression for Parameter Estimation in Epidemic Models
Allan Peter Engsig-Karup
Compartment models of infectious diseases are often linear in coefficients that are physical parameters or products or ratios of them. We exploit this with closed-form least squares and equip the estimates with distribution-free uncertainty. Physics-informed regression (PIR) fits the parameters to finite-difference derivatives of the data; we introduce physics-informed weak regression (PIWR), which integrates the model in time instead, needs no derivatives of the data, estimates the initial state jointly and is high-order accurate. To quantify uncertainty we wrap both
estimators in split conformal prediction, with a calibration population generated by the physics-based model itself, giving simultaneous intervals for all parameters with a finite-sample coverage guarantee. A residual-adaptive factor, computed without ground truth, scales the intervals to each data set (PIWR-RACC). On the eleven
benchmark models of disease-informed neural networks (DINNs), PIWR or PIR-FD6 is one to four orders of magnitude more accurate than the reported DINN results wherever the sampling resolves the dynamics, in about a millisecond per fit instead of minutes to hours of training, and the 90% intervals achieve 87-92% coverage in all 88 settings. On the S3I3R model of the original PIR study, PIWR lowers the reported PIR errors by about three orders of magnitude. On real data from the 1978 boarding-school influenza outbreak, a variable-projection variant gives calibrated 95% intervals that overlap those of a
Bayesian analysis. On a time-varying transmission
benchmark, a B-spline expansion of beta(t) keeps PIR and PIWR linear; they are more accurate than physics-informed neural networks in three of four cases and comparable in the noisiest, with simultaneous bands for beta(t). The complete conformal calibration takes less than two minutes for every
benchmark.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
What Information Buys in Active Inference
Melih Kandemir and Mustafa Mert Çelikok
Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscience, while the established RL theory builds regret guarantees on confidence sets. This situation leaves open the conditions under which the information-driven exploration found in nature is more advantageous than the exploration of the established RL theory. We put the state-of-the-art methods of these two paradigms, Active Inference (AIF) and Bayes-UCBVI, on an equal footing: both act through the same state abstraction of a tabular MDP with $S$ states, and their greedy forms differ only in the exploration term. We find that AIF's information gain decays too fast to serve as a confidence width, locking greedy
agents that explore by it out of the optimal action. Calibrating the information gain by a square root yields calibrated AIF, which attains regret of order $\sqrt T$ for an exact abstraction, but no better guarantee than Bayes-UCBVI for a fixed reward. When an adversary chooses rewards from the
agent's data, Bayes-UCBVI's value-specific widths lock out, whereas calibrated AIF pays a factor $\sqrt S$, which no goal-memoryless
agent avoids on some MDP whose start states the adversary also chooses. Calibrated information gain thus acts as a reward-agnostic $\ell_1$ confidence set, preferable to value-specific widths when rewards are chosen from the
agent's data, not when they are fixed.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
Shashank Sule, Xinyue Yu, Boonam Shin et al. (+3 more)
Understanding the mechanism behind why
diffusion models memorize their training data remains an open question in generative modeling. In this paper, we examine this question from two perspectives: (a) the model-level perspective concerning the onset of memorization in a model during training, and (b) the sample-level perspective concerning the proclivity of an individual sample to be memorized by a given model. We show that the so-called input loss
curvature, or the Frobenius norm of the Hessian of the
diffusion model's training loss, is an effective metric for quantifying both perspectives on memorization. In particular, we empirically demonstrate that input
curvatures averaged across training and validation samples can identify when a model is likely to start regurgitating its training data when tracked over training time. However, this metric is not sufficient for measuring an individual sample's likelihood of being memorized. To address this perspective, we introduce a counterfactual definition of memorization in
diffusion models, extending the analogous notion from supervised learning proposed by [Feldman, 2019]. We prove that this counterfactual memorization is controlled by the input
curvature of the leave-one-out model, thus establishing a direct theoretical link between a sample's local loss landscape and its generation likelihood. We validate our derived bounds through several empirical experiments and show that input-dependent
curvature provides a proxy for memorization.
Read the full abstract ▾
All authors Shashank Sule, Xinyue Yu, Boonam Shin, Yuxuan Liu, Abolfazl Hashemi, Hayden Schaeffer
Machine Learning (Statistics)
Discrete Diffusion Models are Minimax Optimal Under Sparsity
Luca Blunda, Tyler Farghly, Umut Simsekli et al. (+1 more)
Discrete
diffusion models have emerged as a leading alternative to autoregressive models for generating text and other categorical data, yet their theoretical foundations remain less developed than those of their continuous counterparts. For continuous
diffusion models, a prominent line of theory suggests that these models can adapt to low-dimensional structure in the data, thereby improving statistical efficiency and generalization. In this work, we investigate the discrete analogue of this phenomenon by studying the statistical rates of discrete
diffusion models under sparsity assumptions. By analyzing a nonparametric
estimator derived from a regularized score-
entropy objective, we establish minimax-optimal rates in the sparse regime. The proposed regularization can be viewed as an adaptation of label smoothing techniques introduced for classification, and does not require the knowledge of the sparsity parameter. This shows that discrete
diffusion models can adapt intrinsically to sparsity.
Read the full abstract ▾
All authors Luca Blunda, Tyler Farghly, Umut Simsekli, Alain Durmus
Statistics Theory · Cross-listed from math.PR
On rigidity of infinite exchangeable sequences
Maximillian Newman
We ask when the law of an infinite exchangeable sequence is uniquely determined by a finite-dimensional marginal. For mixtures of a probability kernel, a weak localization condition makes three coordinates determine the directing law. Completeness alone makes four coordinates sufficient. Neither order can be lowered in general. For exponential kernels with a nonnegative sufficient statistic, a Müntz-Szász support condition implies three coordinates suffice. In the scalar case, this result is sharp. We also characterize when completeness holds simultaneously for all scalar mixing laws by a weighted Müntz density condition and show that statistic support alone need not decide completeness. These methods yield sharp finite-dimensional refinements of Freedman's
theorem on rotatable sequences, namely that spherical
symmetry of three coordinates makes an infinite exchangeable sequence rotatable while
symmetry of the first two coordinates is in general insufficient. Finally, for energy-radial densities we give a complete-monotonicity criterion for infinite extendibility and determine the exact uniqueness thresholds for $\ell_q$-norm exponential laws.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CR
Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries
Inderjeet Singh
Finite safety assessments cannot exclude an adversary that preserves unattacked behavior until certification and activates afterward. We characterize this limitation for embodied and tool-use AI. An adversary that matches the complete observation law until observable, irrevocable certification and retains intervention access afterward attains $v\mathbb{E}[(n-S_0)_+]$ expected post-certification events over $n$ trials, where $v$ is the active event probability and $S_0$ the unattacked certification time. The result covers adaptive and privately randomized evaluators. Switching at certification is exactly optimal for a fixed likelihood-ratio test under memoryless, event-increasing interventions without a binding intervention budget. A positive clean-certification requirement forces linear worst-case expected excess when intervention raises event rates over an authorized deployment horizon, even if certification is deferred to the deadline. In simulated RoCo collaboration, switching yields commanded-separation violations in 34 of 36 post-certification episodes. Preregistered randomized AgentDojo trials estimate 1.98 additional security violations per four-task continuation (95% interval [1.67, 2.29]). An
adversarial language-model controller infers hidden certification from noisy evaluation receipts and times intervention accordingly. With permanent revocation, a mixture likelihood-ratio monitor bounds cumulative excess above a prespecified reference rate at square-root order, up to logarithmic factors, on every sample path. A single fixed alternative can permit linear expected excess under stationary attacks. Against a controller observing the monitor's rule and statistic, live mixture monitoring reduces violations from 12.00 to 3.81 per 24 assigned task opportunities. Revocation also reduces tasks executed.
Read the full abstract ▾
Machine Learning (Statistics)
Semi-Amortized Bayesian Inference with Reusable Local Posteriors
Niek Stevenson, Paul-Christian Bürkner, Andrew Heathcote et al. (+1 more)
Amortized
Bayesian inference uses simulations to perform
Bayesian estimation when likelihoods are expensive or analytically unavailable. However, learning the full joint posterior is challenging in high-dimensional models with complex dependencies. We introduce a semi-amortized method that divides parameters into smaller blocks, and uses conditional independence to identify the information needed to infer each block. We train neural
estimators for blocks whose posterior updates are computationally expensive or frequently repeated. Each
estimator learns from simulations of the relevant model components to approximate the block's full conditional posterior, given the relevant observations and the values of the variables on which it depends. A Markov chain then alternates direct draws from these
estimators with conventional updates of the remaining variables. Each neural update uses the current conditioning values and requires neither likelihood evaluation nor an acceptance-rejection step. Where the same conditional structure recurs, one
estimator can be reused across blocks and
datasets, with an output dimension determined by the block rather than the full model. We illustrate the approach in high-dimensional and computationally expensive hierarchical and spatiotemporal models.
Read the full abstract ▾
All authors Niek Stevenson, Paul-Christian Bürkner, Andrew Heathcote, Stefan T. Radev
Machine Learning (Statistics)
Wong–Zakai Convergence and FNO Learning for Stochastic Volterra Equations
Wonjae Lee, Jiuk Jang and Hyungbin Park
This paper studies Wong–Zakai convergence and
Fourier neural operator learning for
stochastic Volterra integral equations (SVIEs) describing random systems with
memory. A Wong–Zakai
theorem establishes convergence of pathwise solutions driven by piecewise linear approximations of Brownian motion to the corresponding SVIE solution. This convergence provides a foundation for using mirror-padded
Fourier neural operators to learn the solution operator mapping initial conditions and driving noise to entire solution trajectories. Numerical experiments demonstrate competitive prediction accuracy and generalization to temporal grids finer than those used for training, while assessing the influence of padding and computational cost. As a follow-up to \citet{LKP:26}, this study extends their operator-learning framework to SVIEs through separate convergence and approximation analyses tailored to convolutional
memory.
Read the full abstract ▾
Statistical Methodology
Linear regression with probabilistic networks for network-linked data
Jingnan Zhang, Shiwen Ye and Danyang Huang
Network-linked data combine a network with
covariates and responses observed for its nodes. Common regression approaches either model dependence among neighboring responses using the observed network or capture similarities associated with network structure without explicitly modeling response dependence. We propose a probabilistic network-linked data analysis framework that incorporates both features while accounting for randomness in network formation. The model uses normalized connection probabilities to weight peer responses and latent positions to capture network cohesion. A two-step likelihood procedure first estimates the network structure and then fits the response model. We establish theoretical guarantees for network recovery in sparse networks with heterogeneous node effects, and prove consistency and joint asymptotic normality of the regression
estimators subject to suitable conditions. The analysis also clarifies when estimating the network preserves the classical convergence rate for spatial autoregressive models. Simulation studies demonstrate the estimation performance of the method and assess its inference accuracy. An analysis of diplomatic network data illustrates its practical application.
Read the full abstract ▾
Machine Learning (Statistics)
Delay Complexity Dimension: Learning with Delayed and Censored Labels
Madhava Gaikwad
We study PAC learning of binary classifiers when labels arrive after random delays and are censored beyond an observation window $W$. How many samples does this observation model require? We propose a distribution-dependent answer, the Delay Complexity Dimension (DCD). The DCD combines the VC dimension with the exact label-information rate of the observation channel. Every learner needs $\Omega(\mathrm{DCD}/\varepsilon^2)$ samples. An oracle pseudo-label ERM matches this rate on the instance family of the lower bound, up to constants and logarithmic factors. On that family the DCD is within a constant factor of the classical censored-ERM complexity, and whether the DCD rate is achievable where the two separate remains open. The theory also yields a quantization scheme under which delay information lowers the residual label variance, an expedition policy with a $(1-1/e)$ guarantee for its version-space objective, and a phenomenon with no analogue in the no-delay setting. Under a fixed collection horizon, a moderate window can strictly beat immediate labels. Experiments on synthetic data and the Criteo conversion logs support these predictions. A budget-matched test of a companion labeler-diversity prediction is reported as a negative result.
Read the full abstract ▾
Machine Learning (Statistics)
Joint Neural Posterior Estimation and Model Selection for Chemical Reaction Networks
J. Arturo Esquivel, Jorge A. Campos-Gonzalez-Angulo, Sergio Pablo-García et al. (+2 more)
Simulation-based
Bayesian inference for Chemical Reaction Networks (CRNs) must simultaneously solve two computationally demanding problems: discriminating between overlapping reaction pathways and estimating reaction rate constants. Classical approaches, such as Sequential Monte Carlo ABC, rely on handcrafted summary statistics and scale poorly across large parameter spaces. To overcome this, we introduce a neural inference architecture. Our pipeline utilizes shared 1D-CNNs and a permutation-invariant Set
Attention mechanism to learn low-dimensional summary statistics from variable-length, multi-experiment time-series data. These statistics simultaneously condition a model selection network to compute posterior model probabilities and model-specific conditional Invertible Neural Networks (cINNs) that estimate parameter posterior distributions. We demonstrate the practical utility of our approach on a real protodeboronation example, illustrating how it handles structural ambiguity and extracts kinetic parameters from competing catalyst deactivation pathways.
Read the full abstract ▾
All authors J. Arturo Esquivel, Jorge A. Campos-Gonzalez-Angulo, Sergio Pablo-García, Alán Aspuru-Guzik, Vianey Leos-Barajas
Statistics Applications
Discrete-Time Survival Modelling with Dynamic Industrial Graphs for Mortgage Credit Risk
Shiqi Fang
Mortgage credit risk may exhibit dependence across economically connected areas, even when they are geographically distant. However, current models typically represent cross-area dependence through fixed geographical neighbourhoods, potentially overlooking economic connections that extend beyond physical proximity and evolve over time. These models may fail to capture shared risks among geographically distant but economically similar areas, and cannot describe a changing dependence of risk; it therefore risks retaining economically obsolete connections while omitting newly emerging ones. This paper uses similarities of industrial composition as a measure of economic proximity and develops a discrete-time survival model with dynamic industrial graphs to capture evolving cross-area dependence in mortgage default risk. The resulting graphs are reconstructed annually, allowing both their
topology and edge weights to change as regional industrial compositions evolve. Jointly combining a proper conditional autoregressive process with a first-order autoregressive process governs the temporal cross-area dependence and the temporal persistence in latent risk simultaneously. The model is estimated using Integrated Nested Laplace Approximation and evaluated on U.S. mortgage loan-level data against the static industrial graph specifications. The results show that the proposed dynamic model achieves better in-sample fit and out-of-sample discrimination, while its estimated parameters indicate substantial graph-induced cross-area dependence and temporal persistence. These findings demonstrate the value of allowing economically defined neighbourhoods to evolve when modelling mortgage default risk.
Read the full abstract ▾
Statistical Methodology
Designing for Prediction: Planning Formulas for Expected Out-of-Sample $R^2$ and $r$
Elizabeth Tipton
Design calculations let researchers gauge, before data is collected, whether a planned study is likely to detect an effect. In this paper, we develop such calculations for predictive models using ordinary least squares and ridge regression under leave-one-out cross-validation. To do so, we derive closed-form expressions relating the expected out-of-sample $R^2$ and the expected correlation between observed and predicted outcomes to four design-stage quantities: sample size, number of predictors, an assumed population $R^2$ ($\rho^2_{in}$) and a design efficiency factor $D^*$ (which summarizes how leverage is distributed across observations). Unlike the classical shrinkage formulas, which depend only on sample size and dimension, these are sensitive to how
covariates are arranged – to imbalance and skew. A simulation study assesses the accuracy of the formulas against realized values under a variety of data generating mechanisms. We apply the methods to a published study predicting personality from smartphone behavior and show that the design formulas would have indicated, before any analyses were run, that least squares could not be expected to beat the sample mean while ridge could. An R package, predpower, implements the calculations.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
A Rate Transfer Method: Convergence Rates in Stochastic Gradient Descent via Stochastic Calculus
Neil Walton
We develop a rate-transfer method that converts finite-time Lyapunov estimates for an associated
stochastic differential equation (SDE) into finite-time convergence bounds for
stochastic gradient
algorithms. Under generic moment and smoothness conditions, the convergence analysis of the
stochastic gradient method reduces to a calculus argument verifying a Lyapunov generator inequality for the SDE. The method recovers established rates for classical
stochastic gradient methods and provides a way to establish convergence rates for new optimization
algorithms through calculus arguments applied to their associated SDEs.
Read the full abstract ▾
Statistical Methodology
From Fourier Amplitudes to Persistence Images for Impact-Echo Defect Detection
Keunsu Kim, Suyoung Choi and Meiyan Kang
Impact-echo testing is a nondestructive testing method used to detect internal defects in concrete structures. However, boundary reflections, structured interference, and measurement noise can produce complex
Fourier amplitude spectra, making defect classification difficult. We constructed a two-dimensional frequency–amplitude persistence image (FA-PI) from the
Fourier amplitude spectrum. Using a fixed parameterization of exact multi-parameter persistent homology, each retained
Fourier component is represented with frequency-bin location as one coordinate and amplitude as the other, and Gaussian kernels are then applied in this two-dimensional space. This construction has a direct analytical connection to the corresponding one-dimensional Gaussian-smoothed
Fourier amplitude spectrum. FA-PI was evaluated on an experimental backfill-void
dataset and a public bridge-deck impact-echo
dataset, and compared with raw and Gaussian-smoothed
Fourier amplitude spectra using a linear classifier with a Softmax output and identical signal-level data splits. The evaluation included original signals and signals corrupted by Gaussian noise, sinusoidal interference, or mixed noise, with matching noise types and signal-to-noise ratios during training, validation, and testing. Performance was assessed using accuracy and macro-averaged precision, recall, and F1-score. Across the two
datasets, FA-PI consistently outperformed the raw and Gaussian-smoothed
Fourier baselines in the noisy-condition summaries. These findings indicate that representing
Fourier spectral components in a two-dimensional frequency–amplitude space can improve impact-echo defect classification beyond frequency-axis smoothing alone.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.IT
Optimal Support Size Testing
Qisheng Wang
We present an optimal tester for support size that determines whether an unknown probability distribution is supported on $n$ elements or $\varepsilon$-far away (in total variation distance) with sample complexity $\Theta(\frac{n}{\varepsilon\log(n)})$, improving the prior best $O(\frac{n}{\varepsilon\log(n)} \log(\min\{\frac{1}{\varepsilon}, n\}))$ due to Ferreira Pinto Jr. and Harms (STOC 2025). As a generalization, we provide a pretty good lower bound on support size.
Read the full abstract ▾
Statistics Theory
Sampling with collective variables: Relative error analysis for umbrella sampling under general conditions
Holden Lee, Matheau Santana-Gijzen and Jonathan Weare
We consider the problem of estimating the expectation of a function (observable) with respect to a high-dimensional probability distribution, under the assumption that a low-dimensional collective variable map captures all the directions of slow mixing for the distribution. More precisely, under the assumption that the fibers of the collective variable map satisfy a log-Sobolev inequality, we establish non-asymptotic relative-error guarantees for this problem using the eigenvector method for umbrella sampling (EMUS). Our analysis applies to both log-Lipschitz observables and indicator of convex sets, and gives complexity bounds that are
polynomial in regularity parameters and inverse relative accuracy, and exponential in the dimension of the collective variable space. As a relative error guarantee, our bounds apply to rare event estimation. We consider Gaussian biasing windows defined on a low-dimensional collective variable space and analyze a modified sparse
matrix version of EMUS. The bounds quantify the effect of the map through mixing of the conditional distribution and fluctuation of the observable along the fibers.
Read the full abstract ▾