Machine Learning (Statistics) · Cross-listed from cs.AI
Evaluating the Ability of Explanations to Disambiguate Models in a Rashomon Set
Kaivalya Rawal, Eoin Delaney, Zihao Fu et al. (+2 more)
Explainable artificial intelligence (XAI) is concerned with producing explanations indicating the inner workings of models. For a Rashomon set of similarly performing models, explanations provide a way of disambiguating the behavior of individual models, helping select models for deployment. However explanations themselves can vary depending on the explainer used, and need to be evaluated. In the paper "Evaluating Model Explanations without Ground Truth", we proposed three principles of explanation evaluation and a new method "AXE" to evaluate the quality of feature-importance explanations. We go on to illustrate how evaluation metrics that rely on comparing model explanations against ideal ground truth explanations obscure behavioral differences within a Rashomon set. Explanation evaluation aligned with our proposed principles would highlight these differences instead, helping select models from the Rashomon set. The selection of alternate models from the Rashomon set can maintain identical predictions but mislead explainers into generating false explanations, and mislead evaluation methods into considering the false explanations to be of high quality. AXE, our proposed explanation evaluation method, can detect this
adversarial fairwashing of explanations with a 100% success rate. Unlike prior explanation evaluation strategies such as those based on model sensitivity or ground truth comparison, AXE can determine when protected attributes are used to make predictions.
Read the full abstract ▾
All authors Kaivalya Rawal, Eoin Delaney, Zihao Fu, Sandra Wachter, Chris Russell
Statistics Applications · Cross-listed from eess.SP
Stable Filtering for Efficient Dimensionality Reduction of Streaming Manifold Data
Nicholas P. Bertrand, Eva Yezerets, Han Lun Yap et al. (+2 more)
Many areas in science and engineering now have access to technologies that enable the rapid collection of overwhelming data volumes. While these
datasets are vital for understanding phenomena from physical to biological and social systems, the sheer magnitude of the data makes even simple storage, transmission, and basic processing highly challenging. To enable efficient and accurate execution of these data processing tasks, we require new dimensionality reduction tools that 1) do not need expensive, time-consuming training, and 2) preserve the underlying geometry of the data that has the information required to understand the measured system. Specifically, the geometry to be preserved is that induced by the fact that in many applications, streaming high-dimensional data evolves on a low-dimensional attractor
manifold. Importantly, we may not know the exact structure of this
manifold a priori. To solve these challenges, we present randomized filtering (RF), which leverages a specific instantiation of randomized dimensionality reduction to provably preserve non-linear
manifold structure in the embedded space while remaining data-independent and computationally efficient. In this work we build on the rich theoretical promise of randomized dimensionality reduction to develop RF as a real, practical approach. We introduce novel methods, analysis, and experimental verification to illuminate the practicality of RF in diverse scientific applications, including several simulated and real-data examples that showcase the tangible benefits of RF.
Read the full abstract ▾
All authors Nicholas P. Bertrand, Eva Yezerets, Han Lun Yap, Adam S. Charles, Christopher J. Rozell
Statistics Applications
Reliability Modeling of Single-Sided Aluminized Polyimide Films during Storage Considering Stress-Induced Degradation Mechanism Transition
Shi-Shun Chen, Dong-Hua Niu, Wen-Bin Chen et al. (+4 more)
Single-sided aluminized polyimide films (SAPF) are widely used in thermal management of aerospace systems. Although the reliability of SAPF in space environments has been thoroughly studied, its reliability in ground environments during storage is always ignored, potentially leading to system failure. This paper aims to investigate the reliability of SAPF in storage environments, focusing on the effects of temperature and relative humidity. Firstly, the relationship between the performance degradation of SAPF and aluminum corrosion is identified. Next, considering the presence of two distinct stages in the influence of temperature on aluminum corrosion, a novel degradation model accounting for the degradation mechanism transition is developed. Additionally, a parameter analysis method is proposed for determining SAPF degradation mechanism based on experimental data. Then, a statistical analysis method incorporating an improved rime optimization
algorithm is employed for parameter estimation, and the reliability model is established. Experimental results demonstrate that the proposed method effectively identifies two distinct stages in the impact of temperature on SAPF performance degradation. Furthermore, the proposed degradation model outperforms traditional degradation models with unchanged degradation mechanism in terms of degradation prediction accuracy, extrapolation capability and robustness, indicating its suitability for describing the degradation pattern of SAPFs.
Read the full abstract ▾
All authors Shi-Shun Chen, Dong-Hua Niu, Wen-Bin Chen, Jia-Yun Song, Ya-Fei Zhang, Xiao-Yang Li, Enrico Zio
Machine Learning (Statistics) · Cross-listed from econ.EM
Automatic debiased machine learning and sensitivity analysis for sample selection models
Jakob Bjelac, Victor Chernozhukov, Phil-Adrian Klotz et al. (+2 more)
In this paper, we extend the Riesz representation framework to causal inference under sample selection, where both treatment assignment and outcome observability are non-random. Formulating the problem in terms of a Riesz representer enables stable estimation and a transparent decomposition of omitted variable bias into three interpretable components: a data-identified scale factor, outcome confounding strength, and selection confounding strength. For estimation, we employ the ForestRiesz
estimator, which accounts for selective outcome observability while avoiding the instability associated with direct propensity score inversion. We assess finite-sample performance through a simulation study and show that conventional double machine learning approaches can be highly sensitive to tuning parameters due to their reliance on inverse probability weighting, whereas the ForestRiesz
estimator delivers more stable performance by leveraging automatic debiased machine learning. In an empirical application to the gender wage gap in the U.S., we find that our ForestRiesz approach yields larger treatment effect estimates than a standard double machine learning approach, suggesting that ignoring sample selection leads to an underestimation of the gender wage gap. Sensitivity analysis indicates that implausibly strong unobserved confounding would be required to overturn our results. Overall, our approach provides a unified, robust, and computationally attractive framework for causal inference under sample selection.
Read the full abstract ▾
All authors Jakob Bjelac, Victor Chernozhukov, Phil-Adrian Klotz, Jannis Kueck, Theresa M. A. Schmitz
Machine Learning (Statistics)
Robust low-rank estimation with multiple binary responses using pairwise AUC loss
The Tien Mai
Multiple binary responses arise in many modern data-analytic problems. Although fitting separate logistic regressions for each response is computationally attractive, it ignores shared structure and can be statistically inefficient, especially in high-dimensional and class-imbalanced regimes. Low-rank models offer a natural way to encode latent dependence across tasks, but existing methods for binary data are largely likelihood-based and focus on pointwise classification rather than ranking performance. In this work, we propose a unified framework for learning with multiple binary responses that directly targets discrimination by minimizing a surrogate loss for the area under the ROC curve (AUC). The method aggregates pairwise AUC surrogate losses across responses while imposing a low-rank constraint on the coefficient
matrix to exploit shared structure. We develop a scalable projected gradient descent
algorithm based on truncated singular value decomposition. Exploiting the fact that the pairwise loss depends only on differences of linear predictors, we simplify computation and analysis. We establish non-asymptotic convergence guarantees, showing that under suitable regularity conditions, leading to linear convergence up to the minimax-optimal statistical precision. Extensive simulation studies demonstrate that the proposed method is robust in challenging settings such as label switching and data contamination and consistently outperforms likelihood-based approaches.
Read the full abstract ▾
Statistical Methodology
Finite-sample Valid Permutation Inference under Multi-way Clustering
Wenxuan Guo, Panos Toulis and Yuhao Wang
Econometric applications with multi-way clustering often feature a small number of effective clusters or heavy-tailed data, making standard cluster-robust and bootstrap inference unreliable in finite samples. In this paper, we develop a framework for finite-sample valid permutation inference in linear regression with multi-way clustering under an assumption of conditional exchangeability of the errors. Our assumption is closely related to the notion of separate exchangeability studied in earlier work, but can be more realistic in many economic settings as it imposes minimal restrictions on the
covariate distribution. We construct permutation tests of significance that are valid in finite samples and establish theoretical power guarantees, in contrast to existing methods that are justified only asymptotically. While our main presentation is under exchangeable errors, our framework encompasses invariance-based tests that remain valid under general forms of heteroskedastic errors. We also extend our methodology to settings with missing data and derive power results that reveal phase transitions in detectability. Through simulation studies, we demonstrate that the proposed tests maintain correct size and competitive power, while standard cluster-robust and bootstrap procedures can exhibit substantial size distortions.
Read the full abstract ▾
Statistical Methodology
Flexible modeling of nonnegative continuous data: Box-Cox symmetric regression and its zero-adjusted extension
Rodrigo M. R. de Medeiros and Francisco F. Queiroz
The Box-Cox symmetric distributions constitute a broad class of probability models for positive continuous data, offering flexibility in modeling skewness and tail behavior. Their parameterization allows a straightforward
quantile-based interpretation, which is particularly useful in regression modeling. Despite their potential, only a few specific distributions within this class have been explored in regression contexts, and zero-adjusted extensions have not yet been formally addressed in the literature. This paper formalizes the class of Box-Cox symmetric regression models and introduces a new zero-adjusted extension suitable for modeling data with a non-negligible proportion of observations equal to zero. We discuss maximum likelihood estimation, assess finite-sample performance through simulations, and develop diagnostic tools including residual analysis, local influence measures, and goodness-of-fit statistics. An empirical application on basic education expenditure illustrates the models' ability to capture complex patterns in zero-inflated and highly skewed nonnegative data. To support practical use, we developed the new BCSreg R package, which implements all proposed methods.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.PR
A Sharp Universality Dichotomy for the Free Energy of Spherical Spin Glasses
Taegyun Kim
We study the free energy for pure and mixed spherical $p$-
spin models with i.i.d.\ disorder. In the mixed case, each $p$-interaction layer is assumed either to have regularly varying tails with exponent $\alpha_p$ or to satisfy a finite $2p$-th moment condition. For the pure spherical $p$-
spin model with regularly varying disorder of tail index $\alpha$, we introduce a tail-adapted normalization that interpolates between the classical Gaussian scaling and the extreme-value scale, and we prove a sharp universality dichotomy for the quenched free energy. In the subcritical regime $\alpha<2p$, the thermodynamics is driven by finitely many extremal couplings and the free energy converges to a non-degenerate random limit described by the NIM (non-intersecting monomial) model, depending only on extreme-order statistics. At the critical exponent $\alpha=2p$, we obtain a random one-dimensional TAP-type variational formula capturing the coexistence of an extremal spike and a universal Gaussian bulk on spherical slices. In the supercritical regime $\alpha>2p$ (more generally, under a finite $2p$-th moment assumption), the free energy is universal and agrees with the deterministic Crisanti–Sommers/Parisi value of the corresponding Gaussian model, as established in [Sawhney-Sellke'24]. We then extend the subcritical and critical results to mixed spherical models in which each $p$-layer is either heavy-tailed with $\alpha_p\le 2p$ or has finite $2p$-th moment. In particular, we derive a TAP-type variational representation for the mixed model, yielding a unified universality classification of the quenched free energy across tail exponents and mixtures.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
Systemic Risk Surveillance
Timo Dimitriadis and Yannick Hoga
Following several episodes of financial
market turmoil in recent decades, changes in systemic risk have drawn growing
attention. Therefore, we propose surveillance schemes for systemic risk, which allow to detect misspecified systemic risk forecasts in an "online" fashion. This enables daily monitoring of the forecasts while controlling for the accumulation of false test rejections. Such online schemes are vital in taking timely countermeasures to avoid financial distress. Our monitoring procedures allow multiple series at once to be monitored, thus increasing the likelihood and the speed at which early signs of trouble may be picked up. The tests hold size by construction, such that the null of correct systemic risk assessments is only rejected during the monitoring period with (at most) a pre-specified probability. Monte Carlo simulations illustrate the good finite-sample properties of our procedures. An empirical application to US banks during multiple crises demonstrates the usefulness of our surveillance schemes for both regulators and financial institutions.
Read the full abstract ▾
Statistical Methodology
Sparsifying transform priors in Gaussian graphical models
Marcus Gehrmann and Håkon Tjelmeland
Bayesian methods constitute a popular approach for estimating the conditional independence structure in Gaussian graphical models, since they can quantify the uncertainty through the posterior distribution. Inference in this framework is typically carried out with Markov chain Monte Carlo (MCMC). However, the most widely used choice of prior distribution for the precision
matrix, the so called G-Wishart distribution, suffers from an intractable normalizing constant, which gives rise to the problem of double intractability in the updating steps of the MCMC
algorithm. In this article, we propose a new class of prior distributions for the precision
matrix, termed ST priors, that allow for the construction of MCMC
algorithms that do not suffer from double intractability issues. A realization from an ST prior distribution is obtained by applying a sparsifying transform on a
matrix from a distribution with support in the set of all positive definite
matrices. We carefully present the theory behind the construction of our proposed class of priors and also perform some numerical experiments, where we apply our methods on a human gene expression
dataset. The results suggest that our proposed MCMC
algorithm is able to converge and achieve acceptable mixing when applied on the real data.
Read the full abstract ▾
Statistics Theory · Cross-listed from quant-ph
Sample Complexity of Composite Quantum Hypothesis Testing
Jacob Paul Simpson, Efstratios Palias and Sharu Theresa Jose
This paper investigates symmetric composite binary
quantum hypothesis testing (QHT), where the goal is to determine which of two uncertainty sets contains an unknown
quantum state. While asymptotic error exponents for this problem are well-studied, the finite-sample regime remains poorly understood. We bridge this gap by characterizing the sample complexity – the minimum number of state copies required to achieve a target error level. Specifically, we derive lower bounds that generalize the sample complexity of simple QHT and introduce new upper bounds for various uncertainty sets, including of both finite and infinite cardinalities. Notably, our upper and lower bounds match up to universal constants, providing a tight characterization of the sample complexity. Finally, we extend our analysis to the differentially private setting, establishing the sample complexity for
privacy-preserving composite QHT.
Read the full abstract ▾
Statistical Methodology · Cross-listed from math.OC
Truncated Multidimensional Trigonometric Moment Problem: A Choice of Bases and the Unique Solution
Guangyu Wu and Anders Lindquist
In this paper, we resolve the Truncated Multidimensional Trigonometric Moment Problem (TMTMP) from a system and signal processing perspective, which serves as the foundation for the Multidimensional Rational Covariance Extension Problem (RCEP). While standard mathematical TMTMPs focus on the existence of atomic measure solutions, system identification requires analytic rational solutions with positive
polynomial coefficients. To overcome the long-standing challenge of characterizing the positive feasible domain under general bases, we propose a novel choice of basis functions and a corresponding estimation scheme via convex optimization. We establish an explicit condition to guarantee the positiveness of the spectral estimate. Crucially, the map from the estimate parameters to the trigonometric moments is proved to be a diffeomorphism, ensuring the existence and uniqueness of the solution. Furthermore, we comprehensively prove the statistical properties of the
estimator, including its consistency, (asymptotic) unbiasedness, convergence rate, and efficiency. The proposed framework is applied to a system identification task, where simulations validate its effectiveness.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.NA
Sampling via Stochastic Interpolants by Langevin-based Velocity and Initialization Estimation in Flow ODEs
Chenguang Duan, Yuling Jiao, Gabriele Steidl et al. (+3 more)
We propose a novel method for sampling from unnormalized Boltzmann densities based on a probability flow ordinary differential equation (ODE) derived from linear
stochastic interpolants. The key innovation of our approach is the use of a sequence of Langevin samplers to enable efficient simulation of the flow. Specifically, these Langevin samplers are employed (i) to generate samples from the interpolant distribution at intermediate times and (ii) to construct, starting from these intermediate times, a robust
estimator of the velocity field governing the probability flow ODE. Theoretically, we provide convergence guarantees for both Langevin components, and establish a non-asymptotic convergence rate for the probability flow ODE. Extensive numerical experiments demonstrate the efficiency of the proposed method on challenging
multimodal distributions across a range of dimensions, as well as its effectiveness in
Bayesian inference tasks.
Read the full abstract ▾
All authors Chenguang Duan, Yuling Jiao, Gabriele Steidl, Christian Wald, Jerry Zhijian Yang, Ruizhe Zhang
Statistics Theory · Cross-listed from math.PR
Beta distribution and associated Stirling numbers of the second kind
Jakub Gismatullin and Patrick Tardivel
This article gives a formula for associated Stirling numbers of the second kind based on the moment of a sum of independent random variables having a beta distribution. From this formula we deduce, using probabilistic approaches, lower and upper bounds for these numbers.
Machine Learning (Statistics) · Cross-listed from cs.NE
Sleep-Based Homeostatic Regularization for Stabilizing Spike-Timing-Dependent Plasticity in Recurrent Spiking Neural Networks
Andreas Massey, Aliaksandr Hubin, Stefano Nichele et al. (+1 more)
Spike-timing-dependent plasticity (STDP) provides a biologically-plausible learning mechanism for spiking neural networks (SNNs); however, Hebbian weight updates in architectures with recurrent connections suffer from pathological weight dynamics: unbounded growth, catastrophic forgetting, and loss of representational diversity. We propose a neuromorphic regularization scheme inspired by the synaptic homeostasis hypothesis: periodic offline phases during which external inputs are suppressed, synaptic weights undergo
stochastic decay toward a homeostatic baseline, and spontaneous activity enables
memory consolidation. We demonstrate that this sleep-wake cycle prevents weight saturation while preserving learned structure. Empirically, we find that low to intermediate sleep durations (10-20% of training) improve stability on MNIST-like
benchmarks in our STDP-SNN model, without any data-specific hyperparameter tuning. In contrast, the same sleep intervention yields no measurable benefit for the surrogate-gradient spiking neural network (SG-SNN). Taken together, these results suggest that periodic, sleep-based renormalization may represent a fundamental mechanism for stabilizing local Hebbian learning in neuromorphic systems, while also indicating that special care is required when integrating such protocols with existing gradient-based optimization methods.
Read the full abstract ▾
All authors Andreas Massey, Aliaksandr Hubin, Stefano Nichele, Solve Sæbø
Statistical Computation
Particle Filtering for a Class of State-Space Models with Low and Degenerate Observational Noise
Abylay Zhumekenov, Alexandros Beskos, Dan Crisan et al. (+2 more)
We consider the discrete-time filtering problem in scenarios where the observation noise is low or degenerate. We focus on the case where the observation equation is a linear function of the state and the data involve additive noise. However, we place minimal assumptions on the hidden state process. For such a class of models we derive new particle filters (PFs) with the key property that their performance is robust to the size of the observation noise. As a consequence, the developed PFs are well-defined in the limiting case of degenerate observation noise. Indicatively, we prove (under assumptions) that the PF applied in this low noise setting inherits the properties of the PF used in the degenerate case. We extend our framework to the case where the hidden states are drawn from a
diffusion process. We illustrate our
algorithms numerically on several examples.
Read the full abstract ▾
All authors Abylay Zhumekenov, Alexandros Beskos, Dan Crisan, Ajay Jasra, Nikolas Kantas
Statistics Theory · Cross-listed from math.OC
Kantorovich Distance via Spanning Trees: Properties and Algorithms
Jérémie Bigot and Luis Fredes
We study optimal transport between probability measures supported on the same finite metric space, where the ground cost is a distance induced by a weighted connected graph. Building on recent work showing that the resulting Kantorovich distance can be expressed as a minimization problem over the set of spanning trees of this underlying graph, we investigate the implications of this reformulation on the construction of an optimal transport plan and a dual potential based on the solution of such an optimization problem. In this setting, we derive an explicit formula for the Kantorovich potential in terms of the imbalanced cumulative mass (a generalization of the cumulative distribution in R) along an optimal spanning tree solving such a minimization problem, under a weak non-degeneracy condition on the pair of measures that guarantees the uniqueness of a dual potential. Our second contribution establishes the existence of an optimal transport plan that can be computed efficiently by a dynamic programming procedure once an optimal spanning tree is known. Finally, we propose a
stochastic algorithm based on simulated annealing on the space of spanning trees to compute such an optimal spanning tree. Numerical experiments illustrate the theoretical results and demonstrate the practical relevance of the proposed approach for optimal transport on finite metric spaces.
Read the full abstract ▾
Statistics Theory
Rank tests for time-varying covariance matrices observed under noise
Markus Reiß and Lars Winkelmann
We consider a $d$-dimensional continuous martingale $X(t)$ with quadratic variation
matrix $\langle X\rangle_t=\int_0^t \Sigma(s)\,ds$ and develop tests for the rank of its spot covariance
matrix $\Sigma(t)$, $t\in[0,1]$. The process $X$ is observed under observational noise, as is standard for microstructure noise models in high-frequency finance. We test the null hypothesis ${\mathcal H}_0:rank(\Sigma(t))\le r$ against local alternatives ${\mathcal H}_{1,n}:\lambda_{r+1}(\Sigma(t))\ge v_n$, where $\lambda_{r+1}$ denotes the $(r+1)$st
eigenvalue and $v_n\downarrow 0$ as the sample size $n\to\infty$. We construct test statistics based on
eigenvalues of carefully calibrated localized spectral covariance
matrix estimates. Critical values are provided non-asymptotically as well as asymptotically via maximal
eigenvalues of Gaussian orthogonal ensembles. The power analysis establishes asymptotic consistency for a separation rate $v_n\thicksim (\underline\lambda_r^{-1/(\beta+1)}n^{-\beta/(\beta+1)})\wedge n^{-\beta/(\beta+2)}$, depending on the Hölder-regularity $\beta$ of $\Sigma$ and a possible spectral gap $\underline\lambda_r\ge 0$ under ${\mathcal H}_0$. A lower bound shows the optimality of this rate. We discuss why the rate is much faster than conventional estimation rates. The theory is illustrated by simulations and a real data example with German government bonds of varying maturity.
Read the full abstract ▾
Statistics Applications
A parsimonious tail compliant multiscale statistical model for aggregated rainfall
Pierre Ailliot, Carlo Gaetan and Philippe Naveau
Modeling rainfall intensity distributions across aggregation scales (from sub-hourly to weekly) is essential for hydrological risk analysis and IDF curves. Aggregation naturally imposes mathematical constraints: return levels must be ordered by time scale, as daily accumulations necessarily exceed sub-daily ones. From a statistical perspective, each aggregation step should ideally not require additional parameters, yet parsimonious models describing the full distribution remain scarce, as most literature focuses on seasonal block maxima. In this study, we propose a parsimonious framework to model all rainfall intensities (low to large) across scales. We utilize the Extended Generalized Pareto Distribution (EGPD), which aligns with extreme value theory for both tails while remaining flexible for the bulk of the distribution. We establish a general result on the behavior of EGPD variables under various aggregation procedures. To overcome the difficulty of direct likelihood inference, we link the EGPD class to Poisson compound sums. This allows the use of the Panjer
algorithm for efficient composite likelihood evaluation. Our approach ensures that return levels do not cross across scales and enables estimation for return periods below annual or seasonal levels. We demonstrate the method using sub-hourly series from six French stations with diverse climates. Only eight parameters are needed per station to capture scales from six minutes to three days. IDF curves above and below the annual scale are provided.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Deep Exploration of Epoch-wise Double Descent in Noisy Data: Signal Separation, Large Activation, and Benign Overfitting
Tomoki Kubo, Ryuken Uda and Yusuke Iida
Deep double descent is one of the key phenomena underlying the generalization capability of deep learning models. In this study, epoch-wise double descent, which is delayed generalization following overfitting, was empirically investigated by focusing on the
evolution of internal structures. Fully connected neural networks of three different sizes were trained on the CIFAR-10
dataset with 30% label noise. By decomposing the loss curves into signal contributions from clean and noisy training data, the epoch-wise evolutions of internal signals were analyzed separately. Three main findings were obtained from this analysis. First, the model achieved strong re-generalization on test data even after perfectly fitting noisy training data during the double descent phase, corresponding to a "benign overfitting" state. Second, noisy data were learned after clean data, and as learning progressed, their corresponding internal activations became increasingly separated in outer layers; this enabled the model to overfit only noisy data. Third, a single, very large activation emerged in the shallow layer across all models; this phenomenon is referred as "outliers," "massive activa-tions," and "super activations" in recent large
language models and evolves with re-generalization. The magnitude of large activation correlated with input patterns but not with output patterns. These empirical findings directly link the recent key phenomena of "deep double descent," "benign overfitting," and "large activation", and support the proposal of a novel scenario for understanding deep double descent.
Read the full abstract ▾
Machine Learning (Statistics)
Structural Dimension Reduction in Bayesian Networks
Pei Heng, Yi Sun and Jianhua Guo
This work introduces a novel technique, named structural dimension reduction, to collapse a
Bayesian network onto a minimum and localized one while ensuring that probabilistic inferences between the original and reduced networks remain consistent. To this end, we propose a new combinatorial structure in directed acyclic graphs called the directed convex hull, which has turned out to be equivalent to their minimum localized
Bayesian networks. An efficient
polynomial-time
algorithm is devised to identify them by determining the unique directed convex hulls containing the variables of interest from the original networks. Experiments demonstrate that the proposed technique has high dimension reduction capability in real networks, and the efficiency of probabilistic inference based on directed convex hulls can be significantly improved compared with traditional methods such as variable elimination and belief propagation
algorithms. The code of this study is open at \href{https://github.com/Balance-H/
Algorithms}{https://github.com/Balance-H/
Algorithms} and the proofs of the results in the main body are postponed to the appendix.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.PR
Wasserstein-p Central Limit Theorem Rates: From Local Dependence to Markov Chains
Yixuan Zhang and Qiaomin Xie
Non-asymptotic central limit
theorem (CLT) rates play a central role in modern machine learning and operations research. In this paper, we study CLT rates for multivariate dependent data in Wasserstein-$p$ ($W_p$) distance, for general $p\ge 1$. We focus on two fundamental dependence structures that commonly arise in practice: locally dependent sequences and geometrically ergodic Markov chains. In both settings, we establish the first optimal $\mathcal O(n^{-1/2})$ rate in $W_1$, as well as the first $W_p$ ($p\ge 2$) CLT rates under mild moment assumptions, substantially improving the best previously known bounds in these dependent-data regimes. As an application of our optimal $W_1$ rate for locally dependent sequences, we further obtain the first optimal $W_1$-CLT rate for multivariate $U$-statistics. On the technical side, we derive a tractable auxiliary bound for $W_1$ Gaussian approximation errors that is well suited for studying dependent data. For Markov chains, we further prove that the regeneration time of the split chain associated with a geometrically ergodic chain has a
geometric tail without assuming strong aperiodicity or other restrictive conditions. These tools may be of independent interests and enable our optimal $W_1$ rates and underpin our $W_p$ ($p\ge 2$) results.
Read the full abstract ▾
Other Statistics
Proactive Anomaly Screen for Multiple Endpoints Using Bayesian Latent Class Modeling: A k-Step Ahead Approach
Yuxi Zhao and Margaret Gamalo
In clinical trials, ensuring the quality and validity of data for downstream analysis and results is paramount, thus necessitating thorough data monitoring. This typically involves employing edit checks and manual queries during data collection. Edit checks consist of straightforward schemes programmed into relational databases, though they lack the capacity to assess data intelligently. In contrast, manual queries are initiated by data managers who manually scrutinize the collected data, identifying discrepancies needing clarification or correction. Manual queries pose significant challenges, particularly when dealing with large-scale data in late-phase clinical trials. Moreover, they are reactive rather than predictive, meaning they address issues after they arise rather than preemptively preventing errors. In this paper, we propose a joint model for multiple endpoints, focusing on primary and key secondary measures, using a
Bayesian latent class approach. This model incorporates adjustments for risk monitoring factors, enabling proactive, $k$-step ahead, detection of conflicting or anomalous patterns within the data. Furthermore, we develop individualized dynamic predictions at consecutive time-points to identify potential anomalous values based on observed data. This analysis can be integrated into electronic data capture systems to provide objective alerts to stakeholders. We present simulation results and demonstrate effectiveness of this approach with real-world data.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
Learning a Stochastic Differential Equation Model of Tropical Cyclone Intensification from Reanalysis and Observational Data
Kenneth Gee and Sai Ravela
Tropical cyclones are among the most consequential weather hazards, yet estimates of their risk are limited by the relatively short historical record. To extend these records, researchers often generate large ensembles of synthetic storms using simplified models of cyclone intensification. Developing such models, however, has traditionally required substantial theoretical effort. Here we explore whether equation-discovery methods, a class of data-driven techniques designed to infer governing equations, can accelerate the process of developing simplified intensification models. Using observational storm data (IBTrACS) together with environmental conditions from reanalysis (ERA5), we learn a compact
stochastic differential equation describing tropical cyclone intensity
evolution. We focus on TCs because their dynamics are well studied and a hierarchy of reduced-order models exist, enabling direct comparison of the learned model to physically-derived counterparts. We find that the learned model simulates synthetic TCs whose intensification statistics and hazard estimates are consistent with observations and competitive with a leading physics-based TC intensification model. Our model also reproduces known nonlinear dynamical behavior of tropical cyclones, including as a saddle node bifurcation as inner core ventilation is increased. This result shows that equation-discovery approaches, when applied directly to storm intensity, can recover not only realistic statistics but also physically meaningful dynamical structure. These findings highlight the potential for data-driven methods to complement existing theory and reduced-order models in the study of extreme weather.
Read the full abstract ▾
Machine Learning (Statistics)
Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds
Xinping Yi, Gaojie Jin, Xiaowei Huang et al. (+1 more)
Understanding the generalization behavior of deep neural networks remains a fundamental challenge in modern statistical learning theory. Among existing approaches, PAC-
Bayesian norm-based bounds have demonstrated particular promise due to their data-dependent nature and their ability to capture algorithmic and
geometric properties of learned models. However, most existing results rely on isotropic Gaussian posteriors, heavy use of spectral-norm concentration for weight perturbations, and largely architecture-agnostic analyses, which together limit both the tightness and practical relevance of the resulting bounds. To address these limitations, in this work, we propose a unified framework for PAC-
Bayesian norm-based generalization by reformulating the derivation of generalization bounds as a
stochastic optimization problem over anisotropic Gaussian posteriors. The key to our approach is a sensitivity
matrix that quantifies the network outputs with respect to structured weight perturbations, enabling the explicit incorporation of heterogeneous parameter sensitivities and architectural structures. By imposing different structural assumptions on this sensitivity
matrix, we derive a family of generalization bounds that recover several existing PAC-
Bayesian results as special cases, while yielding bounds that are comparable to or tighter than state-of-the-art approaches. Such a unified framework provides a principled and flexible way for geometry-/structure-aware and interpretable generalization analysis in deep learning.
Read the full abstract ▾
All authors Xinping Yi, Gaojie Jin, Xiaowei Huang, Shi Jin