Peer Revenue Returns subject to peer review

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Wednesday 29 July

Statistical Methodology

In Search of the Most Balanced Sampling Design

Balanced sampling aims to select random samples in which the estimated totals of the auxiliary variables, weighted by the inverse of the inclusion probabilities, correspond as closely as possible to the known population totals. While several methods, such as rejective sampling, rerandomization, and the cube method, have been proposed to improve balance, identifying the most balanced sampling design under fixed inclusion probabilities remains a challenging combinatorial problem. This problem can be formulated as a linear program defined over the set of all possible samples, but the number of samples grows exponentially with population size, making exact optimization infeasible except for very small populations. To address this issue, we propose a heuristic approach based on a genetic algorithm that iteratively improves the balance of sampling designs by combining minimum support designs with highly balanced candidate samples. Although optimality cannot be guaranteed, the proposed method can substantially improve balance relative to standard procedures such as the cube method. The approach is applicable to both survey sampling and experimental design.
Read the full abstract

Statistical Methodology

An association measure for mixed-type variables

Quantifying the association between a real-valued variable and a categorical variable is a fundamental task in data analysis. Existing methods often rely on parametric assumptions or arbitrary integer encoding, which may lead to unstable results. We propose a label-invariant population measure of association, $\xi'$, specifically designed for the mixed real-valued-categorical setting. The proposed measure is normalized between 0 and 1; it equals 0 if and only if the variables are independent and 1 if and only if the categorical variable is a measurable function of the real-valued one. We also introduce a corresponding sample estimator, $\xi_n'$, computable in $O(n \log n)$ time. These measures are invariant to permutations of category labels and strictly monotone transformations of the real-valued variable. We establish the strong consistency and asymptotic normality of the estimator $\xi_n'$, enabling a computationally efficient, permutation-free Wald test for independence, and an asymptotic confidence interval for the population measure $\xi'$. Extensive simulations and an application to The Cancer Genome Atlas (TCGA) data demonstrate that the proposed method provides coding stability, competitive power, and substantial computational advantages in nominal mixed-type settings.
Read the full abstract

Statistics Theory

Weak but Not Strong Asymptotic Testability

We construct two fixed disjoint families $H_0,H_1$ of stationary ergodic binary process distributions for which a weakly asymptotically consistent test exists, but no strongly asymptotically consistent test exists. The construction combines a synchronizing binary suspension code, an independent i.i.d marker process, and countably many independent slowly switching two-state Markov chains. In particular, this disproves the asymptotic-consistency branch of a conjecture of Ryabko.
Read the full abstract

Statistical Methodology

Bayesian Graphical Models under Positivity Constraints: A Scalable generalized likelihood Approach

We develop a computationally scalable Bayesian framework for precision matrix estimation in Gaussian graphical models under total positivity constraints. To overcome the high computational cost of the Gaussian likelihood, we adopt a generalized Bayesian approach based on the $D$-trace loss, which eliminates the log-determinant term and enables efficient optimization while allowing relaxation of positive definiteness during sampling. Sparsity is induced via spike-and-slab priors, and the resulting generalized posterior is shown to be proper under mild conditions. Our primary contribution is a suite of efficient posterior sampling algorithms tailored to high-dimensional settings. Starting from a component-wise Gibbs sampler, we introduce a novel data augmentation scheme that induces conditional independence among precision matrix entries, enabling joint updates. By exploiting the Gram structure of the sample covariance matrix, we further develop a fast matrix-normal sampler that significantly reduces per-iteration complexity in high-dimensional settings. An interweaving strategy combines augmented and direct updates to improve mixing without sacrificing scalability. Experiments on synthetic and financial data demonstrate substantial computational gains over existing methods, while maintaining competitive estimation accuracy and improved recovery of structured dependencies.
Read the full abstract

All authors Swarnali Raha, Partha Sarkar, Sirani Perera, Kshitij Khare

Machine Learning (Statistics)

Chaos Is a LADDER: Domain Generalization Beyond Invariance via Reweighting

Domain generalization (DG) aims to learn from multiple source domains and generalize to unseen target domains. Most DG methods pursue invariance: they seek a causal representation whose prediction rule is invariant across domains. This principle is effective when the causal mechanism is stable, but becomes restrictive when the domain itself modulates how causal content maps to the response. In this case, directly feeding domain style into the predictor can create misleading shortcuts, since style does not by itself cause the response. Yet the apparent chaos of multiple styles can become a ladder: style can locate the unseen target domain among source domains and guide which domain-dependent prediction rules should be trusted. We propose Latent Adaptive Domain Disentanglement and Environment Reweighting (LADDER), a fixed-model DG pipeline that learns causal/style representations, freezes the encoders, fits source-specific classifiers, and uses an unlabeled target-domain covariate set only at inference to compute weights over these fixed classifiers, with no target labels or model-state updates. We establish theoretical guarantees for source reweighting and validate LADDER on simulations, FMoW, and a location-grouped iWildCam protocol, with gains in overall and group-averaged accuracy.
Read the full abstract

All authors Yuhang Jiang, Fengchuan Zhang, Sanguo Zhang, Guojun Zhu

Machine Learning (Statistics) · Cross-listed from cs.LG

Existence-Field Diffusion Model for Spatial Point Processes with Variable Cardinality

We study generative modeling of spatial point processes (SPP), where both the number of points and their spatial configuration are governed by a joint distribution. While diffusion models have achieved strong performance in modeling complex distributions, extending them to variable-cardinality SPP remains challenging. Existing approaches either sample cardinality before generating locations conditionally, or introduce specialized discrete operations to change the number of points during generation. We propose the existence-field diffusion model (EFDM), which associates each potential point with a continuous variable representing its degree of presence. EFDM uses Gaussian diffusion to jointly model point locations and cardinality through a simple continuous representation. Existence probabilities can increase or decrease in both forward and reverse diffusion, without explicit point-addition or point-deletion steps. The same construction naturally accommodates categorical attributes, allowing locations, existence, and attributes to be modeled together. Experiments demonstrate improved molecular stability and validity over the compared baselines on the molecular dataset, alongside competitive performance on real-world spatial and synthetic datasets.
Read the full abstract

All authors Xiaoyin Pan, Christian R. Shelton, Rakshith Mahishi, Chengkuan Hong

Machine Learning (Statistics)

Origins and mitigation of double descent in reduced order modeling

Latent low-dimensional structure in datasets of natural and engineered systems enables their sparse sensing, or full-state reconstruction from historical data and very few carefully chosen localized measurements. Depending on the reconstruction algorithm, sensor locations, and measurement noise, the reconstruction risk curves demonstrate a diversity of patterns including a dramatic peak in error known as double descent in Machine Learning literature. Here we explore those scenarios under a unified Data-Noise Averaging theory. Qualitatively, we formulate sufficient criteria for double descent to emerge through a catastrophic amplification of a pathological signal in reconstruction. Quantitatively, we predict the detailed risk curves at a fraction of computational cost, trace reconstruction instability to individual sensors and their combinations, and provide regularization mechanisms to mitigate the instability. We demonstrate results for both static reconstruction of Sea Surface Temperature patterns and time integration of a reduced order model of a PDE.
Read the full abstract

Statistics Theory

A Lower Bound on the Sliced Wasserstein Comparison Constant for Translated Measures

We provide closed-form expressions for standard 1-Wasserstein distance ($W_1$) and Sliced 1-Wasserstein distance ($SW_1$) for translated probability measures on $\mathbb{R}^d$. Furthermore, we construct an explicit lower bound for the constant $c_d$ established by Carlier, Figalli, Mérigot, and Wang between $W_1$ and $SW_1$ [1] for translated probability measures on $\mathbb{R}^d$. Sliced Wasserstein distances are computationally efficient and used in image generation and other applications, which raises natural questions in computational efficiency versus theoretical correctness. A lower bound on the constant $c_d$ gives information about the cost of that efficiency.
Read the full abstract

Statistical Methodology · Cross-listed from econ.EM

Specification Testing for Dyadic Regression Models

This paper develops omnibus specification tests for linear conditional-mean models with undirected dyadic data. We establish a uniform projection theorem that reduces the dyadic process to its latent first-order node projections under shared-node dependence. We then show that a raw first-order node-multiplier bootstrap is valid when this node component is nondegenerate but double-counts dyad-specific variation when dyads are independent. An exact covariance decomposition motivates a corrected Gaussian bootstrap that is valid in both regimes. The resulting Kolmogorov-Smirnov and Cramér-von Mises tests are consistent against fixed alternatives and have nontrivial power against rate-appropriate local alternatives. Simulations show that the corrected Kolmogorov-Smirnov test provides the most stable size control while retaining substantial local power. An application to the Lazega law-firm network rejects additive linear and quadratic specifications but finds no remaining misspecification after including an economically relevant interaction.
Read the full abstract

Statistical Methodology

Optimizing Efficiency and Convergence in MMRM: Practical Considerations for Longitudinal Data Analysis

Mixed models for repeated measures (MMRM) are a popular method for analyzing longitudinal data in clinical trials. However, practical challenges, such as small sample sizes, large numbers of time points, and selection of variance-covariance structure for within-subject errors, often present barriers to model convergence and valid inference. This article evaluates different options for using MMRM in different scenarios through extensive simulation studies and an application on diabetes trial data. We demonstrate that the empirical bias-reduced coefficient covariance adjustment with the heterogeneous autoregressive covariance structure yields near-nominal coverage with high convergence rates for moderate and large sample designs. For small sample sizes, the simple model with baseline covariates and treatment by time point interaction achieves good efficiency and high probability of convergence. Based on these results, we provide practitioners with actionable guidance for applying MMRM to clinical trial data.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

High-Order Markov Blanket Discovery via a k-Order Relaxation of the Faithfulness Assumption

The problem of learning the graphical Markov blanket (MB) of a variable from data has applications in many areas such as structure learning for Bayesian networks and Markov random fields, causal discovery, and feature selection. However, a common assumption most methods make is that the conditional independencies in the distribution imply the same separation in the graphical structure – also known as the faithfulness assumption. Unfortunately, this assumption can be violated by higher-order dependencies such as XOR and parity-type relations, and – on finite samples – by empirical violations that, in extreme cases, even induce spurious dependencies absent from the true distribution. Therefore, in this paper we propose a "k-order" relaxation of the faithfulness assumption that captures parity type relationships between k+2 variables. We then propose a proof of concept algorithm called k-order Markov blanket (kOMB) that uses this relaxation for MB discovery. Finally, we empirically show how kOMB can recover the MB of a variable under both true and empirical violations of faithfulness. Code available at: https://github.com/lklee9/k-order-Markov-blanket
Read the full abstract

Tuesday 28 July

Machine Learning (Statistics) · Cross-listed from cs.AI

Task- and Session-Level Model Routing: A Common-Interface Hybrid Evaluation of Four Open-Source Routers Across Four Benchmarks

Agentic systems increasingly delegate model selection to a router, yet open-source routers are usually evaluated with different tasks, candidate pools, and execution protocols, limiting direct comparison. We present a common measurement protocol and hybrid evaluation of four router implementations across RouterBench, BFCL v4, tau2-bench, and WebArena. We evaluate 290 frozen tasks against a locked matrix of 2,610 candidate outcomes. Three routers emit constant or near-constant tier assignments; only vLLM Semantic Router varies materially with prompt content, and it has the highest observed success rate on none of the four benchmarks. Always-Mid matches Aurelio exactly on three benchmarks and within 0.003 on the fourth. For vLLM, task-level superiority tests detect no task-specific advantage over a share-matched content-blind allocation; equivalence is established only on WebArena at the protocol-declared five-percentage-point margin. The results show that, under these configurations and controls, observed gains track selected-tier composition more closely than demonstrated task-specific targeting. Fixed-tier baselines and selected-tier distributions are therefore necessary controls in router evaluation; the findings are scoped to these configurations, candidate pool, and frozen benchmark samples, not to routing paradigms in general.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

Randomly initialized autoencoders: fixed points and edge-of-chaos

In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises questions of existence, stability, and basins of attraction of these fixed points. These questions are addressed via the contractive properties of autoencoders, and are closely related to the notion of edge-of-chaos. Edge-of-chaos (EoC) is an important notion in the theory of DNNs. It describes the critical regime separating ordered and chaotic signal propagation through a randomly initialized network. Initialization at or near this critical regime offers several theoretical and practical advantages, including stability of the network w.r.t. perturbations of the input. EoC was previously introduced for broad classes of neural networks using mean-field averaging methods. In this paper we modify the notion of EoC for the study of autoencoders. Specifically, we introduce local and global EoC for autoencoders that control local (small) and global (arbitrary) perturbations of the input respectively. The study of stability of autoencoders falls within the scope of nonlinear problems in Random Matrix Theory (RMT). Our analysis of local EoC is based on spectral techniques of RMT, whereas global EoC is studied by employing Sudakov-Fernique inequality for Gaussian processes.
Read the full abstract

All authors Leonid Berlyand, Roman Sarapin, Yitzchak Shmalo, Victor Slavin, Sasha Sodin

Statistical Methodology · Cross-listed from cs.LG

Early Cycle Charge Trajectory Generative Prediction and Full Life Cycle Health Management of Iron-Chromium Flow Batteries Based on FlowBD-E1

Long-duration stationary energy storage requires batteries whose degradation can be detected before substantial capacity loss has accumulated. Iron-chromium redox flow batteries are attractive for this role because they use abundant and low-cost active species, yet their operation is shaped by slow chromium kinetics, hydrogen evolution, membrane crossover and electrolyte imbalance. These coupled processes gradually reshape the full charge voltage/current (V/I) trajectory, but most battery prognostic studies either focus on lithium-ion cells or compress ageing into scalar capacity and state-of-health (SOH) labels. Here we study an industrial 33 kW Fe-Cr redox flow battery and introduce FlowBD-E1, an early-cycle generative forecasting framework that predicts complete future charge V/I trajectories from only the first few cycles. The model combines a multi-scale convolutional encoder, a lifecycle Transformer and an age-aware FiLM decoder, and we compare three deployment strategies: single-step latent extrapolation (SLE), recursive latent forecasting (RLF) and teacher-forced updating (TFU). Using the first 9 of 289 cycles, RLF achieved a joint V/I mean absolute percentage error (MAPE) of 0.731% over the remaining lifecycle and produced SOH estimates below 1% MAPE. Ablation and independent-sequence tests showed that the age-aware generative architecture outperformed LSTM and TCN baselines and retained sub-percent errors under industrial validation. These results suggest that early-cycle trajectory generation can turn a short commissioning record into a long-horizon diagnostic signal for flow-battery management.
Read the full abstract

Statistics Applications · Cross-listed from physics.ao-ph

Projected climate memory and inherited warm-tail risk in accelerated European summer warming

European summer warming reflects interactions among background change, persistent ocean–land–circulation states, and same-season variability. We develop an empirical reduced-dynamics framework that decomposes regional summer indicators into inherited slow-state memory, its predictable component, and contemporaneous innovation. Projection-operator theory motivates the decomposition, implemented with finite causal filters, ridge-regularised prediction, and logistic risk models. Using ERA5-derived summer indicators for 28 IPCC AR6 European sub-regions over 1950–2024, with validation on 2006–2024, we find that Mediterranean-state memory improves mean summer-temperature prediction relative to trend-only and ARX baselines. The gain over ARX is modest, while moving-average, exponentially weighted, and tempered filters contain similar annual information, indicating that the data identify useful slow-state memory more robustly than a unique kernel shape. Predictable-state reconstruction is ridge-sensitive and therefore treated diagnostically rather than as a forecasting model. The strongest result concerns warm-tail risk. In parsimonious logistic models, high accumulated Mediterranean memory raises predicted upper-tail event probability by about 8–11 percentage points for annual maximum summer temperature, warm-day frequency, and warm-spell duration at 1-, 3-, and 5-year horizons. Regional bootstrap intervals remain positive for all targets and horizons. Circular-shift placebos yield one-sided probabilities of approximately 0.05–0.14 and do not survive strict family-wise correction across nine tests, so the evidence is moderate rather than decisive. Overall, annual projected memory is not a universal short-horizon predictor, but a physically interpretable inherited risk-loading variable identifying years and regions predisposed to warm-tail outcomes.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

GeoID-PINN: Identifiability-Aware Regional Epidemic Inference with Geographic Coupling

Regional surveillance data reflect local transmission, reporting, seeding, and external infection pressure, which are difficult to identify separately. We introduce GeoID-PINN, a physics-informed neural network (PINN) for susceptible-infectious-recovered-deceased (SIRD) dynamics. The model represents spatial dependence with a row-stochastic source-composition matrix whose rows assign nonnegative source weights that sum to one. We regularize this matrix toward a spatial prior constructed from distance, adjacency, commuting, or lead-lag information. In a four-region simulation with known truth, a compatible distance prior gives source-composition error 0.099. The error rises to 0.159 without regularization and 0.577 under a strongly misspecified prior, while trajectory fit and transmission-scale estimates remain similar. Accurate trajectories therefore do not guarantee recovery of the regional dependence structure. We also evaluate GeoID-PINN retrospectively using COVID-19 data from 64 Louisiana counties. Relative to an autoregressive negative-binomial baseline, Forecast-Trained Geo-PINN reduces mean squared error (MSE) from 32,957 to 11,468 and mean absolute error (MAE) from 70.60 to 57.73. The baseline has lower negative log likelihood (NLL), 5.158 versus 5.346, indicating better distributional fit but worse point accuracy. In a controlled 15-county comparison, county adjacency reduces MSE by 6.85 percent and MAE by 3.1 percent. Similar performance across plausible priors supports structured regularization but not unique edge recovery. These results require prior-sensitivity and observation-model checks before interpretation.
Read the full abstract

Machine Learning (Statistics)

More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility

Neural networks increasingly combine data across populations, time periods, and operating conditions to improve generalization. This raises a reliability question: whether a model refitted on pooled data preserves an action ordering supported by both sources. Case-Based Decision Theory (CBDT) formalizes this requirement through its composition axiom, which requires source-supported preferences to survive their union. We study when this property holds for fixed-representation neural networks with ordinary least squares (OLS) output heads. First, we show that pooled refitting recomputes the inverse-Gram geometry used to weight source evidence, which can reverse shared preferences, and derive exact and approximate preservation conditions. Next, we introduce a scale-invariant Gram mismatch measure for prioritizing candidate pools and geometry-oriented regularization for shaping source geometry during training. Finally, we develop a three-stage audit that traces strict pairwise reversals through decision changes to task-defined utility loss. Experiments spanning a load-based bidding proxy and medical and financial decision proxies reveal stable and reversal-prone pooling regimes: the load audit identifies a measurable nonzero class of source-consensus-relative harmful decisions under the proxy utility, while cross-domain audits show that comparable mismatch can correspond to sharply different preservation rates. Geometry-oriented objectives occupy distinct descriptive accuracy-consistency-geometry-harm operating points. Together, the framework makes compositional reliability measurable and operational through screening, analytic certification, geometry-oriented training, and decision-consequence auditing.
Read the full abstract

All authors Yanli Yan, Yuanzheng Li, Yong Zhao, Hongbo Guo, Shoudong Han

Statistics Theory

Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning

Knowledge graph learning provides a powerful framework for representing and inferring structured knowledge, with broad practical applications. However, the scarcity of relation-specific labeled triples per entity hinders the training of expressive models, and the ad hoc design of scoring functions limits generalizability and lacks theoretical grounding. We address both issues with a theoretically grounded, end-to-end training framework that extends and subsumes existing methods. Our framework is a two-stage procedure: unsupervised pretraining over heterogeneous corpora followed by supervised learning with multiple relation types. We establish a nonasymptotic risk bound that disentangles pretraining representation error from labeled-sample complexity, formally quantifying the benefit of large-scale unlabeled data for downstream knowledge prediction. Synthetic experiments validate each theoretical component, and real-world experiments confirm the effectiveness of our approach on large-scale knowledge graph benchmarks.
Read the full abstract

Statistical Methodology

The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text

Estimating causal effects of linguistic properties from observational text is difficult because the same document can contain both the treatment of interest and the non-treatment textual attributes needed for adjustment. Existing approaches often learn representations from the full text to capture latent confounding, but when treatment status is itself encoded by words in the text, these representations can directly encode treatment. This creates a confounder trap: richer representations can make treated and control documents separable, inducing overlap violations even when the underlying causal problem satisfies overlap. We study latent text treatments that are encoded through lexicons or other treatment-defining lexical information, and propose masking-based adjustment representations that remove this lexical treatment signal before representation learning. We formalize representation-induced overlap failure, prove that deletion masking preserves overlap for bag-of-words/topic-model representations, and characterize replacement masking as a natural relaxation for large language models that hides treatment-defining tokens while preserving word order and context. Across simulations, masking improves overlap diagnostics, stabilizes treatment effect estimates, and reduces bias relative to adjustment methods that learn from the unmasked text.
Read the full abstract

Statistics Applications · Cross-listed from q-bio.NC

Three Failures of Pain Location: Why Its Diagnostic Utility Is Three Quantities, Not One

Patient-reported pain location is diagnostically decisive for some presentations and nearly uninformative for others. The prevailing account treats this as one gradient of diagnostic utility set by anatomical complexity. That explanation conflates three epistemically distinct failures, each with its own mathematics, its own optimal instrument, and its own public-health consequence. In anatomical multiplexing, many structures share one location: a non-identifiable inverse problem. In delocalized amplification - clinically, central sensitization or nociplastic pain - a centrally driven pain-behaviour pattern replaces the peripheral generator: a change of generative model. In referred and atypical displacement, location is hypothesized to shift in a systematic, person-dependent way: a group-conditional bias whose direct evidence is still open. The three are one Bayesian inference problem failing at different nodes - the likelihood, the model class, and the group-conditional prior - with a fourth node at the report itself. The formal development is in a companion paper; this paper states what each model shows and what follows clinically. Re-examination finds that the published "high-utility" accuracy band leans on overstated specificity (Lipton et al., 2003; Bruyninckx et al., 2008; Devillé et al., 2000), so the gradient is real but flatter than drawn. The well-evidenced finding that better detection alone does not improve outcomes when treatment uptake lags is about the care pathway, not perception - a distinction the three-way split makes visible and a single utility number hides. The paper organizes the failures along a why-location-fails axis, distinct from the nociceptive/neuropathic/nociplastic taxonomy (Kosek et al., 2016), and sets out the study that would test the one prediction still open.
Read the full abstract

2607.26297 · 28 Jul · v2. The mathematical appendix is now a standalone companion paper, "One Inference, Four Nodes: Formal Models of Why Pain Location Fails," submitted separately; this version is the clinical and evidential half. Also: specificity re-examination corrected, registered sample raised to ~1250. 47 pages, 6 figures

Statistical Methodology

Studying Competing Events with Federated Cumulative Incidence Curves

Combining electronic health record (EHR) data from multiple institutions is a valuable strategy for conducting post-market safety surveillance of medical products, but privacy concerns limit sharing individual-level data. We develop a novel federated learning (FL) method for multi-site post-market safety surveillance of medical products using competing risks data. We apply this method to study immune-related adverse events (irAEs) following treatment with immune checkpoint inhibitors (ICIs) in patients with auto-immune disease (AID). We provide an algorithm for constructing non-parametric cumulative incidence curves for competing event types, which can be used to compare exposure groups (e.g. treated and untreated) with no sharing of patient-level data across institutions. We incorporate covariate adjustment via inverse propensity weighting, and informative causal comparison using the area under cumulative incidence curves, known as restricted mean time lost. We apply our method to $N=10,281$ cancer patients with no pre-existing endocrine-related AID receiving ICIs across $K=10$ sites from the OneFlorida+ network, comparing patients with a pre-existing non-endocrine AID to those with no pre-existing AID. After covariate adjustment, we found that patients with a pre-existing non-endocrine AID lost 4.8 [95% CI: 4.3,5.2] months of event-free survival time to endocrine irAEs in the first 18 months following treatment, compared to 3.2 [95% CI: 3.1,3.3] months in the group without prior AID. As patients with prior AID were initially excluded from clinical trials of ICIs, our findings provide important new information to clinicians and patients receiving or considering ICI treatment. Our proposed non-parametric federated algorithm is the first to allow investigators to use some of the most crucial non-parametric tools for conducting postmarket safety surveillance across multiple institutions.
Read the full abstract

All authors Malcolm Risk, Shuang Yang, Jiang Bian, Yi Guo, Hyojung Jang, Jingchuan Guo, Xu Shi, Lili Zhao

Statistics Theory

Denoising growth complexity: Data geometry and certified schedules for diffusion sampling

Two central challenges in diffusion-based sampling are the theoretical one of understanding their remarkable effectiveness even in high-dimensional settings, and the practical one of designing algorithms with certified performance guarantees. We show that these questions are intimately connected via the denoising growth complexity ($\mathsf{DGC}$). It is a geometric measure defined by a log-time weighted integral of the derivative of the denoising mean-squared error along the Gaussian heat flow. We show how the $\mathsf{DGC}$ increments lead to a simple and explicit bound on the KL error of an Euler scheme applied to the stochastic innovations representation. The bound is local along the path: each step is controlled by the corresponding $\mathsf{DGC}$ increment and its relative stepsize. This structure allows us to derive KL sampling guarantees for optimized stepsize schedules, both in a simpler single-block setting and in a more refined $K$-block setting. The $\mathsf{DGC}$ function has a natural martingale structure, which we exploit to develop fully data-certified versions of these algorithms. It also admits information-theoretic upper bounds in terms of covariance, rate distortion, metric entropy, and the Poincar'e constant, thereby recovering and sharpening a range of existing diffusion-sampling guarantees, as well as giving new results. In log heat-time, the fine partition limit is governed by an integral involving the square root of the $\mathsf{DGC}$ density, whereas a single-block schedule depends on its ordinary integral. This comparison precisely characterizes when adaptation to data geometry yields substantial computational gains, including logarithmic-to-constant separations for simple Gaussian mixture models.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Top-$k$ Pareto Bandits: Hypervolume Regret for Multi-Objective Slate Selection

We consider a stochastic multi-objective bandit problem where, at each round, the agent selects a slate of $k$ arms and observes their $d$-dimensional reward vectors under semi-bandit feedback. We do not aim at identifying a single optimal arm; instead, we consider the problem of maintaining a small set of actions that jointly approximate the Pareto frontier. We formalize this objective through the dominated hypervolume induced by the selected subset of arms, and define an $\alpha$-approximate hypervolume regret with respect to the best size-$k$ subset achievable in hindsight, where $\alpha = 1 - 1/e$ reflects the approximation guarantee of greedy maximization for monotone submodular functions. To address this problem, we introduce THV-UCB, an optimistic algorithm that selects arms greedily based on optimistic estimates of their marginal hypervolume contributions. We establish a gap-free regret bound $\tilde{O}(d\sqrt{nkT})$ that holds on every instance, together with a gap-dependent bound $\tilde{O}(nk^{2.5}/\Delta_{\min})$ that becomes polylogarithmic in $T$ once the arms are sufficiently well separated. Our results provide theoretical support for using small subsets to approximate Pareto fronts in various multi-objective applications.
Read the full abstract

All authors Nicolas Gutowski, Fabien Chhel, Alexandre Letard, Sylvain Lamprier

Statistical Methodology

Reclaiming the "frequentist" role of marginal likelihood in Bayesian belief revision

In modern Bayesian computation and parametric estimation, the marginal likelihood, serving as the denominator P(D) in Bayes' Theorem, is routinely bypassed via unnormalized proportionality relations. Even within specialized model-selection frameworks where it is explicitly evaluated to compute Bayes Factors, the denominator is treated purely as a static constant. This note evaluates a subtle analytical oversight resulting from this computational convenience. Through the analysis of a simplified, sequential partial-information system, we show that the marginal probability possesses a critical dual layer of information: while the posterior probability determines the local magnitude of a belief update upon a solitary trial, the marginal denominator governs the physical, long-run frequentist cadence of that update across a historical horizon. Discarding the denominator computes what an observer ought to believe once a specific dataset manifests, but erases the data-generating reality that governs how frequently that inferential state occurs in nature. We propose reclaiming the marginal likelihood as an active, real-time regularizer. We introduce three diagnostic measures to regulate recursive online estimation gains, and construct valid mixture probabilities that blend the prior and posterior to function as surprise-activated or conservative regulators. This paired probability framework may offer a robust regularizing mechanism for sequential estimation architectures and quantitative risk management scenarios under non-stationary distribution shifts.
Read the full abstract

Statistics Theory

Conditional copula representations and extremal bounds for multivariate statistical functionals

In this paper, we derive a conditional copula representation for expectations of the form $\mathbb{E}[g(\boldsymbol{X})]$, where $\boldsymbol{X}$ is a random vector with arbitrary marginal distributions and $g$ is a measurable function satisfying suitable integrability conditions. The proposed representation explicitly separates the contributions of the marginal distributions and the dependence structure through conditional copula distributions, yielding a unified quantile–copula framework for a broad class of statistical functionals. This framework encompasses numerous quantities of practical interest, including moments, probabilities, dependence measures, inequality indices, entropy measures, and multivariate functionals. We further establish extremal bounds under fixed marginals by exploiting the concordance order on copulas and characterize the classes of functions for which these bounds apply through the notion of $\Delta$-antitonicity. Finally, several illustrative examples illustrate the versatility of the proposed framework through applications to risk measures, stochastic superiority probabilities, information measures, and option pricing under dependence uncertainty.
Read the full abstract

All authors Roberto Vila, Cira E G Otiniano, Carolyne Brito, Enzo Brasil

Statistics Applications · Cross-listed from cs.CL

A large-scale corpus of religious radio broadcast transcripts from webstream recordings in the United States

Religious radio is a widespread but understudied form of mass communication in the United States, and content-level analysis of it has been constrained by the absence of large-scale transcript data. This Data Descriptor presents a corpus of transcribed English-language religious radio broadcasts captured from live webstreams over a one-month period in July 2025. Fifteen-minute segments were recorded on a rolling schedule from 785 distinct streams, which together rebroadcast the signals of more than two thousand AM and FM stations, yielding over 700,000 recordings and more than 60 million diarized lines of speech. Each recording was transcribed and speaker-diarized with an automated pipeline, and segmented and labeled by programming format and topic using a large language model. The corpus is organized as linked tables of stream metadata, recording metadata, and transcript lines. It supports descriptive study of religious broadcasting across regions and traditions, analysis of how social and political issues are discussed in religious media, and speech-processing research in an underrepresented domain.
Read the full abstract

All authors Samuel Bestvater, Athena Chapekis, Skyler Seets, Anna Lieb, Sono Shah, Aaron Smith

Statistical Methodology

Retrospective Orthogonal Design: Response-Surface Reconstruction from Observational Data

Regression estimates from observational data can depend on specification under multicollinearity, while sequential sums of squares (SS) depend on term order. We introduce Retrospective Orthogonal Design (ROD), which reconstructs conditional mean surfaces on a probability-balanced lattice. ROD preserves observed cell means, completes unsupported cells, applies weighted tensor-product contrasts, and evaluates the reconstructed surface through piecewise-affine interpolation over Freudenthal polyhedra. Resolution and completion are selected jointly by validation among rank-admissible candidates, followed by refitting and evaluation on an untouched test set. For an admissible lattice, $\mathbf{X}^{\top}\mathbf{W}\mathbf{X}=c\mathbf{I}$, yielding specification-invariant contrast effects and unique, order-independent SS within the retained contrast space. Response-free projection calibration maps the fixed reconstruction onto a declared scientific basis and corrects finite-resolution recovery loss. Across 6,480 simulation conditions spanning nine data-generating processes, ROD matched or exceeded polynomial regression in five processes and performed strongest on threshold, sign-interaction, and localized surfaces. For the quadratic-interaction process, mean out-of-sample $R^2$ differed by only $0.0001$, while calibrated coefficient bias remained small across prespecified targets. A Rao-based information adjustment provides dependence-aware sample-size guidance for ROD planning. In a weighted Mincer application, ROD produced the highest out-of-sample $R^2$ point estimate, with substantial interval overlap with polynomial regression, and provided exhaustive SS allocations invariant to term-entry order.
Read the full abstract

All authors Lawrence Fulton, Christopher Fulton, Arvind Sharma, Aleksandar Tomic

Statistics Applications

Statistical Learning of Pediatric Mental Health-Related Emergency Department Visits Across COVID-19 Pandemic Periods

This article presents a statistical learning framework for studying the evolution of pediatric mental health-related emergency department (MHED) visit patterns across the pre-, during-, and post-COVID-19 pandemic periods using population-based administrative health records. The MHED records are formulated as zero-truncated recurrent event data, partitioned into three successive time periods. We develop the modeling framework in a stepwise manner, guided by model fit using a collection of MHED records. The resulting framework progresses from nonparametric marginal rate models to more structured Cox-type regression models for characterizing visit patterns. We ultimately apply stratified regression analysis to investigate changes in visit frequencies and covariate effects across pandemic periods, accounting for prespecified period cut-off points and coarsened individual follow-up information. The proposed framework is motivated by and illustrated using pediatric MHED data throughout the article, providing a practical approach for analyzing recurrent healthcare utilization data with evolving temporal patterns.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.DS

Randomizing the Number of Centers in k-means++

The $k$-means++ algorithm is a standard and widely used seeding method for $k$-means clustering, but for a fixed number $k$ of centers its worst-case expected approximation ratio is $\Theta(\log k)$. We consider the same algorithm when an adversary first fixes the dataset and some $K$; the number of centers $k$ is then chosen uniformly from $\{K,\ldots,2K-1\}$. We prove that $k$-means++ is an $O(1)$-approximation with constant probability in this budget-smoothed setup.
Read the full abstract

Statistical Methodology

The Dirichlet Process as sampling distribution

The Dirichlet process (DP) is the most common bayesian nonparametric prior, however, its properties as sampling distribution have not been studied nor inference on its parameters. Here we use the DP as a data generating model and make bayesian inference on its centering measure and precision parameter. We illustrate with a sequence of histograms as observed data. In particular, we consider simulated and real datasets.
Read the full abstract