Peer Revenue Past citations are no guarantee of future returns

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Thursday 12 February

Statistical Methodology

Scalable Changepoint Detection for Large Spatiotemporal Data on the Sphere

We propose a novel Bayesian framework for changepoint detection in large-scale spherical spatiotemporal data, with broad applicability in environmental and climate sciences. Our approach models changepoints as spatially dependent categorical variables using a multinomial probit model (MPM) with a latent Gaussian process, effectively capturing complex spatial correlation structures on the sphere. To handle the high dimensionality inherent in global datasets, we leverage stochastic partial differential equations (SPDE) and spherical harmonic transformations for efficient representation and scalable inference, drastically reducing computational burden while maintaining high accuracy. Through extensive simulation studies, we demonstrate the efficiency and robustness of the proposed method for changepoint estimation, as well as the significant computational gains achieved through the combined use of the MPM and truncated spectral representations of latent processes. Finally, we apply our method to global aerosol optical depth data, successfully identifying changepoints associated with a major atmospheric event.
Read the full abstract

Statistics Theory · Cross-listed from quant-ph

Reconstruction of finite Quasi-Probability and Probability from Principles: The Role of Syntactic Locality

Quasi-probabilities appear across diverse areas of physics, but their conceptual foundations remain unclear: they are often treated merely as computational tools, and operations like conditioning and Bayes' theorem become ambiguous. We address both issues by developing a principled framework that derives quasi-probabilities and their conditional calculus from structural consistency requirements on how statements are valued across different universes of discourse, understood as finite Boolean algebras of statements.We begin with a universal valuation that assigns definite (possibly complex) values to all statements. The central concept is Syntactic Locality: every universe can be embedded within a larger ambient one, and the universal valuation must behave coherently under such embeddings and restrictions. From a set of structural principles, we prove a representation theorem showing that every admissible valuation can be re-expressed as a finitely additive measure on mutually exclusive statements, mirroring the usual probability sum rule. We call such additive representatives pre-probabilities. This representation is unique up to an additive regraduation freedom. When this freedom can be fixed canonically, pre-probabilities reduce to finite quasi-probabilities, thereby elevating quasi-probability theory from a computational device to a uniquely determined additive representation of universal valuations. Classical finite probabilities arise as the subclass of quasi-probabilities stable under relativisation, i.e., closed under restriction to sub-universes. Finally, the same framework enables us to define a coherent theory of conditionals, yielding a well-defined generalized Bayes' theorem applicable to both pre-probabilities and quasi-probabilities. We conclude by discussing additional regularity conditions, including the role of rational versus irrational probabilities in this setting.
Read the full abstract

Statistical Methodology

Batch-based Bayesian Optimal Experimental Design in Linear Inverse Problems

Experimental design is central to science and engineering. A ubiquitous challenge is how to maximize the value of information obtained from expensive or constrained experimental settings. Bayesian optimal experimental design (OED) provides a principled framework for addressing such questions. In this paper, we study experimental design problems such as the optimization of sensor locations over a continuous domain in the context of linear Bayesian inverse problems. We focus in particular on batch design, that is, the simultaneous optimization of multiple design variables, which leads to a notoriously difficult non-convex optimization problem. We tackle this challenge using a promising strategy recently proposed in the frequentist setting, which relaxes A-optimal design to the space of finite positive measures. Our main contribution is the rigorous identification of the Bayesian inference problem corresponding to this relaxed A-optimal OED formulation. Moreover, building on recent work, we develop a Wasserstein gradient-flow -based optimization algorithm for the expected utility and introduce novel regularization schemes that guarantee convergence to an empirical measure. These theoretical results are supported by numerical experiments demonstrating both convergence and the effectiveness of the proposed regularization strategy.
Read the full abstract

Other Statistics

Bayesian inference for the automultinomial model with an application to landcover data

Multicategory lattice data arise in a wide variety of disciplines such as image analysis, biology, and forestry. We consider modeling such data with the automultinomial model, which can be viewed as a natural extension of the autologistic model to multicategory responses, or equivalently as an extension of the Potts model that incorporates covariate information into a pure-intercept model. The automultinomial model has the advantage of having a unique parameter that controls the spatial correlation. However, the model's likelihood involves an intractable normalizing function of the model parameters that poses serious computational problems for likelihood-based inference. We address this difficulty by performing Bayesian inference through the Double-Metropolis Hastings algorithm, and implement diagnostics to assess the convergence to the target posterior distribution. Through simulation studies and an application to land cover data, we find that the automultinomial model is flexible across a wide range of spatial correlations while maintaining a relatively simple specification. For large data sets we find it also has advantages over spatial generalized linear mixed models. To make this model practical for scientists, we provide recommendations for its specification and computational implementation.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

On the Complexity of Offline Reinforcement Learning with $Q^\star$-Approximation and Partial Coverage

We study offline reinforcement learning under $Q^\star$-approximation and partial coverage, a setting that motivates practical algorithms such as Conservative $Q$-Learning (CQL; Kumar et al., 2020) but has received limited theoretical attention. Our work is inspired by the following open question: "Are $Q^\star$-realizability and Bellman completeness sufficient for sample-efficient offline RL under partial coverage?" We answer in the negative via an information-theoretic lower bound. To identify additional structure that enables sample-efficient offline RL under partial coverage, we introduce a general decision-estimation framework, inspired by model-free decision-estimation coefficients (DEC) for online RL (Foster et al., 2023b; Liu et al., 2025b). Our framework decomposes offline RL complexity into decision complexity and value estimation error. This allows modular study of both sub-problems. Our result not only unifies existing results (Chen and Jiang, 2022; Uehara et al., 2023), but further improves and generalizes them. On the decision complexity side, our improvement includes: the first $\epsilon^{-2}$ sample complexity bound for soft $Q$-learning under partial coverage that improves Uehara et al.'s (2023) $\epsilon^{-4}$ bound, the removal of the need for additional online interaction in the value-gap setting of Chen and Jiang (2022), and new learnable settings beyond the above two cases. On the value estimation side, we provide a new characterization of the role of Bellman completeness under partial coverage, and the first characterization of offline learnability for general low-Bellman-rank MDPs (Jiang et al., 2017; Du et al., 2021; Jin et al., 2021). The latter is a canonical online RL setting that has remained unexplored in offline RL except for special cases. As a side contribution, our techniques give the first analysis of CQL in the function approximation setting.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Empirical Gaussian Processes

Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This kernel function is typically handcrafted from a small set of standard functions, a process that requires expert knowledge, results in limited adaptivity to data, and imposes strong assumptions on the hypothesis space. We study Empirical GPs, a principled framework for constructing flexible, data-driven GP priors that overcome these limitations. Rather than relying on standard parametric kernels, we estimate the mean and covariance functions empirically from a corpus of historical observations, enabling the prior to reflect rich, non-trivial covariance structures present in the data. Theoretically, we show that the resulting model converges to the GP that is closest (in KL-divergence sense) to the real data generating process. Practically, we formulate the problem of learning the GP prior from independent datasets as likelihood estimation and derive an Expectation-Maximization algorithm with closed-form updates, allowing the model handle heterogeneous observation locations across datasets. We demonstrate that Empirical GPs achieve competitive performance on learning curve extrapolation and time series forecasting benchmarks.
Read the full abstract

All authors Jihao Andreas Lin, Sebastian Ament, Louis C. Tiao, David Eriksson, Maximilian Balandat, Eytan Bakshy

Statistics Applications

Enhanced Forest Inventories for Habitat Mapping: A Case Study in the Sierra Nevada Mountains of California

Traditional forest inventory systems, originally designed to quantify merchantable timber volume, often lack the spatial resolution and structural detail required for modern multi-resource ecosystem management. In this manuscript, we present an Enhanced Forest Inventory (EFI) and demonstrate its utility for high-resolution wildlife habitat mapping. The project area covers 270,000 acres of the Eldorado National Forest in California's Sierra Nevada. By integrating 118 ground-truth Forest Inventory and Analysis (FIA) plots with multi-modal remote sensing data (LiDAR, aerial photography, and Sentinel-2 satellite imagery), we developed predictive models for key forest attributes. Our methodology employed a two-tier segmentation approach, partitioning the landscape into approximately 575,000 reporting units with an average size of 0.5 acre to capture forest heterogeneity. We utilized an Elastic-Net Regression framework and automated feature selection to relate remote sensing metrics to ground-measured variables such as basal area, stems per acre, and canopy cover. These physical metrics were translated into functional habitat attributes to evaluate suitability for two focal species: the California Spotted Owl (Strix occidentalis occidentalis) and the Pacific Fisher (Pekania pennanti). Our analysis identified 25,630 acres of nesting and 26,622 acres of foraging habitat for the owl, and 25,636 acres of likely habitat for the fisher based on structural requirements like large-diameter trees and high canopy closure. The results demonstrate that EFIs provide a critical bridge between forestry and conservation ecology, offering forest managers a spatially explicit tool to monitor ecosystem health and manage vulnerable species in complex environments.
Read the full abstract

All authors Maxime Turgeon, Michael Kieser, Dwight Wolfe, Bruce MacArthur

Statistical Methodology · Cross-listed from econ.EM

Improved Inference for CSDID Using the Cluster Jackknife

Obtaining reliable inferences with traditional difference-in-differences (DiD) methods can be difficult. Problems can arise when both outcomes and errors are serially correlated, when there are few clusters or few treated clusters, when cluster sizes vary greatly, and in various other cases. In recent years, recognition of the “staggered adoption” problem has shifted the focus away from inference towards consistent estimation of treatment effects. One of the most popular new estimators is the CSDID procedure of Callaway and Sant'Anna (2021). We find that the issues of over-rejection with few clusters and/or few treated clusters are at least as severe for CSDID as for traditional DiD methods. We also propose using a cluster jackknife for inference with CSDID, which simulations suggest greatly improves inference. We provide software packages in Stata csdidjack and R didjack to calculate cluster-jackknife standard errors easily.
Read the full abstract

All authors Sunny R. Karim, Morten Ørregaard Nielsen, James G. MacKinnon, Matthew D. Webb

Machine Learning (Statistics)

The Implicit Bias of Logit Regularization

Logit regularization, the addition of a convex penalty directly in logit space, is widely used in modern classifiers, with label smoothing as a prominent example. While such methods often improve calibration and generalization, their mechanism remains under-explored. In this work, we analyze a general class of such logit regularizers in the context of linear classification, and demonstrate that they induce an implicit bias of logit clustering around finite per-sample targets. For Gaussian data, or whenever logits are sufficiently clustered, we prove that logit clustering drives the weight vector to align exactly with Fisher's Linear Discriminant. To demonstrate the consequences, we study a simple signal-plus-noise model in which this transition has dramatic effects: Logit regularization halves the critical sample complexity and induces grokking in the small-noise limit, while making generalization robust to noise. Our results extend the theoretical understanding of label smoothing and highlight the efficacy of a broader class of logit-regularization methods.
Read the full abstract

Statistics Theory

General-purpose post-sampling reweighting method for multimodal target measures

When sampling multi-modal probability distributions, correctly estimating the relative probability of each mode, even when the modes have been discovered and locally sampled, remains challenging. We test a simple reweighting scheme designed for this situation, which consists in minimizing (in terms of weights) the Kullback-Leibler divergence of a weighted (regularized) empirical distribution of the samples with respect to the target measure. We show in a theoretical toy model that the convergence in terms of the number of samples follows the Monte Carlo rate. Numerical experiments on a variety of simple benchmarks are provided, exploring the effect of the ambient dimension, number of modes, number of samples and definition of the modes. We find that, particularly in the situation of interest with well-identified separated modes, the method is able to correct the error in the estimation from the initial sampling of the relative probabilities of the modes.
Read the full abstract

Statistics Theory · Cross-listed from econ.EM

Decomposition of Spillover Effects Under Misspecification: Pseudo-true Estimands and a Local-Global Extension

To measure spillovers, researchers often summarize who else was treated with a simple measure, such as the share of treated neighbors. We study what the researcher estimates when that summary is misspecified. We show that the researcher estimates the best approximation to the true policy effect among all functionals of the chosen summary, and that the usual direct and spillover estimates are exactly the components of this approximation. We then specialize this framework to a setting common in applications such as cash transfers: treatment spills over both globally, through market equilibrium, and locally, through network externalities, while the researcher models only one channel. The network estimator still recovers the network spillover, and the equilibrium estimator the equilibrium spillover, even when the other channel is ignored. We illustrate with a simulation calibrated to a large cash-transfer experiment.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.OC

Change-Point Detection via Piecewise Linear Fitting Using MIP

We present a new mixed-integer programming (MIP) approach for offline multiple change-point detection by casting the problem as a globally optimal piecewise linear (PWL) fitting problem. Our main contribution is a family of strengthened MIP formulations whose linear programming (LP) relaxations admit integral projections onto the segment-assignment variables, which encode the segment membership of each data point. This property yields provably tighter relaxations than existing formulations for offline multiple change-point detection. We further extend the framework to multi-dimensional PWL models with shared change-points. Extensive computational experiments on benchmark real-world datasets demonstrate that the proposed formulations achieve reductions in solution times in comparison to the state-of-the-art.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Learning Conditional Averages

We introduce the problem of learning conditional averages in the PAC framework. The learner receives a sample labeled by an unknown target concept from a known concept class, as in standard PAC learning. However, instead of learning the target concept itself, the goal is to predict, for each instance, the average label over its neighborhood – an arbitrary subset of points that contains the instance. In the degenerate case where all neighborhoods are singletons, the problem reduces exactly to classic PAC learning. More generally, it extends PAC learning to a setting that captures learning tasks arising in several domains, including explainability, fairness, and recommendation systems. Our main contribution is a complete characterization of when conditional averages are learnable, together with sample complexity bounds that are tight up to logarithmic factors. The characterization hinges on the joint finiteness of two novel combinatorial parameters, which depend on both the concept class and the neighborhood system, and are closely related to the independence number of the associated neighborhood graph.
Read the full abstract

All authors Marco Bressan, Nataly Brukhim, Nicolo Cesa-Bianchi, Emmanuel Esposito, Yishay Mansour, Shay Moran, Maximilian Thiessen

Statistical Methodology · Cross-listed from eess.IV

Time-resolved aortic 3D shape reconstruction from a limited number of cine 2D MRI slices

Background and Objective: To assess the feasibility and accuracy of reconstructing time-resolved, three-dimensional, subject-specific aortic geometries from a limited number of standard cine 2D magnetic resonance imaging (MRI) acquisitions. This is achieved by coupling a statistical shape model with a differentiable volumetric mesh optimization algorithm. Methods: Cine 2D MRI slices were manually segmented and used to reconstruct subject-specific aortic geometries via a differentiable mesh optimization algorithm, constrained by a statistical shape model. Optimal slice positioning was first evaluated on synthetic data, followed by in-vivo acquisition in 30 subjects (19 volunteers and 11 aortic stenosis patients). Time-resolved aortic geometries were reconstructed, from which geometric descriptors and radial strain were derived. In a subset of 10 subjects, 4D flow MRI data was acquired to provide volumetric reference for peak-systolic shape comparison. Results: Accurate reconstruction was achieved using as few as six cine 2D MRI slices. Agreement with 4D flow MRI reference data yielded a Dice score of (89.9 +/- 1.6) %, Intersection over Union of (81.7 +/- 2.7) %, Hausdorff distance of (7.3 +/- 3.3) mm, and Chamfer distance of (3.7 +/- 0.6) mm. The mean absolute radius error along the aortic arch was (0.8 +/- 0.6) mm. Secondary analysis demonstrated significant differences in geometric features and radial strain across age groups, with strain decreasing progressively with age at values of (11.00 +/- 3.11) x 10-2 vs. (3.74 +/- 1.25) x 10-2 vs. (2.89 +/- 0.87) x 10-2 for the young, mid-age, and elderly groups, respectively. Conclusion: The proposed framework enables reconstruction of time-resolved, subject-specific aortic geometries from a limited number of standard cine 2D MRI acquisitions, providing a practical basis for downstream computational analysis.
Read the full abstract

All authors Gloria Wolkerstorfer, Stefano Buoso, Rabea Schlenker, Jochen von Spiczak, Robert Manka, Sebastian Kozerke

Machine Learning (Statistics) · Cross-listed from cs.GT

Scale-Invariant Fast Convergence in Games

Scale-invariance in games has recently emerged as a widely valued desirable property. Yet, almost all fast convergence guarantees in learning in games require prior knowledge of the utility scale. To address this, we develop learning dynamics that achieve fast convergence while being both scale-free, requiring no prior information about utilities, and scale-invariant, remaining unchanged under positive rescaling of utilities. For two-player zero-sum games, we obtain scale-free and scale-invariant dynamics with external regret bounded by $\tilde{O}(A_{\mathrm{diff}})$, where $A_{\mathrm{diff}}$ is the payoff range, which implies an $\tilde{O}(A_{\mathrm{diff}} / T)$ convergence rate to Nash equilibrium after $T$ rounds. For multiplayer general-sum games with $n$ players and $m$ actions, we obtain scale-free and scale-invariant dynamics with swap regret bounded by $O(U_{\mathrm{max}} \log T)$, where $U_{\mathrm{max}}$ is the range of the utilities, ignoring the dependence on the number of players and actions. This yields an $O(U_{\mathrm{max}} \log T / T)$ convergence rate to correlated equilibrium. Our learning dynamics are based on optimistic follow-the-regularized-leader with an adaptive learning rate that incorporates the squared path length of the opponents' gradient vectors, together with a new stopping-time analysis that exploits negative terms in regret bounds without scale-dependent tuning. For general-sum games, scale-free learning is enabled also by a technique called doubling clipping, which clips observed gradients based on past observations.
Read the full abstract

Machine Learning (Statistics)

Aggregate Models, Not Explanations: Improving Feature Importance Estimation

Feature-importance methods show promise in transforming machine learning models from predictive engines into tools for scientific discovery. However, due to data sampling and algorithmic stochasticity, expressive models can be unstable, leading to inaccurate variable importance estimates and undermining their utility in critical biomedical applications. Although ensembling offers a solution, deciding whether to explain a single ensemble model or aggregate individual model explanations is difficult due to the nonlinearity of importance measures and remains largely understudied. Our theoretical analysis, developed under assumptions accommodating complex state-of-the-art ML models, reveals that this choice is primarily driven by the model's excess risk. In contrast to prior literature, we show that ensembling at the model level provides more accurate variable-importance estimates, particularly for expressive models, by reducing this leading error term. We validate these findings on classical benchmarks and a large-scale proteomic study from the UK Biobank.
Read the full abstract

All authors Joseph Paillard, Angel Reyero Lobo, Denis A. Engemann, Bertrand Thirion

Statistics Theory

High-Probability Minimax Adaptive Estimation in Besov Spaces via Online-to-Batch

We study nonparametric regression over Besov spaces from noisy observations under sub-exponential noise, aiming to achieve minimax-optimal guarantees on the integrated squared error that hold with high probability and adapt to the unknown noise level. To this end, we propose a wavelet-based online learning algorithm that dynamically adjusts to the observed gradient noise by adaptively clipping it at an appropriate level, eliminating the need to tune parameters such as the noise variance or gradient bounds. As a by-product of our analysis, we derive high-probability adaptive regret bounds that scale with the $\ell_1$-norm of the competitor. Finally, in the batch statistical setting, we obtain adaptive and minimax-optimal estimation rates for Besov spaces via a refined online-to-batch conversion. This approach carefully exploits the structure of the squared loss in combination with self-normalized concentration inequalities.
Read the full abstract

Machine Learning (Statistics)

PAC-Bayesian Generalization Guarantees for Fairness on Stochastic and Deterministic Classifiers

Classical PAC generalization bounds on the prediction risk of a classifier are insufficient to provide theoretical guarantees on fairness when the goal is to learn models balancing predictive risk and fairness constraints. We propose a PAC-Bayesian framework for deriving generalization bounds for fairness, covering both stochastic and deterministic classifiers. For stochastic classifiers, we derive a fairness bound using standard PAC-Bayes techniques. Whereas for deterministic classifiers, as usual PAC-Bayes arguments do not apply directly, we leverage a recent advance in PAC-Bayes to extend the fairness bound beyond the stochastic setting. Our framework has two advantages: (i) It applies to a broad class of fairness measures that can be expressed as a risk discrepancy, and (ii) it leads to a self-bounding algorithm in which the learning procedure directly optimizes a trade-off between generalization bounds on the prediction risk and on the fairness. We empirically evaluate our framework with three classical fairness measures, demonstrating not only its usefulness but also the tightness of our bounds.
Read the full abstract

All authors Julien Bastian, Benjamin Leblanc, Pascal Germain, Amaury Habrard, Christine Largeron, Guillaume Metzler, Emilie Morvant, Paul Viallard

Statistical Methodology · Cross-listed from cs.LG

Potential-energy gating for robust state estimation in bistable stochastic systems

We introduce potential-energy gating, a method for robust state estimation in systems governed by double-well stochastic dynamics. The observation noise covariance of a Bayesian filter is modulated by the local value of a known or assumed potential energy function: observations are trusted when the state is near a potential minimum and progressively discounted as it approaches the barrier separating metastable wells. This physics-based mechanism differs from statistical robust filters, which treat all state-space regions identically, and from constrained filters, which bound states rather than modulating observation trust. The approach is especially relevant in non-ergodic or data-scarce settings where only a single realization is available and statistical methods alone cannot learn the noise structure. We implement gating within Extended, Unscented, Ensemble, and Adaptive Kalman filters and particle filters, requiring only two additional hyperparameters. Monte Carlo benchmarks (100 replications) on a Ginzburg-Landau double-well with 10% outlier contamination show 57-80% RMSE improvement over the standard Extended Kalman Filter, all statistically significant (p < 10^{-15}, Wilcoxon test). A naive topological baseline using only well positions achieves 57%, confirming that the continuous energy landscape adds ~21 percentage points. The method is robust to misspecification: even with 50% parameter errors, improvement never falls below 47%. Comparing externally forced and spontaneous Kramers-type transitions, gating retains 68% improvement under noise-induced transitions whereas the naive baseline degrades to 30%. As an empirical illustration, we apply the framework to Dansgaard-Oeschger events in the NGRIP delta-18O ice-core record, estimating asymmetry gamma = -0.109 (bootstrap 95% CI: [-0.220, -0.011]) and showing that outlier fraction explains 91% of the variance in filter improvement.
Read the full abstract

Machine Learning (Statistics)

Estimation of instrument and noise parameters for inverse problem based on prior diffusion model

This article addresses the issue of estimating observation parameters (response and error parameters) in inverse problems. The focus is on cases where regularization is introduced in a Bayesian framework and the prior is modeled by a diffusion process. In this context, the issue of posterior sampling is known to be thorny, and a recent paper proposes a notably simple and effective solution. Additionally, it opens an remarkable flexibility when it comes to estimating observation parameters. The proposed strategy enables to define an optimal estimator for both observation parameters and image of interest. Furthermore, the strategy provides a means for uncertainty quantification. In addition, MCMC algorithms allow for the computation of estimates and properties of posteriors, while offering some guarantees. The paper presents several numerical experiments that clearly confirm the computational efficiency and the quality of both estimates and uncertainty quantification.
Read the full abstract

Machine Learning (Statistics)

Provable Offline Reinforcement Learning for Structured Cyclic MDPs

We introduce a novel cyclic Markov decision process (MDP) framework for multi-step decision problems with heterogeneous stage-specific dynamics, transitions, and discount factors across the cycle. In this setting, offline learning is challenging: optimizing a policy at any stage shifts the state distributions of subsequent stages, propagating mismatch across the cycle. To address this, we propose a modular structural framework that decomposes the cyclic process into stage-wise sub-problems. While generally applicable, we instantiate this principle as CycleFQI, an extension of fitted Q-iteration enabling theoretical analysis and interpretation. It uses a vector of stage-specific Q-functions, tailored to each stage, to capture within-stage sequences and transitions between stages. This modular design enables partial control, allowing some stages to be optimized while others follow predefined policies. We establish finite-sample suboptimality error bounds and derive global convergence rates under Besov regularity, demonstrating that CycleFQI mitigates the curse of dimensionality compared to monolithic baselines. Additionally, we propose a sieve-based method for asymptotic inference of optimal policy values under a margin condition. Experiments on simulated and real-world Type 1 Diabetes data sets demonstrate CycleFQI's effectiveness.
Read the full abstract

Statistical Methodology

Improving the adjusted Benjamini–Hochberg method using e-values in knockoff-assisted variable selection

Considering the knockoff-based multiple testing framework of Barber and Candès [2015], we revisit the method of Sarkar and Tang [2022] and identify it as a specific case of an un-normalized e-value weighted Benjamini-Hochberg procedure. Building on this insight, we extend the method to use bounded p-to-e calibrators that enable more refined and flexible weight assignments. Our approach generalizes the method of Sarkar and Tang [2022], which emerges as a special case corresponding to an extreme calibrator. Within this framework, we propose three procedures: an e-value weighted Benjamini-Hochberg method, its adaptive extension using an estimate of the proportion of true null hypotheses, and an adaptive weighted Benjamini-Hochberg method. We establish control of the false discovery rate (FDR) for the proposed methods. While we do not formally prove that the proposed methods outperform those of Barber and Candès [2015] and Sarkar and Tang [2022], simulation studies and real-data analysis demonstrate large and consistent improvement over the latter in all cases, and better performance than the knockoff method in scenarios with low target FDR, a small number of signals, and weak signal strength. Simulation studies and a real-data application in HIV-1 drug resistance analysis demonstrate strong finite sample FDR control and exhibit improved, or at least competitive, power relative to the aforementioned methods.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient

A variety of widely used optimization methods like SignSGD and Muon can be interpreted as instances of steepest descent under different norm-induced geometries. In this work, we study the implicit bias of mini-batch stochastic steepest descent in multi-class classification, characterizing how batch size, momentum, and variance reduction shape the limiting max-margin behavior and convergence rates under general entry-wise and Schatten-$p$ norms. We show that, without momentum, worst-case convergence and successful classification can only be guaranteed with full-batch gradient. In contrast, momentum enables small-batch convergence to an approximate max-margin solution through a batch-momentum trade-off, though it slows convergence. This approach provides fully explicit, dimension-free rates that improve upon prior results. Moreover, we prove that variance reduction can recover the exact full-batch implicit bias for any batch size, albeit at a slower convergence rate. Finally, we further investigate the batch-size-one steepest descent without momentum, and reveal its convergence to a fundamentally different bias via a concrete data example, which reveals a key limitation of purely stochastic updates. Overall, our unified analysis clarifies when stochastic optimization aligns with full-batch behavior, and paves the way for perform deeper explorations of the training behavior of stochastic gradient steepest descent algorithms.
Read the full abstract

Statistical Methodology

Locally Interpretable Individualized Treatment Rules for Black-Box Decision Models

Individualized treatment rules (ITRs) aim to optimize healthcare by tailoring treatment decisions to patient-specific characteristics. Existing methods typically rely on either interpretable but inflexible models or highly flexible black-box approaches that sacrifice interpretability; moreover, most impose a single global decision rule across patients. We introduce the Locally Interpretable Individualized Treatment Rule (LI-ITR) method, which combines flexible machine learning models to accurately learn complex treatment outcomes with locally interpretable approximations to construct subject-specific treatment rules. LI-ITR employs variational autoencoders to generate realistic local synthetic samples and learns individualized decision rules through a mixture of interpretable experts. Simulation studies show that LI-ITR accurately recovers true subject-specific local coefficients and optimal treatment strategies. An application to precision side-effect management in breast cancer illustrates the necessity of flexible predictive modeling and highlights the practical utility of LI-ITR in estimating optimal treatment rules while providing transparent, clinically interpretable explanations.
Read the full abstract

Statistical Methodology

Representation Learning with Blockwise Missingness and Signal Heterogeneity

Unified representation learning for multi-source data integration faces two important challenges: blockwise missingness and blockwise signal heterogeneity. The former arises from sources observing different, yet potentially overlapping, feature sets, while the latter involves varying signal strengths across subject groups and feature sets. While existing methods perform well with fully observed data or uniform signal strength, their performance degenerates when these two challenges coincide, which is common in practice. To address this, we propose Anchor Projected Principal Component Analysis (APPCA), a general framework for representation learning with structured blockwise missingness that is robust to signal heterogeneity. APPCA first recovers robust group-specific column spaces using all observed feature sets, and then aligns them by projecting shared "anchor" features onto these subspaces before performing PCA. This projection step induces a significant denoising effect. We establish estimation error bounds for embedding reconstruction through a fine-grained perturbation analysis. In particular, using a novel spectral slicing technique, our bound eliminates the standard dependency on the signal strength of subject embeddings, relying instead solely on the signal strength of integrated feature sets. We validate the proposed method through extensive simulation studies and an application to multimodal single-cell sequencing data.
Read the full abstract

Statistical Methodology

High-Dimensional Mediation Analysis for Generalized Linear Models Using Bayesian Variable Selection Guided by Mediator Correlation

High-dimensional mediation analysis aims to identify mediating pathways and to estimate indirect effects linking an exposure to an outcome. In this paper, we propose a Bayesian framework to address key challenges in these analyses, including high dimensionality, complex dependence among omics mediators, and non-continuous outcomes. Furthermore, commonly used approaches assume independent mediators or ignore correlations in the selection stage, which can reduce power when mediators are highly correlated. Addressing these challenges leads to a non-Gaussian likelihood and specialized selection priors, which in turn require efficient and adaptive posterior computation. Our proposed framework selects active pathways under generalized linear models while accounting for mediator dependence. Specifically, the mediators are modeled using a multivariate distribution, exposure-mediator selection is guided by a Markov random field prior on inclusion indicators, and mediator-outcome activation is restricted to mediators supported in the exposure-mediator model through a sequential subsetting Bernoulli prior. Simulation studies show improved operating characteristics in correlated-mediator settings, with appropriate error control under the global null and stable performance under model misspecification. We illustrate the method using real-world metabolomics data to study metabolites that mediate the association between adherence to the Alternate Mediterranean Diet score and two cardiometabolic outcomes.
Read the full abstract

All authors Youngho Bae, Chanmin Kim, Fenglei Wang, Qi Sun, Kyu Ha Lee

Wednesday 11 February

Machine Learning (Statistics) · Cross-listed from cs.LG

Support Vector Data Description for Radar Target Detection

Classical radar detection techniques rely on adaptive detectors that estimate the noise covariance matrix from target-free secondary data. While effective in Gaussian environments, these methods degrade in the presence of clutter, which is better modeled by heavy-tailed distributions such as the Complex Elliptically Symmetric (CES) and Compound-Gaussian (CGD) families. Robust covariance estimators like M-estimators or Tyler's estimator address this issue, but still struggle when thermal noise combines with clutter. To overcome these challenges, we investigate the use of Support Vector Data Description (SVDD) and its deep extension, Deep SVDD, for target detection. These one-class learning methods avoid direct noise covariance estimation and are adapted here as CFAR detectors. We propose two novel SVDD-based detection algorithms and demonstrate their effectiveness on simulated radar data.
Read the full abstract

All authors Jean Pinsolle, Yadang Alexis Rouzoumka, Chengfang Ren, Chistèle Morisseau, Jean-Philippe Ovarlez

Machine Learning (Statistics)

The Cost of Learning Under Multiple Change Points

We consider an online learning problem in environments with multiple change points. In contrast to the single change point problem that is widely studied using classical "high confidence" detection schemes, the multiple change point environment presents new learning-theoretic and algorithmic challenges. Specifically, we show that classical methods may exhibit catastrophic failure (high regret) due to a phenomenon we refer to as endogenous confounding. To overcome this, we propose a new class of learning algorithms dubbed Anytime Tracking CUSUM (ATC). These are horizon-free online algorithms that implement a selective detection principle, balancing the need to ignore "small" (hard-to-detect) shifts, while reacting "quickly" to significant ones. We prove that the performance of a properly tuned ATC algorithm is nearly minimax-optimal; its regret is guaranteed to closely match a novel information-theoretic lower bound on the achievable performance of any learning algorithm in the multiple change point problem. Experiments on synthetic as well as real-world data validate the aforementioned theoretical findings.
Read the full abstract

Statistical Methodology

Who's Winning? Clarifying Estimands Based on Win Statistics in Cluster Randomized Trials

Treatment effect estimands based on win statistics, including the win ratio, win odds, and win difference are increasingly popular targets for summarizing endpoints in clinical trials. Such win estimands offer an intuitive approach for prioritizing outcomes by clinical importance. The implementation and interpretation of win estimands is complicated in cluster randomized trials (CRTs), where researchers can target fundamentally different estimands on the individual-level or cluster-level. We numerically demonstrate that individual-pair and cluster-pair win estimands can substantially differ when cluster size is informative: where outcomes and/or treatment effects depend on cluster size. With such informative cluster sizes, individual-pair and cluster-pair win estimands can even yield opposite conclusions regarding treatment benefit. We describe consistent estimators for individual-pair and cluster-pair win estimands and propose a leave-one-cluster-out jackknife variance estimator for inference. Despite being consistent, our simulations highlight that some caution is needed when implementing individual-pair win estimators due to finite-sample bias. In contrast, cluster-pair win estimators are unbiased for their respective targets. Altogether, careful specification of the target estimand is essential when applying win estimators in CRTs. Failure to clearly define whether individual-pair or cluster-pair win estimands are of primary interest may result in answering a dramatically different question than intended.
Read the full abstract

All authors Kenneth M. Lee, Xi Fang, Fan Li, Michael O. Harhay

Statistics Applications

Regularized Ensemble Forecasting for Learning Weights from Historical and Current Forecasts

Combining forecasts from multiple experts often yields more accurate results than relying on a single expert. In this paper, we focus on forecasting a continuous outcome and introduce a novel regularized ensemble method that extends the traditional linear opinion pool by leveraging both current forecasts and historical performances to set the weights. In contrast to existing approaches that emphasize either current forecasts or past accuracy, our method accounts for both sources simultaneously. It learns weights by minimizing the variance of the combined forecast (or its transformed version) while incorporating a regularization term informed by historical performances. We also show that this approach has a Bayesian interpretation. Different distributional assumptions within this Bayesian framework yield different functional forms for the variance component and the regularization term, adapting the method to various scenarios. In empirical studies on Walmart sales and macroeconomic forecasting, our ensemble outperforms leading benchmark models both when experts' full forecasting histories are available and when experts enter and exit over time. Throughout, we also characterize how the optimal weights depend on current forecasts and historical performance, and use the empirical results to discuss where the framework's strengths lie and when each source of information plays a greater role.
Read the full abstract