Statistics Theory
Functional Bias and Tangent-Space Geometry in Variational Inference
Sean Plummer
Variational inference approximates
Bayesian posterior distributions by projecting onto a tractable family of distributions. While most theoretical analyses evaluate the quality of this approximation using global divergence measures, many applications rely on specific posterior summaries such as expectations, variances, or tail probabilities. We develop a
geometric framework for analyzing the bias of posterior functionals under variational approximations. We show that the leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space induced by the variational family. Functionals aligned with this space incur only second-order bias. For structured mean-field variational families we characterize the tangent space explicitly and show that it consists of block-additive functions of the parameter blocks, while interaction components determine the leading-order bias. Under standard local asymptotic normality conditions we further derive explicit asymptotic expansions for the bias of posterior functionals and show that omitted interaction directions produce first-order distortion of cross-block dependence measures. These results provide a
geometric explanation for several well-known properties of mean-field variational inference, including the systematic distortion of cross-block dependencies.
Read the full abstract ▾
Statistics Applications
Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement
Ronald Sielinski
AI-powered answer engines are inherently non-deterministic: identical queries submitted at different times can produce different responses and cite different sources. Despite this
stochastic behavior, current approaches to measuring domain visibility in generative search typically rely on single-run point estimates of citation share and prevalence, implicitly treating them as fixed values. This paper argues that citation visibility metrics should be treated as sample
estimators of an underlying response distribution rather than fixed values. We conduct an empirical study of citation variability across three generative search platforms–Perplexity Search, OpenAI SearchGPT, and Google Gemini–using repeated sampling across three consumer product topics. Two sampling regimes are employed: daily collections over nine days and high-frequency sampling at ten-minute intervals. We show that citation distributions follow a power-law form and exhibit substantial variability across repeated samples. Bootstrap confidence intervals reveal that many apparent differences between domains fall within the noise floor of the measurement process. Distribution-wide rank stability analysis further demonstrates that citation rankings are unstable across samples, not only among top-ranked domains but throughout the frequently cited domain set. These findings demonstrate that single-run visibility metrics provide a misleadingly precise picture of domain performance in generative search. We argue that citation visibility must be reported with uncertainty estimates and provide practical guidance for sample sizes required to achieve interpretable confidence intervals.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Cross-Domain Uncertainty Quantification for Selective Prediction: A Comprehensive Bound Ablation with Transfer-Informed Betting
Abhinaba Basu
We present a comprehensive ablation of nine finite-sample bound families for selective prediction with risk control, combining concentration inequalities (Hoeffding, Empirical Bernstein, Clopper-Pearson, Wasserstein DRO, CVaR) with multiple-testing corrections (union bound, Learn Then Test fixed-sequence) and betting-based confidence sequences (WSR). Our main theoretical contribution is Transfer-Informed Betting (TIB), which warm-starts the WSR wealth process using a source domain's risk profile, achieving tighter bounds in data-scarce settings with a formal dominance guarantee. We prove that the TIB wealth process remains a valid supermartingale under all source-target divergences, that TIB dominates standard WSR when domains match, and that no data-independent warm-start can achieve better convergence. The combination of betting-based confidence sequences, LTT monotone testing, and cross-domain transfer is, to our knowledge, a three-way novelty not present in the literature. We evaluate all nine bound families on four
benchmarks-MASSIVE (n=1,102), NyayaBench (n=280), CLINC-150 (n=22.5K), and Banking77 (n=13K)-across 18 (alpha, delta) configurations. On MASSIVE at alpha=0.10, LTT eliminates the ln(K) union-bound penalty, achieving 94.0% guaranteed coverage versus 73.8% for Hoeffding-a 27% relative improvement. On NyayaBench, where the small calibration set makes Hoeffding-family bounds infeasible below alpha=0.20, Transfer-Informed Betting achieves 18.5% coverage at alpha=0.10, a 5.4x improvement over LTT + Hoeffding. We additionally compare with split-conformal prediction, showing that conformal methods produce prediction sets (avg. 1.67 classes) whereas selective prediction provides single-prediction risk guarantees. We apply these methods to agentic caching systems, formalizing a progressive trust model where the guarantee determines when cached responses can be served autonomously.
Read the full abstract ▾
Statistical Methodology
Efficient semiparametric estimation of marginal treatment effects with genetic instrumental variables
Ashish Patel, Francis J DiTraglia and Stephen Burgess
Alcohol misuse is a key target of public health strategies aimed at reducing cardiovascular risk. The effect of excessive alcohol consumption on blood pressure may vary systematically with individuals' unobserved propensity to engage in heavy drinking, complicating causal inference with observational data. The marginal treatment effects framework uses an instrumental variable for treatment choice (excessive alcohol consumption) to study how selection into treatment is linked with the treatment effect. We explore the use of a genetic instrument within this framework, which is challenging because genetic compliers (individuals for whom a change in the instrument changes their treatment choice) are likely to be a small proportion of the overall sample. This can lead to greater sampling uncertainty in the tails of the propensity score distribution, i.e., the conditional probability of choosing treatment, and in turn poor estimation of causal estimands that measure heterogeneous treatment effects. We show that the use of efficient influence functions of target estimands improves estimation in terms of robustness to sampling uncertainty in nonparametrically estimated propensity scores. We find evidence of reverse selection on gains: individuals most prone to excessive alcohol consumption experience larger adverse effects on blood pressure.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
The Temporal Markov Transition Field
Michael Leznik
The Markov Transition Field (MTF), introduced by Wang and Oates (2015), encodes a time series as a two-dimensional image by mapping each pair of time steps to the transition probability between their
quantile states, estimated from a single global transition
matrix. This construction is efficient when the transition dynamics are stationary, but produces a misleading representation when the process changes regime over time: the global
matrix averages across regimes and the resulting image loses all information about when each dynamical regime was active. In this paper we introduce the Temporal Markov Transition Field (TMTF), an extension that partitions the series into $K$ contiguous temporal chunks, estimates a separate local transition
matrix for each chunk, and assembles the image so that each row reflects the dynamics local to its chunk rather than the global average. The resulting $T \times T$ image has $K$ horizontal bands of distinct texture, each encoding the transition dynamics of one temporal segment. We develop the formal definition, establish the key structural properties of the representation, work through a complete numerical example that makes the distinction from the global MTF concrete, analyse the bias–variance trade-off introduced by temporal chunking, and discuss the
geometric interpretation of the local transition
matrices in terms of process properties such as persistence, mean reversion, and trending behaviour. The TMTF is amplitude-agnostic and order-preserving, making it suitable as an input channel for convolutional neural networks applied to time series characterisation tasks.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Multi-level meta-reinforcement learning with skill-based curriculum
Sichen Yang and Mauro Maggioni
We consider problems in sequential decision making with natural multi-level structure, where sub-tasks are assembled together to accomplish complex goals. Systematically inferring and leveraging hierarchical structure has remained a longstanding challenge; we describe an efficient multi-level procedure for repeatedly compressing Markov decision processes (MDPs), wherein a parametric family of policies at one level is treated as single actions in the compressed MDPs at higher levels, while preserving the semantic meanings and structure of the original MDP, and mimicking the natural logic to address a complex MDP. Higher-level MDPs are themselves independent MDPs with less stochasticity, and may be solved using existing
algorithms. As a byproduct, spatial or temporal scales may be coarsened at higher levels, making it more efficient to find long-term optimal policies. The multi-level representation delivered by this procedure decouples sub-tasks from each other and usually greatly reduces unnecessary stochasticity and the policy search space, leading to fewer iterations and computations when solving the MDPs. A second fundamental aspect of this work is that these multi-level decompositions plus the factorization of policies into
embeddings (problem-specific) and skills (including higher-order functions) yield new transfer opportunities of skills across different problems and different levels. This whole process is framed within curriculum learning, wherein a teacher organizes the student
agent's learning process in a way that gradually increases the difficulty of tasks and and promotes transfer across MDPs and levels within and across curricula. The consistency of this framework and its benefits can be guaranteed under mild assumptions. We demonstrate abstraction, transferability, and curriculum learning in examples, including MazeBase+, a more complex variant of the MazeBase example.
Read the full abstract ▾
Machine Learning (Statistics)
Micro-Diffusion Compression - Binary Tree Tweedie Denoising for Online Probability Estimation
Roberto Tacconelli
We present Midicoth, a lossless compression system that introduces a micro-
diffusion denoising layer for improving probability estimates produced by adaptive statistical models. In compressors such as Prediction by Partial Matching (PPM), probability estimates are smoothed by a prior to handle sparse observations. When contexts have been seen only a few times, this prior dominates the prediction and produces distributions that are significantly flatter than the true source distribution, leading to compression inefficiency. Midicoth addresses this limitation by treating prior smoothing as a shrinkage process and applying a reverse denoising step that corrects predicted probabilities using empirical calibration statistics. To make this correction data-efficient, the method decomposes each byte prediction into a hierarchy of binary decisions along a bitwise tree. This converts a single 256-way calibration problem into a sequence of binary calibration tasks, enabling reliable estimation of correction terms from relatively small numbers of observations. The denoising process is applied in multiple successive steps, allowing each stage to refine residual prediction errors left by the previous one. The micro-
diffusion layer operates as a lightweight post-blend calibration stage applied after all model predictions have been combined, allowing it to correct systematic biases in the final probability distribution. Midicoth combines five fully online components: an adaptive PPM model, a long-range match model, a trie-based word model, a high-order context model, and the micro-
diffusion denoiser applied as the final stage.
Read the full abstract ▾
Machine Learning (Statistics)
Structural Causal Bottleneck Models
Simon Bing, Jonas Wahl and Jakob Runge
We introduce structural causal bottleneck models (SCBMs), a novel class of structural causal models in which causal effects between high-dimensional variables are mediated by low-dimensional summary statistics, or *bottlenecks*. SCBMs provide a flexible framework for mechanism-specific, target-dependent dimension reduction while remaining estimable via standard learning
algorithms. We prove that bottleneck variables are identifiable up to bijection from observational data, and validate this experimentally. We then study causal effect estimation in linear SCBMs under three standard adjustment strategies. For instrumental variables, we identify a population-level non-identifiability of naïve two-stage least squares when all variables are high-dimensional, and show that bottleneck representations restore identification. For mediation and
backdoor adjustment, we prove that conditioning on bottleneck variables yields valid effect estimates with explicit finite-sample efficiency gains over conditioning on full high-dimensional variables. We argue that SCBMs provide a principled alternative to existing causal dimension reduction frameworks such as causal representation learning and causal abstractions.
Read the full abstract ▾
Machine Learning (Statistics)
Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimation
Adam Rozzio, Rafael Athanasiades and O. Deniz Akyildiz
Maximum marginal likelihood estimation (MMLE) can be formulated as the optimization of a free energy functional. From this viewpoint, the Expectation-Maximisation (EM)
algorithm admits a natural interpretation as a coordinate descent method over the joint space of model parameters and probability measures. Recently, a significant body of work has adopted this perspective, leading to interacting particle
algorithms for MMLE. In this paper, we propose an accelerated version of one such procedure, based on Stein variational gradient descent (SVGD), by introducing Nesterov acceleration in both the parameter updates and in the space of probability measures. The resulting method, termed Momentum SVGD-EM, consistently accelerates convergence in terms of required iterations across various tasks of increasing difficulty, demonstrating effectiveness in both low- and high-dimensional settings.
Read the full abstract ▾
Statistical Methodology
RESAPLE: An Approximate One-Step Restricted Likelihood Estimator of Spatial Dependence for Exploratory Spatial Analysis
Aditya Khan and Meredith Franklin
Spatial autocorrelation diagnostics such as Moran's index and the approximate profile likelihood-based
estimators (APLE) are widely used to assess spatial dependence in areal data. Yet, although Moran's index is frequently applied to regression residuals and APLE is typically formulated for raw outcomes, neither is explicitly derived as an
estimator of residual spatial dependence after adjusting for large-scale trends and
covariates. We propose RESAPLE, a one-step approximate restricted maximum likelihood (REML)
estimator of the spatial error model's spatial dependence parameter $\rho$ based on REML residuals. Because RESAPLE is a Rayleigh coefficient, it retains the interpretability and diagnostic convenience of exploratory indices, while also providing a computationally inexpensive and accurate
estimator of $\rho$ under moderate spatial dependence. We show theoretically that for small to medium sample sizes and well-specified trend models, RESAPLE is a better
estimator of, and competitive test statistic for, residual spatial dependence relative to Moran's index and the APLE across a broad range of spatial configurations. The theory we develop also yields a diagnostic for spatial weight selection, providing guidance towards resolving a longstanding challenge in spatial data analysis. We illustrate the method through simulations on both regular and highly irregular
lattices with a case study using American Community Survey tract-level data.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Impact of Connectivity on Laplacian Representations in Reinforcement Learning
Tommaso Giorgi, Pierriccardo Olivieri, Keyue Jiang et al. (+2 more)
Learning compact state representations in Markov Decision Processes (MDPs) has proven crucial for addressing the curse of dimensionality in large-scale reinforcement learning (RL) problems. Existing principled approaches leverage structural priors on the MDP by constructing state representations as linear combinations of the state-graph
Laplacian eigenvectors. When the transition graph is unknown or the state space is prohibitively large, the graph spectral features can be estimated directly via sample trajectories. In this work, we prove an upper bound on the approximation error of linear value function approximation under the learned spectral features. We show how this error scales with the algebraic connectivity of the state-graph, grounding the approximation quality in the topological structure of the MDP. We further bound the error introduced by the eigenvector estimation itself, leading to an end-to-end error decomposition across the representation learning pipeline. Additionally, our expression of the
Laplacian operator for the RL setting, although equivalent to existing ones, prevents some common misunderstandings, of which we show some examples from the literature. Our results hold for general (non-uniform) policies without any assumptions on the
symmetry of the induced transition kernel. We validate our theoretical findings with numerical simulations on gridworld environments.
Read the full abstract ▾
All authors Tommaso Giorgi, Pierriccardo Olivieri, Keyue Jiang, Laura Toni, Matteo Papini
Machine Learning (Statistics)
Generative Adversarial Regression (GAR): Learning Conditional Risk Scenarios
Saeed Asadi and Jonathan Yu-Meng Li
We propose Generative
Adversarial Regression (GAR), a framework for learning conditional risk scenarios through generators aligned with downstream risk objectives. GAR builds on a regression characterization of conditional risk for elicitable functionals, including
quantiles, expectiles, and jointly elicitable pairs. We extend this principle from point prediction to generative modeling by training generators whose policy-induced risk matches that of real data under the same context. To ensure robustness across all policies, GAR adopts a minimax formulation in which an
adversarial policy identifies worst-case discrepancies in risk evaluation while the generator adapts to eliminate them. This structure preserves
alignment with the risk functional across a broad class of policies rather than a fixed, pre-specified set. We illustrate GAR through a tail-risk instantiation based on jointly elicitable $(\mathrm{VaR}, \mathrm{ES})$ objectives. Experiments on S&P 500 data show that GAR produces scenarios that better preserve downstream risk than unconditional, econometric, and direct predictive baselines while remaining stable under adversarially selected policies.
Read the full abstract ▾
Statistics Theory
Bayesian inference of planted matchings: Local posterior approximation and infinite-volume limit
Zhou Fan, Timothy L. H. Wee and Kaylee Y. Yang
We study
Bayesian inference of an unknown matching $\pi^*$ between two correlated random point sets $\{X_i\}_{i=1}^n$ and $\{Y_i\}_{i=1}^n$ in $[0,1]^d$, under a critical scaling $\|X_i-Y_{\pi^*(i)}\|_2 \asymp n^{-1/d}$, in both an exact matching model where all points are observed and a partial matching model where a fraction of points may be missing. Restricting to the simplest setting of $d=1$, in this work, we address the questions of (1) whether the posterior distribution over matchings is approximable by a local
algorithm, and (2) whether marginal statistics of this posterior have a well-defined limit as $n \to \infty$. We answer both questions affirmatively for partial matching, where a decay-of-correlations arises for large $n$. For exact matching, we show that the posterior is approximable locally only after a global sorting of the points, and that defining a large-$n$ limit of marginal statistics requires a careful indexing of points in the Poisson point process limit of the data, based on a notion of flow. We leave as an open question the extensions of such results to dimensions $d \geq 2$.
Read the full abstract ▾
Statistics Theory
Minimax estimation for Varying Coefficient Model via Laguerre Series
Rida Benhaddou, Khalid Chokri and Jackson Pinschenat
We delve into the estimation of the functional coefficients and inference for varying coefficient model. Applying Laguerre series, we develop an
estimator for the vector of functional coefficients that attains asymptotically optimal convergence rates in the minimax sense. These rates are derived for functional coefficients that belong to Laguerre-Sobolev space. The method is based on approximating the functional coefficients using truncated Laguerre series and choosing empirical Laguerre coefficients that minimize the least squares criterion. In addition, we establish the asymptotic normality of the
estimator for the functional coefficients, construct their confidence intervals, and establish point-wise hypothesis tests about their true values. A simulations study is carried out to examine the finite-sample properties of the proposed methodology. A real data set is considered as well, and results based on the proposed methodology are compared to those based on selected existing approaches.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Breaking the Bias Barrier in Concave Multi-Objective Reinforcement Learning
Swetha Ganesh and Vaneet Aggarwal
While standard reinforcement learning optimizes a single reward signal, many applications require optimizing a nonlinear utility $f(J_1^\pi,\dots,J_M^\pi)$ over multiple objectives, where each $J_m^\pi$ denotes the expected discounted return of a distinct reward function. A common approach is concave scalarization, which captures important trade-offs such as
fairness and risk sensitivity. However, nonlinear scalarization introduces a fundamental challenge for policy gradient methods: the gradient depends on $\partial f(J^\pi)$, while in practice only empirical return estimates $\hat J$ are available. Because $f$ is nonlinear, the plug-in
estimator is biased ($\mathbb{E}[\partial f(\hat J)] \neq \partial f(\mathbb{E}[\hat J])$), leading to persistent gradient bias that degrades sample complexity. In this work we identify and overcome this bias barrier in concave-scalarized multi-objective reinforcement learning. We show that existing policy-gradient methods suffer an intrinsic $\widetilde{\mathcal{O}}(\epsilon^{-4})$ sample complexity due to this bias. To address this issue, we develop a Natural Policy Gradient (NPG)
algorithm equipped with a multi-level Monte Carlo (MLMC)
estimator that controls the bias of the scalarization gradient while maintaining low sampling cost. We prove that this approach achieves the optimal $\widetilde{\mathcal{O}}(\epsilon^{-2})$ sample complexity for computing an $\epsilon$-optimal policy. Furthermore, we show that when the scalarization function is second-order smooth, the first-order bias cancels automatically, allowing vanilla NPG to achieve the same $\widetilde{\mathcal{O}}(\epsilon^{-2})$ rate without MLMC. Our results provide the first optimal sample complexity guarantees for concave multi-objective reinforcement learning under policy-gradient methods.
Read the full abstract ▾
Statistical Methodology
Kantorovich Regression Analysis of Random Distributions with Mixed Predictors
Kaheon Kim and Changbo Zhu
In modern scientific applications, observations often take the form of probability distributions. We study regression problems with distribution-valued responses and mixed distributional and Euclidean predictors. Under quadratic optimal transport, the proposed Kantorovich regression model approximates the response Kantorovich potential from the Wasserstein barycenter as a linear combination of transformed predictor Kantorovich potentials, where the transformations are determined by one-dimensional functional parameters. The formulation naturally accommodates mixed predictors, allowing Euclidean
covariates to enter as scaling coefficients, and extends to multivariate distributions. We characterize functional parameter classes ensuring the intrinsic structure of the model, and establish asymptotic consistency of model parameters through the convex Sobolev loss. Real data applications include a mixed-predictor analysis of housing price distributions and an analysis of two-dimensional temperature distributions, demonstrating the flexibility and interpretability of the proposed framework.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Efficient Credal Prediction through Decalibration
Paul Hofman, Timo Löhr, Maximilian Muschalik et al. (+2 more)
A reliable representation of uncertainty is essential for the application of modern machine learning methods in safety-critical settings. In this regard, the use of credal sets (i.e., convex sets of probability distributions) has recently been proposed as a suitable approach to representing epistemic uncertainty. However, as with other approaches to epistemic uncertainty, training credal predictors is computationally complex and usually involves (re-)training an ensemble of models. The resulting computational complexity prevents their adoption for complex models such as foundation models and multi-modal systems. To address this problem, we propose an efficient method for credal prediction that is grounded in the notion of relative likelihood and inspired by techniques for the calibration of probabilistic classifiers. For each class label, our method predicts a range of plausible probabilities in the form of an interval. To produce the lower and upper bounds of these intervals, we propose a technique that we refer to as decalibration. Extensive experiments show that our method yields credal sets with strong performance across diverse tasks, including coverage-efficiency evaluation, out-of-distribution detection, and in-context learning. Notably, we demonstrate credal prediction on models such as TabPFN and CLIP – architectures for which the construction of credal sets was previously infeasible.
Read the full abstract ▾
All authors Paul Hofman, Timo Löhr, Maximilian Muschalik, Yusuf Sale, Eyke Hüllermeier
Statistical Methodology
Decoupling Distance and Networks: Hybrid Graph Attention-Geostatistical Methods for Spatio-temporal Risk Mapping
Toba Temitope Bamidele, Ezra Gayawan, Femi Barnabas Adebola et al. (+1 more)
Accurate spatial prediction and rigorous uncertainty quantification are central to modern spatial epidemiology and environmental risk analysis. We introduce a statistically principled hybrid modelling framework that integrates the nonlinear,
attention-based representation learning capabilities of a dynamic Graph
Attention Network (GATv2) with a latent Gaussian spatial process from model-based geostatistics (MBG). This framework jointly captures relational dependence encoded in graph structures and continuous spatial dependence governed by physical proximity. We evaluate the proposed model via a controlled simulation study and an applied analysis of malaria prevalence data, comparing its predictive accuracy, calibration, and uncertainty quantification against classical geostatistical models and standalone GATv2 architectures. Our analyses show that GATv2 captures complex nonlinear interactions but fails to account for residual spatial autocorrelation, resulting in miscalibrated predictive distributions. Conversely, geostatistical models provide coherent uncertainty quantification through structured covariance functions yet are constrained by linear predictor assumptions and by their reliance on Euclidean distance to encode spatial structure. By integrating
attention mechanisms and nonlinear features with an explicit probabilistic spatial random field, the hybrid model captured the relational dependence, consistently improved predictive accuracy, and provided more realistic uncertainty quantification in both simulation and applied settings. Overall, the findings demonstrate that the hybrid model constitutes a statistically coherent and empirically robust framework for modelling complex spatial and spatio-temporal processes in settings where both distance-based and structure-based dependencies operate.
Read the full abstract ▾
All authors Toba Temitope Bamidele, Ezra Gayawan, Femi Barnabas Adebola, Olatunji Johnson
Machine Learning (Statistics) · Cross-listed from cs.LG
Beyond the Markovian Assumption: Robust Optimization via Fractional Weyl Integrals in Imbalanced Data
Gustavo A. Dorrego
Standard Gradient Descent and its modern variants assume local, Markovian weight updates, making them highly susceptible to noise and overfitting. This limitation becomes critically severe in extremely imbalanced
datasets such as financial fraud detection where dominant class gradients systematically overwrite the subtle signals of the minority class. In this paper, we introduce a novel optimization
algorithm grounded in Fractional Calculus. By isolating the core
memory engine of the generalized fractional derivative, the Weighted Fractional Weyl Integral, we replace the instantaneous gradient with a dynamically weighted historical sequence. This fractional
memory operator acts as a natural regularizer. Empirical evaluations demonstrate that our method prevents overfitting in medical diagnostics and achieves an approximately 40 percent improvement in PR-AUC over classical optimizers in financial fraud detection, establishing a robust bridge between pure fractional
topology and applied Machine Learning.
Read the full abstract ▾
Machine Learning (Statistics)
Unifying On- and Off-Policy Variance Reduction Methods
Olivier Jeunen
Continuous and efficient experimentation is key to the practical success of user-facing applications on the web, both through online A/B-tests and off-policy evaluation. Despite their shared objective – estimating the incremental value of a treatment – these domains often operate in isolation, utilising distinct terminologies and statistical toolkits. This paper bridges that divide by establishing a formal equivalence between their canonical variance reduction methods. We prove that the standard online Difference-in-Means
estimator is mathematically identical to an off-policy Inverse Propensity Scoring
estimator equipped with an optimal (variance-minimising) additive control variate. Extending this unification, we demonstrate that widespread regression adjustment methods (such as CUPED, CUPAC, and ML-RATE) are structurally equivalent to Doubly Robust estimation. This unified view extends our understanding of commonly used approaches, and can guide practitioners and researchers working on either class of problems.
Read the full abstract ▾
Statistics Theory
Limiting Spectral Distribution of moderately large Kendall's correlation matrix and its application
Raunak Shevade and Monika Bhattacharjee
We establish the limiting spectral distribution of Kendall's correlation
matrices in the moderate high-dimensional regime where the dimension grows slower than the sample size. Our framework allows observations to be independent but not necessarily identically distributed, and accommodates both discrete and continuous data. Unlike existing results developed under i.i.d. observations, our approach remains valid under substantial distributional heterogeneity and also covers certain i.i.d. models beyond previously studied settings. Under mild
symmetry and convergence conditions on some traces, we prove that the empirical spectral distribution of a properly centered and scaled Kendall's correlation
matrix converges weakly almost surely to a deterministic, generally model-dependent limit. The analysis clarifies how distributional heterogeneity influences the limiting spectrum. As an application, we propose a graphical tool for detecting dependence among components in high-dimensional data and show that ignoring heterogeneity may lead to spurious detection of dependence.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Towards plausibility in time series counterfactual explanations
Marcin Kostrzewa, Krzysztof Galus and Maciej Zięba
We present a new method for generating plausible counterfactual explanations for time series classification problems. The approach performs gradient-based optimization directly in the input space. To enforce plausibility, we integrate soft-DTW (dynamic time warping)
alignment with $k$-nearest neighbors from the target class, which effectively encourages the generated counterfactuals to adopt a realistic temporal structure. The overall optimization objective is a multi-faceted loss function that balances key counterfactual properties. It incorporates losses for validity, sparsity, and proximity, alongside the novel soft-DTW-based plausibility component. We conduct an evaluation of our method against several strong reference approaches, measuring the key properties of the generated counterfactuals across multiple dimensions. The results demonstrate that our method achieves competitive performance in validity while significantly outperforming existing approaches in distributional
alignment with the target class, indicating superior temporal realism. Furthermore, a qualitative analysis highlights the critical limitations of existing methods in preserving realistic temporal structure. This work shows that the proposed method consistently generates counterfactual explanations for time series classifiers that are not only valid but also highly plausible and consistent with temporal patterns.
Read the full abstract ▾
Statistical Methodology
Amortized Phylodynamic Inference with Neural Bayes Estimators and Recursive Neural Networks
Alexander E. Zarebski, Thomas Williams and Louis du Plessis
Phylodynamics is used to estimate epidemic dynamics from phylogenetic trees or genomic sequences of pathogens, but the likelihood calculations needed can be challenging for complex models. We present a neural Bayes
estimator (NBE) for key epidemic quantities: the reproduction number, prevalence, and cumulative infections through time. By performing
quantile regression over tree space, the NBE allows us to estimate posterior medians and credible intervals directly from a reconstructed tree. Our approach uses a recursive neural network as a tree
embedding network with a prediction network conditioned on time and
quantile level to generate the estimates. In simulation studies, the NBE achieves good predictive performance, with conservative uncertainty estimates. Compared with a BEAST2 fixed-tree analysis, the NBE gives less biased estimates of time-varying reproduction numbers in our test setting. Under a misspecified sampling model, the NBE performance degrades (as expected) but remains reasonable, and fine-tuning a pre-trained model yields estimates comparable to those from a model trained from scratch, at substantially lower computational cost.
Read the full abstract ▾
Statistics Theory
Size-Location Correlation for Set-Valued Processes: Theory, Estimation, and Laws of Large Numbers under $\rho$-Mixing
Tuyen Luc Tri
We propose a variational framework for analyzing dependence structures of convex compact random sets based on their support functions. The approach relies on the canonical even–odd decomposition on the unit sphere, which separates size-related and location-related components and induces an exact orthogonality in the sphere $L^2(\sigma)$ space. This decomposition yields an additive variance–covariance structure that is intrinsic to set-valued data and cannot be recovered from point-based or selection-based representations. Within this framework, we introduce size, location, and total covariance and correlation indices for random sets, together with compatible $\rho$-mixing coefficients for set-valued processes. The resulting dependence measures are geometrically interpretable, invariant under translations, and free of degeneracies that arise for centrally symmetric sets under classical approaches. Weak and strong laws of large numbers are established under weak stationarity, providing asymptotic stability of Minkowski averages in the $L^2(\sigma)$ support-function norm. The proposed quantities admit natural numerical realizations via directional Monte Carlo and spherical designs. Applications to interval-valued and convex-valued data, including regression with set-valued responses, illustrate how the even–odd decomposition disentangles directional location dependence from size effects beyond what can be captured by finite-dimensional summaries such as the Steiner point.
Read the full abstract ▾
Statistics Theory
Sign Identifiability of Causal Effects in Stationary Stochastic Dynamical Systems
Gijs van Seeventer and Saber Salehkaleybar
We study identifiability in continuous-time linear stationary
stochastic differential equations with a known causal structure. Unlike existing approaches, we relax the assumption of a known
diffusion matrix, thereby respecting the model's intrinsic scale invariance. Therefore, rather than recovering drift coefficients themselves, we introduce edge-sign identifiability: for a given causal structure, we ask whether the sign of a given drift entry is uniquely determined across all observational covariance
matrices induced by parametrisations compatible with that structure. This leads to a trichotomy of edge-sign identifiability: identifiable, non-identifiable, and partially identifiable. This trichotomy introduces the new notion of partial identifiability to the literature, which we show is a genuine category in our setting. Under a notion of faithfulness, we derive criteria to identify membership of each category for general graphs. Applying our criteria to specific causal structures, both analogous to classical causal settings (e.g., instrumental variables) and novel cyclic settings, we determine their edge-sign identifiability and, in some cases, obtain explicit expressions for the sign of a target edge in terms of the observational covariance
matrix.
Read the full abstract ▾
Statistics Theory
Weighted Chernoff information and optimal loss exponent in context-sensitive hypothesis testing
Mark Kelbert and El'mira Yu. Kalimulina
We study binary hypothesis testing for i.i.d. observations under a multiplicative context weight. For the optimal weighted total loss, defined as the sum of weighted type-I and type-II losses, we prove the logarithmic asymptotic $$ L_n^* = \exp\{-n D_C^{\mathrm{w}}(\mathbb{P}, \mathbb{Q}) + o(n)\}, \quad n \to \infty, $$ where $D_C^{\mathrm{w}}$ is the weighted Chernoff information. The single-letter form of the exponent relies on a structural assumption that the weight factorises across observations, $\varphi(x_1^n) = \prod_{i=1}^n \varphi(x_i)$; this restriction is essential for the single-letter representation and should be distinguished from the weaker qualitative description "multiplicative context weight". The proof embeds the weighted
geometric mixtures $\varphi p^\alpha q^{1-\alpha}$ into a likelihood-ratio exponential family and identifies the rate through its log-normaliser. We also derive concentration bounds for the tilted weighted log-likelihood, obtain closed forms for Gaussian, Poisson, and exponential models, and extend the exponent characterisation to finitely many hypotheses.
Read the full abstract ▾
Machine Learning (Statistics)
Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spaces
Hamish Flynn, Joe Watson, Ingmar Posner et al. (+1 more)
We analyze the
Bayesian regret of the Gaussian process posterior sampling reinforcement learning (GP-PSRL)
algorithm. Posterior sampling is a heuristic for decision-making under uncertainty that has been used to develop successful
algorithms for a variety of continuous control problems. However, theoretical work on GP-PSRL is limited. All known regret bounds either have a sub-optimal growth rate, require strong smoothness assumptions, or fail to properly account for the fact that the set of possible system states is unbounded. Through a recursive application of the Borell-Tsirelson-Ibragimov-Sudakov inequality, we show that, with high probability, the states actually visited by the
algorithm are contained within a ball of near-constant radius. We then use the chaining method to control the regret suffered by GP-PSRL under weak smoothness conditions. Our main result is a
Bayesian regret bound of the order $\widetilde{\mathcal{O}}(H\sqrt{\gamma_TT})$, where $H$ is the horizon, $T$ is the number of time steps and $\gamma_T$ is the expected information gain. With this result, we resolve the limitations with prior theoretical work on PSRL, and provide the theoretical foundation and tools for analyzing PSRL in complex settings.
Read the full abstract ▾
All authors Hamish Flynn, Joe Watson, Ingmar Posner, Jan Peters
Statistical Methodology
An objective non-local prior for skew-symmetric models
F. J. Rubio
We propose an objective non-local prior for testing
symmetry against skew-symmetric alternatives. The prior is derived through a formal construction rule by assigning a uniform distribution to a discrepancy-based measure of the shape parameter's effect. This approach avoids the need for user-specified hyperparameters and produces a weakly informative prior tailored to the skew-symmetric family. We illustrate the use of the proposed prior in the context of testing normality against skew-normal alternatives through both a simulation study and a real-data application.
Read the full abstract ▾
Statistical Methodology
A Unified Framework for Density Estimation under Right-Censored Point-Centred Quarter Sampling
Wenzhe Huang, Guochun Shen, Dingliang Xing et al. (+1 more)
While the point-centred quarter method (PCQM) is widely used for density estimation, existing methods for handling right-censored data from truncated search radii rely primarily on a Poisson model assuming complete spatial randomness (CSR), leaving a critical gap for spatially aggregated populations. To address this limitation, we develop a unified likelihood- and moment-based framework for right-censored point-centred quarter sampling under both Poisson and negative binomial distribution (NBD) models. In particular, the proposed NBD-based
estimators explicitly account for spatial aggregation and censoring simultaneously, extending distance-based inference beyond the CSR setting. Extensive simulations and applications to fully mapped forest plots reveal that the NBD-based MLE delivers the most robust overall performance across diverse ecological scenarios. Across more than 100 species from fully mapped forest plots, the proposed NBD-based MLE approximately reduced absolute relative bias by a median of 0.10 compared with existing censored
estimators, representing a relative improvement of over 30%. Ultimately, our framework provides a rigorously validated and practically useful toolkit for analysing censored point-to-tree distance data.
Read the full abstract ▾
All authors Wenzhe Huang, Guochun Shen, Dingliang Xing, Jiangyan Zhao
Machine Learning (Statistics)
Beyond ReinMax: Low-Variance Gradient Estimators for Discrete Latent Variables
Daniel Wang and Thang D. Bui
Machine learning models involving discrete latent variables require gradient
estimators to facilitate backpropagation in a computationally efficient manner. The most recent addition to the Straight-Through family of
estimators, ReinMax, can be viewed from a numerical ODE perspective as incorporating an approximation via Heun's method to reduce bias, but at the cost of high variance. In this work, we introduce the ReinMax-Rao and ReinMax-CV
estimators which incorporate Rao-Blackwellisation and control variate techniques into ReinMax to reduce its variance. Our
estimators demonstrate superior performance on training variational autoencoders with discrete latent spaces. Furthermore, we investigate the possibility of leveraging alternative numerical methods for constructing more accurate gradient approximations and present an alternative view of ReinMax from a simpler numerical integration perspective.
Read the full abstract ▾