Statistical Methodology
A Gentle Introduction to Conformal Time Series Forecasting
M. Stocker, W. Małgorzewicz, M. Fontana et al. (+1 more)
Conformal prediction is a powerful post-hoc framework for uncertainty quantification that provides distribution-free coverage guarantees. However, these guarantees crucially rely on the assumption of exchangeability. This assumption is fundamentally violated in time series data, where temporal dependence and distributional shifts are pervasive. As a result, classical split-conformal methods may yield prediction intervals that fail to maintain nominal validity. This review unifies recent advances in conformal
forecasting methods specifically designed to address nonexchangeable data. We first present a theoretical foundation, deriving finite-sample guarantees for split-conformal prediction under mild weak-dependence conditions. We then survey and classify state-of-the-art approaches that mitigate serial dependence by reweighting calibration data, dynamically updating residual distributions, or adaptively tuning target coverage levels in real time. Finally, we present a comprehensive simulation study that compares these techniques in terms of empirical coverage, interval width, and computational cost, highlighting practical trade-offs and open research directions.
Read the full abstract ▾
All authors M. Stocker, W. Małgorzewicz, M. Fontana, S. Ben Taieb
Statistics Applications
Variance Stabilizing Transformations for Electricity Price Forecasting in Periods of Increased Volatility
Bartosz Uniejewski
Accurate day-ahead electricity price forecasts are critical for power system operation and
market participation, yet growing renewable penetration and recent crises have caused unprecedented
volatility that challenges standard models. This paper revisits variance stabilizing transformations (VSTs) as a preprocessing tool by introducing a novel parametrization of the asinh transformation, systematically analyzing parameter sensitivity and calibration window size, and explicitly testing performance under volatile
market regimes. Using data from Germany, Spain, and France over 2015-2024 with two model classes (NARX and LEAR), we show that VSTs substantially reduce forecast errors, with gains of up to 14.6% for LEAR and 8.7% for NARX relative to untransformed
benchmarks. The new parametrized asinh consistently outperforms its standard form, while rolling averaging across transformations delivers the most robust improvements, reducing errors by up to 17.7%. Results demonstrate that VSTs are especially valuable in volatile regimes, making them a powerful tool for enhancing electricity price
forecasting in today's power
markets.
Read the full abstract ▾
Statistics Theory
Nonparametric Estimation of Joint Entropy via Partitioned Sample-Spacing
Jungwoo Ho, Sangun Park and Soyeong Oh
We propose a nonparametric
estimator of multivariate joint
entropy based on partitioned sample spacing (PSS). The method extends univariate spacing ideas to $\mathbb{R}^{d}$ by partitioning into localized
cells and aggregating within-
cell statistics, with strong consistency guarantees under mild conditions. In
benchmarks across diverse distributions, PSS consistently outperforms $k$-nearest neighbor
estimators and achieves accuracy competitive with recent normalizing flow-based methods, while requiring no training or auxiliary density modeling. The
estimator scales favorably in moderately high dimensions ($d = 10$–$40$) and shows particular robustness to correlated or skewed distributions. These properties position PSS as a practical and reliable alternative to both $k$NN and NF-based
entropy estimators, with broad utility in information-theoretic machine learning tasks such as total-correlation estimation, representation learning, and feature selection.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
Asymptotic analysis of rare events in high dimensions
Anya Katsevich and Alexander Katsevich
Understanding rare events is critical across domains ranging from signal processing to reliability and structural safety, extreme-weather
forecasting, and insurance. The analysis of rare events is a computationally challenging problem, particularly in high dimensions $d$. In this work, we develop the first asymptotic high-dimensional theory of rare events. First, we exploit asymptotic integral methods recently developed by the first author to provide an asymptotic expansion of rare event probabilities. The expansion employs the geometry of the rare event boundary and the local behavior of the log probability density. Generically, the expansion is valid if $d^2\ll\lambda$, where $\lambda$ characterizes the extremity of the event. We prove this condition is necessary by constructing an example in which the first-order remainder is bounded above and below by $d^2/\lambda$. We also provide a nonasymptotic remainder bound which specifies the precise dependence of the remainder on $d$, $\lambda$, the density, and the boundary, and which shows that in certain cases, the condition $d^2\ll \lambda$ can be relaxed. As an application of the theory, we derive asymptotic approximations to rare probabilities under the standard Gaussian density in high dimensions. In the second part of our work, we provide an asymptotic approximation to densities conditional on rare events. This gives rise to simple procedure for approximately sampling conditionally on the rare event using independent Gaussian and exponential random variables.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CE
Making Evidence Actionable in Adaptive Learning Closing the Diagnostic Pedagogical Loop
Amirreza Mehrabi, Jason Wade Morphew, Breejha Quezada et al. (+1 more)
Adaptive learning often diagnoses precisely yet intervenes weakly, producing help that is mistimed or misaligned. This study presents evidence supporting an instructor-governed feedback loop that converts concept-level assessment evidence into vetted microinterventions. The adaptive learning
algorithm includes three safeguards: adequacy as a hard guarantee of gap closure,
attention as a budgeted limit for time and redundancy, and diversity as protection against overfitting to a single resource. We formulate intervention assignment as a binary integer program with constraints for coverage, time, difficulty windows derived from ability estimates, prerequisites encoded by a concept
matrix, and anti-redundancy with diversity. Greedy selection serves low-richness and tight-latency settings, gradient-based relaxation serves rich repositories, and a hybrid switches along a richness-latency frontier. In simulation and in an introductory physics deployment with 1204 students, both solvers achieved full skill coverage for nearly all learners within bounded watch time. The gradient-based method reduced redundant coverage by about 12 percentage points relative to greedy and produced more consistent difficulty
alignment, while greedy delivered comparable adequacy at lower computational cost in resource-scarce environments. Slack variables localized missing content and guided targeted curation, sustaining sufficiency across student subgroups. The result is a tractable and auditable controller that closes the diagnostic pedagogical loop and enables equitable, load-aware personalization at the classroom scale.
Read the full abstract ▾
All authors Amirreza Mehrabi, Jason Wade Morphew, Breejha Quezada, N. Sanjay Rebello
Machine Learning (Statistics)
The Shape of Data: Topology Meets Analytics. A Practical Introduction to Topological Analytics and the Stability Index (TSI) in Business
Ioannis Diamantis
Modern business and economic
datasets often exhibit nonlinear, multi-scale structures that traditional linear tools under-represent. Topological Data Analysis (TDA) offers a
geometric lens for uncovering robust patterns, such as connected components, loops and voids, across scales. This paper provides an intuitive, figure-driven introduction to persistent homology and a practical, reproducible TDA pipeline for applied analysts. Through comparative case studies in consumer behavior, equity
markets (SAX/eSAX vs.\ TDA) and foreign exchange dynamics, we demonstrate how topological features can reveal
segmentation patterns and structural relationships beyond classical statistical methods. We discuss methodological choices regarding distance metrics, complex construction and interpretation, and we introduce the Topological Stability Index (TSI), a simple yet interpretable indicator of structural variability derived from persistence lifetimes. We conclude with practical guidelines for TDA implementation, visualization and communication in business and economic analytics.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
AdamNX: An Adam improvement algorithm based on a novel exponential decay mechanism for the second-order moment estimate
Meng Zhu, Quan Xiao and Weidong Min
This paper studies the exponential decay mechanism of the second-moment estimate in Adam. We propose AdamNX and a time-varying second-moment decay rate that gradually weakens the correction applied to the update scale. Under the assumptions used in our analysis, this mechanism makes the updates approach momentum-SGD-like behavior during the training plateau phase. We report results on the image-classification, object-detection, and semantic-
segmentation tasks, configurations, and comparators included in this paper. These results do not establish multi-seed statistical effects, flatness, or generalization beyond the reported tasks. Our code is open-sourced at https://github.com/mengzhu0308/AdamNX.
Read the full abstract ▾
Statistics Theory
Random sets from the perspective of metric statistics
Daisuke Kurisu, Yuta Okamoto and Taisuke Otsu
Since the seminal work by Beresteanu and Molinari(2008), the random set theory and related inference methods have been widely applied in partially identified econometric models. Meanwhile, there is an emerging field in statistics for studying random objects in metric spaces, called metric statistics. This paper clarifies a relationship between two fundamental concepts in these literatures, the Aumann and Fréchet means, and presents some applications of metric statistics to econometric problems involving random sets.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression
Tingkai Yan, Haodong Wen, Binghui Li et al. (+3 more)
While data scaling laws of large
language models (LLMs) have been widely examined in the one-pass regime with massive corpora, their form under limited data and repeated epochs remains largely unexplored. This paper presents a theoretical analysis of how a common workaround, training for multiple epochs on the same
dataset, reshapes the data scaling laws in linear regression. Concretely, we ask: to match the performance of training on a
dataset of size $N$ for $K$ epochs, how much larger must a
dataset be if the model is trained for only one pass? We quantify this using the effective reuse rate of the data, $E(K, N)$, which we define as the multiplicative factor by which the
dataset must grow under one-pass training to achieve the same test loss as $K$-epoch training. Our analysis precisely characterizes the scaling behavior of $E(K, N)$ for SGD in linear regression under either strong convexity or Zipf-distributed data: (1) When $K$ is small, we prove that $E(K, N) \approx K$, indicating that every new epoch yields a linear gain; (2) As $K$ increases, $E(K, N)$ plateaus at a problem-dependent value that grows with $N$ ($\Theta(\log N)$ for the strongly-convex case), implying that larger
datasets can be repeated more times before the marginal benefit vanishes. These theoretical findings point out a neglected factor in a recent empirical study (Muennighoff et al. (2023)), which claimed that training LLMs for up to $4$ epochs results in negligible loss differences compared to using fresh data at each step, i.e., $E(K, N) \approx K$ for $K \le 4$ in our notation. Supported by further empirical validation with LLMs, our results reveal that the maximum $K$ value for which $E(K, N) \approx K$ in fact depends on the data size and distribution, and underscore the need to explicitly model both factors in future studies of scaling laws with data reuse.
Read the full abstract ▾
All authors Tingkai Yan, Haodong Wen, Binghui Li, Kairong Luo, Wenguang Chen, Kaifeng Lyu
Machine Learning (Statistics) · Cross-listed from cs.LG
Fast and Robust Simulation-Based Inference With Optimization Monte Carlo
Vasilis Gkolemis, Christos Diou and Michael U. Gutmann
Bayesian parameter inference for complex
stochastic simulators is challenging due to intractable likelihood functions. Existing simulation-based inference methods often require large number of simulations and become costly to use in high-dimensional parameter spaces or in problems with partially uninformative outputs. We propose a new method for differentiable simulators that delivers accurate posterior inference with substantially reduced runtimes. Building on the Optimization Monte Carlo framework, our approach reformulates inference for
stochastic simulators in terms of deterministic optimization problems. Gradient-based methods are then applied to efficiently navigate toward high-density posterior regions and avoid wasteful simulations in low-probability areas. A JAX-based implementation further enhances the performance through vectorization of key method components. Extensive experiments, including high-dimensional parameter spaces, uninformative outputs, multiple observations and
multimodal posteriors show that our method consistently matches, and often exceeds, the accuracy of state-of-the-art approaches, while reducing the runtime by a substantial margin.
Read the full abstract ▾
Other Statistics · Cross-listed from q-fin.GN
CBDC Stress Test in a Dual-Currency Setting
Catalin Dumitrescu
This study explores the potential impact of introducing a Central Bank Digital Currency (CBDC) on financial stability in an emerging dual-currency economy (Romania), where the domestic currency (RON) coexists with the euro. It develops an integrated analytical framework combining econometrics, machine learning, and behavioural modelling. CBDC adoption probabilities are estimated using XGBoost and logistic regression models trained on behavioural and macro-financial indicators rather than survey data. Liquidity stress simulations assess how banks would respond to deposit withdrawals resulting from CBDC adoption, while VAR, MSVAR, and SVAR models capture the macro-financial transmission of liquidity shocks into credit contraction and changes in monetary conditions. The findings indicate that CBDC uptake (co-circulating Digital RON and Digital EUR) would be moderate at issuance, amounting to around EUR 1 billion, primarily driven by digital readiness and trust in the central bank. The study concludes that a non-remunerated, capped CBDC, designed primarily as a means of payment rather than a store of value, can be introduced without compromising financial stability. In dual currency economies, differentiated holding limits for domestic and foreign digital currencies (e.g., Digital RON versus Digital Euro) are crucial to prevent uncontrolled euroisation and preserve monetary sovereignty. A prudent design with moderate caps, non remuneration, and macroprudential coordination can transform CBDC into a digital liquidity buffer and a complementary monetary policy instrument that enhances resilience and inclusion rather than destabilising the financial system.
Read the full abstract ▾
Statistics Theory
Local asymptotic normality for discretely observed McKean-Vlasov diffusions
Akram Heidari and Mark Podolskij
We study the local asymptotic normality (LAN) property for the likelihood function associated with discretely observed $d$-dimensional McKean-Vlasov
stochastic differential equations over a fixed time interval. The model involves a joint parameter in both the drift and
diffusion coefficients, introducing challenges due to its dependence on the process distribution. We derive a
stochastic expansion of the log-likelihood ratio using Malliavin calculus techniques and establish the LAN property under appropriate conditions. The main technical challenge arises from the implicit nature of the transition densities, which we address through integration by parts and Gaussian-type bounds. This work extends existing LAN results for interacting particle systems to the mean-field regime, contributing to statistical inference in non-linear
stochastic models
Read the full abstract ▾
Statistical Methodology
Novel Tau-Informed Initialization for Maximum Likelihood Estimation of Copulas with Discrete Margins
Anna van Es and Eva Cantoni
We study Gaussian-copula models with discrete margins, with primary emphasis on low-count (Poisson) data. Our goal is exact yet computationally efficient maximum likelihood (ML) estimation in regimes where many observations contain small counts, which imperils both identifiability and numerical stability. We develop three novel Kendall's tau-based approaches for initialization tailored to discrete margins in the low-count regime and embed it within an inference functions for margins (IFM) inspired start. We present three practical initializers (exact, low-intensity approximation, and a transformation-based approach) that substantially reduce the number of ML iterations and improve convergence. For the ML stage, we use an unconstrained reparameterization of the model's parameters using the log and spherical-Cholesky and compute exact rectangle probabilities. Analytical score functions are supplied throughout to stabilize Newton-type optimization. A simulation study across dimensions, dependence levels, and intensity regimes shows that the proposed initialization combined with exact ML achieves lower root-mean-squared error, lower bias and faster computation times than the alternative procedures. The methodology provides a pragmatic path to retain the statistical guarantees of ML (consistency, asymptotic normality, efficiency under correct specification) while remaining tractable for moderate- to high-dimensional discrete data. We conclude with guidance on initializer choice and discuss extensions to alternative correlation structures and different margins.
Read the full abstract ▾
Statistics Applications
TacEleven: generative tactic discovery for football open play
Siyao Zhao, Hao Ma, Zhiqiang Pu et al. (+4 more)
Creating offensive advantages during open play is fundamental to football success. However, due to the highly dynamic and long-sequence nature of open play, the potential tactic space grows exponentially as the sequence progresses, making automated tactic discovery extremely challenging. To address this, we propose TacEleven, a generative framework for football open-play tactic discovery developed in close collaboration with domain experts from AJ Auxerre, designed to assist coaches and analysts in tactical decision-making. TacEleven consists of two core components: a
language-controlled tactical generator that produces diverse tactical proposals, and a
multimodal large
language model-based tactical critic that selects the optimal proposal aligned with a high-level stylistic tactical instruction. The two components enables rapid exploration of tactical proposals and discovery of alternative open-play offensive tactics. We evaluate TacEleven across three tasks with progressive tactical complexity: counterfactual exploration, single-step discovery, and multi-step discovery, through both quantitative metrics and a questionnaire-based qualitative assessment. The results show that the TacEleven-discovered tactics exhibit strong realism and tactical creativity, with 52.50% of the multi-step tactical alternatives rated adoptable in real-world elite football scenarios, highlighting the framework's ability to rapidly generate numerous high-quality tactics for complex long-sequence open-play situations. TacEleven demonstrates the potential of creatively leveraging domain data and generative models to advance tactical analysis in sports.
Read the full abstract ▾
All authors Siyao Zhao, Hao Ma, Zhiqiang Pu, Jingjing Huang, Yi Pan, Shijie Wang, Zhi Ming
Statistical Methodology
Scalable approximation of the transformation-free linear simplicial-simplicial regression via constrained iterative reweighted least squares
Michail Tsagris and Omar Alzeley
Simplicia-simplicial regression concerns statistical modeling scenarios in which both the predictors and the responses are vectors constrained to lie on the simplex. \cite{fiksel2022} introduced a transformation-free linear regression framework for this setting, wherein the regression coefficients are estimated by minimizing the Kullback-Leibler divergence between the observed and fitted compositions, using an expectation-maximization (EM)
algorithm for optimization. In this work, we reformulate the problem as a constrained logistic regression model, in line with the methodological perspective of \cite{tsagris2025}, and we obtain parameter estimates via constrained iteratively reweighted least squares. Simulation results indicate that the proposed procedure substantially improves computational efficiency-yielding speed gains ranging from $6\times--326\times$-while providing estimates that closely approximate those obtained from the EM-based approach.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Counterfactual Explainable AI (XAI) Method for Deep Learning-Based Multivariate Time Series Classification
Alan G. Paredes Cetina, Kaouther Benguessoum, Raoni Lourenço et al. (+1 more)
Recent advances in deep learning have improved multivariate time series (MTS) classification and regression by capturing complex patterns, but their lack of transparency hinders decision-making. Explainable AI (XAI) methods offer partial insights, yet often fall short of conveying the full decision space. Counterfactual Explanations (CE) provide a promising alternative, but current approaches typically prioritize either accuracy, proximity or sparsity – rarely all – limiting their practical value. To address this, we propose CONFETTI, a novel multi-objective CE method for MTS. CONFETTI identifies key MTS subsequences, locates a counterfactual target, and optimally modifies the time series to balance prediction confidence, proximity and sparsity. This method provides actionable insights with minimal changes, improving interpretability, and decision support. CONFETTI is evaluated on seven MTS
datasets from the UEA archive, demonstrating its effectiveness in various domains. CONFETTI consistently outperforms state-of-the-art CE methods in its optimization objectives, and in six other metrics from the literature, achieving $\geq10\%$ higher confidence while improving sparsity in $\geq40\%$.
Read the full abstract ▾
All authors Alan G. Paredes Cetina, Kaouther Benguessoum, Raoni Lourenço, Sylvain Kubler
Machine Learning (Statistics) · Cross-listed from cs.LG
Laplace Learning in Wasserstein Space
Mary Chriselda Antony Oliver, Michael Roberts, Carola-Bibiane Schönlieb et al. (+1 more)
The
manifold hypothesis posits that high-dimensional data typically resides on low-dimensional sub spaces. In this paper, we assume
manifold hypothesis to investigate graph-based semi-supervised learning methods. In particular, we examine Laplace Learning in the Wasserstein space, extending the classical notion of graph-based semi-supervised learning
algorithms from finite-dimensional Euclidean spaces to an infinite-dimensional setting. To achieve this, we prove variational convergence of a discrete graph p- Dirichlet energy to its continuum counterpart. In addition, we characterize the Laplace-Beltrami operator on asubmanifold of the Wasserstein space. Finally, we validate the proposed theoretical framework through numerical experiments conducted on
benchmark datasets, demonstrating the consistency of our classification performance in high-dimensional settings.
Read the full abstract ▾
All authors Mary Chriselda Antony Oliver, Michael Roberts, Carola-Bibiane Schönlieb, Matthew Thorpe
Machine Learning (Statistics)
Likelihood-guided Regularization in Attention Based Models
Mohamed Salem and Inyoung Kim
The
transformer architecture has demonstrated strong performance in classification tasks involving structured and high-dimensional data. However, its success often hinges on large- scale training data and careful regularization to prevent overfitting. In this paper, we intro- duce a novel likelihood-guided variational Ising-based regularization framework for
Vision Transformers (ViTs), which simultaneously enhances model generalization and dynamically prunes redundant parameters. The proposed variational Ising-based regularization approach leverages
Bayesian sparsification techniques to impose structured sparsity on model weights, allowing for adaptive architecture search during training. Unlike traditional dropout-based methods, which enforce fixed sparsity patterns, the variational Ising-based regularization method learns task-adaptive regularization, improving both efficiency and interpretability. We evaluate our approach on
benchmark vision datasets, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100, demonstrating improved generalization under sparse, complex data and allowing for principled uncertainty quantification on both weights and selection parameters. Additionally, we show that the Ising regularizer leads to better-calibrated probability estimates and structured feature selection through uncertainty-aware
attention mechanisms. Our results highlight the effectiveness of structured
Bayesian sparsification in enhancing
transformer-based architectures, offering a principled alternative to standard regularization techniques.
Read the full abstract ▾
Statistical Methodology
Modeling group heterogeneity in spatio-temporal data via physics-informed semiparametric regression
Marco F. De Sanctis, Eleonora Arnone, Francesca Ieva et al. (+1 more)
In this work we propose a novel approach for modeling spatio-temporal data characterized by group structures. In particular, we extend classical mixed effect regression models by introducing a space-time nonparametric component, regularized through a partial differential equation, to embed the physical dynamics of the underlying process, while random effects capture latent variability associated with the group structure present in the data. We propose a two-step procedure to estimate the fixed and random components of the model, relying on a functional version of the Iterative Reweighted Least Squares
algorithm. We investigate the asymptotic properties of both fixed and random components, and we assess the performance of the proposed model through a simulation study, comparing it with state-of-the-art alternatives from the literature. The proposed methodology is finally applied to the study of hourly nitrogen dioxide concentration data in Lombardy (Italy), using random effects to account for measurement heterogeneity across monitoring stations equipped with different sensor technologies.
Read the full abstract ▾
All authors Marco F. De Sanctis, Eleonora Arnone, Francesca Ieva, Laura M. Sangalli
Statistics Theory
Asymptotic confidence bands for centered purely random forests
Natalie Neumeyer, Jan Rabe and Mathias Trabs
In a multivariate nonparametric regression setting we construct explicit asymptotic uniform confidence bands for centered purely random forests. Since the most popular example in this class of random forests, namely the uniformly centered purely random forests, is well known to suffer from suboptimal rates, we propose a new type of purely random forests, called the Ehrenfest centered purely random forests, which achieve minimax optimal rates. Our main confidence band
theorem applies to both random forests. The proof is based on an interpretation of random forests as generalized U-Statistics together with a Gaussian approximation of the supremum of empirical processes. Our theoretical findings are illustrated in simulation examples.
Read the full abstract ▾
Statistics Theory
Convergence rate of randomized midpoint Langevin Monte Carlo
Ruinan Li, Tian Shen and Zhonggen Su
The randomized midpoint Langevin Monte Carlo (RLMC), introduced by Shen and Lee (2019), is a variant of classical Unadjusted Langevin
Algorithm. It was shown in the literature that the RLMC is an efficient
algorithm for approximating high-dimensional probability distribution $\pi$. In this paper, we establish the exponential ergodicity of RLMC with constant step-size. Moreover, we design a dereasing-step size RLMC and provide its convergence rate in terms of a functional class distance.
Read the full abstract ▾
Statistics Theory
Non-asymptotic Analysis of Poisson randomized midpoint Langevin Monte Carlo
Tian Shen and Zhonggen Su
The task of sampling from a high-dimensional distribution $\pi$ on $\R^d$ is a fundamental algorithmic problem with applications throughout statistics, engineering, and the sciences. Consider the Langevin
diffusion on $\R^d$ \begin{align*} \dif X_t=-\nabla U(X_t)dt+\sqrt{2}dB_t, \end{align*} under mild conditions, it admits $\pi(\dif x)\propto \exp(-U(x))\dif x$ as its unique stationary distribution. Recently, Kandasamy and Nagaraj (2024) introduced a
stochastic algorithm called Poisson Randomized Midpoint Langevin Monte Carlo (PRLMC) to enhance the rate of convergence towards the target distribution $\pi$. In this paper, we first show that under mild conditions, the PRLMC, as a Markov chain, admits a unique stationary distribution $\pi_\eta$ ($\eta$ is the step size) and obtain the convergence rate of PRLMC to $\pi_\eta$ in total variation distance. Then we establish a sharp error bound between $\pi_\eta$ and $\pi$ under the 2-Wasserstein distance. Finally, we propose a decreasing-step size version of PRLMC and provide its convergence rate to $\pi$ which is nearly optimal.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Generalization Bounds for Semi-supervised Matrix Completion with Distributional Side Information
Antoine Ledent, Mun Chong Soo and Nong Minh Hieu
We study a
matrix completion problem where both the ground truth $R$
matrix and the unknown sampling distribution $P$ over observed entries are low-rank
matrices, and share a common subspace. We assume that a large amount $M$ of unlabeled data drawn from the sampling distribution $P$ is available, together with a small amount $N$ of labeled data drawn from the same distribution and noisy estimates of the corresponding ground truth entries. This setting is inspired by recommender systems scenarios where the unlabeled data corresponds to `implicit feedback' (consisting in interactions such as purchase, click, etc. ) and the labeled data corresponds to the `explicit feedback', consisting of interactions where the user has given an explicit rating to the item. Leveraging powerful results from the theory of low-rank subspace recovery, together with classic generalization bounds for
matrix completion models, we show error bounds consisting of a sum of two error terms scaling as $\widetilde{O}\left(\sqrt{\frac{nd}{M}}\right)$ and $\widetilde{O}\left(\sqrt{\frac{dr}{N}}\right)$ respectively, where $d$ is the rank of $P$ and $r$ is the rank of $M$. In synthetic experiments, we confirm that the true generalization error naturally splits into independent error terms corresponding to the estimations of $P$ and and the ground truth
matrix $\ground$ respectively. In real-life experiments on Douban and MovieLens with most explicit ratings removed, we demonstrate that the method can outperform baselines relying only on the explicit ratings, demonstrating that our assumptions provide a valid toy theoretical setting to study the interaction between explicit and implicit feedbacks in recommender systems.
Read the full abstract ▾
Statistics Theory
Reconciliation of Bayes and empirical Bayes interval estimation with application to small area estimation
Aditi Sen, Masayo Y. Hirose and Partha Lahiri
Multi-level normal hierarchical models, also interpreted as mixed effects models, play an important role in developing statistical theory in multi-parameter estimation for a wide range of applications, including small area estimation. In this article, we propose a novel reconciliation framework of the empirical Bayes (EB) and hierarchical Bayes approaches for interval estimation of random effects under a two-level normal model. Our framework shows that a second-order efficient empirical Bayes confidence interval, with EB coverage error of order $O(m^{-3/2})$, $m$ being the number of areas in the area-level model, can also be viewed as a credible interval whose expected posterior coverage is close to the nominal level, provided a carefully chosen prior, referred to as a "matching prior", is placed on the hyperparameters. While existing literature has examined matching priors that reconcile frequentist and
Bayesian inference in various settings, this paper is the first to study matching priors with the goal of interval estimation of random effects in a two-level model. We obtain a matching prior on the variance component that achieves a proper posterior under mild regularity conditions. The theoretical results in the paper are corroborated through a Monte Carlo simulation study and real data analysis.
Read the full abstract ▾
Machine Learning (Statistics)
Recovering Geodesic Distances from Noisy Observations
Charles Fefferman, Jonathan Marty and Kevin Ren
Let $(M,d,\mu)$ be a nice geodesic probability space (such as a
manifold) and $X=\{x_i\}_{i\in [N]} \subset M$ a finite $\mu$-i.i.d. sample. Suppose we observe noisy distance random variables $d'(x_i,x_j)$. When can the true distances be recovered to small additive error? This question has attracted interest in two disparate fields: latent-space graph models and boundary-value inverse problems. We first show that, taking $N \asymp \epsilon^{-2d-2}|\log\epsilon|$ and using runtime $O(N^{2.5+o_d (1)})$, we can recover all distances in $X$ to additive error $O(\epsilon|\log\epsilon|)$ with high probability, assuming $\mathbb{E} d'(x_i, x_j) = F(d(x_i,x_j))$ for an unknown, increasing bilipschitz $F$ and with the $d'(x_i,x_j)$ allowed to be arbitrarily heterogeneous in their distribution as long as they are bounded in subgaussian norm. This generalizes existing work, which required $F$ to be known and made strict assumptions on the distribution of $d'(x_i, x_j)$. Achieving this generalization required a novel “ruler-building” procedure to learn $F$. We extend our results to a missing data case where we randomly lose access to noisy distances. Our second result uses ideas from the first to tackle a partial observability case: $X = Y \cup Z$ with $Z$ sampled from $M$ and $Y$ sampled from an observer subset satisfying visibility conditions that model the assumptions made on the boundary in boundary-value inverse problems. In this setting, we recover all pairwise distances in $Z$ to additive error $O(\epsilon)$ with $|Z| \asymp \epsilon^{-2d-2}|\log\epsilon|$, $|Y| \asymp \epsilon^{-\max\{d_\nu,4\}}$ with $d_\nu$ determined by the volume-growth properties of the probability measure on the observer set, and runtime $O(|Z|^3)$. This is, to our knowledge, the first quantitative guarantee for this observer/observed split under noise, which complements existing stability results for boundary data.
Read the full abstract ▾
Statistical Methodology
Bayesian Variable Selection on Small Sample Trial Data via Adaptive Posterior-Informed Shrinkage Prior
Lingxuan Kong, Yumin Zhang, Chenkun Wang et al. (+1 more)
Identifying variables associated with clinical endpoints is of much interest in clinical trials. With the rapid growth of
cell and gene therapy (CGT) and therapeutics for ultra-rare diseases, there is an urgent need for statistical methods that can detect meaningful associations under severe sample-size constraints. Motivated by data-borrowing strategies for historical controls, we propose the Adaptive Posterior-Informed Shrinkage Prior (APSP), a
Bayesian approach that adaptively borrows information from external sources to improve variable-selection efficiency while preserving robustness across plausible scenarios. APSP builds upon existing
Bayesian data borrowing frameworks, incorporating data-driven adaptive information selection, structure of mixture shrinkage informative priors and decision making with empirical null to enhance variable selection performances under small sample size. Extensive simulations show that APSP attains better efficiency relative to traditional and popular data-borrowing and
Bayesian variable-selection methods while maintaining robustness under linear relationships. We further applied APSP to identify variables associated with peak C-peptide at Day 75 from the Clinical Islet Transplantation (CIT) Consortium study CIT06 by borrowing information from the study CIT07.
Read the full abstract ▾
All authors Lingxuan Kong, Yumin Zhang, Chenkun Wang, Yaoyuan Vincent Tan
Statistics Applications
Scalable Vision-Guided Crop Yield Estimation
Harrison H. Li, Medhanie Irgau, Nabil Janmohamed et al. (+2 more)
Precise estimation and uncertainty quantification for average crop yields are critical for agricultural monitoring and decision making. Existing data collection methods, such as crop cuts in randomly sampled fields at harvest time, are relatively time-consuming. Thus, we propose an approach based on prediction-powered inference (PPI) to supplement these crop cuts with less time-consuming field photos. After training a computer
vision model to predict the ground truth crop cut yields from the photos, we learn a “control function" that recalibrates these predictions with the spatial coordinates of each field. This enables fields with photos but not crop cuts to be leveraged to improve the precision of zone-wide average yield estimates. Our control function is learned by training on a
dataset of nearly 20,000 real crop cuts and photos of rice and maize fields in sub-Saharan Africa. To improve precision, we pool training observations across different zones within the same first-level subdivision of each country. Our final PPI-based point estimates of the average yield are provably asymptotically unbiased and cannot increase the asymptotic variance beyond that of the natural baseline
estimator – the sample average of the crop cuts – as the number of fields grows. We also propose a novel bias-corrected and accelerated (BCa) bootstrap to construct accompanying confidence intervals. Even in zones with as few as 20 fields, the point estimates show significant empirical improvement over the baseline, increasing the effective sample size by as much as 73% for rice and by 12-23% for maize. The confidence intervals are accordingly shorter at minimal cost to empirical finite-sample coverage. This demonstrates the potential for relatively low-cost images to make area-based crop insurance more affordable and thus spur investment into sustainable agricultural practices.
Read the full abstract ▾
All authors Harrison H. Li, Medhanie Irgau, Nabil Janmohamed, Karen Solveig Rieckmann, David B. Lobell
Statistics Theory
Transfer Learning and Locally Linear Regression for Locally Stationary Time Series
Jinwoo Park
This paper investigates locally linear regression for locally stationary time series and develops theoretical results for locally linear smoothing and transfer learning. Existing analyses have focused on local constant
estimators and given samples, leaving the principles of transferring knowledge from auxiliary sources across heterogeneous time-varying domains insufficiently established. We derive uniform convergence for multivariate locally linear
estimators under strong mixing. The resulting error expansion decomposes
stochastic variation, smoothing bias, and a term induced by local stationarity. This additional term, originating from the locally stationary structure, has smaller order than in the Nadaraya-Watson
benchmark, explaining the improved local linear performance. Building on these results, we propose bias-corrected transfer learned
estimators that connect a sparsely observed series with densely observed related sources through a smoothly varying bias function defined over rescaled time and
covariates. An additional refinement shows how local temporal adjustment of this bias enhances stability and enables efficient information borrowing across domains. Simulation studies and an empirical analysis of international fuel prices support the theoretical predictions and demonstrate the practical advantages of transfer learning.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
On the Information Processing of One-Dimensional Wasserstein Distances with Finite Samples
Cheongjae Jang, Jonghyun Won, Soyeon Jun et al. (+3 more)
Leveraging the Wasserstein distance – a summation of sample-wise transport distances in data space – is advantageous in many applications for measuring support differences between two underlying density functions. However, when supports significantly overlap while densities exhibit substantial pointwise differences, it remains unclear whether and how this transport information can accurately identify these differences, particularly their analytic characterization in finite-sample settings. We address this issue by conducting an analysis of the information processing capabilities of the one-dimensional Wasserstein distance with finite samples. By utilizing the Poisson process and isolating the rate factor, we demonstrate the capability of capturing the pointwise density difference with Wasserstein distances and how this information harmonizes with support differences. The analyzed properties are confirmed using neural spike train decoding and amino acid contact frequency data. The results reveal that the one-dimensional Wasserstein distance highlights meaningful density differences related to both rate and support.
Read the full abstract ▾
All authors Cheongjae Jang, Jonghyun Won, Soyeon Jun, Chun Kee Chung, Keehyoung Joo, Yung-Kyun Noh
Statistical Methodology · Cross-listed from econ.EM
Identification-aware Markov chain Monte Carlo
Toru Kitagawa and Yizhou Kuang
Leaving posterior sensitivity concerns aside, non-identifiability of the parameters does not raise a difficulty for
Bayesian inference as far as the posterior is proper, but multi-modality or flat regions of the posterior induced by the lack of identification leaves a challenge for modern
Bayesian computation. Sampling methods often struggle with slow or non-convergence when dealing with multiple modes or flat regions of the target distributions. This paper develops a novel Markov chain Monte Carlo (MCMC) approach for non-identified models, leveraging the knowledge of observationally equivalent sets of parameters, and highlights an important role that identification plays in modern
Bayesian analysis. We show that our identification-aware proposal eliminates mode entrapment, achieving a convergence rate uniformly bounded away from zero, in sharp contrast to the exponentially decaying rates characterizing standard Random Walk Metropolis and
Hamiltonian Monte Carlo. Simulation studies show its superior performance compared to other popular computational methods including
Hamiltonian Monte Carlo and sequential Monte Carlo. We also demonstrate that our method uncovers non-trivial modes in the target distribution in a structural vector moving-average (SVMA) application.
Read the full abstract ▾