Statistics Theory
Nonparametric Regression with Measurement Error in Banach Spaces
Pratim Guha Niyogi and Priyadarshi Dey
We consider nonparametric regression with a Banach-space-valued predictor where the
covariate takes values in a general separable Banach space. Our primary objective is to develop a unified framework for estimating the regression operator without relying on an inner-product structure, thereby extending classical and functional nonparametric regression beyond Euclidean and
Hilbert spaces in the presence of measurement error. We establish the fundamental properties of the proposed
estimator and develop its large-sample theory under conditions formulated in terms of the geometry and local concentration of the Banach-valued predictor. In particular, we study pointwise estimation and inference and discuss the construction of pointwise confidence intervals and uniform confidence bands. Our proposed framework is further extended to accommodate error-in-variables settings, where the predictor is observed with contamination. We also investigate related inverse problems arising from the construction of inverse weighting functions and consider partial contamination models. These developments provide a general framework for nonparametric regression and inference with complex infinite-dimensional predictors and identify several statistical and mathematical challenges that arise when the usual
Hilbert-space structure is unavailable.
Read the full abstract ▾
Statistics Theory
Low Dimensional Sampling under Reconstructed Constraints
Imon Banerjee and Riddhiman Bhattacharyya
We study sampling from a distribution supported on an unknown compact $d$-dimensional $C^2$
manifold $M\subset\mathbb{R}^D$, observed only through i.i.d. uniform points from $M$. We reconstruct the constraint using an adaptive local-convex-hull
estimator and target an ambient distribution penalized by squared distance to the reconstruction. Although the reconstructed set may be nonsmooth or fail to be a
manifold, its Hausdorff accuracy alone suffices to control the Wasserstein error. We quantify the tradeoff between reconstruction accuracy and penalty strength and show that optimal tuning gives an error proportional to the square root of the Hausdorff error. A planar example proves that this dependence is sharp for the proposed scheme. For a $d$ dimensional constraint reconstructed from $N$ observations, the error becomes $\mathcal{O}( (N/N)^{1/d})$. Finally, we prove uniform
geometric ergodicity of a Gaussian random-walk Metropolis–Hastings sampler and combine reconstruction, approximation, and mixing into an explicit finite-time guarantee.
Read the full abstract ▾
Machine Learning (Statistics)
A Functional SVD Framework for Regularized Multivariate Functional PCA with Dual Penalization
Yue Zhao, Hossein Haghbin, Rebecca Sanders et al. (+1 more)
This paper introduces a novel framework for Regularized Multivariate Functional Principal Component Analysis (ReMFPCA) via Functional Singular Value Decomposition (SVD). The proposed method extends existing MFPCA approaches by incorporating a generalized functional SVD within a
Hilbert space framework, enabling simultaneous regularization of both functional principal components (PCs) and their associated PC scores. A key innovation of this framework is the inclusion of a sparsity penalty on the PC scores, which enhances interpretability by filtering out irrelevant subject-specific variations. This dual-penalization strategy represents a significant advancement beyond existing covariance-based eigen decomposition methods, which penalize only the functional PCs. Two power
algorithm implementations, sequential and joint, are proposed, together with a cross-validation approach based on iterative regression for optimal smoothing parameter selection. Comprehensive simulation studies and real data applications demonstrate that the proposed framework substantially improves the extraction of informative and interpretable components, offering methodological and practical benefits for analyzing multivariate functional data across diverse domains.
Read the full abstract ▾
All authors Yue Zhao, Hossein Haghbin, Rebecca Sanders, Mehdi Maadooliat
Machine Learning (Statistics)
From matrix inversion to constraints: provably tighter confidence regions for importance weights in label shift
Mushan Li, Kihyun Han and Yanyuan Ma
Importance weights are essential in domain adaptation under label shift, yet their utility is often undermined by the finite sample uncertainty associated with their estimation. Existing methods typically analyze this uncertainty through Gaussian elimination on interval-valued linear systems, which leads to overly conservative confidence regions and inefficient downstream applications. We propose a paradigm shift from inversion-based inference to a direct
matrix constraint framework. We use this framework to define a joint confidence region and extract marginal intervals via linear programming, deriving provably tighter bounds for importance weights while maintaining exact finite-sample validity. Furthermore, we analyze the confidence region's geometry and provide the theoretical results for its diameter bounds. Evaluated across text, image,
multimodal benchmarks, including AGNews, MNIST, CIFAR-10, N24News, and a real-world autonomous driving
dataset, nuImages, our approach consistently yields shorter confidence intervals and smaller prediction sets than inversion-based methods.
Read the full abstract ▾
Statistical Methodology
Three Case Studies of Two-stage MCMC for Fast Bayesian Inference of Large Spatio-temporal Data
Mostafa Shams, Robert Erhardt and Staci Hepler
High dimensional spatio-temporal models can quickly run up against computational limitations. Recursive or multi-stage
Bayesian algorithms are one way to address this computational challenge. Here we consider an increasingly used two-stage
algorithm. The first stage models locations independently and in parallel across space. Resampling particles from this stage-one model with Metropolis-within-Gibbs methods and suitably computed acceptance ratios can impose spatial dependence across locations at a low computational cost, while targeting the same posterior distribution as the single-stage MCMC
algorithm at a fraction of the overall computational cost. In this paper we show three complete examples of how two-stage MCMC
algorithms can be used to efficiently fit
Bayesian spatio-temporal models to large
datasets. Specifically, we consider: a spatio-temporal self-exciting count model on areal data utilizing intrinsic conditional autoregressive (ICAR) prior distributions; a spatio-temporal model of point-referenced data using a stationary, isotropic distance-based covariance function; and a binary model fit to data on a spatial
lattice utilizing a latent Gaussian process to capture all spatio-temporal dependence. In each example, we define the model and describe the corresponding two-stage MCMC approach. We then compare the resulting posterior samples and computational efficiency with those obtained using a standard single-stage MCMC
algorithm. While varying across the three examples and numerous parameters, the two-stage approach often achieves roughly an order of magnitude higher computational efficiency, with the two-stage and single-stage
algorithms producing closely agreeing posterior samples.
Read the full abstract ▾
Statistics Applications
Zero-Inflated Model Lab: An Interactive Shiny Application for Modelling Count Data with Excess Zeros
Oscar Rodriguez de Rivera
Count data often contain a large number of zeros in ecological and environmental research, arising from factors such as true species absence, imperfect detection, or the rarity of certain events. Choosing and interpreting suitable statistical models, like Poisson, Negative Binomial, zero-inflated, or hurdle models, can be difficult for non-experts because of the models' assumptions and diagnostic requirements. To address this, we introduce an interactive web application built in R with the Shiny framework that helps users analyse, compare, and interpret count-data models that may exhibit zero inflation. The application enables users to upload
datasets, explore distributions and relationships among variables, and fit a variety of models including Poisson, Negative Binomial, zero-inflated, and hurdle approaches. It incorporates tools for model comparison using information criteria, visualisation of coefficient estimates and confidence intervals, assessment of variable importance, and diagnostic checks based on simulated residuals. Importantly, the app provides clear visualisations of both the count-generating and zero-generating processes, making it easier to communicate components that are typically challenging to interpret. To improve accessibility, the tool offers guided explanations of outputs and diagnostics, making it valuable for both research applications and teaching. A case study using environmental count data illustrates how the application helps users identify zero inflation, choose appropriate models, and gain insight into the ecological processes underlying the data. Overall, the application narrows the gap between advanced statistical methods and practical analysis, supporting reproducible workflows and enhancing the clarity and interpretability of models for zero-inflated count data.
Read the full abstract ▾
Statistics Applications · Cross-listed from eess.SY
Recent Advances in Resilient Multi-Energy Systems Against Climate Change
Grant Ruan, Zhengmao Li, Yi Wang et al. (+1 more)
Climate change is a global threat to the long-term sustainable development of energy systems. Recent works have explored the emerging opportunity of coordinating different energy carriers and sectors (e.g. electricity, natural gas, heating, hydrogen, transportation, and water sectors) to unlock the cross-sector flexibility against
climate change. This review has established a holistic framework for resilient multi-energy systems through the lens of nested coupling. It covers the most recent progress in resilience resources, resilience evaluation, resilience-oriented operation & planning, resilience pricing & investment, and real-world implementation. This work differs from prior studies through a full investigation on
climate change impacts (distribution shifts), resilience pricing, and global projects. Within this area, we advocate for a unique and interdisciplinary perspective spanning across energy systems,
climate science, sociology, economics, and data science. At the end, seven major challenges and opportunities are identified, including data deficiency, distributed coordination &
privacy, high-fidelity simulation, and machine learning techniques. Researchers, industrial experts, and policy makers can follow this review to capture the emerging trend and future opportunities in this growing area.
Read the full abstract ▾
All authors Grant Ruan, Zhengmao Li, Yi Wang, Ning Zhang
Machine Learning (Statistics)
PU classification under Non-SCAR: clustering-assisted logistic model with oversampling enhancement
Konrad Furmańczyk and Kacper Paczutkowski
This study addresses the PU classification problem under violations of the SCAR assumption. We investigate logistic regression-based approaches, namely the cluster method and its extensions with strict and non-strict Lasso regularization. The primary contribution of this work is the integration of the SMOTE technique to alleviate class imbalance and systematically assess its impact on the performance of the considered
algorithms. SMOTE is first applied to rebalance the training
dataset. Next, cleaning labels are derived via 2-means clustering. Logistic regression is then trained on the cleaned data, where identified positive instances are augmented with additional true positives and the remaining observations are treated as negative. The experimental evaluation is conducted on 13 real
benchmark datasets and one synthetic
dataset. For comparison, we include the naive approach and the Spy-EM method. The results demonstrate that incorporating SMOTE improves classification performance when the SCAR condition is violated and indicate moderate robustness of the LassoJoint method in this setting.
Read the full abstract ▾
Statistics Theory
Two short proofs of incompatibility for correlation matrices
Cedric Phillips
Let $P_d$ be the elliptope of $d \times d$ correlation
matrices, and, for a standardised law $F$, let $S_d^F$ be the set of correlation
matrices of random vectors with all margins $F$. We give two short proofs. First, an eleven-term representation of $|x|^4$ as a sum of fourth powers of linear forms on $\mathbb{R}^4$ yields $S_{11} \neq P_{11}$ for uniform margins. Second, the six diagonals of the regular icosahedron yield $S_6^F \neq P_6$ for arcsine margins. Together with a result of Devroye and Letac, this gives $S_d^F = P_d$ if and only if $d \le 5$, settling a
conjecture of theirs.
Read the full abstract ▾
Statistics Theory
Unrestricted conditional maximum likelihood estimation for a Fréchet regression model
Marcio Reverbel and Johan Segers
Heavy-tailed response variables are routinely modeled as a function of
covariates, but the asymptotic theory available for the resulting maximum likelihood
estimators sidesteps certain technical difficulties by restricting the parameters to a compact set, which is not the natural parameter space. We consider a Fréchet model whose scale parameter depends log-linearly on a vector of
covariates, while the tail parameter is constant, and we study the conditional maximum likelihood
estimator of the tail parameter and the regression coefficients jointly. We show that this
estimator is consistent over the full, noncompact parameter space, and that it is asymptotically normal, with the inverse Fisher information as limiting covariance
matrix. Consistency is obtained by a specific compactification of the parameter space relying on the
cosmic closure of Euclidean space. The extension of Wald's consistency
theorem needed for that step, from single observations to blocks of observations, is stated in an abstract setting and is possibly of independent interest.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CV
Direct Conditional Transition Sampling for Diffusion Inverse Problems
Qi Yu, Hanlin Wu and Xiaohui Sun
Training-free
diffusion inverse solvers typically choose between local measurement guidance and costly clean-space posterior updates. Independent posterior refresh can improve global correction by sampling a clean conditional and re-noising it, but its practical realization requires probability-flow ODE integration and clean-space Markov chain Monte Carlo (MCMC). We propose Direct Conditional Transition Sampling (DCTS), a direct
stochastic-flow approximation to the same ideal refresh target. Rather than explicitly drawing a clean sample, DCTS estimates the measurement-conditioned clean mean along a short inner path and transports Gaussian source noise directly to the next noisy state. A denoiser-compatible sufficient statistic and a covariance-scaled operator update enable this conditional-mean estimation. Experiments on four inverse problems demonstrate that DCTS achieves competitive reconstruction quality with up to $16.8\times$ speedups over competing methods.
Read the full abstract ▾
Machine Learning (Statistics)
Evaluation of optimisation and Bayesian inference methods for reaction rates in atmospheric chemical mechanisms
Valery Ashu, Wenqing Peng, Zhi-Song Liu et al. (+7 more)
Constraining reaction rate coefficients is a central challenge in the development of explicit atmospheric chemical mechanisms, particularly for autoxidation systems where many reaction pathways are only indirectly observed through high-resolution mass spectrometry. In this study, we evaluate rate-coefficient optimisation methods for a toy-case autoxidation mechanism using synthetic data with known ground truth. Two complementary approaches are compared: ODE-constrained neural-network optimisation, which provides efficient point estimates of uncertain rate coefficients, and the Markov Chain Monte Carlo (MCMC) approach, which samples the posterior distribution of rate coefficients and quantifies parameter uncertainty. The methods are tested using direct concentration observations and mass-spectral observations under different noise levels. For unperturbed and low-noise synthetic observations, both methods converged towards the known rate coefficients, with the neural-network optimiser providing faster point estimates. Under high-noise conditions (with the signal-to-noise ratio approximately S / N = 1), however, MCMC was substantially more robust in recovering the rate coefficients. The posterior analysis shows that mass-spectral aggregation broadens credible intervals even at low noise, and that high-noise mass spectra can leave many individual reaction rates weakly identifiable. Posterior predictive validation nevertheless shows how broad parameter uncertainty constrained by MCMC remains consistent with accurate reproduction of the observable mass spectrum. These results demonstrate that point-estimation and
Bayesian sampling methods provide complementary information: neural-network optimisation is effective for informative data, whereas MCMC is essential for diagnosing uncertainty, non-uniqueness, and identifiability in noisy or aggregated inverse problems.
Read the full abstract ▾
All authors Valery Ashu, Wenqing Peng, Zhi-Song Liu, Heikki Haario, Andreas Rupp, Taiwo Ashu, Petri Clusius, Lukas Pichelstorfer, Zihao Fu, Michael Boy
Statistics Theory · Cross-listed from math.DG
Equivalence on exponential families as Hessian manifolds and classification of exponential families of order 1 with constant Hessian sectional curvature
Koichi Tojo and Taro Yoshino
We introduce an equivalence relation on exponential families, and prove that two exponential families are equivalent in this sense if and only if the corresponding induced Hessian
manifolds are isomorphic. Moreover, we classify exponential families of order 1 with constant Hessian sectional
curvature. To this end, we show that for an exponential family of order 1, it has constant Hessian sectional
curvature if and only if the natural exponential family equivalent to the given family has a quadratic variance function (NEF-QVF). The classification coincides with that of NEF-QVF by Morris (1982) essentially.
Read the full abstract ▾
Machine Learning (Statistics)
Multi-source conformal prediction: leveraging heterogeneity via localization
Rohan Hore, Anirban Chatterjee and Sayantan Choudhury
Many modern prediction tasks involve data from multiple heterogeneous sources, while the test distribution may differ substantially from any individual source. Although heterogeneity poses challenges, it also offers an opportunity: different sources may provide complementary information, with some regions of the feature space better represented in one source than another. We propose Multi-Source Randomly Localized Conformal Prediction (MS-RLCP), which builds on the local coverage properties of randomly localized conformal prediction (RLCP) (Hore and Barber, 2025) and extends it to multiple sources through data-adaptive source selection. Under the widely adopted assumption of a shared response distribution conditional on the features across sources and the test population, we establish finite-sample coverage bounds using an interpretable notion of envelope distribution that captures their aggregate feature-space representation. Our analysis allows the test feature distribution to be absolutely continuous with respect to the envelope, extending beyond mixtures of source distributions. Under additional regularity conditions, we also establish asymptotic test-conditional coverage. Simulations and real-world experiments demonstrate the effectiveness of MS-RLCP across varying levels of data heterogeneity.
Read the full abstract ▾
Statistical Methodology
Bootstrap-Calibrated Spectral Divergence Tests for Online Detection of Covariance Matrix Changes
Mehmet Siddik Cadirci and Martin Singull
A covariance
matrix rarely distorts in a single direction: a shift can expand all variances simultaneously, shift variance along one or two principal directions, displace spectral mass without changing marginal means, or rotate the dependence structure. Default mean-shift detectors are blind to these effects, and no existing online procedure calibrates a family of spectral deviation tests with provable false alarm control across both time and tracking window selection; this paper fills that gap. We consider four spectral deviations: $D_{KL}(P_{1}\|P_{0})$, $D_{KL}(P_{0}\|P_{1})$, Jeffreys, and Bhattacharyya, each evaluated on the
eigenvalues of the empirical relative covariance operator $\widehat{\Sigma}_{0}^{-1/2}\widehat{\Sigma}_{t}\widehat{\Sigma}_{0}^{-1/2}$. Critical values come from a conditional parametric bootstrap that accounts for estimation uncertainty in both the past and tracking windows, an aspect asymptotic approaches typically overlook. The procedure controls the false alarm rate family-by-family over a predefined monitoring period and a range of candidate window sizes; when a single operational window is needed, a power-based criterion selects it. We prove consistency under constant alternatives via local spectral expansions with second-order sensitivity near the null. Simulations are conservative under the null and show detection power depends largely on spectral shape rather than magnitude: $D_{KL}(P_{1}\|P_{0})$ excels under global inflation, while Jeffreys and Bhattacharyya cover a broader range of alternatives. We illustrate the approach on three financial applications: European stock indices, Fama-French sector portfolios, and large-cap technology stocks.
Read the full abstract ▾
Statistical Methodology
Identification of causal policy effects for spatial exposure fields
S.I. Watson and J. Fraser-Govil
Many interventions act on an entire spatial exposure field. We study when their induced laws satisfy policy positivity and what its failure implies for causal evaluation through weighting and structural modelling. We show that, for post-transformation fields that are Gaussian, the only admissible translation that satisfy positivity lie on the Cameron-Martin space of the observed exposure field. Under fine-scale variation and regularity assumptions, these are the only admissible non-decreasing pointwise policies on that scale. Common rules, including binding caps and proportional reductions on the Gaussian scale can therefore violate positivity. The Cameron-Martin norm controls both weight variability and the amplification of disagreement between candidate outcome models into disagreement between policy estimates. Without positivity, bounded continuous response models can make arbitrarily similar observational predictions while their policy estimates remain separated. We also give a policy-specific criterion for structural identification conditional on a realised exposure state. Finally, when a policy violating positivity admits weights in every finite representation, their second moments diverge under nested refinement that recovers the full field. These results motivate assessing identification and sensitivity for the specified policy effect across plausible structural models and under refinement of finite representations.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
Retrieval-Guided Fine-Tuning as Noisy Estimation: Risk bounds and Architectural Analysis
Bhargav Lad and Yifan Hao
Retrieval-Guided Fine-Tuning (RAG-FT) incorporates retrieved data directly into the training objective, but the statistical consequences of noisy
retrieval during training remain theoretically undercharacterized. We study this question by modeling RAG-FT as an estimation problem in a multi-task linear regression framework, using an OLS proxy for single-layer linear self-
attention to obtain finite-sample risk bounds. Under homoscedastic
retrieval noise, we show that
retrieval failure decays exponentially with task separation relative to noise, and derive explicit finite-sample conditions under which RAG-FT achieves lower risk than both target-only and full-corpus training. We then introduce a Distance-Proportional Noise (DPN) model, in which
retrieval quality degrades with rank, and compare two
estimators under the same
retrieval process: the OLS proxy and the literal, uniform-weight forward pass of linear self-
attention. We prove that the
attention estimator's bias diverges as $\Theta(n^{2q})$ even under exact
retrieval, while OLS risk remains $\Theta(d/n)$ for every noise exponent $q>0$. These results locate the instability not in noisy
retrieval itself, but in the fixed, unweighted aggregation of the literal LSA forward pass, which reweighting by reliability empirically removes. We validate the predicted rate separation through direct simulation of the DPN model.
Read the full abstract ▾
Statistical Methodology
A Mathai-type Multivariate Nakagami-$m$ Distribution: Density, Closed-form Estimators, and Asymptotic Efficiency
Hyeonwoo Kim and Hyoung-Moon Kim
In this study, we propose a novel multivariate Nakagami-$m$ distribution whose joint probability density function admits a fully closed-form expression. The proposed density is derived using a Mathai-type construction based on the partial sums of independent gamma random variables. For this distribution, the maximum likelihood
estimator is not available in the closed form. To address this limitation, an asymptotically efficient closed-form
estimator is developed by applying a one-step refinement to a $\sqrt{n}$-consistent initial
estimator. Monte Carlo simulations demonstrate that the proposed
estimator achieves a performance nearly identical to that of the numerically computed maximum likelihood
estimator, while consistently outperforming the closed-form initial
estimators. A real-data application further illustrates its practical utility.
Read the full abstract ▾