Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Tuesday 23 December

Machine Learning (Statistics)

Avoiding the Price of Adaptivity: Inference in Linear Contextual Bandits via Stability

Statistical inference in contextual bandits is challenging due to the adaptive, non-i.i.d. nature of the data. A growing body of work shows that classical least-squares inference can fail under adaptive sampling, and that valid confidence intervals for linear functionals typically require an inflation of order $\sqrt{d \log T}$. This phenomenon – often termed the price of adaptivity – reflects the intrinsic difficulty of reliable inference under general contextual bandit policies. A key structural condition that overcomes this limitation is the stability condition of Lai and Wei, which requires the empirical feature covariance to converge to a deterministic limit. When stability holds, the ordinary least-squares estimator satisfies a central limit theorem, and classical Wald-type confidence intervals remain asymptotically valid under adaptation, without incurring the $\sqrt{d \log T}$ price of adaptivity. In this paper, we propose and analyze a regularized EXP4 algorithm for linear contextual bandits. Our first main result shows that this procedure satisfies the Lai–Wei stability condition and therefore admits valid Wald-type confidence intervals for linear functionals. We additionally provide quantitative rates of convergence in the associated central limit theorem. Our second result establishes that the same algorithm achieves regret guarantees that are minimax optimal up to logarithmic factors, demonstrating that stability and statistical efficiency can coexist within a single contextual bandit method. As an application of our theory, we show how it can be used to construct confidence intervals for the conditional average treatment effect (CATE) under adaptively collected data. Finally, we complement our theory with simulations illustrating the empirical normality of the resulting estimators and the sharpness of the corresponding confidence intervals.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Clust-PSI-PFL: A Population Stability Index Approach for Clustered Non-IID Personalized Federated Learning

Federated learning (FL) supports privacy-preserving, decentralized machine learning (ML) model training by keeping data on client devices. However, non-independent and identically distributed (non-IID) data across clients biases updates and degrades performance. To alleviate these issues, we propose Clust-PSI-PFL, a clustering-based personalized FL framework that uses the Population Stability Index (PSI) to quantify the level of non-IID data. We compute a weighted PSI metric, $WPSI^L$, which we show to be more informative than common non-IID metrics (Hellinger, Jensen-Shannon, and Earth Mover's distance). Using PSI features, we form distributionally homogeneous groups of clients via K-means++; the number of optimal clusters is chosen by a systematic silhouette-based procedure, typically yielding few clusters with modest overhead. Across six datasets (tabular, image, and text modalities), two partition protocols (Dirichlet with parameter $\alpha$ and Similarity with parameter S), and multiple client sizes, Clust-PSI-PFL delivers up to 18% higher global accuracy than state-of-the-art baselines and markedly improves client fairness by a relative improvement of 37% under severe non-IID data. These results establish PSI-guided clustering as a principled, lightweight mechanism for robust PFL under label skew.
Read the full abstract

All authors Daniel M. Jimenez-Gutierrez, Mehrdad Hassanzadeh, David Solans, Mohammed Elbamby, Nicolas Kourtellis, Aris Anagnostopoulos, Ioannis Chatzigiannakis, Andrea Vitaletti

Statistics Applications

Towards Energy Independence: Critical Minerals in the Indian Context

The global impetus for extracting rare earth elements (REEs) is shaping the future of green technologies. From high-efficiency magnets in wind turbines to advanced batteries and solar photovoltaics, REEs are indispensable for a greener world through the energy transition. However, supply chains remain a barrier for the majority of the global population. India is mainly dependent on imports for most REEs. Innovation and recycling efforts in most REEs are still in their early stages. For India, aspiring to Viksit Bharat by 2047, securing sustainable REE access is critical to national energy security and technological independence. This paper explores India's opportunities and challenges in the REE domain, high-lighting underutilized resources such as copper tailings, fly ash, and e-waste. We argue for circular economy pathways that can reduce environmental impacts and strengthen domestic supply. Hindustan Copper Limited (HCL), as India's sole vertically integrated copper producer, is uniquely positioned to pioneer co-recovery of REEs, advance research and development partnerships, and build a comprehensive supply chain. By embedding sustainability, ESG principles, and community trust in its strategy, HCL can evolve into a national champion in this domain.
Read the full abstract

Machine Learning (Statistics)

A Structured Nonparametric Framework for Nonlinear Accelerated Failure Time Models (KAN-AFT)

Accelerated failure time (AFT) models provide a direct and interpretable time-scale description of covariate effects in lifetime data analysis, but classical formulations rely on linear predictors and are therefore limited in their ability to represent nonlinear relationships. Moreover, in heterogeneous clinical settings with complex covariate structures and varying censoring mechanisms, standard survival models such as the Cox proportional hazards model or AFT formulations may be inadequate due to restrictive structural assumptions. We propose a structured nonparametric extension of the AFT framework in which the regression function governing log-survival time is an unknown smooth function represented through Kolmogorov–Arnold representations. We formalize the nonlinear AFT estimand under independent right-censoring and show that the proposed function class strictly contains the classical linear AFT model as a special case. Estimation is carried out through a unified framework that accommodates several censoring-adjusted losses such as Buckley–James, inverse probability of censoring weight and transformation methods. Structural regularization and pruning promote parsimony, and symbolic approximation yields analytic representations of learned component functions. Simulation studies show that the method recovers linear structure when appropriate and captures nonlinear effects when present. Applications to multiple clinical datasets demonstrate competitive predictive performance and transparent covariate-effect estimation.
Read the full abstract

Statistical Methodology

The post-hoc test for local dependence

The concept of independence plays a crucial role in probability theory and has been the subject of extensive research in recent years. Numerous approaches have been proposed to test for independence; however, most of them address the problem only at a global level. From a practical perspective, it is important not only to determine whether the data are dependent but also to identify where this dependence occurs and how strong it is. The graphical presentation of results is another essential aspect that should not be neglected, as it considerably enhances interpretability. The main objective of this work is to propose a solution that considers these aspects simultaneously. Relying on copula-based results, we introduce a novel method for testing global and local statistical independence using the quantile dependence function. Rather than assessing whether the value of the test statistic exceeds a single critical threshold and subsequently deciding whether to reject the independence hypothesis, we introduce so-called critical surfaces that guaranty a locally equal probability of exceeding them under independence. This approach enables a detailed examination of local discrepancies and an assessment of their statistical significance while preserving the overall significance level of the test.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Adaptive Multi-task Learning for Probabilistic Load Forecasting

Simultaneous load forecasting across multiple entities (e.g., regions, buildings) is crucial for the efficient, reliable, and cost-effective operation of power systems. Accurate load forecasting is a challenging problem due to the inherent uncertainties in load demand, dynamic changes in consumption patterns, and correlations among entities. Multi-task learning has emerged as a powerful machine learning approach that enables the simultaneous learning across multiple related problems. However, its application to load forecasting remains underexplored and is limited to offline learning methods, which cannot capture changes in consumption patterns. This paper presents an adaptive multi-task learning method for probabilistic load forecasting. The proposed method can dynamically adapt to changes in consumption patterns and correlations among entities. In addition, the techniques presented provide reliable probabilistic predictions for loads of multiple entities and assess load uncertainties. Specifically, the method is based on vectorvalued hidden Markov models and uses a recursive process to update the model parameters and provide predictions with the most recent parameters. The performance of the proposed method is evaluated using datasets that contain the load demand of multiple entities and exhibit diverse and dynamic consumption patterns. The experimental results show that the presented techniques outperform existing methods both in terms of forecasting performance and uncertainty assessment.
Read the full abstract

Statistical Methodology

Estimation and Inference for Causal Explainability

Understanding how much each variable contributes to an outcome is a central question across disciplines. A causal view of explainability is favorable for its ability in uncovering underlying mechanisms and generalizing to new contexts. Based on a family of causal explainability quantities, we develop methods for their estimation and inference. In particular, we construct a one-step correction estimator using semi-parametric efficiency theory, which explicitly leverages the independence structure of variables to reduce the asymptotic variance. For a null hypothesis on the boundary, i.e., zero explainability, we show its equivalence to Fisher's sharp null, which motivates a randomization-based inference procedure. Finally, we illustrate the empirical efficacy of our approach through simulations as well as an immigration experiment dataset, where we investigate how features and their interactions shape public opinion toward admitting immigrants.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Learning to Reason in LLMs by Expectation Maximization

Large language models (LLMs) solve reasoning problems by first generating a rationale and then answering. We formalize reasoning as a latent variable model and derive a reward-based filtered expectation-maximization (FEM) objective for learning to reason. This view connects EM and modern reward-based optimization, and shows that the main challenge lies in designing a sampling distribution of rationales that justify correct answers. We instantiate and compare three sampling schemes: rejection sampling with a budget, self-taught reasoner (STaR), and prompt posterior sampling (PPS), which only keeps the rationalization stage of STaR that conditions on the correct answer in the prompt. We experiment with LLM-as-a-judge calibration and summarization from feedback tasks, where conditioning on the correct answer provides a strong guidance for generating rationales. Our experiments show the efficacy of PPS over other sampling schemes, and that the sampling scheme can have a significant impact on performance.
Read the full abstract

All authors Junghyun Lee, Branislav Kveton, Anup Rao, Subhojyoti Mukherjee, Ryan A. Rossi, Sunav Choudhary, Alexa Siu

Other Statistics

Change Point Detection and Mean-Field Dynamics of Variable Productivity Hawkes Processes

Many self-exciting systems change because endogenous amplification, as opposed to exogenous forcing, varies. We study a Hawkes process with fixed background rate and kernel, but piecewise time-varying productivity. For exponential kernels we derive closed-form mean-field relaxation after a change and a deterministic surrogate for post-change Fisher information, revealing a boundary layer in which change time information localises and saturates, while post-change level information grows linearly beyond a short transient. These results motivate a Bayesian change point procedure that stabilizes inference on finite windows. We illustrate the method on invasive pneumococcal disease incidence in The Gambia, identifying a decline in productivity aligned with pneumococcal conjugate vaccine rollout.
Read the full abstract

Statistics Theory

Structure-Preserving Nonlinear Sufficient Dimension Reduction for Tensors

We introduce two nonlinear sufficient dimension reduction methods for regressions with tensor-valued predictors. Our goal is two-fold: the first is to preserve the tensor structure when performing dimension reduction, particularly the meaning of the tensor modes, for improved interpretation; the second is to substantially reduce the number of parameters in dimension reduction, thereby achieving model parsimony and enhancing estimation accuracy. Our two tensor dimension reduction methods echo the two commonly used tensor decomposition mechanisms: one is the Tucker decomposition, which reduces a larger tensor to a smaller one; the other is the CP-decomposition, which represents an arbitrary tensor as a sequence of rank-one tensors. We developed the Fisher consistency of our methods at the population level and established their consistency and convergence rates. Both methods are easy to implement numerically: the Tucker-form can be implemented through a sequence of least-squares steps, and the CP-form can be implemented through a sequence of singular value decompositions. We investigated the finite-sample performance of our methods and showed substantial improvement in accuracy over existing methods in simulations and two data applications.
Read the full abstract

Machine Learning (Statistics)

Generative Bayesian Hyperparameter Tuning

\noindent Hyper-parameter selection is a central practical problem in modern machine learning, governing regularization strength, model capacity, and robustness choices. Cross-validation is often computationally prohibitive at scale, while fully Bayesian hyper-parameter learning can be difficult due to the cost of posterior sampling. We develop a generative perspective on hyper-parameter tuning that combines two ideas: (i) optimization-based approximations to Bayesian posteriors via randomized, weighted objectives (weighted Bayesian bootstrap), and (ii) amortization of repeated optimization across many hyper-parameter settings by learning a transport map from hyper-parameters (including random weights) to the corresponding optimizer. This yields a “generator look-up table” for estimators, enabling rapid evaluation over grids or continuous ranges of hyper-parameters and supporting both predictive tuning objectives and approximate Bayesian uncertainty quantification. We connect this viewpoint to weighted $M$-estimation, envelope/auxiliary-variable representations that reduce non-quadratic losses to weighted least squares, and recent generative samplers for weighted $M$-estimators.
Read the full abstract

Statistical Methodology

Assumption-lean covariate adjustment under covariate adaptive randomization when $p = o (n)$

Adjusting for (baseline) covariates with working regression models becomes standard practice in the analysis of randomized clinical trials (RCT). When the dimension $p$ of the covariates is large relative to the sample size $n$, specifically $p = o (n)$, adjusting for covariates even in a linear working model by ordinary least squares can yield overly large bias, defeating the purpose of improving efficiency. This issue arises when no structural assumptions are imposed on the outcome model, a scenario that we refer to as the assumption-lean setting. Several new estimators have been proposed to address this issue. However, they focus mainly on simple randomization under the finite-population model, not covering covariate adaptive randomization (CAR) schemes under the superpopulation model. Due to improved covariate balance between treatment groups, CAR is more widely adopted in RCT; and the superpopulation model fits better when subjects are enrolled sequentially or when generalizing to a larger population is of interest. Thus, there is an urgent need to develop procedures in these settings, as the current regulatory guidance provides little concrete direction. In this paper, we fill this gap by demonstrating that an adjusted estimator based on second-order $U$-statistics can almost unbiasedly estimate the average treatment effect and enjoy a guaranteed efficiency gain if $p = o (n)$. In our analysis, we generalize the coupling technique commonly used in the CAR literature to $U$-statistics and also obtain several useful results for analyzing inverse sample Gram matrices by a delicate leave-$m$-out analysis, which may be of independent interest. Both synthetic and semi-synthetic experiments are conducted to demonstrate the superior finite-sample performance of our new estimator compared to popular benchmarks.
Read the full abstract

Statistics Theory

Convergence analysis of data augmentation algorithms in Bayesian lasso models with log-concave likelihoods

We study the convergence properties of a class of data augmentation algorithms targeting posterior distributions of Bayesian lasso models with log-concave likelihoods. Leveraging isoperimetric inequalities, we derive a generic convergence bound for this class of algorithms and apply it to Bayesian probit, logistic, and heteroskedastic Gaussian linear lasso models. Under feasible initializations, the mixing times for the probit and logistic models are of order $O[(p+n)^3 (pn^{1-c} + n)]$, up to logarithmic factors, where $n$ is the sample size, $p$ is the dimension of the regression coefficients, and $c \in [0,1]$ is determined by the lasso penalty parameter. The mixing time for the heteroskedastic Gaussian model is $O[n(n+p)^3 (p n^{1-c} + n)]$, up to logarithmic factors.
Read the full abstract

Statistics Theory

Optimal Anytime-Valid Tests for Composite Nulls

We consider the problem of designing optimal level-$\alpha$ power-one tests for composite nulls. Given a parameter $\alpha \in (0,1)$ and a stream of $\mathcal{X}$-valued observations $\{X_n: n \geq 1\} \overset{i.i.d.}{\sim} P$, the goal is to design a level-$\alpha$ power-one test $\tau_\alpha$ for the null $H_0: P \in \mathcal{P}_0 \subset \mathcal{P}(\mathcal{X})$. Prior works have shown that any such $\tau_\alpha$ must satisfy $\mathbb{E}_P[\tau_\alpha] \geq \tfrac{\log(1/\alpha)}{\gamma^*(P, \mathcal{P}_0)}$, where $\gamma^*(P, \mathcal{P}_0)$ is the so-called $\mathrm{KL}_{\inf}$ or minimum divergence of $P$ to the null class. In this paper, our objective is to develop and analyze constructive schemes that match this lower bound as $\alpha \downarrow 0$. We first consider the finite-alphabet case~($|\mathcal{X}| = m < \infty$), and show that a test based on universal $e$-process~(formed by the ratio of a universal predictor and the running null MLE) is optimal in the above sense. The proof relies on a Donsker-Varadhan~(DV) based saddle-point representation of $\mathrm{KL}_{\inf}$, and an application of Sion's minimax theorem. This characterization motivates a general method for arbitrary $\mathcal{X}$: construct an $e$-process based on the empirical solutions to the saddle-point representation over a sufficiently rich class of test functions. We give sufficient conditions for the optimality of this test for compact convex nulls, and verify them for Hölder smooth density models. We end the paper with a discussion on the computational aspects of implementing our proposed tests in some practical settings.
Read the full abstract

Machine Learning (Statistics)

Gaussian Process Assisted Meta-learning for Image Classification and Object Detection Models

Collecting operationally realistic data to inform machine learning models can be costly. Before collecting new data, it is helpful to understand where a model is deficient. For example, object detectors trained on images of rare objects may not be good at identification in poorly represented conditions. We offer a way of informing subsequent data acquisition to maximize model performance by leveraging the toolkit of computer experiments and metadata describing the circumstances under which the training data was collected (e.g., season, time of day, location). We do this by evaluating the learner as the training data is varied according to its metadata. A Gaussian process (GP) surrogate fit to that response surface can inform new data acquisitions. This meta-learning approach offers improvements to learner performance as compared to data with randomly selected metadata, which we illustrate on both classic learning examples, and on a motivating application involving the collection of aerial images in search of airplanes.
Read the full abstract

All authors Anna R. Flowers, Christopher T. Franck, Robert B. Gramacy, Justin A. Krometis

Statistics Theory

Decomposing Degree Assortativity in Sparse Spatial Networks

Spatial networks are typically assortative: well-connected nodes link to other well-connected nodes, and the usual reading is sorting, popular nodes seeking each other out. In space there is a rival explanation: nearby nodes draw on the same pool of potential neighbors, so their degrees move together even when popularity and location are unrelated. This paper asks when the two can be told apart from network data, and gives a three-part answer. First, in a sparse random-connection model in which latent popularity and position are coupled by a copula, we prove that the limiting assortativity splits exactly into an intensity-covariance channel and a shared-neighbor channel; because the first mixes sorting with density effects, we define the sorting contribution as the increment produced by the dependence when mean degree and transitivity are held fixed, exactly zero at independence. Second, we prove that three observable summaries (mean degree, transitivity, assortativity) identify the model globally, including on the boundary of no dependence, by a computer-assisted proof in exact rational arithmetic that also verifies the conditions for valid inference; simulations confirm the asymptotics. Third, on data the machinery acts as a gate: no sorting estimate is reported unless the model first fits. In eleven metropolitan location-based networks the model class is rejected and direct estimates find essentially no dependence; the observed assortativity is evidence of neither sorting nor shared opportunity. In a national co-authorship network the verdict reverses: productive authors concentrate where researcher density is high, yet graph-only attribution is again refused, the misfit pointing to the team structure of multi-author papers.
Read the full abstract

Machine Learning (Statistics)

Computationally efficient goodness-of-fit tests through kernelized Stein discrepancy

Models with intractable normalizing constants are widely used in statistics and machine learning. Assessing the adequacy of such models poses significant challenges: obtaining samples from the fitted model often requires sophisticated sampling algorithms. Moreover, model fitting sometimes requires iterative numerical optimization, making bootstrap procedures that require repeated refitting computationally expensive. In this paper, we leverage the kernel-based testing framework to develop a general semiparametric goodness-of-fit test based on the kernelized Stein discrepancy. We establish the consistency and the asymptotic null distribution of the test statistic under general nuisance estimation. To produce a level-$\alpha$ test, we propose a novel influence-adjusted wild bootstrap that requires neither refitting the model nor sampling from it. We prove the consistency of the proposed bootstrap test procedure under the null and the alternative, and characterize its limiting power under contiguous local alternatives. Across simulations ranging from classical normality testing to models with intractable likelihoods, the proposed test delivers competitive or superior power at a computational cost orders of magnitude lower than that of existing approaches. We illustrate the method by assessing the adequacy of a protein signaling network model for reverse-phase protein array data from lung adenocarcinoma tumors. As a complementary insight, we show that the SKSD test can be regarded as a nonparametric score test under exponentially tilted models, connecting score-based and distance-based goodness-of-fit testing.
Read the full abstract

Statistical Methodology

A Markov-switching dynamic matrix factor model for the high-dimensional matrix time series

In this study, we propose a novel model called the Markov-switching dynamic matrix factor (Ms-DMF) model, which serves the dual purpose of structural interpretation and prediction for high-dimensional matrix time series. When estimating the parameters of the Ms-DMF model, an EM (expectation maximization) algorithm was used to get a quasi-maximum likelihood estimation, where all the parameters are estimated jointly. A filtering and smoothing algorithm is used to compute the posterior expectations corresponding to the latent regimes and factors. The consistency, convergence rates, and limit distributions of the estimated parameters are established under mild conditions. The effectiveness of this estimation method is also validated by rigorous numerical simulations. Furthermore, we apply the Ms-DMF model to an international trade flow network. Compared to existing matrix factor models, our approach not only identifies the main import and export centers, but also recognizes the trade cycles between these centers. This provides profound insights and analytical capabilities to advance research in the field of international trade.
Read the full abstract

All authors Chaofeng Yuan, Sainan Xu, Xingbing Kong, Jianhua Guo

Statistical Methodology

Multidimensional Stochastic Dominance Test Based on Center-outward Quantiles

Stochastic dominance (SD) provides a quantile-based partial ordering of random variables and has broad applications. Its extension to multivariate settings, however, is challenging due to the lack of a canonical ordering in $\mathbb{R}^d$ ($d \ge 2$) and the set-valued character of multivariate quantiles. Based on the multivariate center-outward quantile function in Hallin et al. (2021), this paper proposes new first- and second-order multivariate stochastic dominance (MSD) concepts through comparing contribution functions defined over quantile contours and regions. To address computational and inferential challenges, we incorporate entropy-regularized optimal transport, which ensures faster convergence rate and tractable estimation. We further develop consistent Kolmogorov-Smirnov and Cramér- von Mises type test statistics for MSD, establish bootstrap validity, and demonstrate through extensive simulations good finite-sample performance of the tests. Our approach offers a theoretically rigorous, and computationally feasible solution for comparing multivariate distributions.
Read the full abstract

Statistical Methodology

Variable selection in frailty mixture cure models via penalized likelihood estimation

Variable selection naturally arises as a useful subject when faced with data with massive predictor space. In addition to the massive dimensionality, the data may be characterized by intra-subject correlation, and cure fraction, which are ubiquitous in longitudinal studies with recurrent events defining the endpoint of interest. However, variable selection methods simultaneously adjusting for intra-subject correlation, and cure fraction are rare. We propose a comprehensive variable selection method for frailty mixture cure models based on penalized least squares approximation via the generalized linear mixed model methodology. The method provides shrinkage estimation and selection of fixed effects in the incidence and the latency submodels, adjusting for intra-subject correlation using a random effect term. The random effect is shared between the incidence and the latency, incorporating a flexible choice of covariance structure, allowing intra-subject correlation to be modeled as either time-invariant or time-varying. Estimation is facilitated by a penalized semiparametric restricted maximum likelihood method using an expectation-maximization algorithm. Two penalty functions, namely the adaptive least absolute shrinkage and selection operator (adaptive lasso), and the smoothly clipped absolute deviation (SCAD) are studied in the proposed method. Simulation studies are considered, benchmarking the method against an oracle procedure to access its finite sample performance. The practical utility of the method is illustrated using data on recurrent events from a breast cancer gene expression study. In the presence of a relatively large predictor space, results show that the method yields plausible interpretability in whole, as opposed to an unpenalized model.
Read the full abstract

Monday 22 December

Machine Learning (Statistics)

Diffusion Models in Simulation-Based Inference: A Tutorial Review

Diffusion models have recently emerged as powerful learners for simulation-based inference (SBI), enabling fast and accurate estimation of latent parameters from simulated and real data. Their score-based formulation offers a flexible way to learn conditional or joint distributions over parameters and observations, thereby providing a versatile solution to various modeling problems. In this tutorial review, we synthesize recent developments on diffusion models for SBI, covering design choices for training, inference, and evaluation. We highlight opportunities created by various concepts such as guidance, score composition, flow matching, consistency models, and joint modeling. Furthermore, we discuss how efficiency and statistical accuracy are affected by noise schedules, parameterizations, and samplers. Finally, we illustrate these concepts with case studies across parameter dimensionalities, simulation budgets, and model types, and outline open questions for future research.
Read the full abstract

All authors Jonas Arruda, Niels Bracher, Ullrich Köthe, Jan Hasenauer, Stefan T. Radev

Machine Learning (Statistics)

Fast and Exact Least Absolute Deviations Line Fitting via Piecewise Affine Lower-Bounding

Least-absolute-deviations (LAD) line fitting is robust to outliers but computationally more involved than least squares regression. Although the literature includes linear and near-linear time algorithms for the LAD line fitting problem, these methods are difficult to implement and, to our knowledge, lack maintained public implementations. As a result, practitioners often resort to linear programming (LP) based methods such as the simplex-based Barrodale-Roberts method and interior-point methods, or on iteratively reweighted least squares (IRLS) approximation which does not guarantee exact solutions. To close this gap, we propose the Piecewise Affine Lower-Bounding (PALB) method, an exact algorithm for LAD line fitting. PALB uses supporting lines derived from subgradients to build piecewise-affine lower bounds, and employs a subdivision scheme involving minima of these lower bounds. We prove correctness and provide bounds on the number of iterations. On synthetic datasets with varied signal types and noise including heavy-tailed outliers as well as a real dataset from the NOAA's Integrated Surface Database, PALB exhibits empirical log-linear scaling. It is consistently faster than publicly available implementations of LP based and IRLS based solvers. We provide a reference implementation written in Rust with a Python API.
Read the full abstract

Statistics Theory

Deconvolution in unlinked linear models

Unlinked regression, in which covariates and responses are observed separately without known correspondence, has recently gained increasing attention. Deconvolution, on the other hand, is a fundamental and challenging problem in nonparametric statistics with the aim of estimating the distribution of a latent random variable $Z$ based on observations contaminated by some additive noise. The complexity of this task is heavily influenced by the smoothness of the noise distribution and often leads to slow estimation rates. In this paper, we combine the recent unlinked linear regression problem with the classical deconvolution framework. Specifically, we study nonparametric deconvolution under the assumption that $Z$ is a linear function of an observable multidimensional covariate. This structural constraint allows us to introduce a nonparametric estimator of the distribution of $Z$ which achieves the parametric rate of convergence in the Wasserstein distance of order 1, where the smoothness of the noise does not affect the rate. Furthermore, we introduce nonparametric estimators for the unconditional density of $Z$ and the conditional density of $Z$ given an observed response. This allows us to study the problem of estimating the value of the latent linear predictor, whose link to the observed response is not accessible. Through several simulations, we illustrate the fast convergence rate of our deconvolution estimator and the performance of the proposed conditional estimators of the latent predictor in different simulation scenarios.
Read the full abstract

Machine Learning (Statistics)

Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Function

We consider a generalization of the classifier-based density-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. The problem with most loss functions used for this task is that they implicitly define a relationship between the optimal classifier and the target quasiprobabilistic density ratio which is discontinuous or not surjective. We address these problems by introducing a convex loss function that is well-suited for both probabilistic and quasiprobabilistic density ratio estimation. To quantify performance, an extended version of the Sliced-Wasserstein distance is introduced which is compatible with quasiprobability distributions. We demonstrate our approach on a real-world example from particle physics, of di-Higgs production in association with jets via gluon-gluon fusion, and achieve state-of-the-art results.
Read the full abstract

Statistical Methodology

Causal Inference with the Napkin Graph

Unmeasured confounding can render identification strategies based on adjustment functionals invalid. We study the "Napkin" graph, a causal structure that encapsulates features of M-bias, instrumental variables, and classical back-door and front-door settings, yet identifies the average treatment effect through a nonstandard ratio of two g-formulas. We develop influence-function-based estimators for this functional, including doubly-robust one-step and targeted minimum loss-based estimators that remain asymptotically linear under slower-than-parametric nuisance estimation using machine learning. A distinguishing feature of the Napkin graph is that it imposes a generalized independence restriction, known as a Verma constraint, rather than ordinary conditional independence restrictions, on the observed data distribution. We develop semiparametric efficiency theory for causal effects under a moment restriction corresponding to this Verma constraint, characterizing the orthocomplement of the tangent space, deriving the class of influence functions, and obtaining the semiparametric efficiency bound. More broadly, our analysis provides a framework for semiparametric inference in causal models defined by Verma constraints and demonstrates how such restrictions may yield efficiency gains. Simulations confirm the estimators' theoretical properties and demonstrate substantial efficiency gains. A real-data application using the Finnish Life Course Study estimates the effect of educational attainment on income. An accompanying R package, napkincausal, implements our methods.
Read the full abstract

All authors Anna Guo, Lin Liu, David Benkeser, Razieh Nabi

Machine Learning (Statistics) · Cross-listed from cs.RO

Gaussian Variational Inference with Non-Gaussian Factors for State Estimation: A UWB Localization Case Study

This letter extends the exactly sparse Gaussian variational inference (ESGVI) algorithm for state estimation in two complementary directions. First, ESGVI is generalized to operate on matrix Lie groups, enabling the estimation of states with orientation components while respecting the underlying group structure. Second, factors are introduced to accommodate heavy-tailed and skewed noise distributions, as commonly encountered in ultra-wideband (UWB) localization due to non-line-of-sight (NLOS) and multipath effects. Both extensions are shown to integrate naturally within the ESGVI framework while preserving its sparse and derivative-free structure. The proposed approach is validated in a UWB localization experiment with NLOS-rich measurements, demonstrating improved accuracy and comparable consistency. Finally, a Python implementation within a factor-graph-based estimation framework is made open-source (https://github.com/decargroup/gvi_ws) to support broader research use.
Read the full abstract

All authors Andrew Stirling, Mykola Lukashchuk, Dmitry Bagaev, Wouter Kouw, James R. Forbes

Statistical Methodology

Sample size reassessment in Bayesian hybrid clinical trials

The use of historical controls offers a valuable alternative when traditional randomized controlled trials are not feasible. However, such approaches may introduce bias due to temporal changes in patient populations, diagnostic criteria, and/or treatment standards. Hybrid designs, which combine a concurrent control arm with historical control data, can help mitigate the possible bias. We propose a novel Bayesian two-arm randomized clinical trial design incorporating an interim analysis. At the interim analysis, a new criterion derived from the Hellinger distance is used to quantify the similarity between historical and concurrent control data outcomes. This measure informs both (1) the variance function of the control prior distribution in the final analysis and (2) the sample size reassessment for the second stage of the trial. The proposed approach is designed to accommodate both continuous and binary endpoints and is assessed through extensive simulation studies. Results demonstrate the method flexibility and robustness in adapting to varying degrees of historical-control heterogeneity.
Read the full abstract

All authors Marco Ratta, Pavel Mozgunov, Sandrine Boulet, Moreno Ursino

Statistics Applications · Cross-listed from cs.CV

RANSAC Scoring Functions: Analysis and Reality Check

We revisit the problem of assigning a score (a quality of fit) to candidate geometric models – one of the key components of RANSAC for robust geometric fitting. In a non-robust setting, the “gold standard” scoring function, known as the geometric error, follows from a probabilistic model with Gaussian noises. We extend it to spherical noises. In a robust setting, we consider a mixture with uniformly distributed outliers and show that a threshold-based parameterization leads to a unified view of likelihood-based and robust M-estimators and associated local optimization schemes. Next we analyze MAGSAC++ which stands out for two reasons. First, it achieves the best results according to existing benchmarks. Second, it makes quite different modeling assumptions and derivation steps. We discovered, however that the derivation does not correspond to sound principles and the resulting score function is in fact numerically equivalent to a simple Gaussian-uniform likelihood, a basic model within the proposed framework. Finally, we propose an experimental methodology for evaluating scoring functions: assuming either a large validation set, or a small random validation set in expectation. We find that all scoring functions, including using a learned inlier distribution, perform identically. In particular, MAGSAC++ score is found to be neither better performing than simple contenders nor less sensitive to the choice of the threshold hyperparameter. Our theoretical and experimental analysis thus comprehensively revisit the state-of-the-art, which is critical for any future research seeking to improve the methods or apply them to other robust fitting problems.
Read the full abstract

Statistics Theory · Cross-listed from econ.EM

Numerical Analysis of Test Optimality

In nonstandard testing environments, researchers often derive ad hoc tests with correct (asymptotic) size, but their optimality properties are typically unknown a priori and difficult to assess. This paper develops a numerical framework for determining whether an ad hoc test is effectively optimal - approximately maximizing a weighted average power criterion for some weights over the alternative and attaining a power envelope generated by a single weighted average power-maximizing test. Our approach uses nested optimization algorithms to approximate the weight function that makes an ad hoc test's weighted average power as close as possible to that of a true weighted average power-maximizing test, and we show the surprising result that the rejection probabilities corresponding to the latter form an approximate power envelope for the former. We provide convergence guarantees, discuss practical implementation and apply the method to the weak instrument-robust conditional likelihood ratio test and a recently-proposed test for when a nuisance parameter may be on or near its boundary.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Guardrailed Uplift Targeting: A Causal Optimization Playbook for Marketing Strategy

This paper introduces a marketing decision framework that optimizes customer targeting by integrating heterogeneous treatment effect estimation with explicit business guardrails. The objective is to maximize revenue and retention while adhering to constraints such as budget, revenue protection, and customer experience. The framework first estimates Conditional Average Treatment Effects (CATE) using uplift learners, then solves a constrained allocation problem to decide whom to target and which offer to deploy. It supports decisions in retention messaging, event rewards, and spend-threshold assignment. Validated through offline simulations and online A/B tests, the approach consistently outperforms propensity and static baselines, offering a reusable playbook for causal targeting at scale.
Read the full abstract