Machine Learning (Statistics) · Cross-listed from cs.LG
Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
Alexandru Meterez, Depen Morwani, Jingfeng Wu et al. (+3 more)
Increasing the batch size during training – a ”batch ramp” – is a promising strategy to accelerate large
language model pretraining. While for SGD, doubling the batch size can be equivalent to halving the learning rate, the optimal strategy for adaptive optimizers like Adam is less clear. As a result, any batch-ramp scheduling, if used at all, is typically tuned heuristically. This work develops a principled framework for batch-size scheduling and introduces Seesaw: whenever a standard scheduler would halve the learning rate, Seesaw instead multiplies it by $1/\sqrt{2}$ and doubles the batch size, preserving loss dynamics while reducing serial steps. Theoretically, we provide, to our knowledge, the first finite-sample proof of equivalence between learning-rate decay and batch-size ramp-up for SGD on noisy linear regression, and we extend this equivalence to normalized SGD, a tractable proxy for Adam, under a variance-dominated regime observed in practice. Empirically, on 150M/300M/600M-parameter models trained at Chinchilla scale using a constant (critical) batch size, Seesaw matches cosine decay at equal FLOPs while reducing wall-clock time by $\approx 36\%$, approaching the theoretical limit implied by our analysis.
Read the full abstract ▾
All authors Alexandru Meterez, Depen Morwani, Jingfeng Wu, Costin-Andrei Oncescu, Cengiz Pehlevan, Sham Kakade
Statistical Methodology
Loss functions arising from the index of agreement
Hristos Tyralis and Georgia Papacharalampous
We examine the theoretical properties of the index of agreement loss function $L_W$, the negatively oriented counterpart of Willmott's index of agreement, a common metric in environmental sciences and engineering. We prove that $L_W$ is bounded within [0, 1], translation and scale invariant, and estimates the parameter $\Bbb{E}_{F}[\underline{y}] \pm \Bbb{V}_{F}^{1/2}[\underline{y}]$ when fitting a distribution. We propose $L_{\operatorname{NR}_2}$ as a theoretical improvement, which replaces the denominator of $L_W$ with the sum of Euclidean distances, better aligning with the underlying
geometric intuition. This new loss function retains the appealing properties of $L_W$ but also admits closed-form solutions for linear model parameter estimation. We show that as the correlation between predictors and the dependent variable approaches 1, parameter estimates from squared error, $L_{\operatorname{NR}_2}$ and $L_W$ converge. This behavior is mirrored in hydrologic model calibration (a core task in water resources engineering), where performance becomes nearly identical across these loss functions. Finally, we suggest potential improvements for existing $L_p$-norm variants of the index of agreement.
Read the full abstract ▾
Machine Learning (Statistics)
Fast and Scalable Score-Based Kernel Calibration Tests
Pierre Glaser, David Widmann, Fredrik Lindsten et al. (+1 more)
We introduce the Kernel Calibration Conditional Stein Discrepancy test (KCCSD test), a non-parametric, kernel-based test for assessing the calibration of probabilistic models with well-defined scores. In contrast to previous methods, our test avoids the need for possibly expensive expectation approximations while providing control over its type-I error. We achieve these improvements by using a new family of kernels for score-based probabilities that can be estimated without probability density samples, and by using a conditional goodness-of-fit criterion for the KCCSD test's U-statistic. We demonstrate the properties of our test on various synthetic settings.
Read the full abstract ▾
All authors Pierre Glaser, David Widmann, Fredrik Lindsten, Arthur Gretton
Statistical Methodology
Response to Discussions of "Causal and Counterfactual Views of Missing Data Models"
Razieh Nabi, Rohit Bhattacharya, Ilya Shpitser et al. (+1 more)
We are grateful to the discussants, Levis and Kennedy [2025], Luo and Geng [2025], Wang and van der Laan [2025], and Yang and Kim [2025], for their thoughtful comments on our paper (Nabi et al., 2025). In this rejoinder, we summarize our main contributions and respond to each discussion in turn.
Read the full abstract ▾
All authors Razieh Nabi, Rohit Bhattacharya, Ilya Shpitser, James M. Robins
Statistical Methodology
Hierarchical shot-noise Cox process mixtures for clustering across groups
Alessandro Carminati, Mario Beraha, Federico Camerlenghi et al. (+1 more)
Clustering observations across partially exchangeable groups of data is a routine task in
Bayesian nonparametrics. Previously proposed models allow for clustering across groups by sharing atoms in the group-specific mixing measures. However, exact atom sharing can be overly rigid when groups differ subtly, introducing a trade-off between clustering and density estimates and fragmenting across-group clusters, particularly at larger sample sizes. We introduce the hierarchical shot-noise Cox process (HSNCP) mixture model, where group-specific atoms concentrate around shared centers through a kernel. This enables accurate density estimation within groups and flexible borrowing across groups, overcoming the density-clustering trade-off of previous approaches. Our construction, built on the shot-noise Cox process, remains analytically tractable: we derive closed-form prior moments and an inter-group correlation, obtain the marginal law and predictive distribution for latent parameters, as well as the posterior of the mixing measures given the latent parameters. We develop an efficient conditional MCMC
algorithm for posterior inference. We assess the performance of the HSNCP model through simulations and an application to a large
galaxy dataset, demonstrating balanced across-group clusters and improved density estimates compared with the hierarchical Dirichlet process, including under model misspecification.
Read the full abstract ▾
All authors Alessandro Carminati, Mario Beraha, Federico Camerlenghi, Alessandra Guglielmi
Machine Learning (Statistics)
Inverse Problems for Partial Differential Equations with Jump Discontinuities in Coefficients via Two-Stage Physics-Informed Deep Learning and Statistical Mixture Models
Zhikun Zhang, Guanyu Pan, Xiangjun Wang et al. (+2 more)
This work proposes a two-stage physics-informed deep learning framework that combines neural-network-based sampling with statistical inference and constrained parameter refinement. In the first stage, a dual-network physics-informed architecture is used, where a main network approximates the PDE solution and an auxiliary coefficient sub network provides a relaxed continuous surrogate of the true discontinuous coefficient field. A gradient-adaptive weighting strategy is incorporated into the physics residual to improve residual training and enhance sampling reliability near possible discontinuity regions. The sampled coefficient values are then analyzed using
Bayesian learning for Gaussian mixture models and birth-death Markov chain model selection, which estimate the number of coefficient regimes and provide heuristic search intervals for coefficient values and candidate transition regions. In the second stage, the inverse problem is reformulated as a constrained physics-informed
estimator, in which the coefficient is represented explicitly as a hard piecewise-constant function over the spatiotemporal domain. Numerical experiments on different PDE types with jump-discontinuous coefficients demonstrate that the proposed framework achieves accurate parameter estimation with acceptable computational costs compared to existing methods. This work provides an effective integrated workflow for inverse problems governed by PDEs with discontinuous parameter structures, particularly in nonstationary and heterogeneous systems.
Read the full abstract ▾
All authors Zhikun Zhang, Guanyu Pan, Xiangjun Wang, Yong Xu, Guangtao Zhang
Statistical Methodology
Accurate Bayesian inference for tail risk extrapolation in time series
David L. Carl, Simone A. Padoan and Stefano Rizzelli
Accurately quantifying tail risks-rare but high-impact events such as financial crashes or extreme weather-is a central challenge in risk management, with serially dependent data. We develop a
Bayesian framework based on the Generalized Pareto (GP) distribution for modeling threshold exceedances, providing posterior distributions for the GP parameters and tail
quantiles in time series. Two cases are considered: extrapolation of tail
quantiles for the stationary marginal distribution under beta-mixing dependence, and dynamic, past-conditional tail
quantiles in heteroscedastic regression models. The proposal yields asymptotically honest credible regions, whose coverage probabilities converge to their nominal levels. We establish the asymptotic theory for the
Bayesian procedure, deriving conditions on the prior distributions under which the posterior satisfies key asymptotic properties. To achieve this, we first develop a likelihood theory under serial dependence, providing local and global bounds for the empirical log-likelihood process of the misspecified GP model and deriving corresponding asymptotic properties of the Maximum Likelihood
Estimator (MLE). Simulations demonstrate that our
Bayesian credible regions outperform naive
Bayesian and MLE-based confidence regions across several standard time series models, including ARMA, GARCH, and Markovian copula models. Two real-data applications-to U.S. interest rates and Swiss electricity demand-highlight the relevance of the proposed methodology.
Read the full abstract ▾
Machine Learning (Statistics)
Local Causal Discovery for Statistically Efficient Causal Inference
Mátyás Schubert, Tom Claassen and Sara Magliacane
Causal discovery methods can identify valid adjustment sets for causal effect estimation for a pair of target variables, even when the underlying causal graph is unknown. Global causal discovery methods focus on learning the whole causal graph and therefore enable the recovery of optimal adjustment sets, i.e., sets with the lowest asymptotic variance, but they quickly become computationally prohibitive as the number of variables grows. Local causal discovery methods offer a more scalable alternative by focusing on the local neighborhood of the target variables, but are restricted to statistically suboptimal adjustment sets. In this work, we propose Local Optimal Adjustments Discovery (LOAD), a sound and complete causal discovery approach that combines the computational efficiency of local methods with the statistical optimality of global methods. First, LOAD identifies the causal relation between the targets and tests if the causal effect is identifiable by using only local information. If it is identifiable, it finds the possible descendants of the treatment and infers the optimal adjustment set as the parents of the outcome in a modified forbidden projection. Otherwise, it returns the locally valid parent adjustment sets. In our experiments on synthetic and realistic data LOAD outperforms global methods in scalability, while providing more accurate effect estimation than local methods.
Read the full abstract ▾
Statistical Methodology
Reframing cross-world independence in multimediator causal models
En-Yu Lai, Jih-Chang Yu and Yen-Tsung Huang
Multi-mediator causal models provide a general framework for decomposing causal effects along specific mechanisms, but effect identification in the presence of treatment-induced confounding remains conceptually challenging. Path-specific effects (PSEs) provide one important formulation of such causal decomposition. In particular, existing approaches differ substantially in their counterfactual assumptions and in the interventions used to define effects through specific pathways. We use Single-World Intervention Graphs (SWIGs) as a unifying analytic device to formalize and compare three approaches to PSE identification: classical path-specific effects based on cross-world independence assumptions, interventional effects based on randomized mediator interventions, and separable effects based on decomposing the exposure into distinct causal components. Through this unified analysis, we show that these seemingly different approaches can, under appropriate conditions, lead to the same identification formulas despite relying on different causal assumptions and intervention semantics. The SWIG representation further reveals how each approach resolves the counterfactual conflicts created by treatment-induced confounding. Our results highlight a fundamental distinction between statistical identification and causal interpretation: the same observed-data functional can represent different causal questions depending on how the underlying counterfactual worlds are constructed. This unified perspective clarifies relationships among major approaches in multimediator settings and guides
alignment of the estimand, identification assumptions, and scientific mechanism of interest.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
On the Identifiability of Tensor Ranks via Prior Predictive Matching
Eliezer da Silva, Arto Klami, Diego Mesquita et al. (+1 more)
Selecting the latent dimensions (ranks) in tensor factorization is a central challenge that often relies on heuristic methods. This paper introduces a rigorous approach to determine rank identifiability in probabilistic tensor models, based on prior predictive moment matching. We transform a set of moment matching conditions into a log-linear system of equations in terms of marginal moments, prior hyperparameters, and ranks; establishing an equivalence between rank identifiability and the solvability of such system. We apply this framework to four foundational tensor-models, demonstrating that the linear structure of the PARAFAC/CP model, the chain structure of the Tensor Train model, and the closed-loop structure of the Tensor Ring model yield solvable systems, making their ranks identifiable. In contrast, we prove that the symmetric
topology of the Tucker model leads to an underdetermined system, rendering the ranks unidentifiable by this method. For the identifiable models, we derive explicit closed-form rank
estimators based on the moments of observed data only. We empirically validate these
estimators and evaluate the robustness of the proposal.
Read the full abstract ▾
All authors Eliezer da Silva, Arto Klami, Diego Mesquita, Iñigo Urteaga
Statistical Methodology
Additive Density Regression
Eva-Maria Maier, Alexander Fottner, Sonja Greven et al. (+1 more)
We present a structured additive regression approach to model conditional densities given scalar
covariates, where only samples of the conditional distributions are observed. This links our approach to distributional regression models for scalar data. The model is formulated in a Bayes
Hilbert space – preserving nonnegativity and integration to one under summation and scalar multiplication – with respect to an arbitrary finite measure. This allows to consider, amongst others, continuous, discrete and mixed densities. Our theoretical results include asymptotic existence, uniqueness, consistency, and asymptotic normality of the penalized maximum likelihood
estimator, as well as confidence regions and inference for the (effect) densities. For estimation, we propose to maximize the penalized log-likelihood corresponding to an appropriate multinomial, or equivalently, Poisson regression model, which we show to approximate the original penalized maximum likelihood problem. We apply our framework to a motivating gender economic data set from the German Socio-Economic Panel Study (SOEP), analyzing the distribution of the woman's share in a couple's total labor income given
covariate effects for year, place of residence and age of the youngest child. As the income share is a continuous variable having discrete point masses at zero and one for single-earner couples, the corresponding densities are of mixed type.
Read the full abstract ▾
All authors Eva-Maria Maier, Alexander Fottner, Sonja Greven, Almond Stöcker
Statistical Methodology
ROC Analysis with Covariate Adjustment Using Neural Network Models: Evaluating the Role of Age in the Physical Activity-Mortality Association
Ziad Akram Ali Hammouri, Yating Zou, Rahul Ghosal et al. (+2 more)
The receiver operating characteristic (ROC) curve and its summary measure, the Area Under the Curve (AUC), are well-established tools for evaluating the efficacy of biomarkers in biomedical studies. Compared to the traditional ROC curve, the
covariate-adjusted ROC curve allows for individual evaluation of the biomarker. However, the use of machine learning models has rarely been explored in this context, despite their potential to develop more powerful and sophisticated approaches for biomarker evaluation. The goal of this paper is to propose a framework for neural network-based
covariate-adjusted ROC modeling that allows flexible and nonlinear evaluation of the effectiveness of a biomarker to discriminate between two reference populations. The finite-sample performance of our method is investigated through extensive simulation tests under varying dependency structures between biomarkers,
covariates, and referenced populations. The methodology is further illustrated in a clinically case study that assesses daily physical activity - measured as total activity time (TAC), a proxy for daily step count-as a biomarker to predict mortality at three, five and eight years. Analyzes stratified by sex and adjusted for age and BMI reveal distinct
covariate effects on mortality outcomes. These results underscore the importance of
covariate-adjusted modeling in biomarker evaluation and highlight TAC's potential as a functional capacity biomarker based on specific individual characteristics.
Read the full abstract ▾
All authors Ziad Akram Ali Hammouri, Yating Zou, Rahul Ghosal, Juan C. Vidal, Marcos Matabuena
Statistics Theory
Strong consistency of pseudo-likelihood parameter estimator for univariate Gaussian mixture models
Jüri Lember, Raul Kangro and Kristi Kuljus
We consider a new method for estimating the parameters of univariate Gaussian mixture models. The method relies on a nonparametric density
estimator $\hat{f}_n$ (typically a kernel
estimator). For every set of Gaussian mixture components, $\hat{f}_n$ is used to find the best set of mixture weights. That set is obtained by minimizing the $L_2$ distance between $\hat{f}_n$ and the Gaussian mixture density with the given component parameters. The densities together with the obtained weights are then plugged in to the likelihood function, resulting in the so-called pseudo-likelihood function. The final parameter
estimators are the parameter values that maximize the pseudo-likelihood function together with the corresponding weights. The advantages of the pseudo-likelihood over the full likelihood are: 1) its arguments are the means and variances only, mixture weights are also functions of the means and variances; 2) unlike the likelihood function, it is always bounded above. Thus, the maximizer of the pseudo-likelihood function – referred to as the pseudo-likelihood
estimator – always exists. In this article, we prove that the pseudo-likelihood
estimator is strongly consistent.
Read the full abstract ▾
Statistical Computation · Cross-listed from math.NA
Preconditioned Conjugate Gradient methods for the estimation of General Linear Models
Paolo Foschi
The use of the Preconditioned Conjugate Gradient (PCG) method for computing the Generalized Least Squares (GLS)
estimator of the General Linear Model (GLM) is considered. The GLS
estimator is expressed in terms of the solution of an augmented system. That system is solved by means of the PCG method using an indefinite preconditioner. The resulting method iterates a sequence Ordinary Least Squares (OLS) estimations that converges, in exact precision, to the GLS
estimator within a finite number of steps. The numerical and statistical properties of the
estimator computed at an intermediate step are analytically and numerically studied. This approach allows to combine direct methods, used in the OLS step, with those of iterative methods. This advantage is exploited to design PCG methods for the estimation of Constrained GLMs and of some structured multivariate GLMs. The structure of the
matrices involved are exploited as much as possible, in the OLS step. The iterative method then solves for the unexploited structure. Numerical experiments shows that the proposed methods can achieve, for these structured problems, the same precision of state of the art direct methods, but in a fraction of the time.
Read the full abstract ▾
Statistics Theory
The geometry of PLS shrinkages
Paolo Foschi
The geometrical structure of PLS shrinkages is here considered. Firstly, an explicit formula for the shrinkage vector is provided. In that expression, shrinkage factors are expressed a averages of a set of basic shrinkages that depend only on the data
matrix. On the other hand, the weights of that average are multilinear functions of the observed responses. That representation allows to characterise the set of possible shrinkages and identify extreme situations where the PLS
estimator has an highly nonlinear behaviour. In these situations, recently proposed measures for the degrees of freedom (DoF), that directly depend on the shrinkages, fail to provide reasonable values. It is also shown that the longstanding
conjecture that the DoFs of PLS always exceeds the number PLS directions does not hold.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Interaction Concordance Index: Performance Evaluation for Interaction Prediction Methods
Tapio Pahikkala, Riikka Numminen, Parisa Movahedi et al. (+2 more)
Consider two sets of entities and their members' mutual affinity values, say drug-target affinities (DTA). Drugs and targets are said to interact in their effects on DTAs if drug's effect on it depends on the target. Presence of interaction implies that assigning a drug to a target and another drug to another target does not provide the same aggregate DTA as the reversed assignment would provide. Accordingly, correctly capturing interactions enables better decision-making, for example, in allocation of limited numbers of drug doses to their best matching targets. Learning to predict DTAs is popularly done from either solely from known DTAs or together with side information on the entities, such as chemical structures of drugs and targets. In this paper, we introduce interaction directions' prediction performance
estimator we call interaction concordance index (IC-index), for both fixed predictors and machine learning
algorithms aimed for inferring them. IC-index complements the popularly used DTA prediction performance
estimators by evaluating the ratio of correctly predicted directions of interaction effects in data. First, we show the invariance of IC-index on predictors unable to capture interactions. Secondly, we show that learning
algorithm's permutation equivariance regarding drug and target identities implies its inability to capture interactions when either drug, target or both are unseen during training. In practical applications, this equivariance is remedied via incorporation of appropriate side information on drugs and targets. We make a comprehensive empirical evaluation over several biomedical interaction data sets with various state-of-the-art machine learning
algorithms. The experiments demonstrate how different types of affinity strength prediction methods perform in terms of IC-index complementing existing prediction performance
estimators.
Read the full abstract ▾
All authors Tapio Pahikkala, Riikka Numminen, Parisa Movahedi, Napsu Karmitsa, Antti Airola
Statistical Methodology
Evaluating Policy Effects under Network Interference without Network Information: A Transfer Learning Approach
Tadao Hoshino
This paper develops a sensitivity analysis framework that transfers the average total treatment effect (ATTE) from source data with a fully observed network to target data whose network is completely unknown. The ATTE represents the average social impact of a policy that assigns the treatment to every individual in the
dataset. We postulate a
covariate-shift type assumption that both source and target
datasets share the same conditional mean outcome. However, because the target network is unobserved, this assumption alone is not sufficient to pin down the ATTE for the target data. To address this issue, we consider a sensitivity analysis based on the uncertainty of the target network's degree distribution, where the extent of uncertainty is measured by the Wasserstein distance from a given reference degree distribution. We then construct bounds on the target ATTE using a linear programming-based
estimator. The limiting distribution of the bound
estimator is derived via the functional delta method, and we develop a wild bootstrap approach to approximate the distribution. As an empirical illustration, we revisit the social network experiment on farmers' weather insurance adoption in China by Cai et al. (2015).
Read the full abstract ▾
Machine Learning (Statistics)
Row-wise Fusion Regularization: An Interpretable Personalized Federated Learning Framework in Large-Scale Scenarios
Runlin Zhou, Letian Li and Zemin Zheng
We study personalized
federated learning for multivariate responses where client models are heterogeneous yet share variable-level structure. Existing entry-wise penalties ignore cross-response dependence, while
matrix-wise fusion over-couples clients. We propose a Sparse Row-wise Fusion (SROF) regularizer that clusters row vectors across clients and induces within-row sparsity, and we develop RowFed, a communication-efficient
federated algorithm that embeds SROF into a linearized ADMM framework with
privacy-preserving partial participation. Theoretically, we establish an oracle property for SROF-achieving correct variable-level group recovery with asymptotic normality-and prove convergence of RowFed to a stationary solution. Under random client participation, the iterate gap contracts at a rate that improves with participation probability. Empirically, simulations in heterogeneous regimes show that RowFed consistently lowers estimation and prediction error and strengthens variable-level cluster recovery over NonFed, FedAvg, and a personalized
matrix-fusion baseline. A real-data study further corroborates these gains while preserving interpretability. Together, our results position row-wise fusion as an effective and transparent paradigm for large-scale personalized
federated multivariate learning, bridging the gap between entry-wise and
matrix-wise formulations.
Read the full abstract ▾
Statistics Applications · Cross-listed from econ.EM
Dynamic Spatial Treatment Effect Boundaries: A Continuous Functional Framework from Navier-Stokes Equations
Tatsuru Kikuchi
I develop a comprehensive theoretical framework for dynamic spatial treatment effect boundaries using continuous functional definitions grounded in Navier-Stokes partial differential equations. Rather than discrete treatment effect
estimators, the framework characterizes treatment intensity as a continuous function $\tau(\mathbf{x}, t)$ over space-time, enabling rigorous analysis of propagation dynamics, boundary
evolution, and cumulative exposure patterns. Building on exact self-similar solutions expressible through Kummer confluent hypergeometric and modified Bessel functions, I establish that treatment effects follow scaling laws $\tau(d, t) = t^{-\alpha} f(d/t^\beta)$ where exponents characterize
diffusion mechanisms. Empirical validation using 42 million TROPOMI satellite observations of NO$_2$ pollution from U.S. coal-fired power plants demonstrates strong exponential spatial decay ($\kappa_s = 0.004$ per km, $R^2 = 0.35$) with detectable boundaries at 572 km. Monte Carlo simulations confirm superior performance over discrete parametric methods in boundary detection and false positive avoidance (94% vs 27% correct rejection). Regional heterogeneity analysis validates diagnostic capability: positive decay parameters within 100 km confirm coal plant dominance; negative parameters beyond 100 km correctly signal when urban sources dominate. The continuous functional perspective unifies spatial econometrics with mathematical physics, providing theoretically grounded methods for boundary detection, exposure quantification, and policy evaluation across environmental economics, banking, and healthcare applications.
Read the full abstract ▾
Statistical Methodology
Average Treatment Effect Estimation with Non-binary Instrumental Variables
Mei Dong, Lin Liu, Dingke Tang et al. (+3 more)
Non-binary instrumental variables, especially continuous ones, are common in practice. A binary recoding induces a Wald ratio but may discard useful variation and reduce efficiency. Although fully nonparametric approaches can in principle use the entire instrument, they often require high-dimensional nuisance estimation which can be unstable with rich
covariates. We address this problem by developing a generalized Wald estimand for binary treatments that uses the full variation in a non-binary instrument. Under standard instrumental-variable assumptions and a homogeneity condition, the estimand yields a common identification formula for categorical and continuous instruments. We further develop its semiparametric efficiency theory and construct a locally efficient debiased
estimator using risk-minimization reparameterizations and double cross-fitting to accommodate flexible machine learning while improving numerical stability. The central technical challenge is that the many Wald ratios generated by a non-binary instrument must agree, thereby imposing overidentifying restrictions on the observed-data law. In this setting, characterizing the tangent space is nonstandard: it requires a second-order parametric submodel, a construction that, to our knowledge, has not been standard in semiparametric efficiency theory. Simulations show stable performance across sample sizes and greater efficiency than
estimators based on dichotomized instruments. In an application to the Princess Margaret Cancer Centre lung cancer cohort, associational analyses link excess body weight to lower two-year mortality, a seemingly protective pattern often called the obesity paradox. The proposed instrumental-variable analysis instead suggests increased mortality, pointing to residual confounding behind this paradox.
Read the full abstract ▾
All authors Mei Dong, Lin Liu, Dingke Tang, Geoffrey Liu, Wei Xu, Linbo Wang
Machine Learning (Statistics) · Cross-listed from cs.LG
Jet Functors and Weil Algebras in Automatic Differentiation: A Geometric Analysis
Amandip Sangha
We present a differential-
geometric formulation of automatic differentiation (AD) based on jet functors and Weil
algebras. In this framework, forward- and reverse-mode differentiation arise naturally as pushforward and cotangent pullback, while higher-order differentiation corresponds to evaluation in a Weil
algebra. This construction provides a unified, coordinate-free view of derivative propagation and clarifies the algebraic structure underlying AD. All results are realized in modern JAX code, where the Weil-mode formulation computes all mixed derivatives in a single forward pass with cost linear in the
algebra dimension. The resulting implementation achieves algebraically exact and numerically stable differentiation with predictable scaling, demonstrating that
geometric abstraction can yield more efficient and transparent computational differentiation systems. Code is available at https://git.nilu.no/
geometric-ad/jet-weil-ad
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Active Measuring in Reinforcement Learning With Delayed Negative Effects
Daiqi Gao, Ziping Xu, Aseel Rawashdeh et al. (+2 more)
Measuring states in reinforcement learning (RL) can be costly in real-world settings and may negatively influence future outcomes. We introduce the Actively Observable Markov Decision Process (AOMDP), where an
agent not only selects control actions but also decides whether to measure the latent state. The measurement action reveals the true latent state but may have a negative delayed effect on the environment. We show that this reduced uncertainty may provably improve sample efficiency and increase the value of the optimal policy despite these costs. We formulate an AOMDP as a periodic partially observable MDP and propose an online RL
algorithm based on belief states. To approximate the belief states, we further propose a sequential Monte Carlo method to jointly approximate the posterior of unknown static environment parameters and unobserved latent states. We evaluate the proposed
algorithm in a digital health application, where the
agent decides when to deliver digital interventions and when to assess users' health status through surveys.
Read the full abstract ▾
All authors Daiqi Gao, Ziping Xu, Aseel Rawashdeh, Predrag Klasnja, Susan A. Murphy
Statistics Theory · Cross-listed from econ.EM
Debiased Kernel Estimation of Spot Volatility in the Presence of Infinite Variation Jumps
B. Cooper Boniece, José E. Figueroa-López and Tianwei Zhou
Volatility estimation is a central problem in financial econometrics, but becomes particularly challenging when jump activity is high, a phenomenon observed empirically in highly traded financial securities. In this paper, we revisit the problem of spot
volatility estimation for an Itô semimartingale with jumps of unbounded variation. We construct truncated kernel-based
estimators and debiased variants that extend rate-optimal spot
volatility estimation to a wider range of jump activity indices, from the previously available bound $Y<4/3$ to $Y<20/11$. Rate-suboptimal CLTs are also established for $Y>20/11$. Compared with earlier work, our approach achieves smaller asymptotic variances through the use of more general kernels and an optimal choice for the bandwidth convergence rate, and also has broader applicability under more flexible model assumptions. A comprehensive simulation study confirms that our procedures outperform competing methods in finite samples.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Nonparametric Data Attribution for Diffusion Models
Yutian Zhao, Chao Du, Xiaosen Zheng et al. (+2 more)
Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs. Existing methods for
diffusion models typically require access to model gradients or retraining, limiting their applicability in proprietary or large-scale settings. We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images. Our approach is grounded in the analytical form of the optimal score function and naturally extends to multiscale representations, while remaining computationally efficient through convolution-based acceleration. In addition to producing spatially interpretable attributions, our framework uncovers patterns that reflect intrinsic relationships between training data and outputs, independent of any specific model. Experiments demonstrate that our method achieves strong attribution performance, closely matching gradient-based approaches and substantially outperforming existing nonparametric baselines. Code is available at https://github.com/sail-sg/NDA.
Read the full abstract ▾
All authors Yutian Zhao, Chao Du, Xiaosen Zheng, Tianyu Pang, Min Lin
Machine Learning (Statistics) · Cross-listed from cs.LG
Policy Regularized Distributionally Robust Markov Decision Processes with Linear Function Approximation
Jingwen Gu, Yiting He, Zhishuai Liu et al. (+1 more)
Decision-making under distribution shift is a central challenge in reinforcement learning (RL), where training and deployment environments differ. We study this problem through the lens of robust Markov decision processes (RMDPs), which optimize performance against
adversarial transition dynamics. Our focus is the online setting, where the
agent has only limited interaction with the environment, making sample efficiency and exploration especially critical. Policy optimization, despite its success in standard RL, remains theoretically and empirically underexplored in robust RL. To bridge this gap, we propose Distributionally Robust Regularized Policy Optimization
algorithm (DR-RPO), a model-free online policy optimization method that learns robust policies with sublinear regret. To enable tractable optimization within the softmax policy class, DR-RPO incorporates reference-policy regularization, yielding RMDP variants that are doubly constrained in both transitions and policies. To scale to large state-action spaces, we adopt the $d$-rectangular linear MDP formulation and combine linear function approximation with an upper confidence bonus for optimistic exploration. We provide theoretical guarantees showing that policy optimization can achieve
polynomial suboptimality bounds and sample efficiency in robust RL, matching the performance of value-based approaches. Finally, empirical results across diverse domains corroborate our theory and demonstrate the robustness of DR-RPO.
Read the full abstract ▾
All authors Jingwen Gu, Yiting He, Zhishuai Liu, Pan Xu
Machine Learning (Statistics)
A novel Information-Driven Strategy for Optimal Regression Assessment
Benjamín Castro, Camilo Ramírez, Sebastián Espinosa et al. (+3 more)
In Machine Learning (ML), a regression
algorithm aims to minimize a loss function based on data. An assessment method in this context seeks to quantify the discrepancy between the optimal response for an input-output system and the estimate produced by a learned predictive model (the student). Evaluating the quality of a learned regressor remains challenging without access to the true data-generating mechanism, as no data-driven assessment method can ensure the achievability of global optimality. This work introduces the Information Teacher, a novel data-driven framework for evaluating regression
algorithms with formal performance guarantees to assess global optimality. Our novel approach builds on estimating the Shannon mutual information (MI) between the input variables and the residuals and applies to a broad class of additive noise models. Through numerical experiments, we confirm that the Information Teacher is capable of detecting global optimality, which is aligned with the condition of zero estimation error with respect to the – inaccessible, in practice – true model, working as a surrogate measure of the ground truth assessment loss and offering a principled alternative to conventional empirical performance metrics.
Read the full abstract ▾
All authors Benjamín Castro, Camilo Ramírez, Sebastián Espinosa, Jorge F. Silva, Marcos E. Orchard, Heraldo Rozas
Statistical Methodology
Hypothesis testing for the uniformity of random geometric graph
Mingao Yuan
Random
geometric graphs are widely used in modeling geometry and dependence structure in networks. In a random
geometric graph, nodes are independently generated from some probability distribution $F$ over a metric space, and edges link nodes if their distance is less than some threshold. Most studies assume the distribution $F$ to be uniform. However, recent research shows that some real-world networks may be better modeled by nonuniform distribution $F$. Moreover, graphs with nonuniform $F$ have notably different properties from graphs with uniform $F$. A fundamental question is: given a network from a random
geometric graph, is the distribution $F$ uniform or not? In this paper, we approach this question through hypothesis testing. This problem is particularly challenging due to the inherent dependencies among edges in random
geometric graphs, a property not present in classic random graphs. We propose the first statistical test. Under the null hypothesis, the test statistic converges in distribution to the standard normal distribution. The asymptotic distribution is derived using the asymptotic theory of degenerate U-statistics with a kernel function dependent on the number of nodes. This technique is different from existing methods in network hypothesis testing problems. In addition, we present a method for efficiently calculating the test statistic directly from the adjacency
matrix. We also analytically characterize the power of the proposed test. The simulation study shows that the proposed uniformity test has high power. Real data applications are also provided.
Read the full abstract ▾