Machine Learning (Statistics)
LoMC: Localized Multidirectional Correction for Refusal Suppression in Routed Foundation Models
Yan Hong, Kedong Xiu, Wei Li et al. (+6 more)
We study controlled post-training refusal suppression in routed MoE and hybrid-MoE foundation models, aiming to increase non-refusal target-response behavior while preserving general capability under a compact intervention footprint. Existing broad direction-based edits can perturb general-purpose computation, whereas support-only expert edits often lack sufficient capacity to correct heterogeneous refusal representations. To address this limitation, we introduce Localized Multidirectional Correction (LoMC), a support-gated intervention framework that follows a support-then-correction execution order: it first identifies a compact edit support, then aggregates prototype correction directions into layer-wise correction directions, and finally applies rank-one layer-wise correction only within the selected support. By using the edit support as a structural gating constraint, LoMC increases correction capacity without expanding the intervention scope. Experiments on text-only and
multimodal safety
benchmarks across four routed backbones show that LoMC substantially improves non-refusal target-response behavior while maintaining general capability under a compact intervention footprint.
Read the full abstract ▾
All authors Yan Hong, Kedong Xiu, Wei Li, Jun Lan, Huijia Zhu, Shuheng Zhou, Zhongcai Lyu, Weiqiang Wang, Jianfu Zhang
Statistics Theory · Cross-listed from math.PR
On McDiarmid's Inequality under Dependence via Approximate Tensorization of Entropy
Valentin Roth
We argue that dependent versions of McDiarmid's inequality are a useful but underutilized tool in mathematical statistics, learning theory and theoretical computer science. To make this point, we first highlight that approximate tensorization of
entropy (ATE) implies McDiarmid's via the
Entropy Method. Second, we derive McDiarmid's inequality for non-isotropic Gaussian random vectors $X \sim \mathcal N(\mu, \Sigma)$ through ATE with a constant of the order of the condition number of $\Sigma$. We both independently obtain this ATE through a simple application of
stochastic localization and also discuss how a more general ATE for the Gibbs sampler due to Ascolani et al., 2026 generalizes McDiarmid's-like concentration to strongly log-concave and log-smooth probability measures. We then apply the resulting concentration inequalities to resolve a question on the concentration of $\operatorname{sign}(X)$ posed by Simone Bombari, investigate Erdős-Rényi graphs under dependence and prove a Dvoretzky-Kiefer-Wolfowitz-type inequality for observations from a joint measure fulfilling ATE and continuous marginal CDFs. For the class of strongly log-concave and log-smooth measures, this result improves upon a prior Dvoretzky-Kiefer-Wolfowitz-type inequality for non-i.i.d. observations due to Bobkov and Götze, 2010, by establishing the expected $1/\sqrt{n}$-rate of convergence under weak dependence instead of $n^{-1/3}$.
Read the full abstract ▾
Statistical Methodology
Bayesian machine learning approach for recurrent events studies using Soft Bayesian Additive Regression Trees (SBART)
MengXing Chen, Debajyoti Sinha and Antonio Linero
Recurrent event data frequently arise in biomedical studies, where individuals may experience multiple recurrences of the same type of events, such as recurrent hospitalizations. This article introduces a nonparametric method for recurrent events under a
Bayesian ensemble learning framework, called Soft
Bayesian Additive Regression Trees (SBART), which combines multiple soft decision trees to achieve high predictive accuracy and a smooth
estimator of the underlying intensity of the recurrent events. The proposed model represents the conditional intensity function of the non-homogeneous Poisson process as the product of a time-constant baseline, a subject-specific frailty random effect, and a nonparametric component capturing potentially nonlinear
covariate effects and unknown interactions among
covariates and time. A two-layer data augmentation scheme is employed to efficiently incorporate the SBART component within our computational
algorithm. Simulation studies demonstrate that our method, called RecSBART in short, achieves superior accuracy in estimating cumulative intensity compared to existing approaches, even when our modeling assumptions are not true. With the
Bayesian analysis of a study of recurrent hospitalizations of colorectal cancer patients, we further demonstrate our RecSBART method's ability to reveal and interpret the underlying complex relationships among
covariates in a recurrent events study.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.DS
A unified complexity bound for logconcave sampling
Yunbum Kook and Santosh S. Vempala
We give a simple, unified, and nearly tight bound for sampling arbitrary logconcave distributions from a warm start using the In-and-Out
algorithm along with exponential lifting. The main new ingredient in the analysis is an improved bound on the Poincaré constant of a lifted distribution. As a consequence, the resulting convergence rate is nearly tight for both constrained settings (e.g., Gaussian restricted to a convex body) and well-conditioned settings (e.g., strongly logconcave and smooth densities).
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Two-Layer Linear Auto-Regressive Models Estimate Latent States
Yahya Sattar, Sunmook Choi, Leo Maynard-Zhang et al. (+3 more)
Auto-regressive models have emerged as powerful tools for sequential data, from
language to
video. Understanding how and why these models learn latent representations remains an open theoretical question. In this work, we demonstrate that when trained by empirical risk minimization on data from partially observed linear dynamical systems, two-layer linear auto-regressive models naturally learn to approximate Kalman filtering. In particular, we show that the learned hidden representation coincides, up to a similarity transformation, with the state estimates produced by the optimal (Kalman) filter, even though the model has no explicit knowledge of the underlying dynamics or state. The result follows from three main insights. First, we establish that the Kalman filter is well approximated by an auto-regressive model with bounded truncation error. Second, we show that despite non-convexity, the two-layer optimization landscape is benign, i.e., all stationary points are either strict saddles or global minima. Finally, as our main contributions, we provide finite-sample guarantees on prediction error, parameter estimation error, and latent state recovery. Numerical simulations support the theoretical results and demonstrate that the latent representations of auto-regressive models recover state estimates.
Read the full abstract ▾
All authors Yahya Sattar, Sunmook Choi, Leo Maynard-Zhang, Yassir Jedra, Maryam Fazel, Sarah Dean
Machine Learning (Statistics) · Cross-listed from cs.LG
How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings?
Julia Kostin, Kasra Jalaldoust, Elias Bareinboim et al. (+2 more)
Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on. Recent work in causality-based domain generalization has shown how shared causal structure between domains can induce invariant predictors, e.g., models on a subset of features which have stable risk across structured domain shifts. However, the extent to which such population-level causal invariances can lead to gains in finite-sample settings remains underexplored. In particular, in practice we often have access to a few labeled target samples, a setting called supervised domain adaptation (sDA). In this paper, we explore when (full or partial) causal knowledge can provably improve supervised domain adaptation. As a first step, we study linear regression, where full or partial causal knowledge specifies a collection of invariant or possibly invariant feature subsets, each yielding a source-trained candidate predictor. We derive matching upper and lower bounds showing that finite-sample gains are governed by the target-risk margins separating the candidates, together with the finite-source estimation error. When these margins are sufficiently large relative to $n_Q$, an adaptive aggregation procedure can match the best candidate predictor while avoiding negative transfer relative to target-only learning. On the other hand, when the margins are too small, no
algorithm can reliably exploit the candidate collection to obtain faster finite-sample rates. We further connect these margins to structural shift magnitude in linear SCMs and validate the theory on real-world causal
benchmarks.
Read the full abstract ▾
All authors Julia Kostin, Kasra Jalaldoust, Elias Bareinboim, Samory Kpotufe, Fanny Yang
Statistical Methodology
Restricted Multivariate Spatial Modeling
Jihyeon Kwon and Harrison Quick
When modeling health events in small areas, the conditional autoregressive (CAR) framework of Besag, York, and Mollié (BYM) is widely used. For multiple outcomes, the multivariate CAR (MCAR) extension accommodates dependence among diseases that share risk factors, in addition to spatial dependence, and can also jointly model demographic subgroups for a single disease, allowing information to be borrowed across related populations. However, recent studies have shown that the BYM CAR model can be overly informative, leading to excessively precise estimates. While the MCAR model is expected to be more informative due to additional information shared across subgroups, its level of informativeness has not been previously quantified. We propose a framework to measure MCAR model informativeness as an extension of prior work and introduce a method to control it, ensuring the model contributes comparably to each subgroup. We achieve this through a reparameterization of the MCAR model within a computationally efficient framework. We demonstrate how the MCAR model compares with the BYM CAR model in terms of informativeness and oversmoothing and highlight the advantages of the restricted MCAR model using county-level heart disease death data stratified by race and sex.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Physics-Informed Neural Networks for Chemotherapy Pharmacokinetics: Benchmarking the Clinical Estimator and Exposing Parameter Identifiability
Riya Bisht and Dhruv Agarwal
Physics-Informed Neural Networks (PINNs) are an attractive tool for partial-observation problems in biology, where the governing dynamics are known but some compartments cannot be measured. Chemotherapy pharmacokinetics (PK) is a clean instance: drug concentration in
plasma is routinely measured, but concentration in tissue – which determines tumour kill and off-target toxicity – is not. We
benchmark a PINN against the standard clinical baseline (nonlinear least-squares on the analytical biexponential
plasma solution, hereafter NLS) and a physics-agnostic neural baseline (a data-only MLP) on two PK problems. On the linear two-compartment problem, NLS is near-optimal; the PINN matches it to within a small constant factor while also producing the tissue curve in a single training pass, whereas the data-only MLP fails on tissue by roughly 10x. On a Michaelis-Menten extension (saturable elimination), the biexponential closed form no longer exists, so NLS is mis-specified and silently returns meaningless rate constants. The PINN instead exposes a deeper fact: the Michaelis-Menten two-compartment model is non-identifiable from
plasma alone, and the PINN reports this honestly by converging to a basin with k12 -> 0. Adding two sparse tissue observations largely resolves identifiability: across five seeds the PINN recovers k21 to within 1% of truth and Vmax, Km to within one standard-deviation bar, while k12 moves in the correct direction (0.02 -> 0.82) but remains ~2 sigma below truth – a recovery the closed-form NLS
estimator cannot attempt at all, because its biexponential ansatz describes only
plasma. Our claim is not that PINNs beat NLS. It is that PINNs offer a uniform recipe that ties the textbook
estimator on the textbook problem, exposes structural identifiability that the textbook
estimator hides, and absorbs heterogeneous measurements within a single loss.
Read the full abstract ▾
Statistical Methodology
Computationally tractable robust differentially private mean estimation
Kelly Ramsay
We develop a new, differentially private mean
estimator called the balloon mean. The main features of the balloon mean are that it is computationally tractable and enjoys robustness to outlying observations. It is based on an iterative clipping procedure over expanding Mahalanobis balls, or “balloons.” The method satisfies zero-concentrated differential
privacy and depends on a small number of interpretable tuning parameters. We provide theoretical guarantees under heavy-tailed and contaminated elliptical models, characterizing its statistical performance and robustness to outliers. Extensive simulations demonstrate that the balloon mean is robust to heavy-tailed and contaminated data, and outperforms existing differentially private mean
estimators in contaminated settings.
Read the full abstract ▾
Machine Learning (Statistics)
Epistemic Uncertainty Is Not the Reducible Kind
Robin Young
The standard taxonomy of predictive uncertainty defines epistemic uncertainty as the part removable by collecting more data, while the standard measure identifies it with a mutual-information term. We prove the definition and the measure are extensionally inconsistent. On an explicit construction, the measure assigns all uncertainty to the epistemic class, yet no quantity of training data reduces it. Reducibility is instead a property of the pair (uncertainty, acquisition class), and the dichotomy resolves into three parts: aleatoric, sample-reducible epistemic, and mechanism-reducible epistemic uncertainty. An exact identity for the value of an observation shows that in-distribution data never reduces mechanism-irreducible uncertainty and generically increases it. Ensemble disagreement, the deployed epistemic estimate, tracks the training procedure rather than the epistemic term. It collapses to zero beneath a positive truth under consistent training, and equals hyperparameter-scaled initialization noise under interpolation. A finite-sample falsification test and seed-swept experiments confirm the theory.
Read the full abstract ▾
Statistics Applications · Cross-listed from astro-ph.EP
Quantifying Surface Heterogeneity Across Asteroid (101955) Bennu using Candidate Site Remote Sensing Data
Emma-Catherine Belhadfa, Neil E. Bowles, Katherine A. Shirley et al. (+3 more)
The OSIRIS-REx mission acquired spatially resolved (2-10 m spot sizes) visible-near infrared (VNIR) and thermal infrared (TIR) spectra across four candidate sampling sites on asteroid (101955) Bennu: Nightingale, Osprey, Sandpiper, and Kingfisher. To quantify heterogeneity across a small body (about 500 m radius) like Bennu, we explore remotely observed spectral data to draw conclusions about the mineralogical composition and key physical processes that drive surface variability. We derive diagnostic band parameters from the OSIRIS-REx Visible and Infrared Spectrometer and the OSIRIS-REx Thermal Emission Spectrometer
datasets to quantify compositional and physical variability across sites and assess their mineralogical context. The VNIR spectra exhibit similar overall reflectance shapes but systematic differences in spectral slopes and the 2.74 micron OH absorption. TIR emissivity spectra reveal modest but statistically significant shifts in the Christiansen Feature, silicate stretching, and bending band positions, indicating differences in silicate composition, hydration state, and Mg/Fe relative abundance. Principal component analysis separates each site into distinct clusters in multivariate band-parameter space, whereas K-means clustering identifies intra-site spectral sub-populations. Welch's Analysis of Variance and Hotelling's tests confirm that band-parameter variations between sites are significant. These results reveal that Bennu's surface preserves measurable spectral heterogeneity at 2-10 m scales, with site-to-site variations in hydration indicators and silicate band positions. The spectral properties of Nightingale encompass the full range observed across all four sites, establishing a remote sensing baseline for contextualizing laboratory analyses of the returned sample within Bennu's broader composition diversity and alteration history.
Read the full abstract ▾
All authors Emma-Catherine Belhadfa, Neil E. Bowles, Katherine A. Shirley, Amy A. Simon, Victoria E. Hamilton, Hannah H. Kaplan
Statistics Applications
Estimating Individualized Treatment Effects in Acute Ischemic Stroke with Causal Transformation Models (TRAM-DAG): A Multi-Centre Observational Study with External RCT Validation
Lisa Herzog, Oliver Dürr, Pascal Bühler et al. (+4 more)
Personalized medicine in acute ischemic stroke requires moving beyond average treatment effects (ATE) to individualized treatment effect (ITE) estimates to support treatment decisions. In acute ischemic stroke, mechanical thrombectomy has been shown to be more effective on average than lysis in randomized controlled trials (RCTs), such as the MR CLEAN study. We aim to identify which individual patients benefit most from mechanical thrombectomy compared to lysis. The outcome of interest is the modified Rankin Scale (mRS) at three months, an ordinal measure of functional disability (0: no symptoms, 6: death). We demonstrate that causal transformation models on directed acyclic graphs (TRAM-DAG) can be used for ITE estimation after being fitted on observational MAGIC multi-center stroke patient data. To ensure comparability with the MR CLEAN population, which we use for validation, we train the TRAM-DAG on a MAGIC sub-population with NIHSS at admission >= 6, corresponding to one inclusion criterion of MR CLEAN. The fitted model is then used to estimate ITEs for stroke patients in the MR CLEAN population. While these ITE estimates cannot be confirmed experimentally, we show that their average is consistent with the trial's reported ATE. Furthermore, the ITE estimates correctly rank trial patients by their observed frequency of a good outcome (mRS at three months <= 2). These findings support the use of causal models like TRAM-DAG for personalized decision-making in stroke care and highlight their ability to bridge the gap between observational evidence and clinical trials.
Read the full abstract ▾
All authors Lisa Herzog, Oliver Dürr, Pascal Bühler, Hakim Baazaoui, Julian Deseö, Susanne Wegener, Beate Sick
Statistical Methodology
Extending Prais-Winsten Regression to Panel Data with Higher-Order Autoregressive Errors: A Simulation Study
Ariel Linden
We extend the Prais-Winsten AR(k) generalized least squares (GLS) transformation to panel data within the Beck-Katz panel-corrected standard error (PCSE) framework and implement the method in the community-contributed Stata package xtpraisk. As the panel extension of Prais-Winsten, xtpraisk is the natural comparator to xtscc, the panel extension of Newey-West and implementation of the Driscoll-Kraay
estimator. We conduct a Monte Carlo simulation to validate the statistical properties of xtpraisk and compare its finite-sample performance with xtscc. The simulation spans autoregressive orders 1-3, three autocorrelation scenarios, three panel sizes, six series lengths, and five effect sizes, with 2,000 replications per condition. Across all conditions, xtpraisk achieved higher power than xtscc while maintaining near-nominal Type I error rates, confidence interval coverage, and standard error calibration. In contrast, xtscc exhibited systematic standard error underestimation and inflated Type I error at short series lengths, with both deficiencies worsening as autoregressive order increased. Both
estimators were essentially unbiased. Misspecification of the autoregressive order did not degrade xtpraisk's inferential performance, and cross-panel correlation and panel size had negligible effects on the relative performance of either
estimator. The results indicate that xtpraisk is preferable when both statistical efficiency and valid inference are priorities, particularly under persistent higher-order autocorrelation and short to moderate series lengths.
Read the full abstract ▾
Statistical Methodology
Inferring resource selection and utilization distributions from irregular and error-prone animal tracking data
Fanny Dupont, Brett T. McClintock, Jan-Ole Fischer et al. (+3 more)
Habitat selection and space use are fundamental to understanding animal distribution. Traditional methods for quantifying habitat preferences from telemetry data assume regular sampling and negligible measurement error. However, these assumptions are routinely violated in marine systems. Practitioners typically regularize and filter the data before fitting models, but these two-step procedures do not propagate uncertainty from the filtering stage and can yield biased estimates. Habitat-driven Langevin
diffusion models offer an elegant alternative, naturally accommodating irregular sampling. However, incorporating measurement error via a state-space formulation is challenging because habitat
covariates depend on the latent true locations. We address this using the Laplace approximation to simultaneously integrate over true locations and account for habitat
covariates along latent paths, yielding a single-stage framework efficiently implemented in Template Model Builder (TMB). By doing so, we provide the first TMB implementation capable of handling
covariates that depend on latent variables, allowing inference via fast and efficient maximum likelihood estimation. Simulations show that our approach outperforms the two-step method, recovering habitat-selection parameters even under substantial measurement error and missing data, with more accurate utilization distributions and trajectory reconstructions. Applied to narwhal (Monodon monoceros) telemetry data, the two-step method substantially shrinks the habitat selection coefficient towards zero, while our unified approach recovers a much stronger signal. Our framework offers a computationally efficient solution to long-standing challenges of measurement error and temporal irregularity in habitat selection inference, applicable across a wide range of taxa and environments.
Read the full abstract ▾
All authors Fanny Dupont, Brett T. McClintock, Jan-Ole Fischer, Marianne Marcoux, Nigel E. Hussey, Marie Auger-Méthé
Statistics Theory
Integrated expectile-based measures of inequality
Ignacio Cascos and Marco Tarsia
Expectiles provide a class of asymmetric location functionals that incorporate the magnitude of deviations and admit a natural
geometric interpretation. Building on their structural consistency with the convex
stochastic order, this paper introduces a family of integrated expectile functionals for measuring risk, dispersion, and inequality. The proposed functionals admit analytical representations as integrals of expectiles across asymmetry levels and, for a distinguished subclass,
geometric representations in terms of weighted areas of
star-shaped sets encoding distributional asymmetry. This approach yields a new class of expectile-based inequality indices, constituting a natural counterpart to classical Gini-type measures while preserving desirable monotonicity and consistency properties. Empirical counterparts are derived in closed form and admit explicit decompositions over finite samples. The framework extends naturally to multivariate settings through directional expectile constructions, leading to measures capable of capturing genuinely joint forms of multivariate dispersion and inequality.
Read the full abstract ▾
Statistical Methodology
ShrinkageTrees: An R Package for Bayesian Tree Ensembles for Survival Analysis and Causal Inference
Tijn Jacobs
ShrinkageTrees is an R package for
Bayesian tree ensembles in survival analysis and causal inference. The package implements
Bayesian additive regression tree models for right- and interval-censored survival outcomes within an accelerated failure time (AFT) framework, with optional decomposition into prognostic and treatment-effect components for causal inference. Two complementary forms of regularisation are available: regularisation of the tree structure, via depth-penalising priors and Dirichlet splitting priors, and regularisation of the step heights, via global-local shrinkage priors. ShrinkageTrees provides the first implementation of the Horseshoe Forest, which places a horseshoe prior on the step heights. These regularisation strategies extend
Bayesian tree ensembles to high-dimensional settings. An efficient Rcpp backend, multi-chain MCMC, and S3 methods support the full workflow: fitting, prediction, causal effect estimation, and convergence diagnostics.
Read the full abstract ▾
Statistical Methodology
Bayesian nonparametric Mallows model for clustering preference data
Lorenzo Zuccato, Veronica Vinciotti and Valeria Vitelli
Preference learning refers to the learning of latent patterns from ranking and preference data of different kinds. Typical aims of preference learning are to infer a shared consensus ranking, to learn individual-level preferences, and to perform unsupervised clustering. The Mallows model is among the few approaches that can achieve all these objectives jointly. Previous work has developed computationally tractable methods for
Bayesian inference based on a MCMC Metropolis-Hastings scheme, where clustering is performed via a finite mixture of Mallows models. Inference on the number of clusters is then conducted a posteriori. Here we propose a
Bayesian nonparametric Mallows model, based on a Dirichlet process mixture model. This allows joint inference on the number of non-empty clusters and on the clustering allocation, as well as posterior inference on cluster-specific parameters. The implementation of the proposed sampling
algorithm is integrated into the existing R package BayesMallows, which also supports data in the form of incomplete rankings and pairwise comparisons. Simulated data show good performance of the nonparametric model compared to a finite mixture model in terms of recovery of the correct number of clusters, while empirical data on movie ratings show the model's effectiveness in providing personalized movie recommendations on discarded ratings.
Read the full abstract ▾
Statistical Methodology
Bayesian Triangulation Splines: Spatial Adaptation on Irregular Domains
Sihyeon Pyeon, Sunwoo Lim and Seonghyun Jeong
Conventional nonparametric regression methods for two-dimensional non-rectangular domains often overlook domain geometry and allow smoothing across boundaries. In spatial and geostatistical applications, this assumption is frequently invalid because domain boundaries typically constrain interactions among observations. Accommodating spatially varying smoothness is also substantially more challenging than in the univariate setting, and most existing methods do not adequately capture this local structure of the target function. To address these challenges, we propose
Bayesian triangulation splines, which constructs locally adaptive splines over a polygonal domain. The method employs constrained Delaunay triangulations to respect boundary geometry and adapt to heterogeneous smoothness. A carefully designed prior further improves empirical performance. Under a global Sobolev smoothness assumption, we show that the proposed method achieves the optimal posterior contraction rate and adapts to unknown smoothness. We also show that the method exhibits ideal spatial adaptation in the sense that it achieves the oracle rate for inhomogeneous or locally varying structural features. Crucially, this oracle guarantee is not specific to constrained Delaunay triangulations, but holds over any triangulation satisfying weak shape-regularity conditions. Simulation studies confirm that the proposed method outperforms existing approaches by achieving higher estimation accuracy while maintaining low model complexity.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from econ.TH
Market Design for AI: Beyond the Copyright Binary
Yan Dai, Maryam Farboodi, Negin Golrezaei et al. (+1 more)
How can we design a
market of human-generated content for use in training AI models that both enables technological progress and preserves individual incentives for high-quality content creation? Existing approaches take polar positions: a "free-for-all" model based on fair use and a "strong intellectual property rights" model. We show that both fail: Free-for-all does not compensate creators, and—by modeling as a static Stackelberg game—strong intellectual property rights also underpower creative incentives. We find this especially true for more innovative creators, a phenomenon we term the "originality penalty." Extending this insight to a dynamic model, we find another
market failure undermining AI model performance, even for an initially good model: Such a model induces greater reliance by humans on AI-assisted creation, resulting in homogenized content feeding back into training, which degrades the model performance—a "curse of precision." We further propose a
market design with a data intermediary negotiating collectively with the AI firm and subsidizing innovative contributions, thus restoring efficiency.
Read the full abstract ▾
All authors Yan Dai, Maryam Farboodi, Negin Golrezaei, Sepehr Shahshahani