Peer Revenue Impact factor: volatile

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Tuesday 29 September

Statistical Methodology

Statistics of multivariate extremes under random censoring

We study tail dependence of a $d$-dimensional random vector whose coordinates are subject to random right censoring. Along each fixed direction the censored problem reduces exactly to a univariate one, and the observed data determine the radius and whether it was produced by the event or censoring vector. An ordinary Kaplan–Meier product limit therefore estimates the joint tail probability in that direction, in every dimension, and with no multivariate survival function, no smoothing and no tuning parameter beyond the threshold. The theory of this directional estimator is formulated under an arbitrary marginal standardization and conditions only imposed on the standardized laws, in particular for any max-domain of attraction. We prove uniform consistency and functional weak convergence at the square root of the effective number of joint extremes, allowing the standardization to be estimated. A multiplicative standardization recovers the heavy-tailed theory, whereas a standardization built from the marginal (non-directional) Kaplan–Meier estimators requires no marginal tail model and targets the normalized tail copula itself. The standardization error is negligible for the former, and for the latter under a mild condition on the joint censoring. Simulation studies validate the finite-sample performance of the estimator. An application to the National Flood Insurance Program dataset comprised of claims generated by Hurricane Ian estimates the joint upper tail of building and contents losses from indemnities, which are subject to capping, simultaneously in both coordinates.
Read the full abstract

Statistical Methodology

Target-Aware Sequential Inference: Pooled versus Stratified Anytime-Valid Designs

Sequential studies with heterogeneous strata may define the estimand under a target mixture that differs from the sampling distribution. We organize the design problem by guarantee class: one pooled confidence sequence for a single prespecified target, shared pooled inference for a prespecified target set, and stratum-resolved inference when local guarantees must remain available. For target-set pooled inference we derive minimax, gap-aware, and range-capped shared proposals and an adaptive tracking rule. A specialized augmented off-policy score removes between-stratum mean heterogeneity asymptotically, recovering Neyman allocation for efficient single-target pooled inference. Robust directional certification over a convex target set is a classical intersection-union problem and does not require an M-fold vertex split. We derive a Gaussian information lower bound and an exact finite-support characteristic information that coincides with the bounded nonparametric KL-inf limit, and state sufficient conditions for first-order attainment of the Gaussian constant. Within the local-CS architecture, regularly varying boundaries imply a general allocation law with the 2/3 exponent for root-n variance-adaptive widths. Simulations and a PromptEval replay illustrate the resulting guarantee-efficiency frontier.
Read the full abstract

Statistics Applications

Feeling Left Behind? Territorial Disadvantage, Well-Being and Cohesion across European Regions

Left-behind places are usually identified using economic, demographic and accessibility indicators, but these may not align with well-being and social cohesion. We combine ten years of territorial indicators with subjective measures derived from georeferenced social-media data for over 1,200 NUTS-3 regions in 28 European countries (2013-2023). Regional profiles and within-between panel models reveal uneven relationships: disadvantaged regions can display contrasting levels of well-being and cohesion, national context alters cross-country associations, and between-region differences diverge from within-region change. The findings show that aggregate measures conceal distinct forms and trajectories of territorial disadvantage.
Read the full abstract

All authors Stefano Maria Iacus, Giuseppe Porro, Haodong Qi, Devika Jain

Machine Learning (Statistics) · Cross-listed from cs.LG

Counterfactual Probing for Parallel Unmasking with Hidden Forest Structure

Masked generative models offer parallel token prediction, but accurate parallel sampling must account for dependencies among tokens. When dependencies are unknown, finding safe batches also costs model evaluations. We study whether total evaluations, including discovery, can be sublinear in sequence length $N$; sublinear sequential depth then follows. We consider discrete distributions with hidden forest structure, accessed through a fixed approximate conditional oracle. Under explicit regularity conditions and uniform Hellinger error bounds, for any fixed target accuracy $\varepsilon\in(0,1/8]$ and sufficiently large $N$, our sampler achieves seed-averaged total-variation error at most $\varepsilon$, with total masked-state submissions and sequential depth both bounded by $\widetilde{O}(N^C \varepsilon^{-a})$ for constants $0<C<1$ and $a>0$. These guarantees use polynomial vocabulary size and an edge-response lower bound set by $N$ and $\varepsilon$. The sampler shares evaluations of hypothetical reveals across dependence tests to identify safe parallel batches without requiring full recovery of the hidden forest. A tunable parameter trades probing cost against irreversible commit rounds. In the same class, any admissible irreversible product-commit sampler attaining the same seed-averaged accuracy requires $\Omega(N^c \varepsilon^b)$ counterfactual submissions or commit rounds in the worst case, for constants $c,b>0$.
Read the full abstract

Statistical Methodology

Characterizing Survivor Principal Strata Over Time: Identification, Efficient Estimation, and Sensitivity Analysis

Longitudinal studies with functional outcomes subject to truncation-by-death frequently report time-indexed survivor average causal effect estimates, defined among individuals who would survive regardless of treatment assignment. Because the survivor stratum can change over time, the resulting effect estimates may pertain to populations with different characteristics. We propose accompanying survivor average causal effect estimates with measures of covariate divergence between the survivor and baseline populations. We study standardized mean differences (SMDs) and the Kullback-Leibler (KL) divergence as moment-based and distributional summaries, respectively. We show that these divergence measures are identified under a subset of standard principal stratification assumptions and that their identification involves nuisance functions already estimated in survivor analyses. We derive the efficient influence functions, enabling construction of one-step bias-corrected estimators with appealing large-sample properties and allowing machine learning for nuisance estimation. We demonstrate that in randomized studies, one-step estimation of SMDs is robust to arbitrary nuisance model misspecification, whereas the KL divergence and broader density ratio estimands are more sensitive to nuisance misspecification. Additionally, we provide a sensitivity analysis framework that partially relaxes a monotonicity assumption required for identification, and construct one-step estimators for the resulting bounds. Through simulation studies, we demonstrate the finite-sample performance of the proposed estimators and illustrate how divergence measures can improve the interpretability of survivor analyses. Finally, we assess covariate divergence in survivor strata over time in amyotrophic lateral sclerosis (ALS) trials, using data from the Pooled Resource Open-Access ALS Clinical Trials database.
Read the full abstract

All authors Keith Barnatchez, Willow Butler, Julia A. Geller, Elizabeth A. Stuart

Statistical Methodology

When Noise Estimation Hides Basis Misspecification in Repeated Bayesian Inverse Problems

Basis-restricted priors in Bayesian inverse problems can lose coverage when the truth has components outside the basis. We show that estimating the observation-noise variance can hide this loss. Under a linear forward model, when the in-span prior variance dominates the noise, the maximum-likelihood noise estimate absorbs the out-of-basis energy in the complement of the model range. Residual-magnitude and observation-coverage checks then stay near nominal while field coverage falls. We study repeated problems sharing one forward operator and one basis, fixed independently of the tested data. After projection onto the complement, and conditionally on the fitted noise scale, every exact test is a test of the scale-free direction of the residuals. We test the shape of their sample spectrum with John's sphericity statistic. Under Gaussian noise its null model is exact at finite sample size, and we derive its null mean and its power at proportional dimension. On synthetic problems and in a preregistered GEBCO topography study, the test detects structured out-of-basis variation that cross-validation and observation-coverage checks largely miss. It cannot detect Gaussian out-of-basis variation that is isotropic in the complement, since that is indistinguishable from a change of noise scale.
Read the full abstract

Statistical Methodology

Sequential Supersaturated Screening Experiments

Supersaturated screening experiments study many candidate factors with few runs. The experimenter must first identify the active factors, then optimize the response. One-shot regularized regression tends to select many factors, and the second-order response surface model in the selected factors is then large and needs many further runs to fit. We propose Sequential Supersaturated Screening ($S^3$), a two-stage framework for supersaturated designs. $S^3$ builds each screening round by coordinate exchange under a new positive-cone design criterion that uses no practitioner-chosen Welch calibration constant. It then removes low-importance factors one round at a time, using a graduated quantile rule. The criterion scores a design by how far its column correlations sit above the Welch lower bound, and it adapts to the current number of runs and candidate factors. We give a round-by-round bound on the probability that a noise factor is ever fixed, under conditions on noise survival and commitment at each round, together with an explicit upper bound on the total number of Stage 1 runs. Across simulated screening problems and the Borehole benchmark, $S^3$ achieves lower Type I error and higher $F_1$ scores than one-shot cross-validated Lasso (LassoCV). It also improves optimization quality in most settings and runs faster.
Read the full abstract

All authors Songqiao Han, Kalliopi Mylona, Steven Gilmour, Matteo Borrotti

Statistical Methodology

SPARK: A General Goodness-of-Fit Assessment via Residual Projection

Goodness-of-fit testing is a basic tool for assessing whether a fitted procedure has captured the systematic information contained in the covariates. While traditional theory has largely focused on parametric regression models, modern data analysis increasingly relies on flexible black-box learners, whose predictive success alone is insufficient to assess model accuracy. In this paper, we propose SPARK, a general framework for goodness-of-fit testing that applies to traditional statistical models and general black-box learning procedures, continuous and binary responses, and low- and high-dimensional predictors. Based on a debiasing strategy, the residuals from an initial fit of a learning procedure are projected onto nearly orthogonal directions to extract any remaining signal. To capture information across all projection directions, we propose a kernel-based projection method and establish both its asymptotic properties and the consistency of a bootstrap procedure. Comprehensive simulations and real data analyses illustrate the effectiveness and flexibility of our proposed method.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.NA

The double descent and Runge phenomena in overparametrized polynomial interpolation

The Runge phenomenon in polynomial interpolation is often considered a classical analogue of the double descent phenomenon in machine learning. In this note, we explore overparameterized polynomial interpolation in three popular polynomial bases: Monomial, Chebyshev and Legendre basis with coefficients that are minimal in the $\ell^2$-norm (and, for the monomial basis, also those minimal in the $\ell^1$-norm). We present our results primarily for equidistant and Chebyshev data points, but many results are independent of the exact form of sampling.
Read the full abstract

Machine Learning (Statistics)

A Finslerian Approach for Embedding Directed Data

Many datasets carry an intrinsic directionality: citations point backward in time, cells differentiate along lineages, and traffic follows preferred routes. Spectral embedding methods, including most of their extensions to directed graphs, discard this information: they symmetrize the data and map it into a Euclidean space where asymmetry cannot be represented. We instead model directed data as sampled from a Finsler manifold, whose distance depends on the direction of travel, and study the kernel operator built from this asymmetric distance. Through a moment expansion of this operator, we show that its symmetric and antisymmetric parts separate geometry from direction. As the bandwidth of the kernel vanishes, the symmetric part converges to a weighted Laplacian, recovering diffusion maps in the Riemannian case, while the antisymmetric part converges to a first-order transport operator that encodes the directionality. We prove that the corresponding graph operators, built from finitely many samples, converge uniformly and almost surely to these limits. For Randers metrics, this vector field is explicit and yields an embedding algorithm recovering both the manifold structure, from the spectrum of the symmetric part, and the underlying drift. We illustrate the approach on synthetic directed graphs and point-clouds.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

RLTL;DR: Self-improvement by Internalizing Self-generated Feedback

The common paradigm of reinforcement learning with verifiable rewards (RLVR) is to let agents make multiple attempts at a task, and optimize towards the successful ones. This becomes problematic in the realms of self-improvement, where tasks are so difficult that the agent has a low or even no chance of success, and where there are no teacher models or example solutions to distill from. In this paper, we introduce RLTL;DR. After each failed attempt, we show the policy the verifier outputs and let it write its own feedback, in the form of a single TL;DR insight. The next rollout is conditioned on all previous insights, and we sequentially sample rollouts until a solution is found. Moreover, we enable backpropagation on the in-context insights to internalize a direct task to insight mapping. On challenging tool-calling and coding datasets (filtered to Pass@128=0), standard GRPO training of a Qwen 3.5 9B Thinking policy stays flat at a Pass@1 of 0% to 1%. RLTL;DR breaks through this learning barrier, achieving a Pass@1 of 14-31% with insights in context during training and, crucially, 12-13% when no insight is in context at eval time. We identify that the key is the task to insight internalization. To study this further, we reduce our approach to SFTL;DR, training only on (task, insight) tuples, without showing or backpropagating on any rollouts. Training on only 4k of these tuples recovers almost the full performance of RLTL;DR and classical SFT on full rollouts. This demonstrates a promising compacted training paradigm of the form "on this sort of task, keep this sort of thing in mind", which we hope to inspire future research on.
Read the full abstract

All authors Michael Kirchhof, Eleonora Gualdoni, Andrew Szot, Khashayar Gatmiry, Aryo Lotfi, Abbas Kazerouni, Omar Attia, Sanjoy Chowdhury, Alexander Toshev

Statistical Methodology

On the use of G-computation in small randomized controlled trials with many covariates

In contemporary randomized controlled trials (RCTs), the number of patients is often small relative to the number of baseline covariates collected. In such settings, maximum likelihood estimators of conditional treatment effects in generalized linear models, along with their standard errors, may exhibit substantial bias. This study examines whether similar bias arises in G-computation estimators of marginal treatment effects, which are known for their robustness to model misspecification when applied with canonical GLMs. We develop theoretical insights, drawing on recent literature on G-computation under proportional asymptotic regimes, in which the number of covariates grows with the sample size. Specifically, we characterize the bias of the standard G-computation estimator and the leave-one-out G-computation estimator under such high-dimensional settings. Monte Carlo simulations using linear and logistic outcome models are conducted to evaluate practical remedies for G-computation estimators, including covariate selection, cross-fitting, leave-one-out cross-fitting, and small sample corrections to standard errors. Further insights are derived from a re-analysis of the BestAIR trial data. Our findings provide guidance for the application of G-computation in modern RCTs, particularly when addressing challenges posed by limited sample size. Keywords: Covariate adjustment, covariate selection, high-dimensional, proportional asymptotics, targeted learning, cross-fitting.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.AI

XU-RS: Explaining Credal Width in Random-Set Language Models

Uncertainty estimates tell us how unsure a model is, but not why. Without knowing which parts of an input influences a model's uncertainty, we cannot tell whether that uncertainty score depends on input features that are relevant for the task. We study this problem in randomset classifiers built using pretrained language models. These classifiers assign probability to individual answers and to groups of answers, producing lower and upper probabilities for each answer; The difference between these probabilities, called credal width, is used to represent epistemic uncertainty about an answer arising from limited training data. We propose XU-RS, a framework that attributes an answer's credal width to the input tokens (words or word pieces) supplied to a language model. XU-RS uses Expected Gradients (a standard feature attribution method) to estimate how input tokens contribute to credal width. The proposed framework is evaluated on a MedQA dataset using SmolLM3-3B and Llama-2-7B models, demonstrating that setting the embedding of a token ranked highly by XU-RS to zero (zero-masking) causes larger changes in credal width than zero-masking randomly selected tokens. In addition, we show that normalisation can cause other answer groups to influence an answer's width, reveal how token attribution can mask numerical errors, and provide diagnostic checks to verify whether a token ranked highly by XU-RS meaningfully explains model uncertainty.
Read the full abstract

All authors David Achara, Maryam Sultana, Alexander D. Rast, Fabio Cuzzolin

Machine Learning (Statistics)

Ornstein-Uhlenbeck Is Hard to Beat, Yet Superlinear Drift Ships Lower Transport Costs

Brešar and Mijatović \cite{bresar2025} show that Ornstein–Uhlenbeck diffusion is hard to beat in forward convergence under assumptions that exclude superlinear drift. We instead test superlinear Langevin diffusions for score-based image generation, computing their conditional scores numerically from a Fokker–Planck equation. In our experiments, the superlinear models beat the Ornstein–Uhlenbeck baseline on empirical Wasserstein distance across nearly the entire tested grid and show less variation across diffusion horizons. The “hard to beat” verdict of \cite{bresar2025} thus fails to be universal.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Why Adaptive Optimizers Underestimate Rare Tokens

In the softmax output layer, a rare token receives a small positive logit gradient on most steps and a much larger negative gradient on the few steps when it is the target. SGD simply adds these contributions. Coordinate-wise adaptive methods such as Adam, RMSProp, and sign descent instead divide each update by a running estimate of its magnitude, and that estimate is largest immediately after the token appears. This imbalance has two effects. At the level of the whole output layer, we characterize which optimizers preserve the mean output embedding: every method whose update is linear in past gradients does, as do Kronecker-factored and orthogonalized methods such as Shampoo and Muon. Adam, Adafactor, Lion, and sign descent do not, and for these methods we obtain an exact step-by-step expression for the change. At the level of an individual rare token, the same normalization shifts the training fixed point. In the unigram model, sign descent lowers the logit of every token that occurs in fewer than half of the minibatches at a constant expected rate. For RMSProp with periodic arrivals, we can solve the fixed point in closed form: if a token is absent for at least two consecutive minibatches, its equilibrium probability is strictly below its data frequency for every learning rate, and the ratio tends to $\kappa/(2(e^{\kappa/2}-1))$. Here $\kappa$ is the mean number of steps between occurrences divided by the second-moment time constant $1/(1-\beta_2)$. In the same model, SGD and AMSGrad retain the unbiased fixed point. We test these predictions both in a unigram model and in a small language model trained from a known generating distribution. With random arrivals, the bias is larger than the periodic formula predicts; in the language model, the optimizers with the biased fixed point also fit the generating distribution less well.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Physical Muon: Orthogonalization as an Equilibrium Computation

Physical neural networks and analog in-memory computing could reduce the energy cost of neural network training. Realizing this potential, however, requires optimizers that combine effective learning with physical implementability. SGD fits local analog updates but struggles on transformers, while Adam family is unstable against analog bias. Muon offers strong training performance, but its Newton–Schulz orthogonalization relies on dense matrix-matrix products. To address this obstacle, we introduce Physical Muon, which computes the orthogonalization as the equilibrium of a continuous-time flow. Random probes approximate the flow using matrix-vector products, reciprocal reads, and local rank-1 writes. To test whether this replacement preserves training performance, we evaluate it on a 10.95M-parameter transformer. The dense flow's mean validation cross-entropy is 0.0085 above Newton–Schulz across nine seeds per method; the probe implementation is 0.0188 above the control across two seeds. Circuit simulations further reproduce the flow dynamics and yield comparable training behavior.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from math.OC

A stochastic subgradient method with optimal failure exponent

Fix a target accuracy $\varepsilon$, a gradient-noise level $s$, and a horizon $N$. We wish to design algorithms which minimize the probability of observing a suboptimality gap which exceeds the target accuracy, i.e., $\mathcal{E}_N = -\log \sup_{f,P} \mathbb{P}_P(f(x_A) - f_\star \ge \varepsilon)$, with the noise law $P$ known only to be sub-Gaussian. We single out a uniformly averaged schedule which is harmonic (of the form $h_k = R^2/(\varepsilon (N+m-k))$) and prove, via an optimized exponential supermartingale argument, that it attains the optimal exponent $\mathcal{E}_N^\star = \varepsilon^2 N (1+o(1))/(2R^2 s^2)$. Optimality is certified by a matching impossibility result: under Gaussian noise, a gradient-masking change of measure caps the exponent of every algorithm at the same leading order. In a small-noise limit, our setting degenerates into the adversarial-error model of Gösgens and van Parys (2025) for subgradient methods.
Read the full abstract

Statistical Methodology

Debiased and Simultaneous Inference for Heterogeneous Factorial Effect Modifiers via Residualized Walsh-Hadamard Scores

In factorial randomized experiments with $K$ binary treatment components, we study which baseline covariates modify component main effects and interactions. We define effect-modifier cells $(j,S)$, $j\in[p]$ and $\emptyset\neq S\subseteq[K]$, through the coefficients $\theta^\ast_{jS}$ of population linear projections of conditional Walsh–Hadamard contrasts $\tau_{S}$ onto standardized covariates. Using baseline-robust direct scores, we develop frequentist inference for all $p(2^K-1)$ cells. First, we prove a debiased central limit theorem with global insensitivity: the residualization baseline enters the cross-fitted score through an exactly conditionally mean-zero perturbation. The influence-function remainder therefore requires only foldwise weighted $L_2$ consistency of the baseline estimator, with no prescribed polynomial rate or nuisance product-rate condition. A separate, explicit penalty-scale condition controls the baseline's effect on the score Lasso. Second, under explicitly stated uniform nuisance, baseline, studentization, and influence-array conditions derived from bounded sparse-regression primitives, we establish a high-dimensional Gaussian approximation over the full grid, calibrated by a Walsh-spectral multiplier bootstrap. This yields simultaneous confidence bands and Romano–Wolf step-down selection of modified cells with strong familywise error control, without effect-heredity assumptions. The factorial design is fixed in the asymptotic analysis, while the covariate dimension may grow; all $2^K-1$ contrasts are covered jointly.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

High-Dimensional Simulation-Based Inference in Latent Spaces

Neural simulation-based inference (SBI) has been widely successful in inferring a relatively small number of interpretable parameters from potentially high-dimensional observations, such as images or time series. Accordingly, representation learning in SBI has focused almost exclusively on compressing the observations used to condition the posterior. More recently, however, SBI has begun to target increasingly high-dimensional parameter spaces, raising the complementary question of whether the inference target itself should be compressed. Our answer is a practical merger of SBI and latent generative modeling, which learns a low-dimensional representation of the simulator parameters, performs posterior inference directly in this latent space, and maps posterior samples back to the original parameter space. We characterize the conditions under which latent-space inference recovers the desired target posterior and systematically study its empirical trade-offs. Across four case studies and three generative families, we compare latent and standard estimators while controlling for network capacity, regularization, optimization, and training compute. At matched training compute, latent-space inference achieves accuracy and marginal calibration comparable to direct target-space inference while sampling up to more than an order of magnitude faster.
Read the full abstract

All authors Lars Kühmichel, Stefan T. Radev, Bhanu Prasanna Koppolu, Masoumeh Davoudi, Jerry M. Huang, Paul-Christian Bürkner

Statistical Methodology

The MCMC convergence graph: a diagnostic for multimodal posteriors

Multimodality is common in many scientific and engineering applications. However, standard Markov chain Monte Carlo (MCMC) convergence diagnostics often penalise multimodality in practice by conflating overall poor mixing with groups of chains that have mixed well within distinct modes. To diagnose multimodality rather than flag it as a sampling failure, we introduce the MCMC convergence graph: a graph-based diagnostic that computes pairwise R-hat values between chains and summarises them in a graph, revealing sets of chains that explore the same posterior mode. We implement the method in the probabilistic programming framework Stan and demonstrate its use on worked examples ranging from regression models with inherent multimodality in the data, to a pharmacokinetic model for which multimodality reflects non-identifiability. The graph-based diagnostic, which we make freely available in the R package mcmcConvergenceGraph and which accepts MCMC output from any sampler, provides both graphical and numerical summaries of MCMC behaviour for diagnosing multimodality.
Read the full abstract

Statistical Methodology

The INSIDE assumption under all positive coding: interpretation and partial empirical assessment

Mendelian randomisation (MR) implemented through instrumental variable (IV) analysis is a popular strategy for strengthening causal inference in observational studies. A key assumption for several MR estimators, including MR-Egger regression, is the INstrument Strength Independent of Direct Effect (INSIDE) assumption. However, there is no established empirical test for assessing the plausibility of this assumption. Moreover, INSIDE depends on how genetic variants are coded (i.e., on the choice of the effect allele), which is often arbitrary and therefore hampers assessing the plausibility of this assumption on substantive grounds. In this paper, we show that the all-positive coding scheme (i.e., for all variants, choosing the allele positively associated with the exposure as the effect allele), which is typically used in MR-Egger, is equivalent to a coding-invariant model that can be given a natural interpretation because the direct effect parameters under this coding scheme are in the same direction as the bias of individual-variant ratio estimators. Moreover, using both theoretical arguments and simulations, we show that, under commonly assumed data-generating models in the MR methodological literature, heteroscedasticity of instrument-outcome coefficients according to instrument-exposure coefficients is a feature of at least some types of INSIDE violation, indicating that heteroscedasticity tests could contribute to assessing the plausibility of the INSIDE assumption. We further highlight specific cases where the test would not work. We illustrate its application by re-analysing a real dataset assessing the causal effect of large particle high density lipoprotein cholesterol on age-related macular degeneration.
Read the full abstract

All authors Fernando Pires Hartwig, George Davey Smith, Frank Dudbridge, Jack Bowden

Machine Learning (Statistics) · Cross-listed from cs.LG

A Sharp Transition in Data Reconstruction under Differential Privacy

Data reconstruction attacks have empirically been successful in recovering training samples from learned models, raising privacy concerns and motivating defenses with guarantees that remain valid against future threats. While differential privacy (DP) provides formal protection, choosing the privacy budget remains a challenge: small budgets severely reduce utility, but it is hard to quantify how large the budget can be without allowing accurate reconstruction. In this work, we study informed attackers who aim to reconstruct a single $d$-dimensional training sample from a $\rho$-zero-concentrated DP model, knowing all other training data. Our main contribution is to establish a sharp transition at $\rho \asymp d$ for data reconstruction: on the one hand, we derive entropy-based lower bounds for any private mechanism and any attack, characterizing a set of target priors for which reconstruction is information-theoretically impossible for $\rho \ll d$; on the other hand, we analyze a simple attack on private linear regression with output perturbation, showing that reconstruction is practically feasible for $\rho \gg d$. Remarkably, the transition moves to $\rho \asymp s$ for data lying in an $s$-dimensional subspace, demonstrating that the privacy budget guaranteeing adequate protection must be assessed in terms of the effective dimension of the data. We validate our findings via experiments on synthetic data and natural images (CIFAR-10, ImageNet).
Read the full abstract

Statistical Methodology

Popularity Regression-Based Latent Space Model for Large-Scale Sparse Network Analysis

Degree heterogeneity is one of the most important properties of network data. It is widely observed that degree heterogeneity is often related to the nodal features. In this study, we investigate the estimation and statistical inference for a popularity regression-based latent space model with nodal features. Given the complex dependence structure induced by the latent space model, we aim to derive analytically tractable objective functions that effectively account for this structure for sparse networks. Specifically, we propose a total of four estimators. The first two estimators are developed by utilizing only the first-order structure of the network (e.g., the nodal degree), while the last two estimators are developed by leveraging the higher-order network structures (i.e., reciprocity and transitivity). Rigorous asymptotic theory is established based on various non-standard U-statistics. We find that different estimators might have different convergence rates. The extension to higher-order moments-based estimators is also discussed. Extensive numerical experiments and a real data analysis of link prediction for an author citation network are conducted for illustration purposes.
Read the full abstract

All authors Zhan Gao, Danyang Huang, Rui Pan, Hansheng Wang

Statistical Methodology

When Stress Tests Miss the Risk: Statistical Scenario Analysis for Financial Portfolios

Scenario analysis is widely used to stress test financial portfolios, yet conventional approaches often summarize scenario gains using point estimates that overlook dependence between stressed and unstressed risk factors. We develop a machine learning framework for quantifying uncertainty in realized next-day scenario gains through prediction intervals whose widths can be calibrated online. Adaptive conformal scenario analysis (ACSA) calibrates scenario-specific quantile predictions and provides a long-run empirical coverage guarantee over realized scenarios. Our main method, kernel scenario analysis (KSA), estimates scenario-conditional quantiles directly from the specified stress and current market information. KSA can also be combined with ACSA to obtain online-calibrated prediction intervals. In experiments designed to reflect real-world markets, conventional scenario analysis can substantially understate risk, including for portfolios that appear safe under standard stress tests. Our proposed methods achieve better calibration and sharper intervals than empirical-quantile baselines. Overall, the framework moves scenario analysis beyond point estimates by quantifying predictive uncertainty and providing tools for statistically validating scenario gains.
Read the full abstract

Other Statistics

Evidence Over Ideology: A Quantitative Assessment of LTNs and Blanket 20 mph Regimes in London

We evaluate the road safety implications of Low Traffic Neighbourhoods (LTNs) and blanket 20 mph speed limits in London using injury-collision records (2020-2024). Because London implemented 20 mph limits primarily via sign-only orders without physical traffic calming, this work evaluates the blanket statutory model rather than lower urban speeds in principle. For LTNs, we apply a causal pipeline combining naive pre/post comparisons, Empirical Bayes shrinkage for Regression to the Mean (RTM), and spatial Difference-in-Differences. Naive comparisons suggest substantial crash reductions, but Empirical Bayes adjustment reveals much of this reflects mean reversion. After correction, only one zone exhibits a statistically significant inside-area reduction. We find no consistent evidence of boundary-road crash displacement nor uniform inside safety gains. Given the modest corrected benefits, small journey-time increases on boundary roads (approximately 10 seconds per vehicle) monetarily negate safety gains. For 20 mph versus 30 mph roads, five associational methods across 14 contexts show that the aggregate difference in the fatal or serious (KSI) collision share is negligible (0.6 percentage points). Controlling the false discovery rate, four contexts display robustly higher KSI shares on 20 mph roads: A-roads, junctions, pedestrian crossings, and single carriageways, with no context showing a robust reduction. In movement corridors where road geometry cues higher speeds, sign-only limits fail to self-enforce. These findings highlight a structural "wrong-road problem." Indiscriminate sign-only limits impose economic drag without proportionate safety returns. Transport authorities should move away from blanket statutory defaults and return to targeted physical engineering on high-harm corridors.
Read the full abstract

Statistical Methodology

Bayesian inference of risk differences for multi-group bilateral data

Bilateral data from paired body parts are common and correlated in clinical studies. Classical frequentist methods perform poorly in small or sparse datasets. This paper develops a Bayesian framework for bilateral data with multiple groups under Dallal's model. We derive three objective priors (uniform, Jeffreys', and Bernardo's reference priors) and propose a range-based posterior testing procedure, combined with a decision rule, to test the homogeneity of risk differences, with the equivalence margin calibrated. Monte Carlo simulations evaluate empirical Type I error rates, powers, and interval estimation properties. Results show that the Bayesian methods achieve accurate coverage probabilities, narrower confidence intervals, and better Type I error control than the frequentist Wald approach, especially in small-sample and sparse-data settings. We illustrate the methodology with two real datasets. The proposed framework provides a robust and flexible tool for multi-group bilateral data analysis.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Interacting particle guidance for sampling reward-tilted generative priors

Inference-time steering adapts pretrained diffusion and flow-based models to new tasks, e.g., to generate samples from a conditional distribution or samples with desired properties, without retraining. This can be formalized as sampling from a reward-tilted generative prior. As exact sampling from this distribution is intractable, guidance-based methods rely on approximations producing biased samples, and sequential Monte Carlo (SMC) methods correct for this bias using importance weights. However, while exact in the large particle limit, SMC suffers from weight degeneracy and particle collapse in practice. We propose interacting particle guidance (IPG), which replaces reweighting with transport. The particles interact through an additional drift, derived from the Feynman–Kac PDE to cancel the reweighting term, and remain unweighted. Choosing the drift in a reproducing kernel Hilbert space yields a closed-form solution that is cheap to compute, with negligible overhead compared to SMC. We demonstrate the method on Gaussian mixtures with known posteriors, and on high-dimensional image inpainting and protein structure inference tasks.
Read the full abstract

All authors Adhithyan Kalaivanan, Zheng Zhao, Jens Sjölund, Fredrik Lindsten

Machine Learning (Statistics) · Cross-listed from cs.LG

Pointwise or Pairwise: When Do Pairwise Losses Help Reward Learning, Provably?

Pairwise losses are increasingly used for reward learning even when pointwise rewards are observed, with mixed empirical results. When and why do pairwise losses outperform pointwise losses? We study this question in a grouped offline contextual-bandit setting allowing multiple actions per context, capturing many reward learning scenarios. We compare Value Regression (VR), which regresses observed rewards pointwise, with Value Difference Regression (VDR), which regresses reward differences between a pair of actions sampled under the same context. We consider a semiparametric model where the mean reward is the sum of a learnable action-dependent component and an arbitrary context-dependent yet action-independent nuisance, capturing context-specific disturbances. Using a unified localized analysis, we prove finite-sample regression guarantees for finite and linear function classes and translate them into offline-regret bounds. For finite classes, VDR eliminates the misspecification term in the VR bound and improves a reward-scale-dependent error term by averaging over actions within each context, a benefit absent from the corresponding VR term. For linear classes, neither method uniformly dominates: within-context differencing removes nuisance-induced bias but may increase estimation variance relative to using absolute rewards when the misspecification is sufficiently low. This yields a feature geometry-dependent bias-variance tradeoff, which we corroborate with numerical experiments.
Read the full abstract

All authors Junghyun Lee, Minsoo Ha, Sanghwa Kim, Yeongjong Kim, Eunjee Lee, Seiyun Shin, Kwang-Sung Jun

Statistics Applications · Cross-listed from econ.EM

Are there asymmetries in euro area monetary policy transmission?

We answer the question posed in the title with a nonlinear mixed-frequency vector autoregression, estimated with Bayesian additive regression trees. The model combines monthly macro-financial variables with quarterly bank lending survey data, and identifies the dynamic responses from high-frequency policy surprises. Sign asymmetry dominates; a tightening produces monetary policy transmission mostly in line with the theoretical predictions, whereas easing of any size produces mostly insignificant responses. Peak effects vary with initial conditions, while several common, predefined regime splits do not yield considerable differences in the propagation of the shocks.
Read the full abstract

Statistical Methodology

CovEsts: Nonparametric Autocovariance Estimation and Analysis

The paper introduces the R package CovEsts, which implements several nonparametric estimators for the autocovariance function. First, it presents the theoretical foundations of the implemented estimators, their properties, assumptions, and potential limitations. Next, it outlines the structure of the package and its key functions, including several methods for estimating autocovariance functions, constructing corresponding bootstrap confidence regions, and correcting the provided estimators. The package also includes diagnostic tools, such as several metrics for comparing estimators, and additional functions for broader use. The article illustrates a high degree of flexibility of the package in the selection of function parameters and the tuning of the estimators. Applications of selected estimators and package functions are illustrated using simulated data, yearly sunspot counts, and US unemployment increments data.
Read the full abstract