Papers

Electronic health record (EHR) phenotyping algorithms support clinical prediction, observational research, and quality improvement, but manual chart review for gold-standard validation is costly. For algorithms targeting rare outcomes, algorithm-negative patients often greatly outnumber algorithm-positive patients. We propose a design-based validation framework that fully verifies algorithm-positive patients and reviews a simple random sample (SRS) of algorithm-negative patients. Horvitz-Thompson estimators of false-negative (FN) and true-negative totals yield estimates of sensitivity, specificity, negative predictive value, positive predictive value, and accuracy. We derive finite-population-corrected variance estimators and confidence intervals (CIs), with sample-size planning for a prespecified CI half-width. An optional Phase 2 extension uses pilot data to assess whether risk-guided Neyman allocation may improve precision. In simulations, the design-based approach substantially reduced bias relative to naive verified-sample analyses and achieved near-nominal CI coverage. Adaptive allocation reduced root mean squared error and CI width when FNs were concentrated across risk strata, but offered little benefit when the risk signal was weak. We illustrate the framework using a breast cancer recurrence algorithm in 3,336 patients, with endpoint-specific algorithm-negative review budgets of 125, 150, and 175 charts. The framework combines SRS-based validation and precision planning with optional pilot-guided adaptive allocation.

Despite their empirical success, why diffusion models overcome the bottlenecks of classical score-based samplers remains unclear. In this work, we leverage Gaussian distributions to isolate this phenomenon. We establish 2-Wasserstein convergence bounds for optimized hyperparameters, showing that diffusion processes achieve a sampling error of $O(\sqrt{d\lambda_{\max}}\log N/N)$, where $d$ is the dimension, $N$ the number of sampling steps, and $\lambda_{\max}$ the largest eigenvalue of the target covariance matrix. Unadjusted and underdamped Langevin dynamics suffer from an additional $\sqrt\kappa$ factor, where $\kappa$ is the condition number. These rates follow from spectral bounds which are sharp: we confirm them via matching first-order asymptotics as $N\rightarrow\infty$. Our analysis provides a rigorous characterization, in the Gaussian setting, of how time-dependent score trajectories remove condition-number dependence during sampling. By contrast, in the learning phase, we show that estimating the unnoised score by gradient descent leads to essentially the same estimator as estimating a noisy score, which suggests that the benefits of noising do not come from the learning phase.

Archetypal Analysis (AA) represents observations as convex combinations of extremal data-driven profiles, yielding interpretable low-dimensional descriptions of complex datasets. Classical AA relies on a least-squares objective, which is poorly suited to discrete observations such as binary, count, and categorical data. We introduce an efficient likelihood-based framework for AA supporting Bernoulli, Poisson, and multinomial observation models. Our optimization scheme employs local quadratic approximations of the negative log-likelihood, enabling constrained updates through sequential minimal optimization (SMO) and an active-set method. Scalability is improved by bounding the active set while preserving simplex feasibility. We further introduce a cross-validated predictive likelihood criterion for selecting the number of archetypes, providing a principled alternative to reconstruction-error heuristics and stability-based diagnostics. Synthetic experiments demonstrate computational efficiency and accurate recovery of model complexity. Applications to single-cell RNA sequencing, microbiome composition, and somatic mutation data show that the learned archetypes capture interpretable domain-specific structures while achieving competitive likelihood fits and stable solutions. Overall, the proposed framework enables efficient likelihood-based archetypal analysis of discrete data, complemented by predictive likelihood-based model selection.

We detail scalable methods for approximating the quadratic entropy $p^T d p$ for arbitrary distributions $p$ and common distances $d$ of negative type. We focus on the Euclidean and spherical geodesic cases, which both use random feature embeddings and projections to dramatically improve computational complexity within a simple framework. Amortization of a single large matrix multiplication and control variates further enable computation at large scale with low memory and runtime in situations where $d$ is held constant while $p$ varies. We demonstrate this with a comparison against direct pair sampling and bibliometric/scientometric examples on Open Graph Benchmark datasets, revealing papers, fields, and institutions with both particularly narrow and broad interdisciplinary reach from their citations and text features alone.

Graphical approaches to causal abstraction transform a low-level causal directed acyclic graph (DAG) over many measured variables into a smaller, high-level DAG whose nodes cluster the original variables and whose edges summarize the causal relations between clusters. Such cluster DAGs are easier to interpret, but learning them requires finding the clusters and recovering the edges between them. Madaleno et al. (2026) learn the interventional coarsening (the cluster DAG that merges variables the interventions cannot distinguish) in two constraint-based phases: first the clusters, then the edges. We introduce COARSE, the first score-based method for this task: it keeps the two-phase structure but, under linear Gaussian assumptions, swaps the constraint-based edge phase for a score-based one. We show that the interventions themselves identify a causal order over the clusters, and learning the edges reduces to a single local search per cluster under a cluster-level BIC score. We prove that the procedure runs in polynomial time and, provided the variables affected by each intervention are correctly identified, that it is consistent. On synthetic and real-world interventional data, COARSE matches state-of-the-art edge recovery given enough samples, with an edge phase up to two orders of magnitude faster, including on dense graphs with hundreds of nodes.

From previous results it follows that probability distributions on metric spaces featuring unique Fréchet means are dense among measures admitting means in both the quadratic Wasserstein metric and the total variation metric. Additionally, we show that the converse also holds on complete finite-dimensional noncontractible Alexandrov spaces with curvatures bounded from below, which encompass compact manifolds without boundaries and nonmanifold shape spaces: Probability distributions featuring nonunique Fréchet means are also dense in both the Wasserstein metric and the total variation metric. Moreover, in either metric, no nonempty open set of probability measures admits a continuous selection of means. This sharpens a previous result on zero reach of the Dirac embedding. Our argument combines cut-locus avoidance for atoms with a topological obstruction. For Riemannian manifolds, all of our conclusions hold for all exponents $1 < p < \infty$, also.

We consider the problem of learning linear dynamical systems under adversarial contamination from a single trajectory of length $T$. While identification of linear dynamical systems itself is well-studied, the problem of robust system identification under adversarial contamination is relatively less explored. In this work, we study the setting where a fraction of the $T$ observations are contaminated by adversarial outliers. We propose different estimators based on relaxations of least-trimmed squares along with an alternating minimization algorithm. Furthermore, we also propose two estimators which exploit the group-sparsity (through penalization/hard-constraints) of the outliers. For the estimator with group-sparse penalty, we derive non-asymptotic error bounds which establish its robustness to outliers. We also show empirically that the proposed estimators work well in practice.

We consider the problem of learning structured linear dynamical systems over convex sets $\mathcal{K}$, where only a small subset of the observations are available at each time point. An estimator which minimizes a bias-corrected, potentially non-convex objective function is proposed. Non-asymptotic bounds are obtained for the statistical error, which depend on the local complexity of $\mathcal{K}$, the trajectory length $T$, and the sub-sampling probability $p$. Convergence of the projected gradient descent algorithm is also established. The general theory is applied to settings where (i) $\mathcal{K}$ is a subspace, (ii) $\mathcal{K}$ is the set of bi-isotonic matrices, and (iii) $\mathcal{K}$ is the set of matrices whose rows are formed by sampling Lipschitz functions. We show meaningful recovery of the transition matrix is possible for values of $T$ much smaller than what is required in the unconstrained case, and for $p = o(1)$.

Kernel Stein discrepancies (KSDs) provide a versatile tool for comparing distributions. One of their main applications is in quantifying the goodness-of-fit (GoF) between a data-generating distribution and a prescribed target distribution. In this work, we study the related problem of conditional GoF quantification: given only a (possibly non-normalized) conditional target model, without information on the distribution of its covariates, and samples from a joint distribution, the goal is to assess how well the conditional distribution of the samples matches the target. To tackle this setting, we present a framework that allows lifting unconditional KSDs to the conditional setting through an operator-valued kernel on the covariate space, going beyond the known Euclidean case. We establish that our suggested statistic vanishes if and only if the conditional model and the true conditional distribution agree for almost all covariates and deploy it to test conditional GoF on smooth manifolds and on discrete spaces. Our experiments on level, power, and runtime demonstrate the viability of testing on these domains using the proposed statistic.

Accurate covariance estimation is crucial for spatial data analysis. While parametric methods can suffer from model misspecification leading to wrong conclusions, nonparametric approaches are often neglected in practice as they rely on the estimation of a large number of covariance parameters and often face finite-sample bias issues. In this paper, we study the bias properties of sample covariance estimators for stationary lattice processes on $\mathbb{Z}^2$, when the true mean parameter is unknown and has to be estimated. We derive exact formulas for the finite-sample biases of sample covariance estimators and show that their expectations are linear combinations of population covariances determined by spatial lag and sample size. Based on this characterization, we propose jointly bias-corrected covariance estimators that are nearly unbiased. Additionally, we derive formulas for the mean-squared error and prove asymptotic normality results that show asymptotic equivalence for the estimators with and without bias correction. Simulations demonstrate substantial reductions of bias and often also in terms of MSE, particularly under strong spatial dependence. A kriging application to topography data further illustrates the practical benefits of the proposed approach.

A compromised LLM inference server can leak model weights by encoding payload bits in otherwise plausible token choices. A replay of the same prompt in a trusted server can expose such deviations, but benign numerical nondeterminism also causes token mismatches. Patient attackers can therefore hide within normal variation unless evidence is combined across responses. We introduce a prompt-level e-process that calibrates whole-response mismatch events on trusted benign traffic and accumulates evidence sequentially while, under a calibration-transfer assumption, controlling the probability of any false alarm over an unbounded monitoring horizon. We evaluate it on four models against a seed-blind attack and a stronger seed-aware attack that hides payload bits only in near-ties to remain stealthy, analyzing the channel capacity vs detectability trade-off. Compared with a hard per-token alarm, the e-process combines weak evidence across responses while providing explicit anytime false-alarm control.

We present the postshock R package, which implements and extends a donor-based framework for forecasting when a structural shock is known and the target response of interest is not yet observed. The package estimates shock effects from historical donor episodes, balances donors using specified matching features, and transfers the resulting adjustment to a target-series forecast. It provides integrated workflows for conditional mean forecasting through ARIMA and ARIMAX models and conditional variance forecasting through GARCH-X models. Additional functionality includes structured donor pools, control-shock regressors, automated GARCH-X order selection, processed data objects, and reproducible empirical workflows.

Conventional statistical modeling typically assumes a one-to-one mapping between input and output variables, where one-to-one is used in the predictive sense that a specified input results in a unique output. Many scientific and engineering systems, however, exhibit non-one-to-one(noto) input-output relations. In a noto relation, the same input may correspond to multiple admissible outputs, so the conventional statistical models may not be appropriate. This paper develops manifold regression, a parametric modeling framework for such noto input-output relations, which represents the underlying input-output relation through a latent manifold. The manifold is specified upto an unknown finite dimensional parameter and we estimate the unknown parameters from ordinary input-output observations by regularized profile optimization with a robust solver. Once the manifold has been learned, prediction is obtained by slicing the estimated manifold along a specified coordinate value. Because a slice may contain multiple latent roots, the resulting manifold prediction is naturally set-valued. This formulation extends predictive learning beyond conventional one-to-one statistical models while containing classical regression and inverse prediction as special cases. Theoretical results are established to justify the manifold regression model and its sliced prediction sets; and case studies are used to demonstrate stable performance across representative noto cases.

Softmax attention, at the heart of Transformers, has demonstrated remarkable capabilities. Yet its underlying mechanisms remain only partially understood. Recent theoretical work studies Gaussian prompts, where the infinite-prompt limit reduces softmax attention to a linear map, but also removes the query-dependent selection that distinguishes it from linear attention. This work studies the infinite-prompt limit of softmax attention on Gaussian mixtures, which retain the tractability of Gaussian data while introducing latent structure, multimodality, and nonlinear dependencies. We show that softmax attention can represent and learn, via gradient-based methods, optimal solutions to a range of statistical tasks, including supervised classification and denoising. Our results highlight two complementary capabilities of softmax attention: it can recover linear tasks as effectively as its simpler linear counterpart, while also exploiting query-dependent context selection to solve nonlinear tasks beyond the reach of linear attention.

Zador's celebrated theorem is a cornerstone of optimal quantisation: it establishes both the weak limit of the empirical distribution of an optimal $n$-point quantiser in $R^d$ and the decay rate of the associated $L_s$-mean quantisation error. In large dimension, however, observing this asymptotic behaviour requires an astronomically large sample size. We prove that, for spherically symmetric target distributions, optimisation over all spherically symmetric distributions is a convex problem and derive an equivalence theorem that both characterises global optimality and yields a constructive algorithm. We show that, for moderate $n$, random quantisers uniformly distributed on a sphere of suitably chosen radius $R$ perform exceptionally well and, over a broad range of values of $n$, are numerically certified to be optimal among all random quantisers. Their expected distortion has an explicit integral representation that can be evaluated to arbitrary precision, and we prove concentration across random quantisers: the distortion variance tends to zero as $n\to\infty$ for fixed $d$. For $s=2$, both the optimal radius and the associated minimum expected distortion admit exact expressions. For general $s$, the optimal radius can be determined efficiently, and extreme-value theory provides useful approximations when $n$ grows with $d$. Depending on this growth rate, $R$ either converges to zero or approaches a positive limit that is independent of $s$.

We study the multivariate convolution model with additive independent noise, multidimensional signal and noise, aiming to recover the signal's cumulative distribution function from contaminated observations. The noise is ordinary smooth with anisotropic regularity, and the signal belongs to an anisotropic Nikol'skii density class. We extend to the multivariate setting an approximate minimum \(L^1\)-distance estimator based on integrated kernel density estimation, and establish matching upper and lower bounds for the \(L^1\)-risk over anisotropic Nikol'skii classes. To our knowledge, this is the first minimax rate, not a sharpening of existing bounds. Although the lower-bound scheme is classical, the test-function construction is novel: it distinguishes no active component from at least one active component. The rate is governed by a censoring mechanism through the positive part, a noise filter in a harmonic-mean representation of the exponent. We further propose a fully data-driven, rate-adaptive procedure selecting an optimal bandwidth vector over the full Nikol'skii scale. On Nikol'skii product density classes, the \(L^1\)-distance between cumulative distribution functions and the \(1\)-Wasserstein distance share the same minimax rate, though not equivalent in general, via coordinate-wise decoupling of the \(1\)-Wasserstein cost on product measures. These findings reveal a link between the two distances, whose full understanding remains an open question.

This paper examines job waiting times on the Jean Zay supercomputer at IDRIS (Institut du Développement et des Ressources en Informatique Scientifique http://www.idris.fr/), driven by the need to improve scheduling strategies for crossfacility scientific workflows. Drawing on large workload traces, we show that job waiting times are not independent of one another but are instead correlated through job bursts-batches of identical jobs submitted together, also called “flurries”. We introduce a method for quantifying the effect of these bursts on waiting times, and find a strong Pearson correlation between waiting time and a job's rank within its burst. This correlation is above 0.8 for more than 40% of the jobs. We further derive an exact formula for the Slurm fairshare factor based on a convolution product, showing that the burst penalty operates on a much shorter timescale than fairshare itself-pointing to a distinct throttling mechanism specifically targeting bulk submissions. Once burst effects are accounted for, we use Welch's t-test to find that partition occupancy and number of requested cores have a significant effect on waiting time, though the effect is not significant for all partitions.

Reliable predictive uncertainty in Laplace approximations depends critically on the prior precision, yet selecting it requires a posterior sweep that is prohibitively expensive for neural networks with billions of parameters. Under the Maximal Update Parametrization ($\mu\mathrm{P}$), we derive a rescaling of the prior covariance that makes the selected precision stable as model width grows. This leads to $\sigma\mathrm{Transfer}$: we select the precision on a smaller model and zero-shot transfer it to the much larger model, i.e., without searching for the precision on the larger model at all. We show convergence of the prior kernel, posterior covariance, selected precision, and posterior-derived decisions under explicit conditions, and verify $\sigma\mathrm{Transfer}$ across regression, image classification, and Transformer readouts. For example, measured precision-sweep speedups reach $\sim 5000\times$ when transferring from width 128 to 4096 on MNIST, at a target-NLL degradation of $0.002$; transferring from a public 1B to 7B model gives a median search speedup of $\sim 2.3\times$ (up to $\sim 330\times$), with a mean measured target-NLL increase below $10^{-4}$ across ten tasks. The same posterior stability also enables transfer of acquisition, OOD-detection, and abstention decisions without constructing a target posterior.

Machine unlearning updates a trained model after records are deleted, aiming to match exact retraining without repeating the full training procedure. We develop Gaussian mechanisms for Newton updates under sequential deletion requests. Using Gaussian differential privacy (GDP) and its adaptive composition rule, we show that the full sequence of released models is statistically difficult to distinguish from matched exact retraining. To calibrate these mechanisms for empirical risk minimization, we derive upper bounds on the error of the Newton approximation relative to exact retraining and on how this error changes after each deletion batch. Independent Gaussian noise is calibrated using bounds on the full residual at each release, whereas Gaussian random walk noise uses smaller bounds on residual increments. These bounds yield allocations minimizing the worst-case maximum noise variance across releases under the resulting GDP certification constraints. With count-based bounds, the random walk asymptotically matches the worst-case variance of a single release at deletion cap $M$, while independent noise incurs an additional factor of order $M$. Set-based bounds can reduce the noise variances by using gradients and Hessians of the deleted records. For singleton deletion, we further show that count-based independent noise, count-based random walk noise, and set-based independent noise are minimax among fixed Gaussian covariances under their respective residual or increment bounds. With set-based bounds, allowing variances to adapt to deleted records can improve on every fixed covariance by a factor of order $(\log M)^2$ on some data sequences. The residual and noise bounds also yield parameter and predictive consistency relative to exact retraining, uniformly over deletion policies. Simulations and a credit default data analysis evaluate bounds, noise variances, and estimation errors.

A watermark detector can flag a text even when only a small fraction of its tokens are watermarked. We study sequential auditing to identify local passages whose watermark proportions exceed a prespecified threshold. Each alarm fixes a candidate passage's endpoint, while its start is chosen from earlier tokens. Reusing the same evidence to locate and test a passage can invalidate inference. We propose FissionGuard, which uses data fission to construct an alarm value and a validation value from each token-level $p$-value. The alarm values guide alarms and localization. The validation values yield passage-level $p$-values through partial conjunction, excluding as many of the strongest signals as the proportion null allows. These $p$-values are used to test the corresponding hypotheses at levels assigned by an online multiple-testing procedure. Under the stated watermark generation model, allowing dependence among token-level $p$-values at watermarked positions, we prove finite-sample false discovery rate (FDR) control among reported passages at every fixed finite horizon. We also derive a finite-sample lower bound on power for testing nonnull candidates. Experiments on synthetic streams and Mistral and Qwen generations across four marking layouts show gains in watermarked-token coverage over sample splitting, with empirical FDR below the target.

An agent learns from data that many parameters of her model explain equally well. Berk's theorem states that her belief concentrates on the parameters that best fit the data, but does not select between them. We show that, with a continuum of parameters, the posterior concentrates on the best-fitting parameters with the smallest local learning coefficient. We propose a definition of narratives, subsets of the best-fitting parameters on which the learning coefficient is constant. Narratives are characterized by the number of coincidences they require (complexity) and how exactly each must hold (robustness). When the agent's actions affect her data, every long-run action is a singular equilibrium, a best reply to a belief supported on the least complex and most robust narratives. Singular equilibrium refines Berk-Nash and self-confirming equilibrium by restricting overly complex or knife-edge off-path beliefs, and its strict uniform version characterizes uniformly stable actions.

We study inference on the strength of Gaussian multiplicative chaos from one geometric graph with unobserved vertex positions. In the planar model, the edge-count statistic has a piecewise deterministic limit with a transition at $\gamma=1$, whereas degree quantiles consistently estimate $\nu=\gamma^2/2$ throughout the subcritical range. Fractional moments give uniform finite-sample risk bounds. At finite resolution, common graph envelopes calibrate tests over continuous parameter boxes with unknown Poisson intensity. They give simultaneous coverage under adaptive refinement for a fixed statistic, and conditional coverage for an independently piloted size–degree residual on a fixed partition. For the exact periodic FFT–pixel approximation, we prove explicit total-variation rates for the annealed spatial Cox law and the observed graph law, uniformly on every compact subcritical range $0\le\gamma\le\Gamma<2$. The rates allow intensity to grow with resolution and imply uniform asymptotic coverage after recalibration at each resolution. Numerical studies examine parameter refinement, residual calibration, and multiresolution stability.

To test conditional independence of $X$ and $Y$ given a text or an image $Z$, one conditions on an embedding $\psi(Z)$ in place of $Z$. The embedded test is valid if $Z$ is independent of $X$ or of $Y$ given $\psi(Z)$, which cannot be confirmed from data, and when this fails, the rejection probability under the null hypothesis can tend to one. We study this failure, and show that focusing on a specific form of dependence relaxes what the embedding must retain. For a residual correlation test inspired by the Generalised Covariance Measure, validity only requires that the parts of $\mathbb{E}[X \mid Z]$ and $\mathbb{E}[Y \mid Z]$ missed by $\mathbb{E}[X \mid \psi(Z)]$ and $\mathbb{E}[Y \mid \psi(Z)]$ are uncorrelated. Otherwise, we treat the discarded information as an omitted variable. Under the null hypothesis, the bias equals the absolute correlation of the missed parts times the geometric mean of two partial $R^2$ values. This identity yields a robust test valid under a declared tolerance for the geometric mean, which, like a sensitivity parameter, is not identified from the data. On synthetic data and text embeddings, the robust test holds its level approximately. On text generated by a language model, under an exact null hypothesis, every embedding, even the generator's own states, biases the embedded test.

Spatio-temporal tracking data has opened new possibilities for detecting complex tactical patterns in football, yet modeling the interactive movements of multiple players remains challenging. This paper proposes a framework combining graph neural networks (GNNs) with a sequential model to classify match phases on a second-by-second basis across a seven-class taxonomy. Match phase classification is tactically meaningful, and the availability of rule-based labels across 203 matches makes it a suitable testbed for a systematic comparison of adjacency constructions and message-passing layers, a question that has received limited attention in existing research. Our selected GNN-LSTM model outperforms all aggregated-feature baselines, including XGBoost and a Long Short-Term Memory (LSTM) network, as the strongest baseline scores 4.6% lower in macro F1. Graph representations using a domain-informed Delaunay triangulation that approximates passing lanes, paired with a custom Spatial Edge-Augmented Convolution (SEAConv) layer that injects edge attributes directly into messages, achieve the best performance by capturing spatial dependencies while limiting uninformative messages from redundant edges. Integrated Gradients attributions indicate the model uses spatial player configurations, particularly horizontal positioning, in combination with possession and ball-status indicators. This work offers an automated solution for fine-grained tactical analysis, reducing the need for manual tagging and providing deeper insight into dynamic team behavior.

Deep randomized models fix hidden-layer parameters through random initialization and learn only closed-form readouts, typically adding depth by stacking random trans formations without target-aware control of hidden-state evolution. We propose LAIR Net, the Leaky Alignment-Impulse Residual Network, which mixes a shallow learned anchor into each hidden state through a leaky residual transition. We derive a depth uniform bound on input-perturbation sensitivity and use controlled simulations to attribute gains over a randomized baseline to the anchor rather than recursion or added capacity. Benefits emerge when a nonlinear target structure is learnable at the available noise level and diminish for nearly linear targets or dominant noise. Across 23 benchmark datasets, LAIR-Net achieves the best average rank among eight randomized networks and twelve conventional models, with relative performance associated with the same nonlinear-structure and noise quantities identified in simulation.

Sparse identification of nonlinear dynamics (SINDy) is a data-driven framework for discovering governing dynamics from time-series data by identifying a sparse subset of candidate dynamical terms from a prespecified library. In this work, we develop a statistical inference framework for quantifying the reliability of dynamical terms selected by SINDy through hypothesis tests and confidence intervals. A key difficulty is that using the same noisy trajectory for both selecting dynamical terms and assessing their statistical significance can introduce selection bias. Post-selection inference provides a principled framework for addressing such bias, and we propose PSI-SINDy, a post-selection inference method tailored to SINDy. Direct application of existing post-selection inference techniques is challenging because SINDy involves measurement error in the candidate terms and shared noise between the response and design. To address these challenges, PSI-SINDy uses data thinning to decompose a single observed trajectory into four mutually independent views with distinct roles in selection and inference. This construction enables inference for selected dynamical terms while accounting not only for selection bias but also for measurement-error and shared noise effects. We establish the theoretical validity of PSI-SINDy under stated conditions and evaluate its performance through numerical experiments on simulated and experimental dynamical-system data.

Population gradient flow is a common tool for reasoning about how neural networks adapt, including after pretraining. We show that it can mispredict finite-batch stochastic gradient descent (SGD) qualitatively, and we trace the discrepancy to a specific mechanism. In a two-unit ReLU regression, a source task drives the two neurons toward positive proportionality and a target task rewards separating them. After source training for time $T$, gradient flow recovers on the target in time linear in $T$. Online SGD with batch size $b$ and step size $\eta$ in both phases instead fails with high probability throughout a horizon of order $e^{c/\eta}$ once $T \gtrsim \log(b/\eta)$, uniformly on an explicit set of initializations with Gaussian probability above one percent. For each fixed $T$, small-step SGD still recovers, so the failure requires the joint limit of small steps and long pretraining. At the target clone, the population instability is carried entirely by inputs on which the two ReLU gates disagree. For units at angle $\delta$ these inputs form a wedge of probability $\delta/\pi$, and weight decay shrinks the angle exponentially during pretraining. On every other input both units receive the same random linear update, which contracts their separation in conditional expectation. Bounding the cumulative probability of sampling the wedge along the exact online recursion, without a diffusion approximation, shows that recovery with fixed probability from an identical source-gradient-flow checkpoint, within $e^{c/\eta}$ updates, requires $Nb \gtrsim e^{\lambda T}$ target samples and batch size $b \gtrsim \eta e^{\lambda T}$, where $N$ counts updates and $\lambda$ is the weight decay. In simulations, recovery is approximately a function of the disagreement budget $b\delta/\eta$ and saturates in the horizon.

Outside the linear-Gaussian regime, conditional sampling from Gaussian processes (GPs) is challenging. Recent methods such as FlowGP (Moss et al., (2026)) can condition on arbitrary non-linear and non-Gaussian statements, but at considerable cost: an expensive iterative and high-dimensional diffusion that requires hand-specified kernel hyperparameters. In this paper, we alleviate two significant drawbacks of FlowGP by (1) introducing kernel approximations that enable scaling to high-resolution domains and (2) proposing a way to obtain the marginal likelihood by measuring the work needed to steer the diffusion towards conditioning statements. We enable, for the first time, hyperparameter optimisation within FlowGP and demonstrate our approach on probabilistic downscaling from areal summary statistics, PDE solution inference on irregular domains, and recovery of sea level anomaly fields from non-Gaussian satellite observations.

Synthetic tabular generators are commonly optimized for distributional fidelity, but statistical similarity alone does not guarantee preservation of causal effects. In this paper, we study whether causal fidelity can be improved directly within a fully generative tabular model. Causal Fidelity is defined with respect to a target estimand as the discrepancy between inferential distributions obtained from real and synthetic data, and theoretical results show that high statistical fidelity does not generally imply high causal fidelity. We then propose a causal-fidelity-aware training framework which adds a causal discrepancy penalty to the generative objective. The framework is instantiated with a causal-penalized TabDDPM and optimized using an on-policy score-function estimator. We further establish conditions under which causal regularization improves expected causal fidelity. Experiments across diverse treatment-effect simulations and two benchmark datasets evaluate the ability of our method to improve causal fidelity while preserving competitive statistical fidelity.

Many applications of language models hinge not on individual samples but on the expectation of a test functional under the model. Estimating such expectations reliably can be computationally expensive. In this paper, we show how to make estimation more efficient by exploiting the next-token conditional probabilities which are available as a by-product of sampling. We do so through potentials: real-valued functions on prefixes that decompose the test functional additively. We construct an estimator whose variance depends on the chosen potential, and derive conditions under which a potential reduces this variance. We then develop practical potentials for several estimands and applications, and demonstrate substantial variance reductions across several estimands at comparable computational cost.