Papers

Gradient-based attribution provides input-specific information about model parameters, but computing it via backpropagation for many samples is expensive, while static importance scores discard input dependence. We derive KPSA (Kernelized Parameter Sensitivity Atlas) from a block decomposition of the empirical neural tangent kernel. It measures normalized parameter-group sensitivity, embeds those sources in an explicit query space, and uses a lightweight kernel to estimate sensitivities for unseen inputs. We derive bounds that separate finite-source from irreducible representation error and give margin-based top-$k$ recovery. This paper studies two interpretability applications. Firstly, KPSA identifies parameter groups with high susceptibility to perturbations across vision, forecasting, and protein-modeling setups. Across a fivefold range of parameter budgets ($0.02$-$0.1\%$), its selected groups attain $39$-$85\%$ of the perturbation susceptibility of a first-order backpropagation oracle. In vision and forecasting, these parameter groups form sparse sensitivity circuits that extend across depth and vary systematically with the input. Secondly, we use KPSA on an activation-attribution problem for MLP features in transformers. It retains $81\%$ of the gap between a backpropagation oracle and a constant-sensitivity baseline on vision while increasing batched query-time throughput from 551 to 2,041 images/s and reducing peak GPU memory from 4.55 to 0.44GB. On the protein-modeling task, it recovers $68\%$ of this gap. An ablation of KPSA's kernel aggregation shows that nearest-neighbor retrieval in KPSA space can remain competitive. The empirics show that compact kernel atlases can store useful input-dependent parameter sensitivity while amortizing repeated gradient attribution, a step toward scalable conditional interpretability. Code can be found at https://github.com/Axym-Labs/kpsa .

We establish a nonasymptotic Gaussian approximation bound over centered Euclidean balls for sums of independent, not necessarily identically distributed, random vectors in \(\mathbb R^p\). Let \(S_n\) denote the sum, let \(\Sigma=\operatorname{Var}(S_n)\), and let \(G\sim N(0,\Sigma)\). Define the effective rank and the normalized squared spectral mass by \[ r(\Sigma)=\frac{\operatorname{tr}\Sigma}{\|\Sigma\|}, \qquad v(\Sigma)=\frac{\operatorname{tr}(\Sigma^2)}{\|\Sigma\|^2}. \] Under a uniform directional fourth-moment condition, covariance-balance assumptions, and the spectral conditions \(r(\Sigma)\le C_\Sigma v(\Sigma)\) and \(v(\Sigma)\ge6\), we prove \[ \sup_{z\ge0} \left| \mathbb P\{\|S_n\|^2\le z\} - \mathbb P\{\|G\|^2\le z\} \right| \le C\,\frac{r(\Sigma)^{3/2}}{n}. \] The constant depends only on the fixed parameters in the assumptions and is independent of the ambient dimension and individual small eigenvalues of the covariance matrix. The bound is uniform at the endpoint \(v(\Sigma)=6\), without a logarithmic loss. Its main feature is the combination of the \(n^{-1}\) rate with dependence on effective rank, including this endpoint. Consequently, the bound can vanish when \(r(\Sigma)=o(n^{2/3})\), even when \(p\) is much larger than \(n\). The same estimate extends to random elements of a separable Hilbert space with trace class covariance. The proof builds on the Fourier and symmetrization methods of \citet{GoetzeZaitsev2014}, combined with moment matching, truncation, and comparison estimates that retain the covariance structure.

Large language models are stochastic by nature. Repeated executions under the same experimental conditions may produce different responses. We propose a framework that can be applied across the model lifecycle, from task design to residual behavior and continuous monitoring. The method consists of building an empirical response distribution estimated through Monte Carlo simulation. Semantic interventions are introduced to study how these distributions change. We illustrate the framework with two different tasks, borrower classification and music album recommendation, and quantify the resulting distributional changes. The framework is linked to model risk management and governance by measuring robustness and control effectiveness, and characterizing residual behavior. Since it relies only on observed responses, it can also be applied to black-box models accessed through commercial APIs.

Aggregating independently computed sub-sample inferences is a well-known challenge in large-scale Bayesian computation, federated learning, and opinion pooling. The core problem is to approximate a distribution whose density is proportional to a product of individual densities. We propose Product Monte Carlo (PMC), an algorithm for computing arbitrary expectations under a product of densities using only Monte Carlo draws from individual densities. We turn the intractable product expectation into a regression problem by expressing it through a family of computable conditional expectations. Besides the point estimate, our fully Bayesian regression formulation provides coherent uncertainty quantification and principled diagnostics. We establish theoretical guarantees on consistency and characterize convergence rates for the PMC estimator. Across simulated and real-data experiments varying the number, dimensionality, and heterogeneity of the component distributions, PMC achieves higher estimation accuracy, better-calibrated uncertainty coverage, and improved scalability at a lower computational cost than existing methods. In addition, PMC requires no density evaluation, making it applicable to simulation-based inference, as we demonstrate in an application with protein images.

Leave-one-out (LOO) cross-validation, a standard tool for Bayesian model assessment, evaluates each observation using a LOO predictive distribution derived from the model itself. In this work, we introduce an alternative construction in which posterior predictive simulation connects the Bayesian model to an external pretrained predictive learner. Our method, Predictive-Context LOO (PC-LOO), fits a Bayesian model once, generates posterior-predictive synthetic data, and uses them as context for the learner to approximate each LOO predictive density. Under an oracle construction, averaging contexts generated from the LOO posterior recovers the exact LOO predictive density. Practical PC-LOO replaces the oracle predictor by the pretrained learner and the LOO posterior by the full posterior. We quantify these errors through learner compatibility $C_i$ and deletion influence $V_i$, obtaining $H^2 (p^{LOO}_{i} , q^{PC}_{i}) \leq C_{i} + V_{i}$. The identity also holds for any fixed subset of $M$ context points, allowing $M \ll n - 1$ as the learner's context budget, and scalability to large datasets. Experiments with TabICL support this decomposition, deletion influence decays approximately as $n^{-1}$, while learner compatibility can remain limiting at large $n$. PC-LOO remains accurate with reduced context sizes, while model comparison accuracy is governed by differences in approximation error across models rather than their absolute errors. More broadly, PC-LOO establishes posterior predictive simulation as a principled interface for integrating pretrained predictive learners into Bayesian workflows.

Efficient sampling from Bayesian posterior distributions in high dimensions remains a hard problem, despite advances in theory and algorithms. Multiple steps are usually required to draw unbiased samples, for example, iterations of an MCMC algorithm or integration steps in a diffusion-based amortised sampler. However, scalable one-step amortised inference remains a challenging open problem. In this work, we build upon recent progress in data-driven generative modelling to offer a possible solution for one-step amortised sampling. Our one-step implicit sampler (OSIS) uses short-time MCMC dynamics to define a local target for the evolution of an implicitly modelled distribution. In the asymptotic limit, the modelled distribution is a fixed point of the MCMC kernel and thus samples from the target. We analyse the convergence properties of OSIS and show connections with other generative modelling and sampling algorithms. We study various design choices and show that OSIS performs well compared to multi-step amortised samplers and non-amortised MCMC baselines while being far more efficient at sampling time.

We develop standard errors for estimates of causal effects in settings where the distribution of treatment exposures is known but potentially exhibits complicated dependence patterns across units. Such dependence arises, for example, in settings with spillovers/interference across units or when the treatment assignment is spatially correlated. Our variance estimator generalizes several properties of the Neyman variance estimator for completely randomized experiments to general designs: it is unbiased for an upper bound on the variance from the design-based perspective; the conservativeness of this upper bound depends on the heterogeneity of treatment effects; and it is also valid under certain forms of super-population sampling of the potential outcomes. Our estimator reduces to standard approaches in simple designs, such as the Neyman variance estimator in completely randomized experiments and the cluster-robust variance estimator in cluster-randomized experiments. We show that the proposed estimator is admissible among the class of quadratic-form variance estimators, and derive asymptotic conditions under which it is consistent for an upper bound on the asymptotic variance of the treatment effect estimator. In simulations and applications based on two empirical studies, one with peer effects and another with a spatially correlated treatment, we find that the proposed variance estimator performs well, and in some cases yields standard errors that are 2-3 times shorter than those of existing alternatives.

Social influence, the tendency of our social environment to sway our beliefs and behaviors, is often cited as a driver of population-level trends in opinions or beliefs. Many quantitative theories of social influence assume that an agent interacts with its entire social environment. This assumption is often unrealistic. For example, users of online platforms typically see posts from only a small subset of the accounts they follow at any given time. We introduce a simple probabilistic agent-based model of social influence that explicitly models unobserved attention mechanisms. The first of these latent mechanisms is selection: each agent interacts with only one other agent at a time from its social environment. The second is retention: the agent chooses whether or not to ignore the opinion of the other. Our model admits an efficient maximum-likelihood algorithm which allows us to fit it to opinion traces even when these mechanisms are unobserved. On synthetic data, omitting either latent mechanism reduces the overall strength of social influence governed by the inferred interaction kernel, which hides the evidence for the underlying nonlinearity of the social influence. We study a selection of previously analyzed empirical datasets and curate several new ones. We show that models that include the latent mechanisms are often more predictive on these datasets; indeed, they typically improve prediction far more than the choice of interaction kernel does. Modeling latent attention increases held-out predictive likelihood on 14 of these 15 datasets. Each mechanism changes the size of the inferred kernel in a predictable direction, while its effect on the kernel's shape depends on the data.

Masked diffusion models (MDMs) generate discrete data by iteratively unmasking multiple tokens in parallel using a learned mask predictor. Their output carries two errors: parallelization error due to parallel sampling, and estimation error from the mask predictor learned with finite samples. Existing theory treats these errors separately, and the available statistical bounds typically scale with the number $V^L$ of length-$L$ sequences over a vocabulary of size $V$, suffering from the curse of dimensionality. In this work, we establish end-to-end statistical guarantees showing that transformer-based MDMs learn structured distributions with a sample complexity governed by their latent structure. Specifically, we model this structure through mixtures of hidden Markov trees, which capture latent dependencies within components and heterogeneity across components. When the data distribution is well approximated by such a mixture, a transformer-based MDM with parallel unmasking schedules achieves $\varepsilon$-accurate sampling in KL divergence with $\widetilde{O}(LK_{\mathrm{par}}/\varepsilon)$ training samples, where $K_{\mathrm{par}}$ is polynomial in $L$, $V$, the number of hidden states, and the number of mixture components. The transformer network and sampling procedure require no knowledge of the tree topologies or component parameters, thereby adapting to the unknown latent structure automatically. Overall, our results provide theoretical justification for the empirical effectiveness of MDMs on real-world data.

Large-scale tensors arising in science and engineering are often too large to form or store explicitly. Their multilinear contractions, however, can frequently be evaluated efficiently, which makes randomized Frobenius-norm estimation a practical alternative to exact computation. In this paper, we study estimation of the squared Frobenius norm of a real order-$d$ tensor from such contractions using independent real, complex, or quaternion Gaussian probes. For a rank-one tensor, one normalized sample is a product of $d-1$ independent Gamma variables. This gives exact relative-variance factors $3^{d-1}-1$, $2^{d-1}-1$, and $(3/2)^{d-1}-1$, respectively. Under the same budget of underlying real Gaussian vectors, the quaternion probe has smaller rank-one variance than four independent real probes for $d\ge3$; a real-channel representation identifies the gain as modewise pooling. For orthogonal CP tensors, the exact relative variance equals the corresponding rank-one factor divided by the participation-ratio effective rank and the number of samples. For arbitrary real tensors, we prove sharp uniform moment bounds, attained at rank one, and obtain Chebyshev guarantees. A classical block-median construction then gives two-sided exponential confidence for the modified estimator. Numerical examples test the rank-one and effective-rank formulas. An implicit infinite-series kernel tensor illustrates contraction-based norm evaluation; EEG maps provide a qualitative spatial interpolation illustration.

METR's 50% time horizon measures the human completion time of software tasks that an AI solves with 50% probability, allowing AI capabilities to be expressed in interpretable units. On 228 tasks and 26 AIs, we recompute the time horizons using splines and item-response theory to relax the assumption that the AI difficulty of a task depends linearly on the log of human time. Our fitted spline can be interpreted as a function that converts human time to AI difficulty; it is nearly flat in a region from 2–30 min but close to linear elsewhere. Hence, a time-horizon jump from 3 min to 30 min is much easier than one from 30 min to 5 hours despite the same multiplier of $10 \times$. Overall, we contribute time-horizon point estimates that perform better under a cross-validated suite of proper scoring rules, as well as diagnostic plots for assessing time horizons' construct validity. We suggest that time horizons be interpreted together with the diagnostic plots, especially as new time-horizon-based benchmarks are proposed or existing ones grow to include longer tasks.

Density ratios quantify distribution shift from a probability-mass point of view, whereas displacement fields describe, from a dynamical point of view, how one distribution is transported onto another. Although both offer complementary insights, they are usually estimated separately, and converting one into the other requires post-processing. In this paper, we estimate the density ratio between a target and a base distribution by parametrizing it through a displacement field acting on the base: the log-ratio is modeled as minus the Stein operator of the base applied to the field, up to a normalizing constant. This gives both statistical and dynamical descriptions of the distribution shift through a single convex optimization problem. Iterating this estimate-and-move step gives two inference algorithms: push-forward moves the model and corrects a pretrained sampler without retraining it, whereas pull-back moves the data closer to the base and fits a transformation model one layer at a time. Applications to distribution shift in simulation-based inference and to nonlinear independent component analysis illustrate the benefits and limitations of the approach.

We present a topology-aware system based on mixup barcodes and persistent homology for financial decision making. The suggested approach uses topological summaries obtained from Takens delay embeddings of a univariate price series to quantify structural changes between reference and current market regimes. The 1-Wasserstein distance between persistence diagrams, a mixup barcode disruption index, and persistence entropy divergence are combined to provide a novel stress score. A Golden Cross trading method is dynamically modulated by this score, which transforms a binary buy/sell signal into a continuous position-sizing process. At embedding settings chosen by maximizing in-sample Sharpe over a 72-point grid, in-sample assessment on the S&P 500 and Bitcoin yields Sharpe ratios of 1.116 and 1.238, respectively, with much lower maximum drawdown compared with both Buy-and-Hold and the standard Golden Cross baseline. Our findings imply that topological summaries of market geometry include useful information that goes beyond traditional trend markers.

Generating data with desired properties for a logistic regression model under controlled conditions is challenging. We propose an integrated iterative bisection algorithm that determines values of the intercept and regression coefficients capable of generating data with a specified prevalence and predictive accuracy. The performance of the proposed method is demonstrated through simulation studies. We further show that the choice of parameter values, or, in a Bayesian setting, the choice of priors for these parameters, induces a particular level of predictive accuracy. This relationship should therefore be taken into account when generating data for model evaluation under controlled conditions.

Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The conditional marginals over the missing variables are the key ingredient for computing Bayes risk and Value of Information (VOI), the expected gain from acquiring one more observation before deciding. We present the Marformer, a Transformer trained to directly predict conditional marginals given any set of observed values. Like BERT, which is trained to predict missing words from context, the Marformer constructs a hidden-vector representation for each distribution $p(X_i)$ and iteratively refines it through attention to other distributions $p(X_j)$. Unlike generative approaches, the Marformer does not model the full joint distribution, requires no domain knowledge of the data-generating process, and makes all predictions in a single forward pass. We evaluate across three synthetic domains with missing data—Bayesian networks, discretized multivariate Gaussians, and structured annotation data. The Marformer can match or outperform classical missing-data methods, even when those methods are given the true model family and prior that generated the synthetic data. We also evaluate on a real annotation dataset, where the Marformer outperforms the evaluated baselines at the largest training size. In both cases, the Marformer is substantially faster than the evaluated generative baselines.

Target trial emulation has become a standard framework for causal inference from observational data. Within this paradigm, the clone-censor-weight (CCW) methodology provides a practical way to deal with complex treatment and adherence definitions when treatment regimes are not distinguishable at baseline. Despite its increasing use, the statistical interpretation of CCW remains limited. In this work, we formalize the CCW methodology from a statistical viewpoint based on admissible sets and stochastic interventions. We characterize the causal estimands targeted by CCW, establish consistency of CCW estimators under standard identification assumptions, and provide theoretical guarantees for bootstrap-based inference methods.

Koopman operator theory provides a linear framework for analyzing nonlinear dynamical systems and has become a major tool for data-driven modeling. A central challenge, however, is that finite-dimensional approximations computed by methods such as extended dynamic mode decomposition (EDMD) require the dictionary to be specified a priori. Recent machine-learning approaches address this limitation by learning the dictionary from data, predominantly using artificial neural network (ANN) autoencoder architectures. Although kernel methods offer an alternative with greater interpretability and tractability for theoretical analysis, they have received little attention in this setting. We introduce extended dynamic mode decomposition with kernel-based dictionary learning (EDMD-kDL), a kernel-based method for learning finite-dimensional Koopman embeddings directly from data. The method combines ideas from collocation methods and bilevel optimization to simultaneously learn a kernel dictionary and the corresponding Koopman approximation. We evaluate EDMD-kDL against state-of-the-art ANN-based approaches on a range of numerical experiments, including global sea-surface-temperature forecasting and learning directly from video data. Across all tested settings, EDMD-kDL achieves performance comparable to or better than the ANN-based methods. Moreover, in contrast to standard kernel methods, the proposed approach is scalable to large datasets by design since the size of the required kernel matrices depends on the number of collocation points rather than the size of the training dataset.

We consider policy optimization for online episodic tabular Markov decision processes (MDPs) with adversarial losses and bandit feedback. Policy optimization updates the policy locally at each state and avoids optimization over the occupancy-measure polytope, but its existing regret bounds are larger by a factor of the horizon $H$ than those of occupancy-measure-based algorithms. We close this gap by using regularized $Q$-functions, which allow us to control the stability of the local updates jointly over all state-action pairs rather than separately at each state. The resulting algorithm attains high-probability regret bounds of $\widetilde O(\sqrt{HS(H+A)T})$ for known transitions and $\widetilde O(HS\sqrt{AT})$ for unknown transitions, where $S$ is the number of states, $A$ the number of actions, and $T$ the number of episodes. Both bounds improve the horizon dependence of existing policy optimization bounds, and the latter matches the best-known bound. We further extend the algorithm to adversarial linear-mixture MDPs and obtain the same improvement in the horizon dependence.

Randomized controlled trials (RCTs) identify treatment effects without confounding but are often small, whereas observational studies (OBS) are large but may be confounded. Many estimators combining a small RCT with a large OBS have been developed for the average treatment effect (ATE) and the conditional ATE (CATE). However, existing ATE estimators either make assumptions on the OBS or do not borrow enough power from them. The CATE has been studied less than the ATE. Existing CATE methods either assume the OBS are unconfounded, rely on a model of the confounding function, or accept bias in exchange for lower variance. We therefore propose a framework that, without special assumptions on the OBS, fuses the OBS and the RCT by preserving the unbiasedness of RCT-based estimation while borrowing power from the large OBS to boost precision. Applying this principle, we build an ATE estimator, AIPW-Fusion, with closed-form weights and confidence intervals, and two CATE learners, DR-Fusion and R-Fusion. Experiments corroborate our findings.

We consider stochastic nonsmooth nonconvex composite optimization, which includes several important problems such as constrained optimization and the regularized training of neural networks. The objective is the sum of a possibly nonsmooth nonconvex Lipschitz function and a convex regularizer, and the function is accessed through stochastic gradients or function values. The goal is to find a point that satisfies a Goldstein-type stationarity condition designed for composite objectives. To our knowledge, no oracle complexity bound for this setting is known under first-order access, and existing complexities under zeroth-order access are suboptimal. To handle this issue, we employ the framework of online-to-nonconvex conversion, which chooses update directions by an online learner and is known to achieve optimal rates for noncomposite problems. We extend the framework to our composite scenario by introducing new losses for the learner, which contain the regularizer itself rather than its linearization and for which a variant of online mirror descent achieves low regret. We show that the resulting algorithm finds such a point with $O(\delta^{-1}\varepsilon^{-3})$ stochastic gradient queries or $O(d\delta^{-1}\varepsilon^{-3})$ function-value queries, where $\delta$ is the Goldstein radius, $\varepsilon$ is the stationarity tolerance, and $d$ is the dimension. These rates match the optimal ones for noncomposite nonsmooth nonconvex optimization, demonstrating that the additional convex regularizer does not worsen the oracle complexity. We also give rates for the smooth case and present numerical experiments.

Given independent and identically distributed samples samples from a probability distribution in a potentially high-dimensional real space, we study the problem of testing whether the distribution is concentrated near a real algebraic complete intersection of prescribed dimension, bounded degree, and bounded condition number. We design an explicit and effective learning procedure which either certifies the nonexistence of such a manifold, up to a controlled relaxation of the approximation threshold, or returns a candidate regression manifold with controlled geometric complexity. Equivalently, the procedure tests the manifold hypothesis within this hypothesis class. The proposed procedure relies on quantitative geometric estimates for regular polynomial systems, which lead to a tractable auxiliary optimization problem. We then develop a data-driven algorithm to solve this auxiliary optimization problem, establishing explicit bounds on its sample and arithmetic complexity.

Multiple imputation (MI) is widely used for handling missing data in medical research, with variance commonly estimated using Rubin's rules (RR). RR is simple to apply, but under uncongeniality between the imputation and analysis procedures, its variance estimate can be either too small or too large, leading to miscalibrated confidence intervals. Robins and Wang (RW) developed an estimating-equation variance estimator that does not require congeniality under the conditions of their parametric imputation framework. Practical use of RW, however, has been limited by the lack of accessible software and by the additional variance calculations required for modern imputation workflows. We develop an R package, rw, construct RW variance components for normal and logistic conditional imputation models fitted through mice, and propose an approximate RW variance procedure for predictive mean matching (PMM). The PMM procedure preserves the original imputations and selected donors. For the variance calculation, we define differentiable donor probabilities and use expected analysis scores to approximate the cross-moment between analysis and imputation scores. Analysis scores are summed by recorded donor to account for donor reuse. We evaluate the proposed methods in benchmark simulations based on the original RW setting and in validation-study simulations motivated by electronic health record data. RR coverage was up to approximately 23 percentage points below the nominal 95% level in some settings and above nominal in others. The proposed RW estimators generally produced coverage closer to the nominal level in larger samples under both parametric and PMM imputation, while retaining the same completed-data point estimates as RR.

Bayesian Optimization (BO) is a popular method for efficiently optimizing expensive black-box objectives. However, BO utilizing standard Gaussian Processes is ill-suited for doubly stochastic Cox Processes that are often used in spatio-temporal problem spaces. We introduce INLA-SPDE Spatio-Temporal Bayesian Optimization (ISBO): the first scalable BO framework for spatio-temporal data, that models the log-intensity with a Log-Gaussian Cox Process(LGCP) and performs inference via Integrated Nested Laplace Approximation and Stochastic Partial Differential Equations (INLA-SPDE) approach. Using a Matern field on meshes yields a sparse Gaussian Markov Random Field, where INLA provides fast and accurate posterior inference throughout sequential optimization. ISBO stably locates high-intensity regions and the peak of the latent intensity with minimal evaluations. A time-varying Upper Confidence Bound acquisition with masking avoids revisits, while penalized-complexity priors regularize early rounds. Experiments on synthetic and real-world spatio-temporal datasets show accurate peak discovery, intensity recovery, and substantial speedups over an RKHS-based baseline, positioning ISBO as a practical choice for BO with point-process data.

A verifier that accepts or rejects whole answers reveals little: under a flat prior over $k$-bit answers, zero error needs $2^k-1$ verifications. The usual remedy is to solve related source tasks, either all first, as a curriculum does, or interleaved with verification. We price this remedy in information and in calls. With an exact verifier, the least causal information that any interleaving of source calls and $n$ verifications needs to succeed with probability $s$ is a list rate-distortion function, attained by one observation before any verification. It lower-bounds the expected number of binary source calls, which designed sources meet within $1+\log_25$ calls for unique answers and within a logarithmic term in general, where no additive constant suffices. With an exact verifier and fixed sources, moving every call before the first verification preserves all hard caps on calls, although interleaving can save unboundedly many expected calls; under a noisy verifier, source-first protocols can lose unbounded factors in information and in error. For linear banks over $\mathbb{F}_2$, optimal accuracy has a closed form, and after a polynomial-time reduction the budget profile is computable in time $2^{O(h^2)}\operatorname{poly}(J,k+h)$ for $J$ sources and nuisance dimension $h$. In these banks, for zero error under a hard cap, the calls beyond the rounded-up information price are exactly those spent on nuisance. Every numbered result apart from two clauses about the planner is machine-checked in Lean 4, assuming two published results. Used as a ruler, the frontier shows a small transformer using all delivered bits at latent dimension $5$ and none at $11$ within fixed training budgets; in a test with predictions recorded before training, low XOR degree of the target bits did not suffice for their use.

Classical CUSUM relies on the log-likelihood ratio of the underlying distributions, which cannot generally be computed from finite pre- and post-change samples alone. We propose diffusion-integrated score CUSUM (DI-SCUSUM), a training-free detector. We add Gaussian noise to the samples to form two smooth density estimates and calculate their Hyvärinen scores exactly, without training a score network. For each incoming observation, we sample a diffusion time, perturb the observation, and use the importance-weighted score difference as an increment in the DI-SCUSUM recursion. Under the assumption that observations follow the fixed empirical distributions, the post-change mean increment is proportional to the Kullback-Leibler (KL) divergence from the smoothed post-change to the smoothed pre-change empirical distribution. We establish exponential false-alarm scaling and a first-order delay bound that, for a fixed threshold and increment scaling, is inversely proportional to the KL divergence. In the calibrated anisotropic Gaussian simulation, DI-SCUSUM nearly matches likelihood-ratio CUSUM and reduces the measured detection delay by about 91% relative to score-based CUSUM. On MNIST and Oxford-IIIT Pet, DI-SCUSUM also has lower empirical conditional detection delay than SCUSUM at comparable false-alarm levels.

Equity correlation matrices are foundational to quantitative risk, yet the operational default of a single Gaussian-copula structure fit to the full sample has a structural defect: it forces asymptotic tail-independence and so cannot represent the joint moves that drive risk in stress episodes. We argue that the dependence of equity returns is not one object but three distinct under lower-tail, central, and upper-tail conditions, and that any estimator aimed at tail-sensitive applications should be regime-aware. We develop a decomposition that fits a dedicated dependence matrix to each regime, using a tail-dependence estimator grounded in multivariate regular variation in the two tails and a time-varying conditional-correlation model in the central regime, and recombines the three at the unconditional regime probabilities to form a steady-state matrix. The construction identifies tail-regime dependence that the Gaussian-copula benchmark structurally suppresses. Demonstrated on water- and energy-sector Exchange Traded Funds (ETF) constituents, the tail regimes reorganize the commodity partition produced by the central regime, identifying the pairs where the Gaussian-copula default systematically fails to identify joint extreme co-movement.

In many machine learning applications, it is necessary to guard against worst-case scenarios and predictions that could result in substantial losses. In principle, this can be achieved by training risk-averse predictive models that minimize loss functions such as conditional value-at-risk (CVaR), rather than relying on models that perform well on average. In practice, however, the effectiveness of this approach to risk aversion is undermined by the learner's uncertainty regarding the true loss distribution and, consequently, the true CVaR. To achieve reliable risk-aversion, we propose a method in which this (epistemic) uncertainty is represented in terms of credal sets, i.e., sets of probability distributions. More specifically, we develop an efficient yet reliable learner that produces predictions in the form of credal sets and combine it with a novel decision rule that maps each credal set to a single predictive distribution for CVaR minimization. Across classification, under distribution shift, and in reinforcement learning, our approach reliably avoids catastrophic decisions, while sacrificing little in expected performance.

Particle filters are a standard tool for nonlinear state estimation, but their resampling step is discrete, preventing gradient-based learning in variational sequential Monte Carlo. We introduce Differentiable Systematic Resampling (DSR), a temperature-controlled relaxation of systematic resampling, that preserves the CDF-ordered, banded structure of systematic resampling while enabling full gradient flow. DSR converges to exact systematic resampling as the temperature vanishes, and we prove a pointwise exponential convergence rate for the induced bias. Compared to optimal-transport-based differentiable resampling, DSR avoids iterative solvers and has substantially lower computational overhead. Experiments on stochastic dynamical systems and real-world handwriting data show that DSR achieves comparable or superior filtering and dynamics learning performance.

Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation-based inference to evaluate the posterior distribution of the input parameters based on real-world observations of the simulator outputs. However, inference becomes challenging when individual simulator evaluations are computationally expensive. In such cases, a Bayesian optimization-based active learning approach with Gaussian process surrogate models has been used to maximize the information obtained from a limited simulation budget. Recently, gradients of simulator outputs with respect to input parameters have become increasingly available, yet they are rarely exploited for inference. Even though we only need to learn the simulator input-output relationship, gradient information can provide an additional valuable signal to guide the active learning procedure. This is of particular interest in the case of expensive simulators, when sample efficiency is crucial. In this paper, we demonstrate how incorporating gradient information into the Gaussian process surrogate accelerates Bayesian optimization-based inference under a limited simulation budget. Our results show significant improvement in convergence speed from using gradient information. For reverse-mode differentiation, the inference efficiency gains are maintained when accounting for the additional computational cost. In contrast, for forward-mode differentiation, the inference speed-up does not outweigh the computational costs. These results indicate that gradient-enhanced surrogates are beneficial primarily in problems where the number of parameters exceeds the output dimensionality, where reverse-mode differentiation is efficient.

Randomized-withdrawal trials re-randomize induction responders to continued treatment or withdrawal. Baseline characteristics are often reported for the induction population but unavailable for responders, limiting population-adjusted indirect comparisons of maintenance outcomes. We describe re-randomization-probability weighting, which combines induction-population calibration with inverse weighting by known maintenance assignment probabilities. Coding induction non-response as failure defines durable response: response at both induction and maintenance. The design weights reproduce induction-population covariate totals in randomization expectation, allowing population adjustment using reported induction baselines. Conditional maintenance probabilities can also be estimated, although different induction treatments generate different responder populations. A synthetic three-trial demonstration reduced durable-response bias from 0.106 to 0.001 and from 0.055 to 0.001 in two comparisons. Full-procedure simulations showed interval coverage of 91.5% - 94.3%; stress scenarios illustrated bias from omitted covariates and incorrect assignment weights. The method connects established design weighting and aggregate dilution approaches to population-adjusted comparison across randomized-withdrawal and treat-through designs. It requires compatible durable-response endpoints and credible transport and modeling assumptions. Withdrawal after different induction regimens is not automatically a common comparator. Where no valid anchor exists, unanchored comparison is appropriate only if its demanding adjustment assumptions are defensible.