Papers

Inverse optimization estimates the weights of an objective function that explain observed decisions as optimal solutions, and is used in a variety of fields. For mixed-integer linear programs (MILPs), existing methods aim to reproduce the observations as optimal solutions, and thus learn compromise weights when the observations are suboptimal. We propose outperformance inverse optimization, which instead seeks weights that induce, at each state, an optimal solution outperforming the observed action in every component. We give a loss function that can be evaluated with forward-problem oracles alone and is thus applicable to MILPs, together with gradient-based and DC optimization algorithms for minimizing it. For weights inducing a unique outperforming optimal solution at all observations, we prove that the probability of failing to induce such a solution at a new state (the generalization error) is bounded by a quantity inversely proportional to the number of observations, and that this bound is tight in the number of observations up to logarithmic factors. In experiments on synthetic and real data, the proposed methods improve the prediction of solutions outperforming the actions over existing methods.

Human learning is a dissipative dynamical process: mastery accumulates through practice, decays through forgetting, and propagates across interdependent concepts. We model it as a nonlinear dissipative system of ordinary differential equations whose parameters are mechanistically meaningful (a transfer-of-practice matrix, per-concept forgetting rates, and a saturating practice-response gain), and we study when those parameters can actually be recovered from data. For two concepts, we prove a constructive structural identifiability theorem for the associated inverse problem under explicit excitation conditions, together with monotonicity, robustness and L-stability results. We derive an implicit L-stable scheme and a batched solver numerically equivalent to the per-trajectory formulation yet two orders of magnitude faster. The empirical study is two-sided. Under the theorem's excitation conditions, synthetic recovery is exact. On large observational benchmarks it is not. An apparently strong recovery, with forgetting rates correlating with topic error rates at Spearman $\rho = 0.83$, is refuted by four controls: it survives destroying the temporal order of the data, is matched by a classical Bayesian baseline, and is unaffected by removing real timestamps. A single shared forgetting rate attains the same accuracy, and replaying the real schedule with identical true rates reproduces most of the correlation. A positive control shows that the controls detect forgetting where it is observable. We trace the effect to the stationary structure of the model and show that it is consistent with the degeneration anticipated by the theory in the absence of designed excitation.

Bayesian neural networks need not be fully stochastic to be universal conditional density approximators, but it remains open which parameters should be stochastic. We learn this split by applying deep weight factorization to the prior scales, which are the standard deviations of the parameter priors, while fitting the functional prior to a Gaussian process with a maximum mean discrepancy objective. A parameter whose prior scale falls below a cutoff becomes deterministic and is optimized during inference, so the regularizer sparsifies stochasticity rather than capacity. We give a certificate for universal conditional density approximation that is checkable in linear time, together with a minimal repair when it fails. We further show that the common hybrid scheme of sampling some parameters and optimizing the others is stochastic approximation for a type-II maximum a posteriori objective, and that coupled step sizes can leave a tracking error that does not vanish as the step size shrinks. On a bimodal target, the learned split stays close to an unconstrained reference across all budgets and is insensitive to the cutoff, while random masks that distribute the same prior scales across layers are worse by up to two orders of magnitude. On UCI benchmarks, our method performs on par with a fully stochastic network while keeping about half of its parameters deterministic.

The spatial distribution of waiting pedestrians has two primary drivers: environmental preference and interaction with other pedestrians. Spatial point processes naturally capture both spatial heterogeneity and repulsive interaction. In previous work (Sickert Karam et al., arXiv:2606.14532, 2026), we proposed a Gibbs model with inhomogeneous intensity and a modified Diggle-Gates-Stibbard interaction function, which reproduces many phenomena in replicated patterns of pedestrians waiting at a train station. Its single interaction function acts as an effective interaction, averaging over behavioral regimes such as interactions within social groups and among strangers. In this article, we show how groups affect distance-based summary statistics and take first steps towards accounting for them. We detect groups from pairwise distances and contact durations, replace each group by a “macro-pedestrian” at its average position, and refit the model. This yields a larger interaction range and better agreement in nearest-neighbor statistics than the original model, although the two fits concern different datasets and centroid-based distances likely overstate repulsion. This first attempt thus resolves some discrepancies but falls short of a fully adequate solution, opening avenues for further research.

We develop a non-asymptotic theory for nonparametric drift estimation in discretely observed multi-dimensional Lévy-driven stochastic differential equations using sparse ReLU neural networks. Under exponential \(\beta\)-mixing and an exponential-tail condition on the Lévy measure, we establish an oracle inequality for the least-squares estimator. The jump component changes both the martingale structure and the concentration regime relative to diffusion models: continuous martingales are replaced by discontinuous martingales, while the relevant empirical-process fluctuations are sub-exponential rather than sub-Gaussian. We handle these difficulties without truncating the observed increments, using an exponential-supermartingale argument for compensated Poisson integrals and \(\psi_1\)-chaining. Despite the weaker concentration, the resulting statistical bound is, up to constants, of the same order as the corresponding diffusion bound. For drift functions with hierarchical compositional structure, this yields intrinsic-dimensional convergence rates. The framework allows infinite jump activity and, in some cases, infinite variation. Finally, we establish a minimax lower bound on a fixed non-trivial compound-Poisson submodel. Together with the upper bound, this shows that the estimator is minimax optimal up to logarithmic factors.

Statistical inference under adaptively collected data is becoming increasingly popular across e-commerce and mobile health. Many methods exist for discrete action settings, ranging from weighting to debiasing approaches. Despite the ubiquity of continuous actions in experimentation from optimal pricing to precision dosing, to the best of our knowledge, methods that support statistical inference under continuous action, adaptively collected data remain underdeveloped. In this work, we extend kernel-smoothed doubly robust estimation from i.i.d. data to adaptively collected data with continuous actions. We study a family of adaptively weighted estimators, establish their mean squared error rates, asymptotic normality, and characterize an estimation lower bound as a function of regret. We conclude with recommendations for experimental design informing efficiency and regret, and support our theory with an extensive simulation study.

Causal discovery becomes particularly challenging when the available sample size is small relative to the number of variables. This challenge also arises in the linear non-Gaussian acyclic model (LiNGAM), an identifiable framework for causal discovery from observational data. DirectLiNGAM estimates a causal order, which arranges variables so that causes precede their effects, by sequentially identifying an exogenous variable and removing its linear effect from the remaining variables. We establish a structural limitation of this procedure: when the number of variables exceeds the sample size, repeated residualization necessarily becomes degenerate before the full causal order can be determined. Our analysis further reveals that each residual can be reconstructed using only a graph-determined subset of variables already placed earlier in the causal order, termed the active boundary. This result motivates AdaPS-LiNGAM (Adaptive Predecessor Selection LiNGAM), which reconstructs each residual directly from the original observations using an adaptively chosen sparse subset of those earlier variables. The same subset-selection principle is also applied to the final pruning step for edge estimation. Experiments on synthetic data demonstrate that AdaPS-LiNGAM provides accurate causal-structure recovery in sample-limited settings and degrades more gradually as the sample size decreases.

Mean field limits describe the training dynamics of wide neural networks through the evolution of the empirical distribution of their parameters. Although functional central limit theorems characterize the asymptotic fluctuations of this distribution, quantities of practical interest are typically nonlinear observables of the parameter distribution rather than the distribution itself. In this work, we show how these mean field fluctuations propagate to finite dimensional nonlinear observables for shallow neural networks trained by stochastic gradient descent. Working in the weighted Sobolev space in which the limiting fluctuation process is constructed, we apply a functional Delta method under ordinary Fréchet differentiability, without requiring Lions derivatives with respect to the measure variable. We obtain a central limit theorem for the observables and, under a suitable representation of their differentials, an explicit covariance formula inherited from the underlying mean field fluctuation theory. We also study whether prescribed quantities of interest can be recovered from the selected observations. Under a constant rank assumption, we prove that a quantity of interest factors locally through the observation functional if and only if, throughout a neighborhood, the kernel of the differential of the observation is contained in that of the quantity of interest. Thus, a differential condition expressed directly in the ambient Sobolev space yields an exact nonlinear local factorization. These results provide a framework both for quantifying finite-width uncertainty on observable, statistically or physically meaningful quantities and for assessing whether the chosen observations contain the information required to identify them.

In the analysis of multivariate data, factor models represent a powerful technique for both reducing dimensionality and facilitating qualitative understanding of the latent determinants governing the observed data. Interpretability, however, is often hindered by the non-identifiability of factor loading matrices due to rotational invariance. A prevailing strategy to address this issue is to consider positive lower triangular structures. A relaxation of the latter condition has been recently proposed in the literature with the so called generalized lower triangular structure. However, its Bayesian implementation presents substantial challenges, requiring complex reversible-jump algorithms and making prior elicitation difficult because of the limited interpretability of the model parameters. In this paper, we introduce a generalized lower triangular process prior that allows both global and within-component sparsity structures for learning both the number of factors and possible sparsity structures within the vector of observations. The proposed approach provides straightforward prior parameters elicitation exploiting possibly different prior information on the rank and sparsity characteristics. We also explore connections with the Indian buffet process, providing further insight into the sparsity structure induced by the proposed prior. Posterior computation can be performed resorting to a Gibbs sampler with Metropolis-Hastings moves. The efficacy of the proposed method is demonstrated through comprehensive simulations and the analysis of the Big Five Personality Test data.

Transformers have in-context learning capabilities, where some known learning algorithms can be executed in the forward pass through the model. Recent work shows that transformers can exactly perform Lloyd's algorithm for $k$-means clustering with $n$ points in $d$ dimensions with an embedding size $d_{\textsf{emb}} = d+k$ (thus, requiring attention projection matrices of size $(d+k)^2$). In this work, we build upon this result in the following ways: First, we present an equally expressive but smaller transformer that executes Lloyd's algorithm with embedding size $d_{\textsf{emb}} = (d + \lceil \log_2 k \rceil)$. Next, we train these transformers to learn the clustering algorithms given a distribution of clustering tasks, and theoretically characterize and empirically validate the factors affecting the convergence and in-distribution generalization of learning algorithms based on stochastic gradients. Finally, we probe the general clustering abilities of these learned algorithms (in the form of transformers), and try to understand situations where they succeed and fail.

Mid-prefill pruning can reduce the sequence processed by deeper transformer layers, but attention concentration alone does not certify that discarded context is dispensable. We formulate EntroPrefill as a Renyi-guided proposal mechanism coupled to explicit constraints on discarded attention mass. Sink-isolated, regularized head pooling respects grouped-query attention while exposing a quantitative trade-off between specialization and worst-head coverage. We derive a mixture-to-head deletion envelope, a computable upper bound on feasible token removal, and a finite-sample observer guarantee that remains valid when the pruning layer is selected adaptively. We then establish a conditional transformer perturbation bound with explicit sufficient Lipschitz constants and a first-token decision-margin corollary. A counterexample shows why shallow observations alone cannot imply an unconditional future-output guarantee. The systems analysis distinguishes query-head unions, physical page allocation, and KV-transfer payload, and gives an arithmetic break-even condition for pruning. This manuscript is theoretical in scope: it defines the procedure, its assumptions, and its formal limits, but does not report measured acceleration or task-accuracy preservation. Experiments are reserved for subsequent validation of the assumptions, approximation tightness, and end-to-end resource trade-offs.

Kernel methods are among the most powerful tools in machine learning and statistics, with a large number of successful applications. Their immense success stems from the flexible function class associated to each kernel—its reproducing kernel Hilbert space (RKHS)—which facilitates statistical analysis, as well as from their computational tractability and applicability to many domains. Multiple notions (such as characteristic, $L_p$-universal, and integrally strictly positive definite) capture the expressivity of kernels and their RKHSs and play a key role in understanding the statistical properties of kernel methods; these concepts and their relations are well-understood for bounded kernels. Even though unbounded kernels have received significant attention over the past decade (for instance, in the construction of kernel-based discrepancy and dependence measures such as the maximum mean discrepancy, the Hilbert-Schmidt independence criterion, and the kernel Stein discrepancy), surprisingly little is known about the relations of these notions in the unbounded case. In the present paper we tackle this severe bottleneck, establishing their relations under mild assumptions.

This paper studies transfer learning for high-dimensional quantile regression with limited target samples and heterogeneous source domains. We propose a sample-level transfer learning method (SL-TL) that combines sample selection and importance weighting. Unlike existing source-level approaches that include or exclude entire source domains, SL-TL identifies transferable samples within each source domain and incorporates them through adaptive importance weights. The proposed weights rely only on one-dimensional densities associated with the quantile loss, avoiding the estimation of high-dimensional density ratios. We establish error bounds for SL-TL estimators and characterize the effective sample size contributed by source domains. In the oracle setting, we show that SL-TL achieves faster $\ell_2$-convergence rates than existing competitors under the considered regimes. For the unknown setting, we develop an implementable algorithm based on sample splitting and cross-fitting procedures. Extensive simulations and a real data analysis demonstrate the finite-sample performance and robustness of SL-TL.

Structured recovery phenomena, such as restricted isometry properties in compressed sensing, have shown that high-dimensional objects can often be reconstructed from remarkably low-dimensional linear measurements. This work develops an analogous recovery framework for low-rank signed measures on $\mathbb{R}^2$, defined here as measures that can be expressed as finite sums of product measures with one-dimensional factors. The framework is based on linear operators, termed "silhouette operators," that map a measure to a fixed finite collection of one-dimensional linear pushforwards. The main results show that a suitably chosen collection of $2k$ projected marginals suffices to identify every compactly supported rank-$\le k$ signed measure, that this number is optimal, and that the projection directions cannot be chosen arbitrarily. The framework is also extended to higher-dimensional sums of product measures by establishing sufficient conditions under which collections of pairwise marginals identify the full model. Building on this framework, a computationally efficient estimator, termed "silhouette mixture estimation" (SME), is introduced for constructing a low-rank empirical measure from data by matching its one-dimensional projected marginals to the corresponding empirical marginals in Wasserstein distance. When combined with one-dimensional density estimators, SME yields an efficient nonparametric density estimator that performs strongly relative to a range of parametric, nonparametric, and deep-learning baselines in settings of moderate dimension and sample size.

We study the accuracy of Gauss-Newton curvature in ridge-regularized nonlinear least squares. Under local regularity and persistence of level-set curvature magnitude along an exact-fit section, we prove uniform coexistence of two curvature regimes. Global minimizers exist, and every global minimizer has relative Hessian error below $(1+\sqrt2)/8$, while the same low-cost set contains a point with an indefinite Hessian and relative error at least $15/8$. One positive ridge cap works for all independent center and label perturbations in fixed neighborhoods and every positive ridge weight up to the cap. These neighborhoods do not shrink as the ridge weight tends to zero. A pointwise certificate based on the current prediction level set controls the normal, mixed, and tangent parts of the Hessian correction. We prove a sharp relative-error bound over the stated pointwise class when the prediction map and ridge vary. Analytic examples describe the roles of output alignment, curvature orientation, and persistence. A separate structural result gives full Jacobian row rank throughout low-cost sets and exact interpolation near a rank-deficient reference.

The Global Ocean Observing System is key to understanding oceanic variability and climate change. The ocean is vast and often sparsely and irregularly sampled in time and space by various instruments and systems. Statistical models, especially spatio-temporal ones, are useful for enabling inferences, forecasts and decisions from sparse oceanic observations. This article focuses on the statistical treatment of two types of in situ observations in the Global Ocean Observing System: from profiling floats and surface drifters, focusing on the Argo and Global Drifter Programs. We describe the spatio-temporal models that have been developed in recent years for these data, to give a picture of the statistical challenges faced in modern physical oceanography. We will discuss in detail two types of reference frames when modeling such data: Eulerian and Lagrangian, the former of which is more appropriate for profiling floats and the latter for surface drifters.

We compare two locked survival prediction models in a target population before target survival outcomes are available. At a prespecified horizon, the estimand is the target Brier-risk contrast. Under a bounded conditional log-odds shift model on a prespecified deployment summary, we derive a sharp identified set preserving the shared unidentified target outcome law. Direct identification is never wider than separately identifying the risks and subtracting their bounds, with strict tightening under a Brier-specific same-side-1/2 condition. For right-censored source data, conditional Cox censoring estimation, inverse-probability-of-censoring-weighted logistic outcome modeling, and a joint pairs bootstrap yield simultaneous confidence envelopes over a finite sensitivity grid. In simulations, separate-to-direct width ratios ranged from 1.00 to 5.73 across controlled prediction geometries. Targeted simulations showed finite-sample undercoverage of the outer envelope at the small, heavily censored non-small-cell lung cancer (NSCLC) information scale (0.847-0.861 versus 0.95 nominal), compared with 0.946 at the Rotterdam-GBSG scale. In the cross-institutional NSCLC application, all 40 prespecified evaluations resulted in DEFER despite reduced identification uncertainty. In a supporting Rotterdam-to-GBSG analysis, candidate superiority was certified under small sensitivity allowances; one locked configuration yielded ADOPT CANDIDATE under direct identification but DEFER under separate-risk subtraction. Direct identification can materially reduce identification uncertainty and change the operational conclusion when signal and sampling precision are sufficient, while retaining DEFER when directional certification is unsupported.

Early cognitive response after repetitive transcranial magnetic stimulation (rTMS) may predict later cognitive change in Alzheimer disease, but the incremental value of domain-specific response is unclear. We analyzed data from 112 randomized participants; 101 entered a Week-5 landmark cohort and 91 had complete predictors and outcomes at 8, 16, and 24 weeks post-treatment. Nested predictor-set comparisons used ridge regression to evaluate baseline information, trial variables, early total-score response, and domain-specific response. Model-class comparisons included ridge, multi-task elastic net, restricted random forest, and longitudinal mixed effects. Validation used repeated treatment-site-stratified 5-fold cross-validation with five repetitions, nested hyperparameter tuning, and paired participant-level bootstrap comparisons with 10,000 resamples. Adding Week-3 and Week-5 ADAS-Cog total-score response improved prediction. The early total-response ridge achieved overall MAE 3.067, RMSE 3.885, and $R^2=0.384$. Replacing two total-response variables with 22 component-level changes worsened performance (MAE 3.363, RMSE 4.269, $R^2=0.257$). The paired MAE difference was -0.297 points (95% CI -0.485 to -0.115). Mixed effects was competitive (MAE 3.162), with an uncertain overall difference from total-response ridge. Sensitivity analyses using absolute future scores and excluding Week-3 response supported the main findings. In this sample, early aggregate ADAS-Cog response was more informative than a higher-dimensional domain-specific representation. The model is an internally validated early-response updating tool, not a pretreatment selection model or evidence of causal rTMS efficacy.

Sparse regression must balance prediction accuracy with reproducible variable selection. We introduce the Lambert penalty by limiting how quickly the retained fraction of a scalar score increases after variable entry. Maximizing retention under this scale-equivariant constraint yields a logarithmic transition from exact zero to an identity tail. The bounded penalty has a fixed shape and a single penalty parameter selected by training-only five-fold cross-validation. We derive an explicit scalar update, a sharp weak-convexity bound, a sufficient condition for multivariate uniqueness, and conditional convergence of exact cyclic coordinate descent. For fixed dimension, global minimizers are uniformly equivalent to the oracle estimator over an asymptotic tuning band. Gaussian simulations compare Lambert with LASSO, ridge, elastic net, MCP, and SCAD. Lambert favors prediction for strong equal signals and usually improves support recovery over SCAD; MCP often gives better support recovery, and convex methods remain competitive with weak signals or strong correlation. In a QSAR toxicity application, repeated nested cross-validation gives competitive prediction with compact, stable models, although error differences from MCP and SCAD are small. The empirical design followed exploratory development; independent confirmation remains necessary.

Data collection is often costly or logistically demanding, limiting both the questions researchers can pursue and how precisely they can answer them. Prediction-powered inference (PPI) can reduce the amount of data needed for precise parameter estimation by combining labeled data with machine learning predictions. However, ignoring dependence within clusters can produce confidence intervals that cover the true parameter less often than their nominal rate. We introduce Cluster-Robust PPI++, which provides standard errors in closed form and asymptotically valid confidence intervals under arbitrary dependence within independent clusters, requiring no bootstrap or resampling. Our central contribution is to accommodate partially labeled clusters, a common empirical setting in which clusters contain both labeled and unlabeled units. As units are dependent within clusters, partially labeled clusters violate the independence assumption of PPI++. We also show how precision increases depend on the labeling design, and derive a cluster-aware power tuning rule that minimizes asymptotic variance. In an application to television news, standard PPI++ confidence intervals have coverage below 60%, whereas Cluster-Robust PPI++ can achieve nominal 95% coverage.

Conditional density estimation targets the full distribution of a response given covariates, as required, for example, for per-galaxy photometric redshifts. We develop a scalable Bayesian estimator based on the logistic Gaussian process. The log conditional density has a separable covariance: a Matérn kernel along the response, represented in a truncated Fourier basis on a circle, and a covariate kernel represented by Nyström features, which accommodate non-stationary kernels with input-dependent amplitudes and length scales. Instead of a Laplace or variational approximation, we sample the latent field of this finite-feature model. Given the hyperparameters, its posterior is strongly log-concave with a uniformly bounded Hessian, and we draw from it by simulating kinetic Langevin dynamics with symmetric minibatch splitting in Kronecker-whitened coordinates. Marginal-likelihood gradients follow from Fisher's identity as posterior expectations. Under the conditions of our analysis their bias is controlled by the sampler's step size and run length, and the predictive averages over the non-Gaussian latent posterior instead of a Gaussian around its mode. On photometric-redshift benchmarks with up to 3.9 million training observations, trained on a single GPU, the estimator is competitive with state-of-the-art tabular foundation models on density and calibration metrics.

Signals that predict whether a chain-of-thought (CoT) trace is correct are compared by AUC, but deploying one requires a threshold with a guarantee. We ask what distribution-free selective guarantees deliver for CoT verifiers at realistic calibration budgets of tens to a few hundred labelled problems, using seven open models, five verifier signals and 37,000 graded traces. The central observation is validity by abstention: an $(\alpha,\delta)$-valid procedure that issues a certificate with probability $P_{\rm fire}$ bounds the failure probability of an issued certificate only by $\delta/P_{\rm fire}$, so a certificate that rarely fires can be valid and wrong every time it is used. In a simulation with known risk the standard certificate fails in at most 0.3% of calibration draws but in up to 69% of those in which it fires. A certification floor and a lattice condition for Benjamini-Hochberg conformal selection explain why certificates abstain at these budgets, and the data bear them out: the standard certificate returns nothing or a large accepted set, and an unreadable residual-stream probe buys two to three times the coverage of the readable signals, an edge a cross-fitted reconstruction cannot recover linearly from the readable features. We then give a floor-started fixed-sequence certificate, valid without monotonicity assumptions, that covers more than the Bonferroni certificate on every model-signal pair and raises coverage at the non-vacuous target $0.75\pi_0$ from 0.05 to 0.16, although the floor keeps absolute coverage small. Finally, a certificate cannot see what matters after deployment: under benchmark shift the error among accepted traces tracks the new task's base error, and under best-of-$n$ selection against the verifier it rises past the target while the empirical failure frequency stays below $\delta$, because abstention absorbs the failures.

Canonical Correlation Analysis (CCA) is a fundamental method for multiview shared space learning. However, its strict reliance on paired data poses a significant limitation, as such data is often difficult to obtain or entirely unavailable. In this paper, we present Unpaired CCA (UCCA), a novel method that learns linear projections to maximize the correlation of the true underlying pairing without access to any paired samples during training. We first establish theoretical results connecting the Quadratic Assignment Problem (QAP) to CCA. Leveraging these theoretical insights, we derive a practical method to maximize correlation exclusively from unpaired data. To the best of our knowledge, UCCA is the first approach to learn maximally correlated projections in a strictly unpaired setting. We validate UCCA on real-world multi-modal datasets, demonstrating that it significantly outperforms recent unpaired alignment baselines in recovering the underlying true correlation. This work fills a critical gap between traditional statistical multiview learning and the growing field of unpaired data learning.

First order Langevin algorithms for constrained sampling in machine learning, such as projected Langevin Monte Carlo which are based on discretizations of reflected Langevin dynamics, require differentiable log densities that limits their applicability. This paper introduces reflected anchored Langevin dynamics (RALD), a reflected diffusion that converges to non-differentiable targets on constrained domains. The method uses a smooth anchored reference potential and multiplies the drift and noise covariance of its reflected Langevin dynamics by the same state dependent scaling factor. Its Euler-Maruyama discretization with projection gives reflected anchored Langevin Monte Carlo (RALMC) algorithm. We prove explicit convergence bounds and iteration complexity for RALMC in the 2-Wasserstein distance to the target distribution. Numerical experiments are provided to illustrate the theoretical predictions and the empirical performance of the method.

We develop a bias-aware digital-twin calibration and control framework for multiscale bioprocess models within a biological systems-of-systems (Bio-SoS) paradigm. The digital twin is represented by a stochastic differential equation (SDE) model and calibrated from sparse, discrete observations using quasi-likelihood estimation and adjoint sensitivity analysis. SDE generator-based moment expansions characterize truncation-induced parameter bias, while forward-backward adjoints quantify how calibration uncertainty propagates to value functions and policy performance. The resulting parameter-error distribution supports both policy-directed adaptive experimental design and uncertainty-aware policy optimization through a second-order Gaussian-averaged objective. We characterize the asymptotic behavior of the resulting exploration criterion and derive a physical-system performance under the optimized policy. To implement these ideas, we develop an Actor-Simulator algorithm that jointly updates model parameters, selects informative experiments, and optimizes control policies. Numerical studies demonstrate improved calibration accuracy, sample efficiency, and control performance relative to state-of-the-art baselines.

Drive-by structural health monitoring infers the condition of a bridge from the response of an instrumented vehicle, whose measurements are dominated by the road roughness and the vehicle dynamics. The bridge responses, contact forces and bridge parameters depend on one another through the vehicle-bridge interaction (VBI), so estimating them jointly is desirable. This paper proposes a novel filter in the Universal Filter family, the Unscented Universal Filter (UUF), for joint input-state-parameter estimation of nonlinear systems with unknown inputs. The unknown input is estimated by weighted least squares from the innovation, without any evolution model or statistics, and the system inversion imposes no rank condition on the feedforward matrix. The nonlinear propagation is linearised by the scaled unscented transform, and the linearisation residuals are propagated into the error covariances. For the acceleration-only configuration, auxiliary integrators form pseudo-measurements from the acceleration measurements themselves. The UUF is applied to the coupled VBI system, with the contact forces as the unknown input and the local stiffness of the bridge as the tracked parameter, from the responses of the vehicle body and at most a few bridge responses. In numerical case studies on a simply supported beam and the Old Ada Bridge, a steel truss bridge in Japan, the UUF is compared with classical and state-of-the-art joint estimation filters, all tuned by the same practical strategy. Across the damage severities, locations, road classes and noise levels examined, the UUF yields the lowest total error in most scenarios, and its error varies little with damage severity, location and road class. On the Old Ada Bridge, it identifies the stiffness of a damaged member from acceleration-only measurements, without the suspension parameters of the vehicle, and with a rank-deficient feedforward matrix.

Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity. Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed. We close this gap: individual world latents can be provably recovered with no reconstruction, no decoder, and no labels. The key condition is Structural Diversity: different latents leave distinct dependency footprints on observations, just as no two snowflakes are alike. Building on the linear identifiability that LeJEPA provides, we prove that under Structural Diversity, DSReg (Dependency-Sparsity Regularization) recovers individual world latents up to signed permutation, without reconstruction or a decoder. It applies post hoc to any linearly identified representation, reusing trained checkpoints at no loss over joint training, and establishes the first fully identifiable JEPA that recovers every world latent. Moreover, as a condition on dependency footprints, Structural Diversity is strictly weaker than all structural conditions of prior identifiable latent variable models. Across synthetic regimes, world model probes, learned visual encoders, and external renderers, DSReg preserves dense prediction while improving individual-latent recovery and downstream use with scales.

Conditional density estimation describes how the entire distribution of a response changes with covariates, and in imaging studies also with location. Logistic Gaussian processes (LGP) give a flexible prior for such densities. However, the normalizing constant of an LGP has no closed form, so existing methods approximate or replace the likelihood. We propose the exact likelihood finite-rank LGP (ExFR-LGP), which writes the log density as the sum of two bivariate functions, one of the response and a location-varying linear index of the covariates, and one of the response and the location. Each function is assigned a finite-rank Gaussian process prior that is piecewise linear on a regular grid, enabling the normalizing constant, the conditional mean and the quantiles to enjoy closed form representations. Posterior samples are drawn from the exact posterior under this prior via a Gibbs sampler that updates the Gaussian components by elliptical slice sampling. When the true log density is the sum of two such bivariate functions, we show that the posterior contracts at the minimax rate of an $\alpha$-smooth bivariate density up to a logarithmic factor. Numerical experiments using synthetic data demonstrate the effectiveness of ExFR-LGP in conditional density estimation. Furthermore, application to fractional anisotropy responses along the corpus callosum in the Alzheimer's Disease Neuroimaging Initiative data yields covariate-adjusted percentile bands with uncertainty, elucidating how diagnosis changes the distribution along the tract.

Manifold fitting seeks to recover the geometric structure underlying noisy ambient observations. We propose Successive Tangent Space Projection (STSP), an iterative manifold-fitting method that reduces normal noise while preserving tangential variation. The former improves fitting accuracy, whereas the latter helps retain geometric coverage. We characterise the \rev{nearby population fixed-point set} of STSP as a manifold-fitting object. Under uniform sampling from a compact smooth manifold with positive reach and isotropic Gaussian noise, this set lies within $O(\sigma^2)$ of the underlying manifold, and \rev{nearby population orbits} converge geometrically to it. At the finite-sample level, with high probability, fixed points in the local tube lie within $O(\sigma^2)$ of the underlying manifold up to sampling error, and empirical orbits \rev{initialised within that tube} and generated from a fixed reference sample enter and remain in the same neighbourhood. We further develop a multi-scale extension, MS-STSP, designed to reduce curvature bias while preserving the $O(\sigma^2)$ geometric localization order of STSP at both the population and finite-sample levels. Numerical experiments show that STSP compares favourably with competing methods in balancing fitting accuracy and geometric coverage, and that MS-STSP reduces curvature-induced shrinkage, particularly under high noise.

We study the problem of sampling from a target distribution $\pi(x)\propto e^{-f(x)}$ supported on a convex set $ X\subseteq\mathbb R^d$, when the potential $f$ is accessible only through a stochastic first-order oracle. Mirror Langevin algorithms provide a natural approach to constrained sampling by transporting the problem to an unconstrained dual space and discretizing the resulting Mirror Langevin diffusion. Existing implementations, however, typically use a fixed discretization step size and consequently retain a nonvanishing discretization bias at any fixed step size. Moreover, their direct extension to settings with noisy gradient information entails the challenge of controlling both discretization and stochastic-oracle error. We study a stochastic first-order version of the Mirror Langevin Algorithm (sFO-MLA) and, as our main contribution, develop a warm-started two-loop implementation in which an outer loop progressively decreases the step size while an inner loop runs sFO-MLA (with a fixed step size) for an appropriately chosen epoch length. The construction provides a principled schedule linking step sizes and epoch lengths, so that successive epochs warm-start from increasingly accurate distributions rather than repeatedly paying the cost of mixing from a cold start. We establish finite-time Wasserstein guarantees for sFO-MLA that explicitly separate mixing, Euler–Maruyama discretization, and stochastic-gradient errors. These bounds yield a fixed horizon rate of $\widetilde O(T^{-1/2})$ and show that the two-loop scheme removes the associated logarithmic penalty, attaining the canonical $O(T^{-1/2})$ rate under a geometric step-size schedule and corresponding epoch lengths. We illustrate the methodology in two statistically distinct settings.