Papers

We provide a data-driven method for choosing the scale of a Gaussian prior for non-linear Bayesian inverse problems arising from semilinear PDEs. Following \cite{koers2024}, the non-linear model for the parameter $f$ is reparametrized as a linear model for $v = \mathbb{L}u_f$, in which the prior scale can be selected by early stopping at the discrepancy principle \cite{tienstra2025}. We extend the transfer of frequentist guarantees from the linearised problem to the original one beyond Lipschitz solution maps, to maps admitting an arbitrary modulus of continuity, and we provide a general condition on the non-linearity under which such a map exists and is Lipschitz near the truth. The resulting posterior for $f$ contracts adaptively over a range of Sobolev smoothness, and its credible sets have asymptotic frequentist coverage one. We demonstrate our theory in detail for the stationary Schrödinger equation, for which we also provide numerical experiments. We further show how the results apply to the stationary Allen-Cahn equation and to a parabolic Allen-Cahn equation, and we discuss Darcy flow as an edge case outside the semilinear setting. The proposed method thus provides a data-driven way to tune Gaussian priors via early stopping, which is computationally efficient and statistically near-optimal for non-linear problems.

The ROC curve is widely used to assess binary classifiers. Yet for some applications, such as alert systems for monitoring hospitalized patients, conventional ROC analysis cannot meet two key deployment needs: enforcing a constraint on precision to avoid false alarm fatigue and imposing an upper bound on the number of predicted positives to represent the capacity of hospital staff. The usual area under the curve metric also does not reflect asymmetric costs for false positives and false negatives. In this paper we address all three of these issues. First, we show how the subset of classifiers that meet precision and capacity constraints occupy a feasible region in ROC space. We establish the polygon-shaped geometry of this region. We then define the partial area of lesser classifiers, a performance metric that is monotonic with cost and only accounts for the feasible region. Averaging this area over a desired distribution for cost parameters results in the partial volume over the ROC surface, or partial VOROS. In experiments predicting mortality risk from vital sign history on several datasets, we show this cost-aware metric can outperform alternatives at ranking classifiers for in-hospital alerts.

We revisit the problem of parameter estimation for discrete probability distributions with values in $\mathbb{Z}^d$. To this end, we adapt a technique called Stein's Method of Moments to discrete distributions which often gives closed-form estimators when standard methods such as maximum likelihood estimation (MLE) require numerical optimization. These new estimators exhibit good performance in small-sample settings which is demonstrated by means of a comparison to the MLE through simulation studies. We pay special attention to truncated distributions and show that the asymptotic behavior of our estimators is not affected by an unknown (rectangular) truncation domain.

High-dimensional non-convex loss landscapes play a central role in the theory of Machine Learning. Gaining insight into how these landscapes interact with gradient-based optimization methods, even in relatively simple models, can shed light on this enigmatic feature of neural networks. In this work, we will focus on a prototypical simple learning problem, which generalizes the Phase Retrieval inference problem by allowing the exploration of overparametrized settings. Using techniques from field theory, we analyze the spectrum of the Hessian at initialization and identify a Baik-Ben Arous-Péché (BBP) transition in the amount of data that separates regimes where the initialization is informative or uninformative about a planted signal of a teacher-student setup. Crucially, we demonstrate how overparameterization can bend the loss landscape, shifting the transition point, even reaching the information-theoretic weak-recovery threshold in the large overparameterization limit, while also altering its qualitative nature. We distinguish between continuous and discontinuous BBP transitions and support our analytical predictions with simulations, examining how they compare to the finite-N behavior. In the case of discontinuous BBP transitions strong finite-N corrections allow the retrieval of information at a signal-to-noise ratio (SNR) smaller than the predicted BBP transition. In these cases we provide estimates for a new lower SNR threshold that marks the point at which initialization becomes entirely uninformative.

Thermocouples are in widespread use in industry, but they are particularly susceptible to calibration drift in harsh environments. Self-validating thermocouples aim to address this issue by using a miniature phase-change cell (fixed-point) in close proximity to the measurement junction (tip) of the thermocouple. The fixed point is a crucible containing an ingot of metal with a known melting temperature. When the process temperature being monitored passes through the melting temperature of the ingot, the thermocouple output exhibits a "plateau" during melting. Since the melting temperature of the ingot is known, the thermocouple can be recalibrated in situ. Identifying the melting plateau to determine the onset of melting is reasonably well established but requires manual intervention involving zooming in on the region around the actual melting temperature, a process which can depend on the shape of the melting plateau. For the first time, we present a novel machine learning approach to recognize and identify the characteristic shape of the melting plateau and once identified, to quantity the point at which melting begins, along with its associated uncertainty. This removes the need for human intervention in locating and characterizing the melting point. Results from test data provided by CCPI Europe show 100% accuracy of melting plateau detection. They also show a cross-validated R2 of 0.99 on predictions of calibration drift.

Standard dropout applies a fixed, uniform noise rate to all units, ignoring available gradient information. We propose momentum adaptive gradient dropout (MAGDrop), a Bayesian linear layer that adapts per-unit weight noise by tracking an exponential moving average of gradient magnitudes. Units with large recent gradient magnitudes receive less noise to preserve learned features; units with below-average gradient magnitudes receive more noise to promote exploration. This adaptation transforms the classifier head into a data-dependent Bayesian predictor whose mean predictor has no additional inference cost. Our primary contribution is a certificate-aware low-rank posterior, MAGDrop-Subspace, that addresses the tension between strong accuracy and tight bounds from probably approximately correct (PAC)-Bayes theory. A full-head stochastic posterior incurs a Kullback–Leibler (KL) divergence of approximately 8,625 nats. By confining the posterior to an $r$-dimensional subspace constructed from an independent prior-construction sample $S_0$ and certifying on an independent sample $S$ with $S_0\perp S$, MAGDrop-Subspace reduces KL by approximately a factor of 800 while keeping accuracy close to the baseline. On CIFAR-10 with a 13-layer convolutional neural network and $r=50$, MAGDrop-Subspace achieves a PAC-Bayes Gibbs-risk bound of 0.1882 at 90% confidence using binary KL inversion, compared with 0.4439 for the full-head baseline. Mean-predictor accuracy is 82.31%, compared with 83.74% for the baseline. Complementary accuracy experiments on ImageNet-1K show that MAGDrop reaches top-1 accuracy similar to the strong baseline: 78.57% compared with 78.51% on the base Vision Transformer with $16\times16$ patches, and 82.14% compared with 82.23% on ConvNeXt-Base.

For statistical models on circles, we investigate performance of estimators defined as the projections of the empirical distribution with respect to the Wasserstein distance. We develop algorithms for computing the Wasserstein projection estimators based on a formula of the Wasserstein distances on circles. Numerical results on the von Mises, wrapped Cauchy, and sine-skewed von Mises distributions show that the Wasserstein projection estimators attain estimation accuracy comparable to the maximum likelihood estimator. In addition, the Wasserstein projection estimators are found to be robust against noise contamination.

We introduce parametrisation of that property of the available training dataset, that necessitates an inhomogeneous correlation structure for the function that is learnt as a model of the relationship between the pair of variables, observations of which comprise the considered training data. We refer to a parametrisation of this property of a given training set, as its “inhomogeneity parameter”. It is easy to compute this parameter for small-to-large datasets, and we demonstrate such computation on multiple publicly-available datasets, while also demonstrating that conventional “non-stationarity” of data does not imply a non-zero inhomogeneity parameter of the dataset. We prove that - within the probabilistic Gaussian Process-based learning approach - a training set with a non-zero inhomogeneity parameter renders it imperative, that the process that is invoked to model the sought function, be non-stationary. Following the learning of a real-world multivariate function with such a Process, quality and reliability of predictions at test inputs, are demonstrated to be affected by the inhomogeneity parameter of the training data.

Uncertainty quantification (UQ) is crucial for deploying machine learning models in high-stakes applications, where overconfident predictions can lead to serious consequences. An effective UQ method must balance computational efficiency with the ability to generalize across diverse scenarios. Evidential deep learning (EDL) achieves efficiency by modeling uncertainty through the prediction of a Dirichlet distribution over class probabilities. However, the restrictive assumption of Dirichlet-distributed class probabilities limits EDL's robustness, particularly in complex or unforeseen situations. To address this, we propose flexible evidential deep learning ($\mathcal{F}$-EDL), which extends EDL by predicting a flexible Dirichlet distribution – a generalization of the Dirichlet distribution – over class probabilities. This approach provides a more expressive and adaptive representation of uncertainty, significantly enhancing UQ generalization and reliability under challenging scenarios. We theoretically establish several advantages of $\mathcal{F}$-EDL and empirically demonstrate its state-of-the-art UQ performance across diverse evaluation settings, including classical, long-tailed, and noisy in-distribution scenarios.

Modeling hierarchical latent dynamics behind time series data is critical for capturing temporal dependencies across multiple levels of abstraction in real-world tasks. However, existing temporal causal representation learning methods fail to capture such dynamics, as they fail to recover the joint distribution of hierarchical latent variables from single-timestep observed variables. Interestingly, we find that the joint distribution of hierarchical latent variables can be uniquely determined using three conditionally independent observations. Building on this insight, we propose a Causally Hierarchical Latent Dynamic (CHiLD) identification framework. Our approach first employs temporal contextual observed variables to identify the joint distribution of multi-layer latent variables. Sequentially, we exploit the natural sparsity of the hierarchical structure among latent variables to identify latent variables within each layer. Guided by the theoretical results, we develop a time series generative model grounded in variational inference. This model incorporates a contextual encoder to reconstruct multi-layer latent variables and normalize flow-based hierarchical prior networks to impose the independent noise condition of hierarchical latent dynamics. Empirical evaluations on both synthetic and real-world datasets validate our theoretical claims and demonstrate the effectiveness of CHiLD in modeling hierarchical latent dynamics.

The inference of evolutionary histories is a central problem in evolutionary biology. The analysis of a sample of phylogenetic trees can be conducted in Billera-Holmes-Vogtmann tree space, which is a CAT(0) metric space of phylogenetic trees. The globally non-positively curved (CAT(0)) property of this space enables the extension of various statistical techniques. In the problem of nonparametric density estimation, two primary methods, kernel density estimation and log-concave maximum likelihood estimation, have been proposed, yet their theoretical properties remain largely unexplored. In this paper, we address this gap by proving the consistency of these estimators in a more general setting$\unicode{x2014}$CAT(0) orthant spaces, which include BHV tree space. We extend log-concave approximation techniques to this setting and establish consistency via the continuity of the log-concave projection map. We also modify the kernel density estimator to correct boundary bias and establish uniform consistency using empirical process theory.

The use of low-bit quantization has emerged as an indispensable technique for enabling the efficient training of large-scale models. Despite its widespread empirical success, a rigorous theoretical understanding of its impact on learning performance remains notably absent, even in the simplest linear regression setting. We present the first systematic theoretical study of this fundamental question, analyzing finite-step stochastic gradient descent (SGD) for high-dimensional linear regression under a comprehensive range of quantization targets: data, label, parameter, activation, and gradient. Our novel analytical framework establishes precise algorithm-dependent and data-dependent excess risk bounds that characterize how different quantization affects learning: parameter, activation, and gradient quantization amplify noise during training; data quantization distorts the data spectrum and introduces additional approximation error. Crucially, we distinguish the effects of two quantization schemes: we prove that for additive quantization (with constant quantization steps), the noise amplification benefits from a suppression effect scaled by the batch size, while multiplicative quantization (with input-dependent quantization steps) largely preserves the spectral structure, thereby reducing the spectral distortion. Furthermore, under common polynomial-decay data spectra, we quantitatively compare the risks of multiplicative and additive quantization, drawing a parallel to the comparison between FP and integer quantization methods. Our theory provides a powerful lens to characterize how quantization shapes the learning dynamics of optimization algorithms, paving the way to further explore learning theory under practical hardware constraints.

Credit scoring models face a critical challenge: severe class imbalance, with default rates typically below 10%, which hampers model learning and predictive performance. While synthetic data augmentation techniques such as SMOTE and ADASYN have been proposed to address this issue, the optimal augmentation ratio remains unclear, with practitioners often defaulting to full balancing (1:1 ratio) without empirical justification. This study systematically evaluates 10 data augmentation scenarios using the Give Me Some Credit dataset (97,243 observations, 7% default rate), comparing SMOTE, BorderlineSMOTE, and ADASYN at different multiplication factors (1x, 2x, 3x). All models were trained using XGBoost and evaluated on a held-out test set of 29,173 real observations. Statistical significance was assessed using bootstrap testing with 1,000 iterations. Key findings reveal that ADASYN with 1x multiplication (doubling the minority class) achieved optimal performance with AUC of 0.6778 and Gini coefficient of 0.3557, representing statistically significant improvements of +0.77% and +3.00% respectively (p = 0.017, bootstrap test). Higher multiplication factors (2x and 3x) resulted in performance degradation, with 3x showing a -0.48% decrease in AUC, suggesting a "law of diminishing returns" for synthetic oversampling. The optimal class imbalance ratio was found to be 6.6:1 (majority:minority), contradicting the common practice of balancing to 1:1. This work provides the first empirical evidence of an optimal "sweet spot" for data augmentation in credit scoring, with practical guidelines for industry practitioners and researchers working with imbalanced datasets. While demonstrated on a single representative dataset, the methodology provides a reproducible framework for determining optimal augmentation ratios in other imbalanced domains.

Non-Euclidean object-valued time series are playing a growing role in modern data analysis, and periodicity is a fundamental characteristic of many time series. However, quantifying periodicity in general non-Euclidean random objects remains largely unexplored. In this work, we introduce a novel nonparametric framework for quantifying periodicity in random objects within a general metric space that lacks Euclidean structure. Our approach formulates periodicity estimation as a model selection problem and provides methodologies for period estimation, data-driven tuning parameter selection, and periodic component extraction. Our theoretical contributions include deriving uniform convergence rates for general Fréchet regression under temporal dependence, establishing the consistency of period estimation without relying on linearity properties used in the literature for Euclidean data, providing theoretical support for data-driven tuning parameter selection, and deriving uniform convergence results for periodic component estimation. Through extensive simulation studies covering three distinct types of time-varying random objects such as compositional data, networks, and functional data, we showcase the excellent accuracy achieved by our approach in periodicity quantification. Finally, we apply our method to various real datasets, including compositional data arising in U.S. electricity generation, New York City transportation networks, and Germany's water consumption curves, highlighting its practical relevance in identifying and quantifying meaningful periodic patterns.

We study parallel sampling from high-dimensional strongly log-concave distributions. Langevin-based samplers converge rapidly in continuous time, but their discretizations are typically sequential and often require polynomially many steps in the dimension $d$, the target accuracy $\varepsilon^{-1}$, or both. Picard-based parallel sampling methods reduce this sequential depth to polylogarithmic scale by solving for many time-discretization points in parallel; however, existing guarantees often require a polynomial number of processors, leading to substantial memory and gradient-evaluation costs in high dimensions. We show that higher-order Langevin structure can reduce this parallel resource burden while preserving polylogarithmic sequential depth. Our method combines arbitrary-order Langevin dynamics with blockwise Lagrange polynomial interpolation. This sharper discretization reduces the number of parallel points required to achieve a target accuracy. Our results cover both higher-order smooth potentials and ridge-separable potentials, including models such as Bayesian logistic regression and two-layer neural networks, and improve upon the space complexity of the current literature on parallel log-concave sampling.

Parametric factor copula models typically work well in modeling multivariate dependencies due to their flexibility and ability to capture complex dependency structures. However, accurately estimating the linking copulas within these models remains challenging, especially when working with high-dimensional data. This paper proposes a novel approach for estimating linking copulas based on a non-parametric kernel estimator. Unlike conventional parametric methods, our approach utilizes the flexibility of kernel density estimation to capture the underlying dependencies more accurately, particularly in scenarios where the underlying copula structure is complex or unknown. We show that the proposed estimator is consistent under mild conditions and demonstrate its effectiveness through extensive simulation studies. Our findings suggest that the proposed approach offers a promising avenue for modeling multivariate dependencies, particularly in applications requiring robust and efficient estimation of copula-based models.

Data-driven stochastic optimization is ubiquitous in machine learning and operational decision-making problems. Sample average approximation (SAA) and model-based approaches such as estimate-then-optimize (ETO) or integrated estimation-optimization (IEO) are all popular, with model-based approaches being able to circumvent some of the issues with SAA in complex context-dependent problems. Yet the relative performance of these methods is poorly understood, with most results confined to the dichotomous cases of the model-based approach being either well-specified or misspecified. We develop the first results that allow for a more granular analysis of the relative performance of these methods under a local misspecification setting, which models the scenario where the model-based approach is nearly well-specified. By leveraging tools from contiguity theory in statistics, we show that there is a bias-variance tradeoff between SAA, IEO, and ETO under local misspecification, and that the relative importance of the bias and the variance depends on the degree of local misspecification. Moreover, we derive explicit expressions for the decision bias, which allows us to characterize (un)impactful misspecification directions, and provide further geometric understanding of the variance.

Fair, transparent, and explainable decision-making remains a critical challenge in Olympic and Paralympic combat sports. This paper presents FST.ai 2.0, an explainable AI ecosystem designed to support referees, coaches, and athletes in real time during Taekwondo competitions and training. The system integrates {pose-based action recognition} using graph convolutional networks (GCNs), {epistemic uncertainty modeling} through credal sets, and {explainability overlays} for visual decision support. A set of {interactive dashboards} enables human–AI collaboration in referee evaluation, athlete performance analysis, and Para-Taekwondo classification. Beyond automated scoring, FST.ai~2.0 incorporates modules for referee training, fairness monitoring, and policy-level analytics within the World Taekwondo ecosystem. Experimental validation on competition data demonstrates an {85% reduction in decision review time} and {93% referee trust} in AI-assisted decisions. The framework thus establishes a transparent and extensible pipeline for trustworthy, data-driven officiating and athlete assessment. By bridging real-time perception, explainable inference, and governance-aware design, FST.ai~2.0 represents a step toward equitable, accountable, and human-aligned AI in sports.

We consider the situation where we have a limited number of denoising steps, i.e., of evaluations of a diffusion model. We show that two parallel processors or samplers under such limitation can improve the quality of the sampled image. Particularly, the two samplers make denoising steps at successive times, and their information is appropriately integrated in the latent image. Remarkably, our method is simple both conceptually and to implement: it is plug-&-play, model agnostic, and does not require any additional fine-tuning or external models. We test our method with both automated and human evaluations for different diffusion models. We also show that a naive integration of the information from the two samplers lowers sample quality. Finally, we find that adding more parallel samplers does not necessarily improve sample quality.

Matrix-based preconditioned optimizers, such as Muon, have recently been shown to be more efficient than scalar-based optimizers for training large-scale neural networks, including large language models (LLMs). Recent benchmark studies of LLM pretraining optimizers have demonstrated that variance-reduction techniques such as MARS can substantially speed up training compared with standard optimizers that do not employ variance reduction. In this paper, we introduce MARS-M, a new optimizer that integrates MARS-style variance reduction with Muon. Under standard regularity conditions, we prove that MARS-M converges to a first-order stationary point at a rate of $\tilde{\mathcal{O}}(T^{-1/3})$, improving upon the $\tilde{\mathcal{O}}(T^{-1/4})$ rate attained by Muon. Empirical results on language modeling and computer vision tasks demonstrate that MARS-M consistently yields lower losses and improved performance across various downstream benchmarks. The implementation of MARS-M is available at https://github.com/AGI-Arena/MARS/tree/main/MARS_M.

We study a minimal change to an observation-driven Bayesian Dirichlet ARMA (B–DARMA) for compositional time series: replace the raw additive log-ratio (ALR) residual in the moving-average block with a centered innovation that subtracts the Dirichlet conditional ALR mean, available in closed form via digamma identities. We prove a recursion-level first-order equivalence (in $1/\phi$) between the centered specification and a digamma-link DARMA at fixed parameters, under explicit interior and lag-stability conditions. The result clarifies why the two specifications should be predictively indistinguishable in the high-precision regime but does not by itself govern the geometry of the Bayesian posteriors that re-estimation produces. On weekly Federal Reserve H.8 bank-asset shares (October~2015 through October~2025, $T=522$ weeks), predictive performance is statistically indistinguishable across $104$ rolling weekly origins on every accuracy metric examined, while Hamiltonian Monte Carlo divergent transitions are approximately an order of magnitude more frequent under the raw specification, driven by isolated rolling fits at which the raw posterior exhibits localized pathologies. A four-reference sensitivity analysis confirms that predictive equivalence is reference-invariant and that the geometric advantage of centering is preserved across references but varies with the prevalence of pathological raw fits, from a substantial reduction at the loans reference to parity at the cash reference. The practical implication is operational rather than predictive: centering avoids the catastrophic raw-MA divergence spikes that occur at isolated rolling origins, which matters for production workflows in which posterior simulation feeds downstream stress tests. The adjustment is analytic and plug-in, and requires only a local change to the MA innovation calculation.

There has been a surge of recent interest in automatically learning policies to target treatment decisions based on rich individual covariates. In addition, practitioners want confidence that the learned policy has better performance than the incumbent policy according to downstream policy evaluation. However, due to the winner's curse – an issue where the policy optimization procedure exploits prediction errors rather than finding actual improvements – predicted performance improvements are often not substantiated by downstream policy evaluation. To address this challenge, we propose a novel strategy called inference-aware policy optimization, which modifies policy optimization to account for how the policy will be evaluated downstream. Specifically, it optimizes not only for the estimated objective value, but also for the chances that the estimate of the policy's improvement passes a significance test during downstream policy evaluation. We mathematically characterize the Pareto frontier of policies according to the tradeoff of these two goals. Based on our characterization, we design a policy optimization algorithm that estimates the Pareto frontier using machine learning models; then, the decision-maker can select the policy that optimizes their desired tradeoff, after which policy evaluation can be performed on the test set as usual. Finally, we perform simulations to illustrate the effectiveness of our methodology.

We develop a novel approach to tackle the common but challenging problem of conformal inference for missing data in machine learning, focusing on Missing at Random (MAR) data. We propose a new procedure Conformal prediction for Missing data under Multiple Robust Learning (CM–MRL) that combines split conformal calibration with a multiple robust empirical-likelihood (EL) reweighting scheme. The method proceeds via a double calibration by reweighting the complete-case scores by EL so that their distribution matches the full calibration distribution implied by MAR, even when some working models are misspecified. We demonstrate the asymptotic behavior of our estimators through empirical process theory and provide reliable coverage for our prediction intervals, both marginally and conditionally and we further show an interval-length dominance result. We show the effectiveness of the proposed method by several numerical experiments in the presence of missing data.

Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-convex (DC) framework, we present several novel formulations and provide new theoretical insights. In particular, we show the kernelizability and out-of-sample applicability for a PCA-like family of problems. Moreover, we uncover that simultaneous iteration, which is connected to the classical QR algorithm, is an instance of the difference-of-convex algorithm (DCA), offering an optimization perspective on this longstanding method. Further, we describe new algorithms for PCA and empirically compare them with state-of-the-art methods. Lastly, we introduce a kernelizable dual formulation for a robust variant of PCA that minimizes the $l_1$ deviation of the reconstruction errors.

The consistency of posterior distributions in density estimation is at the core of Bayesian statistical theory. Classical work established sufficient conditions, typically combining KL support with complexity bounds on sieves of high prior mass, to guarantee consistency with respect to the Hellinger distance. Yet no systematic theory explains a widely held belief: under KL support, Hellinger consistency is exceptionally hard to violate. This suggests that existing sufficient conditions, while useful in practice, may overlook some key aspects of posterior behavior. We address this gap by directly investigating what must fail for inconsistency to arise, aiming to identify a substantive necessary condition for Hellinger inconsistency. Our starting point is Andrew Barron's classical counterexample, the only known violation of Hellinger consistency under KL support, which relies on a contrived family of oscillatory densities and a prior with atoms. We show that, within a broad class of models including Barron's, inconsistency requires persistent posterior concentration on densities with exponentially high likelihood ratios. In turn, such behavior demands a prior encoding implausibly precise knowledge of the true, yet unknown data-generating distribution, making inconsistency essentially unattainable in any realistic inference problem. Our results confirm the long-standing intuition that posterior inconsistency in density estimation is not a natural phenomenon, but rather an artifact of pathological prior constructions.

Understanding generalization in overparameterized neural networks hinges on the interplay between the data geometry, neural architecture, and training dynamics. In this paper, we theoretically explore how data geometry controls this implicit bias. This paper presents theoretical results for overparametrized two-layer ReLU networks trained below the edge of stability. First, for data distributions supported on a mixture of low-dimensional balls, we derive generalization bounds that provably adapt to the intrinsic dimension. Second, for a family of isotropic distributions that vary in how strongly probability mass concentrates toward the unit sphere, we derive a spectrum of bounds showing that rates deteriorate as the mass concentrates toward the sphere. These results instantiate a unifying principle: When the data is harder to "shatter" with respect to the activation thresholds of the ReLU neurons, gradient descent tends to learn representations that capture shared patterns and thus finds solutions that generalize well. On the other hand, for data that is easily shattered (e.g., data supported on the sphere) gradient descent favors memorization. Our theoretical results consolidate disparate empirical findings that have appeared in the literature.

Structural equation models (SEMs) are fundamental to causal mediation pathway discovery. However, traditional SEM approaches often rely on ad hoc model specifications when handling complex data structures such as mixed data types or non-normal data in which Gaussian assumptions for errors are rather restrictive. The invocation of copula dependence modeling methods to extend the classical linear SEMs mitigates several of key technical limitations, offering greater modeling flexibility to analyze non-Gaussian data. This paper presents a selective review of major developments in this area, highlighting recent advancements and their methodological implications.

Discrete diffusion models have emerged as a powerful class of models and a promising route to fast language generation, but practical implementations typically rely on factored reverse transitions ignoring cross-token dependencies and degrading few-step performance. We propose Latent-Augmented Discrete Diffusion (LADD), which introduces a learnable auxiliary latent channel and performs diffusion over the joint (token, latent) space. The latent variables provide an intermediate representation expressing joint structure while preserving tractable parameterizations. We instantiate LADD with continuous latents (Co-LADD) and discrete latents (Di-LADD), and study two inference schedules: a joint diffusion that denoises data and latents together, and a sequential diffusion that first resolves latents and then samples tokens conditionally. We derive ELBO-style objectives and analyze design choices that balance latent expressivity with diffusion compatibility. In experiments, LADD models yield improvements on unconditional generation metrics as compared to state-of-the-art masked discrete diffusion baselines, and are effective at lower sampling budgets, where unmasking many tokens per step is desirable.

Bayesian optimization (BO) is a powerful framework for estimating parameters of expensive simulation models, particularly in settings where the likelihood is intractable and evaluations are costly. In stochastic models every simulation is run with a specific parameter set and an implicit or explicit random seed, where each parameter set and random seed combination generates an individual realization, or trajectory, sampled from an underlying random process. Existing BO approaches typically rely on summary statistics over the realizations, such as means, medians, or quantiles, potentially limiting their effectiveness when trajectory-level information is desired. We propose a trajectory-oriented BO method that incorporates a Gaussian process surrogate using both input parameters and random seeds as inputs, enabling direct inference at the trajectory level. Using a common random number approach, we define a surrogate-based likelihood over trajectories and introduce an adaptive Thompson Sampling algorithm that refines a fixed-size input grid through likelihood-based filtering and Metropolis-Hastings-based densification. This approach concentrates computation on statistically promising regions of the input space while balancing exploration and exploitation. We apply the method to stochastic epidemic models, a simple compartmental and a more computationally demanding agent-based model, demonstrating improved sampling efficiency and faster identification of data-consistent trajectories relative to parameter-only inference.

We derive explicit comparison bounds for multivariate max-stable distributions with unit-$\alpha$-Fréchet margins. For the Kolmogorov distance, the bounds are expressed through Wasserstein distances between powered de Haan representers, total variation distances between angular measures, and discrepancies of the $\Psi$-functions in the inf–argmax decomposition. On the positive $\ell_\alpha$-sphere, the coefficient multiplying the setwise angular total-variation distance contains no explicit dimension factor for the unnormalised angular measures used here. Separately, for $1\le p<\alpha$, a synchronous de Haan–LePage coupling bounds the $p$-Wasserstein distance between the max-stable laws by an $\alpha$-Wasserstein transport cost between their unpowered de Haan representers. We also compare laws with a common extreme-value copula and different Fréchet indices, obtaining an exact $\ell_1$-Wasserstein formula when $p=1$, and discuss applications to Archimax and clustered Archimax copulas and to Brown–Resnick/Hüsler–Reiss models.