Papers

Electronic health records (EHRs) are central to modern healthcare delivery and research; yet, many researchers lack the database expertise necessary to write complex SQL queries or generate effective visualizations, limiting efficient data use and scientific discovery. To address this barrier, we introduce CELEC, a large language model (LLM)-powered framework for automated EHR data extraction and analytics. CELEC translates natural language queries into SQL using a prompting strategy that integrates schema information, few-shot demonstrations, and chain-of-thought reasoning, which together improve accuracy and robustness. CELEC also adheres to strict privacy protocols: the LLM accesses only database metadata (e.g., table and column names), while all query execution occurs securely within the institutional environment, ensuring that no patient-level data is ever transmitted to or shared with the LLM. On a subset of the EHRSQL benchmark, CELEC achieves execution accuracy comparable to prior systems while maintaining low latency, cost efficiency, and strict privacy by exposing only database metadata to the LLM. Ablation studies confirm that each component of the SQL generation pipeline, particularly the few-shot demonstrations, plays a critical role in performance. By lowering technical barriers and enabling medical researchers to query EHR databases directly, CELEC streamlines research workflows and accelerates biomedical discovery.

We study an information-theoretic minimax problem for finite multivariate Markov chains on $d$-dimensional product state spaces. Given a family $\mathcal B=\{P_1,\ldots,P_n\}$ of $\pi$-stationary transition matrices and a class $\mathcal F = \mathcal{F}(\mathbf{S})$ of factorizable models induced by a partition $\mathbf S$ of the coordinate set $[d]$, we seek to minimize the worst-case information loss by analyzing $$\min_{Q\in\mathcal F}\max_{P\in\mathcal B} D_{\mathrm{KL}}^{\pi}(P\|Q),$$ where $D_{\mathrm{KL}}^{\pi}(P\|Q)$ is the $\pi$-weighted KL divergence from $Q$ to $P$. We recast the above minimax problem into concave maximization over the $n$-probability-simplex via strong duality and Pythagorean identities that we derive. This leads us to formulate an information-theoretic game and show that a mixed strategy Nash equilibrium always exists; and propose a projected subgradient algorithm to approximately solve the minimax problem with provable guarantee. By transforming the minimax problem into an orthant submodular function in $\mathbf{S}$, this motivates us to consider a max-min-max submodular optimization problem and investigate a two-layer subgradient-greedy procedure to approximately solve this generalization. Numerical experiments for Markov chains on the Curie-Weiss and Bernoulli-Laplace models illustrate the practicality of these proposed algorithms and reveals sparse optimal structures in these examples.

We develop new methods to integrate experimental and observational data in causal inference. While randomized controlled trials offer strong internal validity, they are often costly and therefore limited in sample size. Observational data, though cheaper and often with larger sample sizes, are prone to biases due to unmeasured confounders. To harness their complementary strengths, we propose a systematic framework that formulates causal estimation as an empirical risk minimization (ERM) problem. A full model containing the causal parameter is obtained by minimizing a weighted combination of experimental and observational losses–capturing the causal parameter's validity and the full model's fit, respectively. The weight is chosen through cross-validation on the causal parameter across experimental folds. Our experiments on real and synthetic data show the efficacy and reliability of our method. We also provide theoretical non-asymptotic error bounds.

We study the theoretical complexity of simulated tempering for sampling from mixtures of log-concave components differing only by location shifts. The main result establishes the first polynomial-time guarantee for simulated tempering combined with the Metropolis-adjusted Langevin algorithm (MALA) with respect to the problem dimension $d$, maximum mode displacement $D$, and logarithmic accuracy $\log \epsilon^{-1}$. The proof builds on a general state decomposition theorem for $s$-conductance, applied to an auxiliary Markov chain constructed on an augmented space. We also obtain an improved complexity estimate for simulated tempering combined with random-walk Metropolis. Our bounds assume an inverse-temperature ladder with smallest value $\beta_1 = O(D^{-2})$ and spacing $\beta_{i+1}/\beta_i = 1 + O( d^{-1/2} )$, both of which are shown to be asymptotically optimal up to logarithmic factors.

Knowledge Tracing (KT) has been an established problem in the educational data mining field for decades, and it is commonly assumed that the underlying learning process being modeled remains static. Given the ever-changing landscape of online learning platforms (OLPs), we investigate how concept drift and changing student populations can impact student behavior within an OLP through testing model performance both within a single academic year and across multiple academic years. Four well-studied KT models were applied to five academic years of data to assess how susceptible KT models are to concept drift. Through our analysis, we find that all four families of KT models can exhibit degraded performance, Bayesian Knowledge Tracing (BKT) remains the most stable KT model when applied to newer data, while more complex, attention based models lose predictive power significantly faster.

The field of simulation optimization (SO) encompasses various methods developed to optimize complex, expensive-to-sample stochastic systems. Established methods include, but are not limited to, ranking-and-selection for finite alternatives and surrogate-based methods for continuous domains, with broad applications in engineering and operations management. The recent advent of large language models (LLMs) offers a new paradigm for exploiting system structure and automating the strategic selection and composition of these established SO methods into a tailored optimization procedure. This work introduces SOCRATES (Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations), a novel two-stage procedure that leverages LLMs to automate the design of tailored SO algorithms. The first stage constructs an ensemble of digital replicas of the real system. An LLM is employed to implement causal discovery from a textual description of the system, generating a structural `skeleton' that guides the sample-efficient learning of the replicas. In the second stage, this replica ensemble is used as an inexpensive testbed to evaluate a set of baseline SO algorithms. An LLM then acts as a meta-optimizer, analyzing the performance trajectories of these algorithms to iteratively revise and compose a final, hybrid optimization schedule. This schedule is designed to be adaptive, with the ability to be updated during the final execution on the real system when the optimization performance deviates from expectations. By integrating LLM-driven reasoning with LLM-assisted trajectory-aware meta-optimization, SOCRATES creates an effective and sample-efficient solution for complex SO optimization problems.

We present a general framework for Bayesian inference of causal effects that delivers provably robust inferences founded on design-based randomization of treatments. The framework involves fixing the observed potential outcomes and forming a likelihood based on the randomization distribution of a statistic. The method requires specification of a treatment effect model; in many cases, however, it does not require specification of marginal outcome distributions, resulting in weaker assumptions compared to Bayesian superpopulation-based methods. We show that the framework is compatible with posterior model checking in the form of posterior-averaged randomization tests. We prove several theoretical properties for the method, including a Bernstein-von Mises theorem and large-sample properties of posterior expectations. In particular, we show that the posterior mean is asymptotically equivalent to Hodges-Lehmann estimators, which provides a bridge to many classical estimators in causal inference, including inverse-probability-weighted estimators and Hájek estimators. We evaluate the theory and utility of the framework in simulation and a case study involving a nutrition experiment. In the latter, our framework uncovers strong evidence of effect heterogeneity despite a lack of evidence for moderation effects. The basic framework allows numerous extensions, including the use of covariates, sensitivity analysis, estimation of assignment mechanisms, and generalization to nonbinary treatments.

Recent advances in artificial intelligence have relied heavily on increasingly large neural networks, raising concerns about their computational and environmental costs. This paper investigates whether simpler, sparser networks can maintain strong performance by studying Restricted Boltzmann Machines (RBMs) under extreme pruning conditions. Inspired by the Lottery Ticket Hypothesis, we demonstrate that RBMs can achieve high-quality generative performance even when up to 80% of the connections are pruned before training, confirming that they contain viable sub-networks. However, our experiments reveal crucial limitations: trained networks cannot fully recover lost performance through retraining once additional pruning is applied. We identify a sharp transition above which the generative quality degrades abruptly when pruning disrupts a minimal core of essential connections. Moreover, re-trained networks remain constrained by the parameters originally learned performing worse than networks trained from scratch at equivalent sparsity levels. These results suggest that for sparse networks to work effectively, pruning should be implemented early in training rather than attempted afterwards. Our findings provide practical insights for the development of efficient neural architectures and highlight the persistent influence of initial conditions on network capabilities.

We study the adversarial Stochastic Shortest Path (SSP) problem with sparse costs under full-information feedback. In the known transition setting, existing bounds based on Online Mirror Descent (OMD) with negative-entropy regularization scale with $\sqrt{\log S A}$, where $SA$ is the size of the state-action space. While we show that this is optimal in the worst-case, this bound fails to capture the benefits of sparsity when only a small number $M \ll SA$ of state-action pairs incur cost. In fact, we also show that the negative-entropy is inherently non-adaptive to sparsity: it provably incurs regret scaling with $\sqrt{\log S}$ on sparse problems. Instead, we propose a family of $\ell_r$-norm regularizers ($r \in (1,2)$) that adapts to the sparsity and achieves regret scaling with $\sqrt{\log M}$ instead of $\sqrt{\log SA}$. We show this is optimal via a matching lower bound, highlighting that $M$ captures the effective dimension of the problem instead of $SA$. Finally, in the unknown transition setting the benefits of sparsity are limited: we prove that even on sparse problems, the minimax regret for any learner scales polynomially with $SA$.

Large language models (LLMs) can be controlled at inference time through prompts (in-context learning) and internal activations (activation steering). Different accounts have been proposed to explain these methods, yet their common goal of controlling model behavior raises the question of whether these seemingly disparate methodologies can be seen as specific instances of a broader framework. Motivated by this, we develop a unifying, predictive account of LLM control from a Bayesian perspective. Specifically, we posit that both context- and activation-based interventions impact model behavior by altering its belief in latent concepts: steering operates by changing concept priors, while in-context learning leads to an accumulation of evidence. This results in a closed-form Bayesian model that is highly predictive of LLM behavior across context- and activation-based interventions in a set of domains inspired by prior work on many-shot in-context learning. This model helps us explain prior empirical phenomena - e.g., sigmoidal learning curves as in-context evidence accumulates - while predicting novel ones - e.g., additivity of both interventions in log-belief space, which results in distinct phases such that sudden and dramatic behavioral shifts can be induced by slightly changing intervention controls. Taken together, this work offers a unified account of prompt-based and activation-based control of LLM behavior, and a methodology for empirically predicting the effects of these interventions.

Data-driven discovery of model equations is a powerful approach for understanding the behavior of dynamical systems in many scientific fields. In particular, the ability to learn mathematical models from data would benefit systems biology, where the complex nature of these systems often makes a bottom up approach to modeling unfeasible. In recent years, sparse estimation techniques have gained prominence in system identification, primarily using parametric paradigms to efficiently capture system dynamics with minimal model complexity. In particular, the Sindy algorithm has successfully used sparsity to estimate nonlinear systems by extracting from a library of functions only a few key terms needed to capture the dynamics of these systems. However, parametric models often fall short in accurately representing certain nonlinearities inherent in complex systems. To address this limitation, we introduce a novel framework that integrates sparse parametric estimation with nonparametric techniques. It captures nonlinearities that Sindy cannot describe without requiring a priori information about their functional form. That is, without expanding the library of functions to include the one that is trying to be discovered. We illustrate our approach on several examples related to estimation of complex biological phenomena.

Introduction: The inclusion of Breaking in the 2024 Paris Olympic Games introduced a distinctive competition format in which two athletes compete head-to-head in battle rounds. Unlike other aesthetic sports, where athletes receive independent scores for their performances, Breaking judges assess the relative performance quality between two competitors across five predefined evaluation aspects. This may considerably increase their cognitive load and warrants a thorough examination of judging reliability and potential national bias, the tendency of judges to favor athletes from their own country. Methods: Official scoring data from the 2024 Olympic Breaking events were analyzed. Interrater reliability was assessed using the Intraclass Correlation Coefficient, while national bias was estimated using mixed-effects modelling based on judges' individual scores. Results: The analyses revealed low to moderate interrater reliability in Breaking, notably lower than levels reported in other aesthetic sports. Moreover, substantial national bias was detected, with judges displaying a systematic tendency to favor athletes from their own country. Discussion: Although the observed national bias, despite its notable magnitude, likely had only a limited effect on final competition outcomes, the relatively low interrater reliability highlights potential weaknesses in the current scoring framework that may warrant refinement. Nonetheless, Breaking's unique system of aggregating judges' scores into discrete votes appears to reduce the influence of individual judges on competition outcomes, thereby enhancing the system's robustness against unilateral score manipulation.

Recent advances in generative modeling enable neural networks to generate weights without relying on gradient-based optimization. However, current methods are limited by issues of over-coupling and long-horizon. The former tightly binds weight generation with task-specific objectives, thereby limiting the flexibility of the learned optimizer. The latter leads to inefficiency and low accuracy during inference, caused by the lack of local constraints. In this paper, we propose Lo-Hp, a decoupled two-stage weight generation framework that enhances flexibility through learning various optimization policies. It adopts a hybrid-policy sub-trajectory balance objective, which integrates on-policy and off-policy learning to capture local optimization policies. Theoretically, we demonstrate that learning solely local optimization policies can address the long-horizon issue while enhancing the generation of global optimal weights. In addition, we validate Lo-Hp's superior accuracy and inference efficiency in tasks that require frequent weight updates, such as transfer learning, few-shot learning, domain generalization, and large language model adaptation.

In recent years, air pollution, especially the particulate matter $2.5\,\mu\text{m}$ (PM2.5), has become a growing concern mainly in urban areas. Exposure to air pollution is linked to the development of numerous health conditions, such as the aggravation of respiratory diseases, cardiovascular disorders, impaired lung function, and even cancer or early death. Forecasting future levels of PM2.5 has become increasingly important, as it can provide early warnings and help prevent diseases. We propose AQBuc, a dataset for predicting PM2.5 pollution in Bucharest, Romania, introducing an understudied location, as most work in this field has focused on Asian cities. Our dataset allows us to benchmark models for predicting future levels of PM2.5 across various time horizons. We evaluate a broad range of methods, including machine learning models, such as linear regression and ensemble-based methods, deep learning models based on recurrent networks and transformers, and large language models (LLMs). Specialized time-series models handle temporal data better, especially on longer horizons, while tree ensembles are more cost-effective for short horizons. Meanwhile, LLMs struggle to capture continuous sensor dynamics without specialized architectural changes.

The search for life in the Solar System hinges on data from planetary missions. Detecting biosignatures based on molecular identity, isotopic composition, or chiral excess requires measurements that current and planned missions can only partially provide. We introduce a new class of biosignatures, defined by the statistical organization of molecular assemblages and quantified using diversity metrics. Using this framework, we analyze amino-acid diversity across a dataset spanning terrestrial and extraterrestrial contexts. We find that biotic samples are consistently more diverse – and therefore distinct – from their sparser abiotic counterparts. This distinction also holds for fatty acids, indicating that the diversity signal reflects a fundamental biosynthetic signature. It also proves persistent under modeled space-like degradation. Relying only on relative abundances, this biogenicity assessment strategy is applicable to any molecular composition data from archived, current, and planned planetary missions. By capturing a fundamental statistical property of life's chemical organization, it may also transcend biosignatures that are contingent on Earth's evolutionary history.

A specification test for long range dependence (LRD) in functional time series in manifolds has been formulated in \cite{Ruiz-MedinaCrujeiras24} in the spectral domain for fully observed functional data. The asymptotic Gaussian distribution of the proposed test statistics, based on the weighted periodogram operator, under the null hypothesis, and the consistency of the test have been derived. In this paper, we analyze the asymptotic properties of this LRD spectral testing procedure, when the functional data are contaminated, and discretely observed through random uniform spatial sampling.

We present a functional data analysis approach for studying time-dependent, continuous glucose monitoring data with repeated measures for each individual in an experiment. After scaling the glucose concentration curves to the interval [0, 1], we model them by using a Beta distribution with two time-varying parameters. In this context, we develop a local linear maximum likelihood smoothing procedure that is valid when more than one parameter depends on time. Our approach requires much fewer observations than previous functional methods for this setting and is also applicable when only one individual (or a few) is available. We evaluate the performance of our estimator in terms of computation time and model fit using a synthetic dataset as well as a large, real clinical trial dataset. We also compare our method with existing methods in the literature. From a methodological point of view, we contribute to extend local likelihood estimation from one to two time-varying parameters by developing theoretical expressions for estimation and for approximating the leave-one-out cross-validation. Moreover, we show that this kernel-based approach competes with spline-based estimation methods, the dominant line of functional regression models today.

Neural networks are able to approximate any continuous function on a compact set. However, it is not obvious how to quantify the error of the neural network, i.e., the remaining bias between the function and the neural network. Here, we propose the application of extreme value theory to quantify large values of the error, which are typically relevant in applications. The distribution of the error beyond some threshold is approximately generalized Pareto distributed. We provide a new estimator of the shape parameter of the Pareto distribution suitable to describe the error of neural networks. Numerical experiments are provided.

Federated optimization is a constrained form of distributed optimization that enables training a global model without directly sharing client data. Although existing algorithms can guarantee convergence in theory and often achieve stable training in practice, the reasons behind performance degradation under data heterogeneity remain unclear. To address this gap, the main contribution of this paper is to provide a theoretical perspective that explains why such degradation occurs. We introduce the assumption that heterogeneous client data lead to distinct local optima, and show that this assumption implies two key consequences: 1) the distance among clients' local optima raises the lower bound of the global objective, making perfect fitting of all client data impossible; and 2) in the final training stage, the global model oscillates within a region instead of converging to a single optimum, limiting its ability to fully fit the data. These results provide a principled explanation for performance degradation in non-iid settings, which we further validate through experiments across multiple tasks and neural network architectures. The framework used in this paper is open-sourced at: https://github.com/NPCLEI/fedtorch.

In this article, we consider the problem of clustering multi-view data, that is, information associated to individuals that form heterogeneous data sources (the views). We adopt a Bayesian model and in the prior structure we assume that each individual belongs to a baseline cluster and conditionally allow each individual in each view to potentially belong to different clusters than the baseline. We call such a structure ”latent modularity”. Then for each cluster, in each view we have a specific statistical model with an associated prior. We derive expressions for the marginal priors on the view-specific cluster labels and the associated partitions, giving several insights into our chosen prior structure. Using simple Markov chain Monte Carlo algorithms, we consider our model in a simulation study, along with a more detailed case study that requires several modeling innovations.

We introduce two multifidelity trust-region methods based on the Magical Trust Region (MTR) framework. MTR augments the classical trust-region step with a secondary, informative direction. In our approaches, the secondary “magical” directions are determined by solving coarse trust-region subproblems based on low-fidelity objective models. The first proposed method, Sketched Trust-Region (STR), constructs this secondary direction using a sketched matrix to reduce the dimensionality of the trust-region subproblem. The second method, SVD Trust-Region (SVDTR), defines the magical direction via a truncated singular value decomposition of the dataset, capturing the leading directions of variability. Several numerical examples illustrate the potential gain in efficiency.

Data with spatio-temporal attributes are prevalent across many research fields, and statistical models for analyzing spatio-temporal relationships are widely used. Existing reviews focus either on specific domains or model types, creating a gap in comprehensive, cross-disciplinary overviews. To address this, we conducted a systematic literature review following the PRISMA guidelines, searched two databases for the years 2021-2025, and identified 83 publications that met our criteria. We propose a classification scheme for spatio-temporal model structures and highlight their application in the most common fields: epidemiology, ecology, public health, economics, and criminology. Although tasks vary by domain, many models share similarities. We found that hierarchical models are the most frequently used, and most models incorporate additive components to account for spatio-temporal dependencies. The preferred model structures differ among fields of application. We also observe that research efforts are concentrated in only a few specific disciplines, despite the broader relevance of spatio-temporal data. Furthermore, we notice that reproducibility remains limited. Our review, therefore, not only offers inspiration for comparing model structures in an interdisciplinary manner but also highlights opportunities for greater transparency, accessibility, and cross-domain knowledge transfer.

In cognitive psychology and neuroscience, adjudicating between competing theoretical models is a common methodological challenge. Researchers often rely on either first-order direct mapping approaches (e.g., linear regression) or second-order abstraction methods (e.g., Representational Similarity Analysis [RSA]). However, it remains unclear whether or how the nature of the underlying data and feature characteristics affect the performance of these methods. Here, we systematically evaluated regression, RSA, and Pattern Component Modeling (PCM) across distinct data-generating schemes, including first-order linear mappings and geometry-to-first-order transformations with either linear or nonlinear sigmoid readouts, using both univariate behavioral and multivariate fMRI spatial-pattern simulations. Our results suggest that the relative performance of these methods depends on the underlying generative mechanism. Under linear generative assumptions, regression and PCM showed higher model-selection accuracy than RSA. Under nonlinear but order-preserving transformations, rank-based RSA showed an advantage over regression and PCM. We also found that feature multicollinearity affected these methods differently across generative schemes, and that orthogonalizing the predictor space via principal component analysis (PCA) reduced several collinearity-related differences. Finally, analyses of empirical datasets were consistent with the simulation results under approximately linear conditions, with regression showing clearer model discrimination than RSA. Overall, these findings suggest that the relative performance of regression, RSA, and PCM depends on the form of the mapping between features and responses, as well as on the structure of the feature space.

Digital twins are developed to model the behavior of a specific physical asset (or twin), and they can consist of high-fidelity physics-based models or surrogates. A highly accurate surrogate is often preferred over multi-physics models as they enable forecasting the physical twin future state in real-time. To adapt to a specific physical twin, the digital twin model must be updated using in-service data from that physical twin. In this paper, we combine and extend several previous surrogate-related advancements with the goal of demonstrating an end-to-end digital twin (DT) solution for predicting performance of an aircraft structure (the physical asset). To this end, we extend Gaussian process (GP) models to include derivative data, for improved accuracy, with dynamic updating to ingest physical twin data during service. Including derivative data, however, comes at a prohibitive cost of increased covariance matrix dimension. We circumvent this issue through our modified dynamic sparse Cholesky linear system solver. Numerical experiments demonstrate that the prediction accuracy of the derivative-enhanced sparse Cholesky GP method produces improved models upon dynamic data additions. Lastly, we demonstrate the developed algorithm within a DT framework to model fatigue crack growth in an aerospace vehicle, thereby exhibiting through our assembled engineered system how digital twin technologies can be combined in practice.

Ensuring algorithmic fairness remains a significant challenge in machine learning, particularly as models are increasingly applied across diverse domains. While numerous fairness criteria exist, they often lack generalizability across different machine learning problems. This paper examines the connections and differences among various sparsity measures in promoting fairness and proposes a unified sparsity-based framework for evaluating algorithmic fairness. The framework aligns with existing fairness criteria and demonstrates broad applicability to a wide range of machine learning tasks. We demonstrate the effectiveness of the proposed framework as an evaluation metric through extensive experiments on a variety of datasets and bias mitigation methods. This work provides a novel perspective to algorithmic fairness by framing it through the lens of sparsity and social equity, offering potential for broader impact on fairness research and applications.

We extend the approximate residual balancing (ARB) framework to nonlinear models, answering an open problem posed by Athey et al. (2018). Our approach addresses the challenge of estimating average treatment effects in high-dimensional settings where the outcome follows a generalized linear model. We derive a new bias decomposition for nonlinear models that reveals the need for a second-order correction to account for the curvature of the link function. Based on this insight, we construct balancing weights through an optimization problem that controls for both first and second-order sources of bias. We provide theoretical guarantees for our estimator, establishing its $\sqrt{n}$-consistency and asymptotic normality under standard high-dimensional assumptions.

Large Language Models (LLMs) offer a flexible means to generate synthetic tabular data, yet existing approaches often fail to preserve key causal parameters such as the average treatment effect (ATE). In this technical exploration, we first demonstrate that state-of-the-art synthetic data generators, both GAN- and LLM-based, can achieve high predictive fidelity while substantially misestimating causal effects. To address this gap, we propose a hybrid generation framework that combines model-based covariate synthesis (monitored via distance-to-closest-record filtering) with separately learned propensity and outcome models, thereby ensuring that (W, A, Y) triplets retain their underlying causal structure. We further introduce a synthetic pairing strategy to mitigate positivity violations and a realistic evaluation protocol that leverages unlimited synthetic samples to benchmark traditional estimators (IPTW, AIPW, substitution) under complex covariate distributions. This work lays the groundwork for LLM-powered data pipelines that support robust causal analysis. Our code is available at https://github.com/Xyc-arch/llm-synthetic-for-causal-inference.git.

We determine the minimax optimal expected regret in the classic non-stochastic multi-armed bandit with expert advice problem, by proving a lower bound that matches the upper bound of Kale (2014). The two bounds determine the minimax optimal expected regret to be $\Theta\left( \sqrt{T K \log (N/K) } \right)$, where $K$ is the number of arms, $N$ is the number of experts, and $T$ is the time horizon.

We introduce a novel way to combine gradient boosting with mixed effects models, whereby the mean and variance components are learned jointly as functions of covariates via likelihood-based gradients. Gradient Boosted Mixed Models (GBMixed) estimates a nonparametric fixed effects function characterizing the overall mean of the response, while also allowing the random effects covariance matrix along with the residual variance to depend on covariates in a flexible manner. We demonstrate how GBMixed facilitates covariate-dependent random effect predictions, and subsequently point predictions and prediction intervals for individual treatment effects, that can adapt between population-level and cluster-level information. Experiments and applications to two real-world datasets show that GBMixed can accurately recover complex nonlinear fixed effect functions and covariate-dependent covariances in a linear mixed model, while also improving point and probabilistic predictive performance compared with several existing approaches such as parametric linear mixed models, Natural Gradient Boosting, and Gaussian Process Boosting. In simulations where the variance components are designed to vary as a function of covariates, GBMixed reduces the mean squared error in recovering the random effects variance function by a factor of eight relative to linear mixed models and Gaussian Process Boosting.

We introduce a novel framework that transforms the resource-intensive (adversarial) prompt optimization problem into an efficient, amortized inference task. Our core insight is that pretrained, non-autoregressive generative LLMs, such as Diffusion LLMs, which model the joint distribution over prompt-response pairs, can serve as powerful surrogates for prompt search. This approach enables the direct conditional generation of prompts, effectively replacing costly, per-instance discrete optimization with a small number of parallelizable samples. We provide a probabilistic analysis demonstrating that under mild fidelity assumptions, only a few conditional samples are required to recover high-reward (harmful) prompts. Empirically, we find that the generated prompts are low-perplexity, diverse jailbreaks that exhibit strong transferability to a wide range of black-box target models, including robustly trained and proprietary LLMs. Beyond adversarial prompting, our framework opens new directions for red teaming, automated prompt optimization, and leveraging emerging Flow- and Diffusion-based LLMs.