Papers

We study smoothed online prediction with bounded adversarial responses. This widely studied framework bridges i.i.d. sampling and adversarial covariate selection through a smoothness parameter $\textsf{C}_{\textsf{cov}}$, which bounds conditional covariate densities relative to a fixed, unknown base measure. We propose Hedge-Cover, an information-theoretic algorithm that achieves sublinear regret $\widetilde{O}(\sqrt{\text{Pdim}(\mathcal{F}) \textsf{C}_{\textsf{cov}} T})$ for function classes with bounded pseudo-dimension. The algorithm aggregates a carefully constructed family of experts using Hedge, with a prior that links regret to the number of disagreements between a consistent selector and a target function. We bound this number by exploiting covariate smoothness. This answers an open problem posed in \cite{blanchard2025agnostic} on the minimax optimal adaptive regret of the smoothed online regression problem. We establish a matching lower bound for the class of linear predictors. The main intricacy of the lower bound lies in explicitly constructing a challenging sequential covariate distribution supported on mutually orthogonal hyperplanes. This construction may be of independent technical interest. Finally, we revisit the well-specified setting and quantify the effect of response noise. For conditionally $\nu^2$-subGaussian responses, we extend the existing lower bound under realizable responses by showing that the minimax expected regret is $\Omega((1\vee \nu)\sqrt{(\textsf{C}_{\textsf{cov}}-1)dT})$ for a function class of VC dimension $d$. A corresponding upper bound for ERM matches this dependence on $\nu$.

Bayesian state estimation for high-dimensional nonlinear dynamical systems entails a fundamental tension between statistical fidelity and computational tractability, as particle weights can collapse, while Gaussian ensemble updates can miss non-Gaussian posterior structure. Score-based diffusion filters offer a sampling-based alternative, but existing training-free score filters often rely on heuristic likelihood corrections, which can compromise posterior accuracy by neglecting uncertainty about the system state associated with each noisy reverse particle. To address these issues, we propose AECSF, a training-free adaptive ensemble conditional score filter. AECSF constructs an analytically tractable score estimator from the conditional Tweedie identity, which recasts noisy posterior score estimation as estimating the conditional mean of the system state given a noisy reverse particle and the observation. To estimate these conditional means efficiently, AECSF employs a shared adaptive weighted proposal ensemble, while particle-specific conditional weights yield an estimate for each noisy reverse particle without separate proposal sampling. The proposal ensemble is updated using reverse-particle information within the same reverse-diffusion run to improve conditional-mean estimation. Theoretically, we characterize when a fixed weighted proposal measure yields the exact noisy posterior score. Under stated assumptions, we establish a bound relating conditional-mean estimation errors to reverse-sampling endpoint error. Numerical experiments demonstrate that AECSF improves the accuracy of posterior sampling and nonlinear filtering in high-dimensional problems with limited forecast ensembles.

Amortized neural likelihoods enable computationally expensive inference for models with analytically intractable or unspecified likelihoods, but their black-box nature limits interpretability. We introduce a symbolic distillation pipeline that converts trained neural likelihoods into explicit, interpretable expressions optimized for efficient parameter estimation. Our approach uses a recovery-directed objective to guide symbolic regression toward expressions that preserve parameter-recovery accuracy rather than merely approximating the likelihood function. Candidate expressions are evaluated on held-out datasets and selected using a criterion that jointly accounts for expression complexity, parameter-recovery performance, and distributional distance from the learned likelihood. We evaluate the pipeline on the diffusion decision model, a classical cognitive model, whose analytically tractable likelihood provides ground truth for controlled evaluation. The proposed recovery-directed objective improves parameter recovery over standard symbolic-regression objectives. The resulting symbolic likelihoods enable over 100 times faster parameter evaluation than both neural likelihoods and, when available, the exact likelihood, while maintaining a manageable loss in precision. We further demonstrate these computational benefits in Bayesian hierarchical inference on empirical data. Our pipeline provides a lightweight interface for integrating symbolic distillation with existing neural-likelihood estimation methods and can be adapted to a range of simulation-based inference settings.

Standard diagnostic procedures for assessing the goodness of fit of linear models include tests of the null hypothesis of no residual autocorrelation against the alternative of linear dependence. The literature proposes several portmanteau tests for residual autocorrelation in SARMA models, mainly focusing on two cases: short-term and single-seasonal autocorrelation. Since many time series exhibit multiple seasonal patterns, whose periodic components may interact with one another, diagnostic tests for residual autocorrelation in a multi-seasonal framework are needed. However, the literature still lacks portmanteau tests specifically designed for multiple seasonal autocorrelation. This paper addresses this gap by extending classical portmanteau tests to settings with multiple seasonalities. In addition, the proposed approach jointly tests for both short-term and multiple seasonal residual autocorrelation. Test statistics and their asymptotic distributions are defined. Then, Monte Carlo simulations are used to evaluate the performance of the tests and the consistency with the expected results is assessed using statistical tests. The results suggest that the proposed extension can serve as a useful goodness-of-fit diagnostic tool for the class of mSARIMA models. An application to road traffic data is also provided.

This paper develops a general two-point dependent wild bootstrap (DWB) for weakly dependent estimating equations. Its key feature is that the two-point marginal distribution and the latent serial dependence specification can be chosen separately. The construction combines a normalized two-point distribution with a stationary latent Gaussian process via a Gaussian copula transformation, includes dependent Rademacher and Mammen multipliers, and nests the classical iid two-point wild bootstrap as the serially independent case. The induced multiplier autocovariances determine the lag weights in a corresponding heteroskedasticity- and autocorrelation-consistent (HAC) covariance estimator, which coincides exactly with the conditional covariance of the bootstrap estimating-equation sum. We establish first-order bootstrap validity for asymptotically linear estimators by showing that the matched-HAC estimator consistently estimates the long-run covariance and that the bootstrap estimating-equation sum converges conditionally to the same Gaussian limit as its original-sample counterpart, yielding valid HAC-studentized $z$-tests and the corresponding Wald and Lagrange multiplier tests. Monte Carlo experiments in nonlinear generalized method of moments (GMM) and linear regression show that Rademacher DWB generally provides more accurate finite-sample size control for $z$-tests than the Mammen and Gaussian DWB. A GMM application to a nonlinear short-rate mean-reversion model illustrates the practical relevance of the proposed two-point DWB.

Semi-competing risks occur when individuals may experience a non-terminal event and a terminal event, where the terminal event censors the non-terminal event but not vice versa. In the presence of covariate-dependent censoring, augmented inverse probability of censoring weighting (AIPCW) framework has not been developed for the special setting of semi-competing risks which, unlike competing risks, have asymmetric event time structure and complex estimands. Using semiparametric theory on coarsened data, we carefully develop an AIPCW framework for semi-competing risks. When treatment effects are of interest, we further integrate this approach with augmented inverse probability of treatment weighting (AIPTW), yielding a framework for estimating causal estimands under covariate-dependent censoring. The resulting estimators are shown to be doubly robust. Using the proposed framework, we estimate treatment specific risks of: i) non-terminal event, ii) terminal event without the non-terminal event, and iii) terminal event following the non-terminal event. We evaluate the finite sample performance through simulations, and apply the method to data from the Honolulu Asia Aging Study to assess the causal effects of midlife heavy drinking on late life cognitive impairment and mortality.

Climate change may materially affect long-term life insurance liabilities by altering both the level and seasonal pattern of mortality. This article develops a multi-population mortality framework that combines a Hermite spline model with a distributed lag non-linear model to capture age-, region-, and temperature-specific mortality effects. We apply the framework to mortality and temperature data from 15 Spanish NUTS-2 regions and project future mortality under three shared socioeconomic pathway (SSP) scenarios. We then assess the implications for a hypothetical whole life insurance portfolio through expected death-benefit payments and portfolio profit and loss. The results reveal an important seasonal offset: warmer conditions reduce expected payoffs during winter periods but increase them during summer periods, with these effects becoming more pronounced under more severe climate scenarios and for policies issued in later years. Over longer horizons, adverse summer mortality effects become increasingly important. The portfolio analysis further shows that climate-related mortality risk can materially increase the dispersion and downside risk of portfolio outcomes, particularly under SSP5-8.5. These findings highlight the importance of incorporating temperature-related mortality effects into long-term life insurance liability projections and risk assessment.

We study the generalization of spectral gradient descent (SpecGD) in overparameterized matrix classification with corrupted labels. Each input combines a shared low-rank signal with a rank-one sample-specific perturbation, referred to as a shortcut, that enables memorization but does not generalize. We contrast collapsed shortcuts, which share a singular direction, with dispersed shortcuts, which occupy distinct singular directions. Changing only this geometry can reverse the relative generalization of GD and SpecGD: collapsed shortcuts can favor SpecGD, while dispersed shortcuts can favor GD. In the dispersed regime, exact shortcut orthogonality eliminates the signal from the late-stage SpecGD direction, while vanishing random correlations collectively generate a small but generalization-relevant signal through a second-order effect. To identify the direction selected by SpecGD, which the spectral max-margin problem alone does not determine, we combine a refined analysis of its dual with the exponentiated-gradient dynamics of normalized loss weights. Finally, we show that a single SpecGD step can already interpolate and generalize well, while continued training converges to a direction with substantially worse generalization.

Leaderboards rank models by their average scores on benchmark items, and they are consulted repeatedly while the evaluation is still running. Existing confidence intervals for a model's rank control their error rate only if they are computed once, after a number of items chosen in advance. If they are recomputed as results arrive, and the evaluation stops once they look decisive, their error rate exceeds its nominal level. Anytime-valid methods keep their guarantees at all sample sizes simultaneously and hence under any stopping rule. They exist for the accuracy of one model, for one pair of models and for the set of models that may be best. For pairwise battles they also give ranks. None gives ranks when all models are scored on the same items, which makes their scores dependent. We construct rank confidence sequences: for every model, a set of ranks that contains its true rank, simultaneously for all models and at all times, at a chosen error level $\alpha$, in finite samples. The construction combines betting e-processes, one for each ordered pair of models, with closed testing over the possible orderings of the models. It allows any dependence between the models' scores on an item. The method has two advantages. A leaderboard can be inspected after every item without inflating its error rate. The evaluation of each model can stop as soon as the question asked about it is answered, which saves compute. When results are examined only once, halfway through or later, little power is lost relative to fixed-sample methods. The paper quantifies these advantages in simulations and on public leaderboard data.

Stochastic gradient descent (SGD) is widely used to train machine learning models, but subsampling the training data introduces noise into its updates. The strength and applicability of high-probability guarantees therefore depend critically on how the tails of gradient noise are modeled. Reports of heavy-tailed gradient noise in deep learning motivate relaxing the bounded-noise and sub-Gaussian assumptions commonly used in high-probability analyses of SGD. We use Young functions from Orlicz space theory to describe noise tails in a common framework. We model SGD gradient noise by adopting a Young function that preserves the finiteness of all polynomial moments while allowing tails heavier than sub-Weibull, including lognormal distributions. The resulting class is called $\beta$-heavy-tailed, with $\beta$ controlling the tail heaviness. We establish concentration inequalities for $\beta$-heavy-tailed noise and combine them with a uniform bound on the difference between empirical and population gradients along the SGD trajectory to obtain high-probability bounds on optimization and population-risk stationarity for smooth nonconvex losses under trajectory assumptions. The bounds are not restricted to a particular learning-rate decay rule and make explicit the effects of noise tails and learning-rate schedules. Under the Polyak-Łojasiewicz condition, we bound the risk at the last iterate. We also analyze SGD with gradient clipping under the $\beta$-heavy-tailed noise model.

Nonlinear dynamical systems with memory arise across science and engineering, yet uncertainty quantification for efficient forecasting methods such as Next Generation Reservoir Computing (NGRC) remains underdeveloped. We study Bayesian ridge and conformal prediction intervals for NGRC and characterize when their uncertainty estimates agree or differ. In low dimensions, their asymptotic widths are governed by different summaries of the residual distribution, so agreement depends on residual shape rather than dimensionality alone. In high dimensions, regularization introduces a further tradeoff between estimation variance, shrinkage bias, and posterior uncertainty, leading to an explicit transition between regimes where Bayesian intervals are wider or narrower than conformal intervals. We extend these results to quadratic NGRC feature maps and give sufficient conditions for transferring the analysis to temporally dependent forecast windows. Simulations and real-data experiments support the theoretical predictions and illustrate how residual distribution, regularization, dimensionality, and distribution shift affect interval calibration and efficiency. These results provide a principled framework for choosing and interpreting uncertainty quantification methods in reservoir-based forecasting.

We study fully online fixed-confidence causal discovery without any historical observational data. Starting from zero samples, the learner sequentially selects interventions to recover both the causal DAG and its edge weights under a linear-Gaussian structural equation model. We establish an instance-dependent lower bound for any $(\epsilon,\delta)$-correct algorithm and propose FOCUS, which adaptively allocates interventions through an online max–min game. A key contribution is a computable concentration inequality for the accumulated KL divergence involving causal parameters shared across interventions. We prove that FOCUS is $(\epsilon,\delta)$-correct and that its expected stopping time matches the lower bound in its $\Theta(\log(1/\delta))$ dependence up to an instance-dependent constant. Experiments demonstrate improved structure and edge-weight recovery and confirm the predicted stopping-time trend. Our codes are available on https://anonymous.4open.science/r/FOCUS_code-76E5

Linear bandits model sequential decision-making problems with noisy rewards that are linear in the decision variable, where an agent must simultaneously learn about an unknown parameter that governs the mean rewards, while maximizing (expected) rewards over time. Two prominent algorithmic families–upper confidence bound (UCB) and Thompson sampling (TS)–achieve a balance of exploration (to estimate said parameter) and exploitation (utilization of knowledge about it) across time. The quality of estimation of that parameter depends on the eigenvalues of a design matrix. In this paper, we begin by showing that if the inference quality obtained from exploration, encoded in the minimum eigenvalue of the design matrix, grows $\gtrsim \sqrt{t}$ with time $t$, while actions remain sufficiently concentrated for exploitation, then an algorithm produces optimal high-probability $\mathcal{O}(\sqrt{T}\log T)$-regret rate over a time-horizon $T$ for spherical action sets. This analysis is algorithm-agnostic and follows an alternative route to the classical optimism-based elliptical-potential argument for regret analysis. Then, we illustrate that variants of UCB and TS satisfy the inference and concentration properties and in turn, enjoy optimal regret rate. In effect, our results provide a modular framework that can be used to analyze linear bandit algorithms and explicitly connect quality of parameter estimation to optimal regret accumulation.

Deep ReLU networks are piecewise-affine mappings that partition the input space into cells, each characterized by a distinct activation pattern. This structure motivates fitting a local linear model within each cell to preserve predictive accuracy while improving interpretability. The challenge is to identify regimes that are stable, data-adaptive, and easy to explain. We propose REALM, a mixture of linear models whose regimes are induced by neural activation patterns. Because the number of activation cells in a deep neural network (DNN) can grow rapidly with depth, we first distill a deep teacher into a wide, shallow student network (WSSN), then binarize and cluster its hidden-layer activations to define the regimes and fit a linear model within each regime. Since the regimes are discovered from internal structure, the router does not carry the predictive burden. To make regime assignment interpretable, we train a multiclass logistic regression, the explanatory gate, to reproduce the regime assignments. The two-level structure is interpretable at both stages in terms of raw tabular or learned convolutional features: the gate identifies features that determine regime assignments, while the linear models identify features that drive predictions within each regime. We analyze an idealized setting that illustrates a trade-off between partition complexity and stability: as the number of regimes grows, finer partitions can improve approximation but may reduce regime-assignment stability. Experiments on tabular and image datasets show that REALM achieves competitive predictive performance relative to other DNN-guided mixture surrogates and inherently interpretable models while producing stable regime-level explanations.

Nested sampling estimates the evidence by assigning prior volumes to an ordered sequence of likelihood contours. Markov chain constrained-prior samplers, which are used in high-dimensional problems, generate many intermediate states before producing the next classic sample. These intermediate states are discarded because their correlation prevents them from being inserted into the ordered NS sequence without changing its order-statistic law. This paper introduces a novel method of using them to improve evidence estimation, by formulating NS in a Bayesian way and conditioning on phantom samples as Monte Carlo observations. We then present the open-source software package, JAXNS v3, and its implementation choices. We validate the approach on a set of problems, and identify its limitations via ablation. In our experiments, when problem structure is well resolved, conditioning on all retained phantom samples reduces log-evidence RMSE by at least $30\%$, with larger improvements at higher dimensionality. For the tested problems with unresolved structure, full phantom conditioning produces no detectable improvement or deterioration in evidence accuracy. Phantom conditioning produces overconfident evidence uncertainties. We also introduce two dynamic nested sampling allocation schemes. Evidence-improving allocation approximately halves the number of likelihood evaluations required to achieve comparable evidence accuracy relative to uniform allocation. Posterior-improving allocation doubles the classic posterior's Kish effective sample size for $17.5\%$ additional likelihood evaluations.

We study whether benchmarks reliably guide model selection as developers adapt to evaluation feedback across multiple criteria. We find that the worst-case test-set size needed to estimate the best score among $k$ adaptively chosen models, under any convex combination of the criteria, grows exponentially with the number of criteria, reaching the $\Theta(\sqrt{k})$ cost of answering $k$ adaptive statistical queries with only $O(\log k)$ criteria, at fixed accuracy and confidence. In attacks on multi-task large language model benchmarks with five to ten criteria, feedback restricted to nondominated task profiles produces large reused-to-held-out score gaps and frequent false winners. These results challenge a prominent explanation for prior observed reliable benchmark reuse—that developers mainly respond to convincing improvements over the current best—in rich-feedback settings, while leaving open how often ordinary model development encounters this vulnerability.

The neural scaling law relating longer training to better performance through a power law is central to today's large language models (LLMs), yet its origin remains debated. One recent proposal is that power laws can emerge from the strong non-linearity of a single softmax head learning peaked distributions. What happens with multiple softmax functions, as in LLMs, is unclear. Here, we show through toy models that any softmax learning peaked distributions, regardless of its position in the model, can develop logit magnitudes that grow in a power law with exponent $1/3$, becoming a training bottleneck whose loss contribution decays as a power law with the same exponent $1/3$. The overall loss therefore obeys $1/3$ scaling whenever at least one softmax learns peaked distributions. We confirm that many softmax functions in LLMs learn peaked distributions and that LLM loss scaling matches this $1/3$ prediction. Moreover, logit growth dynamics reveal that attention heads, rather than the language modeling head, are the bottleneck likely driving the $1/3$ loss scaling in LLMs. Attention trying to concentrate on specific information, which is the heart of Transformers, may therefore also be the heart of the neural scaling law of training.

The partial area under the receiver operating characteristic curve (pAUC) is an important performance metric for binary classification that summarizes true positive rates within a specific range of false positive rates (FPRs). Classifiers that achieve high pAUC need to be obtained in many real-world applications such as cybersecurity, medical care, and advertising. Although many methods for maximizing the pAUC have been proposed, they typically require both labeled positive and negative data for training. However, in practice, labeled negative data are often difficult to collect due to privacy concerns or the need for high expertise to annotate them. In this paper, we propose a method for maximizing the pAUC from positive and unlabeled (PU) data without negative data. Within an empirical risk minimization framework, we show that the pAUC, including its FPR-dependent thresholds, can be represented using only the positive and marginal densities, and derive an empirical estimator from PU data. A classifier is then trained by maximizing the derived smoothed empirical pAUC estimator. We experimentally demonstrate the effectiveness of the proposed method with ten real-world datasets.

Pooled statistics of Transformer weights obscure how magnitude is distributed across functional channels, while individual weights are too numerous to compare directly. We study the mesoscopic level between them: row and column scale fields, the median-centred log-RMS profiles of a weight matrix over its channels, which together with a global scale and a full balanced core represent the matrix exactly. Across public Pythia checkpoints at four sizes and controlled runs from three initialization families, balancing reveals similar measured core magnitude profiles. A mixture bridge, with its form fixed before the analysis and its coefficients fitted, predicts the pooled-shape departure from field width on held-out runs and data arms of the controlled grid. The indexed fields retain further structure: they align across projections that share a functional channel, and query/key profiles follow reassigned RoPE frequencies rather than fixed matrix coordinates. Training trajectories show early field formation followed by component-dependent broadening or recession. Extending the channel-based analysis to AdamW's second moment reveals related functional organization in its log-space row and column factors. Finally, edits of a frozen checkpoint separate reciprocal scale balance, which preserves the forward computation, from relative channel gain: flattening the gain increases in-distribution loss while preserving matrix norms and the balanced core. Row and column scale fields thus connect pooled magnitude statistics to channel organization and provide coordinates for tracking and testing trained weight structure.

The analysis of experimental or observational data often requires controlling for the effects of variables of secondary interest. Two common approaches to account for such parameters are grouping experimental or observational units into homogeneous groups (blocking), and observing quantitative variables on each one of them (covariance analysis). Often, both can be used and; therefore, comparing their performance poses a research question. Specifically, under randomized complete block designs where the blocks are defined by quantitative variables, there is interest in comparing the two-way Anova (TWA) with analysis of covariance (Ancova) models. Motivated by real-life problems coming from statistical consulting, we compared the performance of linear models considering block effects and those considering covariate effects from analytical and empirical perspectives. Models assumed Gaussian errors and included the effects of T treatments. The formal comparison relied on the theory of orthogonal projections and vector spaces under containment relationships of column spaces of the design matrices induced by conditions on the experimental design or data structure. We found asymptotic and small-sample conditions for the TWA to be at least as precise as the Ancova, as well as for these models to be equivalent. As to the empirical perspective, models were compared through a simulation study motivated by agronomical experiments. The results suggested that when the covariates show complex heterogeneity patterns, the Ancova outperforms the TWA with blocks defined by traditional approaches in terms of precision of location parameters, but not in hypothesis testing accuracy.

Physical systems evolve continuously in time, yet their states are typically observed only at discrete times. Generating trajectories consistent with their probability densities from such observations therefore requires capturing the continuous-time evolution rather than only learning transition mappings between consecutive observations. We propose PhiBE-Flow, a framework that directly estimates the probability velocity field induced by the stochastic differential equation (SDE) which governs this continuous-time distributional evolution. PhiBE-Flow learns from discrete observations using a model-free approach requiring neither known SDE coefficients nor score estimation. We establish convergence guarantees for the method, accounting for both time-discretization and finite-sample errors. We evaluate PhiBE-Flow on systems of increasing complexity, from controlled stochastic numerical systems to Navier–Stokes dynamics and real-world videos. Our results show that PhiBE-Flow accurately recovers probability flows of stochastic dynamics, preserves multiscale physical statistics, and improves video generation performance over representative baselines. The code is available at https://github.com/R1fe/PhiBE-Flow.

Prediction-powered inference (PPI) typically relies on point predictions on unlabeled data. When predictive distributions are available as in a wide range of applications, including predictions from LLMs, we introduce distribution-informed prediction-powered inference (DiPPI), a general framework for further improving statistical efficiency by using predictive distributions as auxiliary information in the spirit of PPI. We characterize the optimal use of this information through score calibration, derive the oracle efficiency for a finite-dimensional representation of the predictive distribution, and provide theoretical guarantees for positive learning with the cross-fitted DiPPI estimator. Through simulations and three real-data applications, we show that DiPPI achieves better efficiency than PPI methods based on point predictions. These gains arise when the predictive distribution contains score-relevant information that is partially lost in point predictions.

The rank $r$ in LoRA is widely treated as a capacity control: a smaller rank is assumed to yield a simpler model that generalizes better. We show that, under hard per-factor norm budgets—the idealization of the weight decay and norm control used in practice—this intuition breaks down. The reason is structural: under such budgets, the updates LoRA can reach are exactly the matrices of rank at most $r$ inside a nuclear-norm ball, and every complexity and displacement functional we analyze is maximized over this set by a rank-one update—so the rank cap never binds. The consequences follow directly. The linear-readout model class we study is identical for every $r \ge 1$, its Rademacher complexity carries no dependence on $r$, and the distance the adaptation can move the source distribution obeys a rank-independent upper bound that we show is sharp. If rank does not control capacity, where does it act? We identify two places. Statistically, replacing the per-factor budgets with a joint budget on the product restores a data-dependent, rank-sensitive complexity bound—though the gain appears only for well-spread feature distributions, and the worst case remains rank-free. Spectrally, rank sets the price of adaptation: canceling the leading singular directions of the pretrained weight requires both sufficient rank and sufficient budget. We bound the smallest rank achieving a desired source–target alignment, with upper and lower bounds that match under two-sided spectral decay. Together, these results recast rank as governing which updates are reachable and what cancellation costs—not how much capacity the model has.

Spectral risk measures, including conditional value-at-risk (CVaR), generate a family of convex criteria for portfolio estimation. We study how the criterion should be chosen when different criteria share the same population minimizer, and whether the efficient criterion can itself be learned from data. We develop a general asymptotic theory for empirical spectral-risk minimization under estimated linear constraints. Under normal scale-mixture elliptical returns, all spectral risk measures identify the same population efficient portfolio, but their empirical minimizers have different sampling distributions. Their asymptotic covariance decomposes into a common component and a positive-semidefinite component scaled by a functional of the spectral measure, reducing efficiency to an optimization over probability measures. We characterize the efficiency-optimal spectral measure and show that single-level CVaR is generally inefficient. We then construct a fully data-adaptive estimator that learns the radial distribution and optimal spectral measure from the same observations used for portfolio estimation, yet has the same first-order distribution as the infeasible oracle. Simulations and an empirical application illustrate the method.

Generating independent, equilibrium samples of molecular systems at scale remains a central obstacle in computational statistical mechanics. Boltzmann Generators address this by pairing a generative model with importance sampling to obtain consistent samples from the target distribution. We introduce Normalizing Flow Flow Maps (NF$^2$M), which combines the strengths of recent stochastic flow maps with the tractability of classic normalizing flows to build a Boltzmann Generator. Unlike most methods, which correct the generative model only at the end, NF$^2$M reweighs each denoising transition as generation proceeds, avoiding wasted compute on trajectories that are ultimately discarded. At each denoising step, a conditional normalizing flow proposes clean configurations given the current noisy state (a simpler task than sampling directly from the target) and its exact likelihood enables correcting each proposal toward the true denoising transition of the target Boltzmann distribution. This is in contrast to most existing methods, whose likelihoods are approximate or expensive to evaluate, undermining the statistical reliability of the correction. We establish consistency of the corrected transitions and bound how approximation errors propagate through the sampling chain. We evaluate NF$^2$M on peptide systems, demonstrating improved sampling efficiency and sample quality.

Vogels et al. (2024) presented an empirical comparison of Bayesian methods for structure learning in undirected Gaussian graphical models. The method implemented in the R package BGGM was run with equal prior probabilities for a null, negative, and positive partial correlation, implying a prior inclusion probability (equivalent to a prior graph density) of 2/3. The other Bayesian methods in the comparison used a prior graph density of 0.2 however. The data-generating densities ranged from 0.01 to 0.1. This resulted in a substantial overestimation of the inclusion probabilities of absent edges by BGGM. For a fair comparison of the performance of the different methods, I repeated the simulation using a prior inclusion probability of 0.2 using BGGM. In this case, the performance of BGGM is comparable with the other Bayesian methods.

Instead of having a single "yes" or "no" result from a test of the global null hypothesis that a function is increasing, we propose a multiple testing procedure of the function's increasingness at several points. If the global null is rejected, then multiple testing provides more information about why. If the global null is not rejected, then multiple testing can provide stronger evidence in favor of increasingness, by rejecting null hypotheses that the function is decreasing. Our approach uses high-level assumptions that apply to a broad class of causal and descriptive statistical models. By inverting the proposed multiple testing procedure that controls the familywise error rate, we also generate "inner" and "outer" confidence sets for the set of points at which the function is increasing. With high asymptotic probability, the inner confidence set is contained within the true set, whereas the outer confidence set contains the true set. We also improve power with stepdown and two-stage procedures. Simulation and empirical examples illustrate the new methodology, and all code is provided.

Multi-objective Bayesian optimization (MOBO) is commonly approached through specialized acquisition functions or scalarization schemes designed to explicitly account for trade-offs among non-preferential objectives. In this work, we show that such complexity might be unnecessary. We propose a framework that extends standard single-objective acquisition functions directly to the multi-objective setting through a hypervolume-based transformation. We further extend hedge strategies for acquisition functions, which are typically used only in single-objective optimization, to the multi-objective regime. Our approach requires minimal modification to existing Bayesian optimization pipelines and avoids the need for bespoke multi-objective formulations. We demonstrate how a broad class of commonly used single-objective acquisition functions and hedge strategies can be adapted in a principled manner to handle multiple objectives, while preserving their intuitive interpretation and computational efficiency. Empirically, we evaluate the proposed methods across a range of synthetic and real-world multi-objective benchmarks. Despite their simplicity, our extensions consistently match or outperform more complex state-of-the-art MOBO methods in terms of optimization performance and sample efficiency. These results suggest that effective multi-objective Bayesian optimization can be achieved by reusing and carefully extending well-established single-objective acquisition strategies, offering a simpler and more flexible alternative to existing approaches.

In forensic science, when crime-scene evidence (CSE) and suspect-related evidence (SRE) is present, it is customary to report on the value of this evidence in the form of a likelihood ratio (LR). The LR can be calculated as the probability of CSE given SRE divided by the probability of CSE given that it was generated by a randomly selected person from an alternative culprit population. In forensic science, this is known as a feature-based LR, and intuitively the LR contrasts "similarity" by "typicality". Since it is generally a problem for feature-based LRs to find appropriate models for the data, one either resorts to score-based LRs or to adjusting the feature-based output post-hoc to well-calibrated output. Either way, the above definition of the LR is broken and interpretation of the LR as similarity between CSE and SRE divided by typicality of CSE is destroyed. Here, we report on progress in obtaining instantly well-performing feature-based LRs using gradient descent in combination with Bayesian probability theory to train a two-level model, the main LR model in forensic science for describing distributions of continuous data. For a dataset of laser-ablation inductively-coupled-plasma mass-spectrometry measurements on glass fragments from forensic casework, we show that our best model on validation data yields much better calibrated feature-based LRs on the test set when compared to state-of-the-art feature-based LR systems trained on the same type of data, and that it improves a factor of 4.5 on average on $C_{\mathrm{llr}}$. For LRs interpretable in terms of "similarity" contrasting "typicality", this is a major advancement. However, a state-of-the-art LR system still performs a factor of 1.5 better on $C_{\mathrm{llr}}$ for this data. We also present future plans to close this remaining gap. In order to facilitate collaboration, we have put relevant code on GitHub.

Random forests (RF) are tree-based models that capture complex, high-order predictor interactions. In biomedical studies, ordered outcomes often have outliers, skewness, or mixtures of continuous and ordinal values (e.g., due to detection limits). Standard RF splitting targets conditional means via squared errors and is sensitive to such irregularities. We propose a Wilcoxon regression tree that selects splits by maximizing a rank-based impurity reduction, equivalent to maximizing the squared Wilcoxon rank-sum statistic. Because the criterion depends only on outcome ranks, it is invariant to monotone transformations and naturally accommodates continuous, ordinal, or mixed outcomes. We develop the Wilcoxon random forest (WRF), which aggregates Wilcoxon regression trees via subsampling and estimates conditional distributions using forest-weighted empirical CDFs. We establish consistency of the WRF distribution estimator under regularity conditions. We evaluate distributional prediction using calibration diagnostics and continuous ranked probability scores, and define an out-of-bag permutation variable-importance measure. Simulations show that the WRF performs comparably to the standard quantile regression forest when errors are symmetric and homoscedastic, and yields improved quantile estimation and calibration when outcomes are skewed or heteroscedastic. We apply the WRF to predict CD4 cell count and HIV viral load six months after antiretroviral therapy initiation in a multicenter Latin American cohort, and the WRF improves threshold probability estimation and accommodates the large mass at the viral load detection limit.