Papers

The record values theory study elements of a time series that exceed all previous observations, which are of particular interest in fields such as sports or climate science. In this paper, we propose a statistical method based on the construction of a Brownian stochastic simulator to reconstruct entire time series solely from such record values, even in a non-stationary case. We then implement a procedure, which can be compared to a Neural-Based Inference (NBI) procedure, to choose the optimal generator hyper parameters. To illustrate our method and motivate its development, we apply it to a glaciological problem. Understanding the past dynamics of glacier fronts is a major challenge to mitigate related mountain hazards, assess water resources, and evaluate contributions to sea-level rise. Field-visible indicators such as moraines provide spatio-temporal evidence of these front position evolution (refered as trajectories) and can be interpreted as the records of a non-stationary process. As a benchmark case, the two hyper parameters of our NBI approach are tuned from the well documented French alpine Glacier des Bossons. Our purely data-based approach offers new perspectives for challenging and further developing physical models of glacier dynamics and inferring the response of glaciers to climate change on centennial to millenial time scales. Beyond the glacier case, it has potential for the various problems for which record series is the sole available data.

Filled pauses (FPs) are a universal feature of spontaneous speech, yet most studies rely on small, single-language corpora, limiting the generalisability of their findings. We analyse ~4,000 hours of parliamentary speech across four related Slavic languages (Croatian, Czech, Polish, Serbian). FP occurrence is obtained via transformer-based automatic detection, while FP rate is modelled using Generalised Estimating Equations (GEE) with Mundlak correction to distinguish within- from between- speaker effects. We replicate a negative association of age and speech rate with FP rate, but find that gender effects are language-specific and directionally opposite to most prior literature. Novel analyses of sentiment, political orientation, and power status reveal a consistent positive association between sentiment and FP rate, alongside parliament-specific modulation by orientation and power status, with opposition speakers tending toward lower FP rates than governing coalition speakers.

We investigate the sharp asymptotic minimaxity of the classical Gavrilov-Benjamini-Sarkar (GBS) step-down multiple testing procedure in sparse Gaussian sequence models. Abraham et al. (2024) recently established sharp asymptotic minimax benchmarks for sparse multiple testing under both the classical beta-min framework and more general heterogeneous signal-strength classes. Although several multiple testing procedures attain these benchmarks, corresponding results for the GBS procedure have remained unavailable. We prove that the GBS procedure is sharply asymptotically minimax under both normalized Hamming loss and the combined $\mathrm{FDP}+\mathrm{FNP}$ loss over the classical beta-min parameter classes. We further establish a sharp asymptotic upper bound for the normalized Hamming risk over the heterogeneous signal-strength classes and show that the GBS procedure attains the corresponding sharp asymptotic minimax benchmark under the combined $\mathrm{FDP}+\mathrm{FNP}$ loss. Our analysis is based on a shifted order-statistic inequality for the GBS critical values together with deterministic and empirical signal-crossing arguments for ordered alternative $p$-values. Unlike existing analyses of Benjamini-Hochberg procedures and empirical Bayes $\ell$-value methods, the proposed approach is intrinsic to the geometry of the GBS step-down procedure and avoids localization of a single implicit rejection threshold. Consequently, our results extend sharp asymptotic minimaxity theory to a genuinely step-down multiple testing procedure whose rejection rule depends on the entire ordered sequence of $p$-values.

When analyzing a manifold learning algorithm for data lying on a smooth, compact, connected Riemannian submanifold $(\mathcal{M}, g)$ of $\mathbb{R}^d$, a key estimate for the geodesic distance $d_g$ is that there exists $K > 0$ such that $0 \leq d_g(p, q)^2 - \|p-q\|^2 \leq K d_g(p, q)^4$ for all $p, q \in \mathcal{M}$. We observe that more generally, when $\mathcal{M}$ is equipped with a smooth symmetric divergence $D$ satisfying a non-degeneracy condition and $g$ is given by $g_p := \frac{1}{2}\mathrm{Hess}_p(D(p, \cdot))$ for all $p \in \mathcal{M}$, there exists $K > 0$ such that $\left| D(p, q) - d_g(p, q)^2 \right| \leq K d_g(p, q)^4$ for all $p, q \in \mathcal{M}$. We demonstrate that this is sufficient for the pointwise convergence of graph Laplacians constructed with $D$ and discuss examples where $D$ is given by the Sinkhorn divergence on a family of probability measures parametrized by a manifold.

The effective reproduction number $R_t$ is one of the most important indicators of epidemic dynamics. Estimating $R_t$, typically from case reports or hospitalization counts, poses a challenging inverse problem. One key issue is lag: $R_t$ acts at the moment of transmission, while the data it generates surface days later. To handle this delay and infer recent infections in real time, popular methods take a Bayesian approach, which can be slow and sensitive to prior specification. As an alternative, we propose ConvRt, a frequentist method for retrospective and real-time estimation. ConvRt deconvolves latent infections and then estimates $R_t$ with successive penalized-likelihood steps, using a spline basis to model smooth curves. Across both stylized and data-driven simulations, we demonstrate favorable performance in point estimation, uncertainty quantification, and runtime. Moreover, by untangling smoothness from future projections, ConvRt enables researchers to assess which qualitative narratives about $R_t$ the data support.

Memory-efficient optimizers such as GaLore train large language models by projecting gradients onto a rank-r subspace recomputed every T steps, assuming this subspace is a slowly drifting object that can be tracked. We show that beyond a small reproducible core, there is no such object. Two estimates of the top-r subspace computed at the same step from disjoint minibatches disagree as much as estimates computed T steps apart (0.73 vs 0.74 of the maximal chordal distance sqrt(2r), at Pythia-160M with r=128): the apparent rotation at each refresh is dominated by estimator noise. This holds across four model families in three architecture classes from 70M to 6.9B parameters, strengthening with scale, and more weakly in a vision transformer. Only ~39 of 128 directions are reproducible across minibatches, and averaging cannot recover the rest: under N-fold averaging the gradient's spectral tail shrinks as N^(-1/4) rather than the N^(-1/2) of pure noise, so no averaging budget makes the subspace well defined. What helps instead follows from treating each refresh as a change of coordinates for Adam's state. Carrying the second moment blindly is provably about (r-k*)/2 worse than the best rotation-blind estimator, while the first moment transports exactly through the rotation, the optimal linear map under isotropic gradients and the rule LDAdam uses. At 1B over 40k steps (3 seeds), full LDAdam reaches 18.7 perplexity at beta2=0.999, beating untransported GaLore after its best beta2 fix (19.3); shortening the second-moment memory to beta2=0.99 helps the refreshing optimizers, though for canonical GaLore the effect is small and a full-rank control reverses it. One measurable fact, subspace non-identifiability, clarifies why GaLore works, which patches work, and what to check before trusting a low-rank assumption: the reproducible rank k*.

Estimating the variance of a conditional expectation is a recurring problem in stochastic simulation, with applications in global sensitivity analysis and Sobol' index estimation. This paper revisits Sobol' index estimation through the lens of nested simulation and develops a unified comparison of classical pick-freeze estimators and nested simulation estimators under a common computational budget. We show that several standard pick-freeze estimators can be interpreted as nested simulation estimators with fixed inner-level sample sizes, enabling direct performance comparisons and clarifying their bias-variance behavior. Building on this perspective, we analyze the standard nested simulation estimator for the Sobol' index numerator and propose two jackknife-based extensions: an unbiased jackknife estimator and a split jackknife estimator that uses an independent preliminary sample to estimate the mean. Under crude Monte Carlo (CMC), the split jackknife estimator attains the canonical mean squared error (MSE) rate, whereas the standard nested simulation and unbiased jackknife estimators attain the slower nested simulation rate. We also characterize the associated allocations of outer- and inner-level simulation effort. Finally, we study the impact of Latin hypercube sampling (LHS), showing that it can improve the standard nested simulation estimator while undermining bias reduction in jackknife-based estimators unless the inner-level sample size grows with the total budget. Numerical experiments corroborate the theory and provide practical guidance on estimator selection for Sobol' index estimation under CMC and LHS.

Recommendation systems and chatbots present content to users, typically using stochastic algorithms that select the content based on user characteristics or context. Examples of content include chat responses, videos, or items available for purchase. Scientists and application developers are often interested in whether characteristics of content increase outcomes such as user engagement. Estimates of such causal effects may guide content providers to generate content that emphasize desirable features. However, in settings with a large content library or where content is generated uniquely for a given user, it can be difficult to use observational data to learn the causal effect of content features, because the content a user sees is tailored to that user, and because content varies in many dimensions. This paper proposes a new method for estimating the impact of content features using observational data, when the algorithm that determines user exposure incorporates some randomization, and when two additional data elements are logged for each user: $(i)$ the identity of at least one item that could have been exposed to the user, but was not (the unexposed item); $(ii)$ an estimate of the ratio of the probability that the unexposed item would have been shown to the probability that the exposed item was shown. We show that causal effects of features are identified in this setting, even in the presence of unobserved confounders that affect both user preferences and the identity of the considered pair of items (exposed and unexposed). Our estimator differs from prior approaches in terms of what data is used and how the estimator is constructed.

Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989). Existing boosting algorithms construct a strong learner using $O(\log(\frac{1}{\epsilon})/\gamma^2)$ calls to a $\gamma$-advantage weak learner, and this round complexity is known to be optimal for generic boosters that succeed on all concept classes (Freund 1995). We show that this lower bound can be circumvented for concept classes that satisfy a mild closure property. Specifically, we present a new boosting algorithm that, for any class $\mathcal{F}$ closed under $O(\log \frac{1}{\gamma})$-XOR, strong learns $\mathcal{F}$ using $O(\log \frac{1}{\epsilon})$ calls to a $\gamma$-advantage weak learner and a single batch of $\tilde{O}(\log(\frac{1}{\epsilon})/\gamma^2)$ additional samples. Our algorithm arises from a new and simple connection between boosting and list-decodable codes. Viewing the target function as a message, we run the weak learner on its encoding and view the resulting weak hypothesis as a corrupted codeword. Feeding this corrupted codeword to a list decoder, we obtain a small list of candidate hypotheses, at least one of which is a strong hypothesis for the original function. Using additional samples, we identify and output this strong hypothesis.

Gene–gene networks are often observed only on a restricted target set, while modern biomedical foundation models provide proxy gene embeddings over substantially larger gene universes. To leverage externally learned representations to improve latent-structure recovery in partially observed target networks, we propose Proxy-Latent Assisted Network Estimation (PLANE), an adaptively weighted joint network–embedding latent variable model. PLANE combines the two sources of information through the common latent positions of the target network and proxy embeddings. Under mild rank conditions, the target network enables the identification of latent positions and loading of all nodes up to an orthogonal rotation. We show that zero-order optimality analyses sharply control the weighted reconstruction loss, but are insufficient to identify the optimal weighting. To understand the network and embedding information trade-off for latent-factor recovery, we analyze blockwise Gram-normalized gradient descent and prove deterministic contraction of aligned, curvature-weighted errors up to an explicit statistical tolerance. We then specialize the weighted statistical error bound to derive the target-block error bound, yielding an optimal, data-adaptive choice of the network embedding weights. Simulations and single-cell perturbation analyses show that informative proxy embeddings improve latent recovery, network reconstruction, and imputation beyond the observed target network.

International trade data, such as the FAO dataset, can be naturally represented as multilayer-dynamic networks, where countries are nodes, trade relationships are edges, different products correspond to different layers, and networks evolve over time. An important problem is how to identify evolving community structures in such multilayer-dynamic trade networks. Motivated by this problem, we study community detection in multilayer-dynamic networks, allowing the community structure to vary across both layers and time. We propose a novel method, MuDySC (Multilayer-Dynamic Spectral Clustering), which smooths the eigenspace projection matrices across adjacent time points and across layers at the same time point. We develop an efficient alternating iterative algorithm and establish both global and local convergence results, with the latter allowing weaker conditions on the tuning parameters when the relevant eigenspaces are sufficiently close and the algorithm is suitably initialized. We apply MuDySC to the FAO data. The analysis reveals clear asymmetry between export and import community structures and highlights both persistent and shifting trade positions of major countries. As an extension to accommodate substantial heterogeneity across network layers, we develop TLC-MuDySC, a tensor-based layer-clustering method that first identifies structurally related layers and then applies MuDySC within the estimated layer groups.

We study randomized experiments with binary outcomes under Neyman's nonparametric model, where covariate measurements are fixed but potential outcomes are random. In this setting we derive the exact variance of the difference-in-means estimator and characterize designs that minimize it. We show that any balanced design satisfying a covariate-balance condition is asymptotically optimal, and we prove that a broad class of blocking designs satisfies this condition under mild smoothness assumptions. Because the variance depends on unknown success probabilities, unbiased variance estimation is impossible. We therefore develop two conservative estimators: a generalization of the Cochran-Mantel-Haenszel (CMH) statistic applicable to any balanced design, and an extension of Robins' variance estimator for blocking designs. We establish conditions under which the CMH-based estimator is asymptotically tight under local alternatives, thereby yielding asymptotically valid confidence intervals. Our theoretical and simulations results show that blocking and other designs that achieve covariate balance and sufficient degree of randomness, perform well when they are equipped with the CMH-based inference. Thus, we provide an experimental design and inference framework for incidence outcomes that is simultaneously variance-optimal, conservative in finite samples and asymptotically tight.

This paper studies the estimation of the $m$th Gini index under finite mixtures of gamma distributions. We derive closed-form expressions for the $m$th Gini index and for the expectation and bias of its non-parametric U-statistic estimator, extending previous results for both single gamma populations and gamma mixture models. We further establish the asymptotic properties of the estimator for gamma mixtures sharing a common rate parameter, including an asymptotic lower bound for the bias, asymptotic unbiasedness, strong consistency, and asymptotic normality. Although these theoretical results require a common rate parameter, a Monte Carlo study also investigates the estimator under mixtures with different rates and compares its performance with bias-corrected and parametric estimators. Finally, the proposed methodology is illustrated through the analysis of an income dataset.

Infinite-width limits are a standard way to reason about neural networks, but it is not automatic that the limiting learner has the same complexity-theoretic inductive bias as large finite networks. We study this question for Bayesian neural networks at the mean-field, or critical feature-learning, scaling. The central quantity is the reduced entropy \[ s_\infty(y,\varepsilon)=\limsup_N -\frac{1}{N}\log \pi_N^0(L\le \varepsilon), \] the intensive prior cost of representing a target function $y$ to population mean-squared error $\varepsilon$. Our main result is a width-robust learnability theorem. At fixed depth, a family of Boolean-cube targets is learnable from polynomially many samples at infinite width if and only if it is learnable at polynomial width, if and only if its reduced entropy is polynomially bounded. Equivalently, up to polynomial slack in accuracy, the Bayesian mean-field learner generalizes exactly on the targets that can be represented by polynomial-size networks. The forward direction is proved by a form of subsampling: from the infinitely many hidden neurons in the mean-field solution, one can select polynomially many representatives and still preserve the learned function on every input simultaneously. At the critical scaling this subsampling has both an “active” component, which keeps the data-dependent low-dimensional statistics, and a “lazy” component, which resamples the entropy-dominated directions from the prior. Thus the infinite-width mean-field limit gives a clean analytic description of learning without introducing spurious width-dependent generalization power.

These lecture notes introduce optimal transport as a mathematical language for actuarial science. They treat losses, premiums, scores, reserves, capital scenarios, climate losses and lifetime distributions as probability measures that can be compared, transported, averaged, stressed and interpolated. The first part develops the main tools: couplings, push-forwards, discrete and continuous Kantorovich problems, duality, Wasserstein distances, quantile transport, barycenters, entropic regularization and statistical optimal transport. The second part applies these tools to risk measures, Wasserstein robustness, pricing and capital, portfolio drift, reserving cash-flow distributions, climate-prevention diagnostics, reinsurance, dependence uncertainty, capital allocation, distributional fairness diagnostics and longevity risk. Later chapters and appendices discuss counterfactual transport, dynamic formulations, unbalanced transport, Schrödinger bridges, cost engineering and computational labs in R. The emphasis is on actuarial modelling choices: the state space, the ground cost, the ambiguity radius, the reference distribution and the interpretation of the transport plan. Transport maps and couplings are used as distributional objects, not as causal claims unless additional assumptions are imposed.

Large language models (LLMs) are increasingly used as synthetic survey respondents, but existing evaluations mostly ask whether individual answers look plausible. We ask a stricter, psychometric question: do LLMs preserve the joint distribution, latent structure, reliability, mediation pathways, and demographic effects of real human survey data? We introduce an open Lithuanian organisational-psychology dataset (n=263 employees; three validated instruments; 68 items, 12 subscales) and condition 37 LLMs on real respondent profiles under a persona-disclosure ladder, presentation, reasoning-effort and cross-language ablations, counterfactual demographic swaps, and a memorization probe. The resulting Psychometric Similarity Score (PSS) is anchored against five statistical baselines, a held-out human ceiling, and the real answers of the same respondents. LLMs reproduce the direction of human psychometric relationships, but statistical baselines with no language model match or beat them. The best LLM (PSS 0.81) is level with a Gaussian copula fit only on other respondents (0.82) and below the real answers of the same 100 people (0.89). A simple classifier separates every LLM sample from humans (median AUC 0.99), and verbatim recall of the items is rare (at most 4.7%) and unrelated to PSS. Counterfactual swaps move answers in the direction of human group differences and by a similar amount (role 0.49 and education 0.33 human SD, vs. 0.46 and 0.44 in humans). Gender swaps move them by 0.06 SD. Tucker's phi on UWES falls inside an item-permutation null for 10 of 37 models. Synthetic respondents compress the response scale (within-respondent SD 0.79 vs. 1.19), lose predictive validity on held-out humans (mean R^2=-0.14 vs. 0.28), and produce significant indirect effects on 4 of 7 mediation paths that are null in humans. LLM samples are therefore not a drop-in replacement for human survey data.

Logit-based watermarking is a widely used mechanism for identifying LLM generated content, yet its effectiveness is governed by a fundamental trade-off between detectability and semantic distortion. Existing analyses provide limited guidance for principled hyperparameter selection, leaving practical deployments reliant on heuristic tuning. In this work, we develop a power-calibrated statistical framework that establishes explicit quantitative relationships between watermark hyperparameters, detection power, and distortion. This characterization transforms watermark design into a guided optimization problem. Building on these results, we derive practical parameter selection procedures that achieve optimal tradeoffs under constraints. Extensive experiments across multiple language models and datasets validate the theory and demonstrate that the proposed framework consistently identifies Pareto-optimal points.

Hierarchical mixture models are a powerful tool for modeling data generated from heterogeneous sources, particularly when the mixing proportion $\boldsymbol{w}$ itself is treated as a random variable with a Dirichlet or Beta-Liouville prior. Such models are widely employed in scenarios where uncertainty in class membership or data-generating processes must be probabilistically quantified. This paper studies the exact marginalization of the mixture weight. For the two-component case we give an $O(n^2)$ dynamic program – and an $O(n \log^2 n)$ FFT variant – for the marginal likelihood, and show that the exact posterior of the weight is a finite mixture of Beta distributions, delivering closed-form posterior summaries, credible intervals and per-observation local false-discovery rates without any sampling. For $K \ge 3$ components we give an exact joint dynamic program. The gain is largest in the small-sample regime the method is built for: on a real multilevel meta-analysis, a pathway-level dysregulation analysis of leukemia gene expression, and a leukemia-derived gene-panel benchmark with known ground truth, the exact interval for the signal proportion is calibrated where EM gives no interval at all (collapsing to a boundary) and Gaussian/Laplace approximations mis-cover, and it is two orders of magnitude faster than the sampler that would match it. On the large prostate-cancer benchmark, where every method has ample data, it agrees with locfdr on the gene ranking while adding a posterior interval for the null proportion.

Marketing systems frequently undergo operational changes that may affect performance, making timely detection of adverse effects essential for decision making. While Bayesian Causal Impact models are widely used to estimate the causal effects of interventions, their ability to support early operational monitoring remains less explored. This paper proposes a simulation based framework to evaluate the sensitivity and detection capabilities of Bayesian causal impact analysis under controlled performance degradations. Using daily traffic data from an abandoned cart marketing journey, we repeatedly perturb the observed outcomes and assess alarm activation probabilities across different effect magnitudes, confidence levels, and detection horizons. Two alarm criteria are analyzed: one based on the proportion of observations falling below the predictive lower bound and another based on consecutive days of negative cumulative impact. Results show that detection performance depends strongly on the interaction between effect size, confidence level, and evaluation horizon. In particular, proportion based criteria become less effective as the monitoring horizon increases, whereas persistence based criteria provide more stable and operationally meaningful detection behavior. The proposed framework extends causal impact analysis beyond retrospective effect estimation, offering a practical methodology for quantifying detection sensitivity and supporting monitoring decisions in dynamic marketing environments.

The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on an often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments prioritizing retention can impede real-time adaptation. Shifting the focus to the Average Lifelong Error (ALE), we formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics. We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias. We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is only one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, which we call Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.

The normalized power prior provides a principled framework for incorporating historical data into Bayesian inference while preserving coherence, but its routine application has been hindered by the need to evaluate an intractable normalizing constant function $C(a_{0})$. Existing approaches typically rely on model-specific marginal likelihood calculations, numerical integration over grids, or auxiliary sampling schemes implemented outside standard Bayesian software. In this paper, we present a computational perspective that exploits a simple and underutilized functional identity: under the unnormalized power prior, the marginal distribution of the power parameter is proportional to the normalizing constant of the normalized power prior. Leveraging this relationship, we propose a sampling-based strategy to approximate the normalized power prior using output from generic Markov chain Monte Carlo (MCMC) algorithms. The resulting approximation can be implemented entirely within all general Bayesian software packages (such as PROC MCMC, BUGS, JAGS, Stan, or NIMBLE), without requiring explicit marginal likelihood evaluation. Several illustrative examples demonstrate the practicality of the approach and highlight its potential to facilitate the routine use of normalized power priors in applied settings.

In survey sampling, the goal is to estimate finite population parameters such as totals, means, and proportions. At the estimation stage, it is common to have access to auxiliary information in the form of covariates known either in aggregate form or for each population unit. These covariates are often used, through models relating them to the variable of interest, to improve efficiency; this approach is known as model-assisted estimation. Modern applications increasingly involve settings where a large number of covariates are observed, sometimes of the same order as the sample size. While this setting offers greater modeling flexibility, it also creates important challenges for inference. In this article, we study variance estimation for the generalized regression (GREG) estimator in high-dimensional regimes. We derive new theoretical results that characterize the high-dimensional asymptotic bias of commonly used variance estimators, including those based on Taylor linearization. Furthermore, under suitable distributional assumptions on the covariates, we show that a cross-validated variance estimator is naturally asymptotically unbiased.

We develop a unified function space theory of deep fully connected neural networks. Functions in our spaces are defined recursively as $\ell^1$-bounded linear combinations of activated functions from preceding layers, with a dictionary of affine functions at the first layer. Unlike existing theories that are largely specialized to homogeneous activations such as the ReLU, our framework provides a meaningful notion of functional complexity for deep networks with a broad range of homogeneous and non-homogeneous activation functions commonly used in practice. This simple construction unites several seemingly disparate ideas from the literature, including norm-based complexity bounds and variational characterizations of depth, and facilitates novel analyses of what kinds of functions deep norm-constrained networks can represent. To this end, we prove a novel representer theorem for our spaces and establish novel function-space complexity bounds showing that the associated function classes remain qualitatively small at arbitrary depth. In the univariate ReLU case, we prove a "depth saturation" result: depth in this setting yields only a small constant rescaling of the function class, with no added functional diversity. As a consequence, we show that deep norm-controlled ReLU functions in any dimension cannot exhibit high frequencies along any direction. This finding reveals that some commonly cited expressivity benefits of depth disappear once network complexity is controlled by an appropriate function space norm, rather than parameter count or other representational costs that permit compounded rescaling across layers. Overall, our results illustrate how a function space perspective yields new structural insights into the relationship between depth and complexity.

Large language models (LLMs) are becoming ubiquitous in engineering and science because they can turn prompts into data analysis code, experimental designs, formulations of optimization problems, among other applications. However, many LLMs suffer from a phenomenon called order dependency, in which the order of phrases in the prompt affects their performance on a given task. To overcome this issue, we introduce a systematic method that uses order-of-addition experiments to quantify the ordering effect of elements in a prompt and identify their best positions. We demonstrate our methodology by constructing two-level fractional factorial designs using state-of-the-art LLMs. We show that order-of-addition experiments can elucidate order dependency in these LLMs, and can help us to identify a high-quality prompt configuration for the task.

Randomized smoothing has emerged as a scalable technique for certifying the adversarial robustness of classifiers. However, its application to regression remains under-explored and faces unique challenges. Existing regression certificates rely on probabilistic acceptance regions and fail to exploit the local geometry of the function. In this work, we present a novel framework for certified robust regression that addresses these limitations. We derive a prediction-centered certificate that guarantees the stability of the smoothed model's prediction and ensures practical computability at test time. We investigate several alternatives for constructing these certificates by explicitly incorporating means, variances, and gradients. In particular, we demonstrate on the MNIST rotation task that utilizing gradient information yields significantly tighter robustness certificates compared to the current state-of-the-art, alpha-smoothing.

What does a discrete diffusion model actually learn? The literature trains denoisers, bridge plug-ins, and concrete scores, each deriving its own ELBO objective under different conventions. This fragmentation makes the object being learned unclear and ELBO values difficult to interpret and compare. All three methods learn the same object in different coordinates: the exact time-reversal of the noising process. We first prove the Oracle Distance identity: for any noising process, the negative ELBO is not just a bound on the likelihood; it equals the data entropy plus the path KL divergence from the reversed noising process to the learned one. Next, for token-factorizing noise, we show that denoiser, bridge plug-in, and concrete score are three coordinates of this reverse process, derive exact conversions between them, and identify the probability distribution learned by each. In particular, contrary to its standard interpretation, the bridge plug-in learns not a denoiser but the cavity distribution: the clean token given the noisy sequence excluding its own position. We confirm the theory by training all three coordinates with masked, uniform, and GIDD kernels on OpenWebText under a shared recipe, and by training uniform-diffusion solvers on four logic tasks. The learned coordinates agree after conversion, while confusing the cavity with the denoiser can severely degrade sampling quality. Together with exact boundary-term formulas, closed-form NELBO calibrations at initialization, and a clock interpretation of time schedules, these results provide a conceptual framework and a mechanical recipe for deriving, implementing, and comparing discrete diffusion processes.

We study the adversarial fake detection model introduced by Mendelson, Paouris and Vershynin. In this model, a genuine sample is $\pmb{X}\sim N(0,\pmb{I}_n)$, while a fake sample is produced as $\pmb{X}+r\pmb{t}({\pmb{X}})$, where the adversary first observes $\pmb{X}$ and then chooses an admissible perturbation $\pmb{t}({\pmb{X}})$ from a prescribed set $\mathscr{T}\subset\mathbb{R}^n$. The central quantity is the detectability radius $r(\mathscr{T})$, which formalizes the transition scale at which fake samples become reliably distinguishable from genuine ones. Mendelson, Paouris and Vershynin introduced the focused width $\widetilde{w}(\mathscr{T})$ as a geometric parameter for this radius and conjectured that, for every origin-symmetric set $\mathscr{T}$, it characterizes $r(\mathscr{T})$ up to universal constants. In this note, we disprove this conjecture for a broad class of discrete sets. More precisely, we consider any origin-symmetric set $\mathscr{T}_n$ lying between the hypercube and the odd integer grid: \begin{equation*} \{-1,1\}^n\subset\mathscr{T}_n\subset ( 2\mathbb{Z}+1)^n. \end{equation*} For every such $\mathscr{T}_n$, we prove that $\frac{\widetilde{w}(\mathscr{T}_n)}{r(\mathscr{T}_n) }\gtrsim \sqrt{\log n}$. Thus, in the Gaussian model, the focused width can overestimate the detectability radius by a $\sqrt{\log n}$ factor and therefore does not characterize it in general. We further show that this logarithmic scale is not intrinsic: in the corresponding non-Gaussian model with product Laplace data, the focused width benchmark can even exceed the detectability radius by at least a polynomial factor of order $n^{1/4}$.

Occupancy ratios correct distribution shift in offline reinforcement learning and are central to off-policy evaluation. Existing primal-dual and minimax methods typically estimate these ratios by enforcing occupancy-balance moments over a critic class. We propose fitted occupancy-ratio evaluation (FORE), a fitted fixed-point method that characterizes the discounted occupancy ratio through an adjoint Bellman recursion. At each iteration, FORE solves a single-level density-ratio objective on one-step-transition data, thereby projecting the adjoint Bellman image onto a log-ratio class in Kullback-Leibler (KL) divergence. Unlike analyses of fitted Q-evaluation, which typically require value-function realizability together with Bellman completeness or projected-operator stability, our central approximation condition is just realizability of the discounted occupancy ratio itself. Under this condition, the population KL-projected recursion contracts in relative entropy toward the true ratio by virtue of the adjoint Bellman operator being a KL-contraction. For the empirical recursion, we establish finite-sample regret bounds that yield convergence in KL up to approximation error and a statistical error governed by the complexity of the ratio hypothesis class. When full coverage fails, we introduce coverage-stopped FORE, which targets the discounted occupancy accumulated before the first uncovered state-action pair and yields a conservative lower bound on target-policy value for nonnegative rewards. The fitted ratio supports direct value estimation by reward reweighting, occupancy-weighted fitted Q-evaluation, and doubly robust estimation that combines the fitted ratio with a fitted Q-function. Together, these results identify discounted occupancy-ratio realizability as a sufficient condition for offline policy evaluation without any completeness assumptions.

Group Relative Policy Optimization (GRPO) is effective when the current policy already samples useful reasoning trajectories, but it stalls on hard prompts whose correct solution modes lie outside the student's on-policy support. We propose TREK (Teacher-Routed Exploration via Forward KL), a simple staged procedure that uses distillation not for imitation but for exploration support expansion. A key advantage of TREK is its generality: because it only consumes verified output trajectories, it can use an external black-box teacher, a white-box teacher, or the same model given additional inference-time context, and it can efficiently identify which hard-prompt samples are most worth consolidating even when teacher internals are unavailable. TREK first identifies prompts where the unaided student has very low pass rate, queries a proposal source to produce verified candidate solutions, keeps the top-$r$ proposals ranked by current student likelihood, applies a short forward-KL phase to pull those verified modes into the student's support, and then returns to standard on-policy GRPO refinement. On mathematical reasoning, TREK with DeepSeek-V4 proposals improves Qwen3 models across all tested scales on AIME 2024 and AIME 2025; for Qwen3-8B, it improves AIME 2025 from 36.9 to 40.3 and AIME 2024 from 47.9 to 51.1 (avg@16), while the self-context variant reaches 38.5 and 49.6 without an external teacher. On agentic tasks, TREK raises ALFWorld success rate from 75.8 to 82.8 and ScienceWorld success rate from 12.5 to 26.7; notably, on the hardest task types, TREK achieves high success rates early in training while unaided GRPO requires substantially more optimization steps to reach comparable levels.

We study estimation and inference for online quantile regression under a one-report user-level $\eps$-locally differentially private ($\eps$-LDP) protocol. The main difficulty is that the standard quantile-regression estimating-equation contribution couples covariates with a residual comparison, so a server that receives only privatized reports cannot form the usual online update. We address this by developing a finite-alphabet channel in which each user computes the contribution locally, applies support-aware stochastic quantization and randomized response to one selected-block category, and sends one report. A public decoder corrects the randomized-response distortion and reconstructs a server-side estimating-equation input with the correct conditional mean. These decoded inputs are then used in projected Polyak-Ruppert averaging. For fixed finite channel designs, we establish local privacy, decoder unbiasedness, consistency, asymptotic normality, and Hessian-free self-normalized inference for prespecified scalar contrasts. Simulations and a New York City taxi-trip illustration show that the private trajectory approaches the nonprivate online reference as the privacy budget grows and outperforms direct Laplace and face-exponential geometric releases in the reported regimes.