Papers

Mixture-of-Experts (MoE) architectures combine specialized predictors through a learned gate and are effective across regression and classification, but for classification with softmax multinomial-logistic gating, rigorous guarantees for stable maximum-likelihood training and principled model selection remain limited. We address both issues in the full-data (batch) regime. First, we derive a batch minorization-maximization (MM) algorithm for softmax-gated multinomial-logistic MoE using an explicit quadratic minorizer, yielding coordinate-wise closed-form updates that guarantee monotone ascent of the objective and global convergence to a stationary point (in the standard MM sense), avoiding approximate M-steps common in EM-type implementations. Second, we prove finite-sample rates for conditional density estimation and parameter recovery, and we adapt dendrograms of mixing measures to the classification setting to obtain a sweep-free selector of the number of experts that achieves near-parametric optimal rates after merging redundant fitted atoms. Experiments on biological protein–protein interaction prediction validate the full pipeline, delivering improved accuracy and better-calibrated probabilities than strong statistical and machine-learning baselines.

The emergence of compositional reasoning in large language models through reinforcement learning with verifiable rewards (RLVR) has been a key driver of recent empirical successes. Despite this progress, it remains unclear which compositional problems are learnable in this setting using outcome-level feedback alone. In this work, we theoretically study the learnability of compositional problems in autoregressive models under RLVR training. We identify a quantity that we call the task-advantage ratio, a joint property of the compositional problem and the base model, that characterizes which tasks and compositions are learnable from outcome-level feedback. On the positive side, using this characterization, we show that compositional problems where correct intermediate steps provide a clear advantage are efficiently learnable with RLVR. We also analyze how such an advantage naturally arises in different problems. On the negative side, when the structural advantage is not present, RLVR may converge to suboptimal compositions. We prove that, in some cases, the quality of the base model determines if such an advantage exists and whether RLVR will converge to a suboptimal solution. We hope our analysis can provide a principled theoretical understanding of when and why RLVR succeeds and when it does not.

We study geometric ergodicity of the Gibbs sampler for linear latent non-Gaussian models (LLnGMs), a class of hierarchical models in which conditional Gaussian structure is preserved through generalized inverse Gaussian (GIG) variance-mixture augmentation. Two complementary routes to geometric ergodicity are developed for the marginal chain on the mixing variables. First, we show that the associated Markov operator is trace-class, and hence admits a spectral gap, over a large portion of the GIG parameter space. Second, for the remaining boundary and heavy-tail regimes, we establish geometric ergodicity via drift and minorization, subject to an explicit null-smallness condition that quantifies how the drift interacts with the null space of the observation operator. Together, these results cover the full GIG parameter space, including the normal-inverse Gaussian, generalized asymmetric Laplace, and Student-$t$ special cases. The geometric ergodicity of this chain underpins the consistency of Gibbs-based stochastic-gradient estimators for maximum likelihood estimation, and we provide conditions that make the required integrability checks transparent. Numerical experiments illustrate the theoretical findings, contrasting mixing efficiency across parameter regimes and probing the role of the null-smallness constant.

Maintainability analysis is a cornerstone of reliability engineering. While the Markov approach is the classical analytical foundation, its reliance on the exponential distribution for failure and repair times is a major and often unrealistic limitation. This paper directly overcomes this critical constraint by investigating and modeling system maintainability using the more flexible and versatile Lindley distribution, which is represented via phase-type distributions. We first present a comprehensive maintainability analysis of a single-component system, deriving precise closed-form expressions for its time-dependent and steady-state availability, as well as the mean time to repair. The core methodology is then systematically generalized to analyze common series and parallel system configurations with n independent and identically distributed components. A dedicated numerical study compares the system performance under the Lindley and exponential distributions, conclusively demonstrating the significant and practical impact of non-exponential repair times on key reliability metrics. Our work provides a versatile and more widely applicable analytical framework for accurate maintainability assessment that successfully relaxes the restrictive exponential assumption, thereby offering greater realism in reliability modeling.

AI agents equipped with tool-calling capabilities are susceptible to Indirect Prompt Injection (IPI) attacks. In this attack scenario, malicious commands hidden within untrusted content trick the agent into performing unauthorized actions. Existing defenses can reduce attack success but often suffer from the over-defense dilemma: they deploy expensive, always-on sanitization regardless of actual threat, thereby degrading utility and latency even in benign scenarios. We revisit IPI through a causal ablation perspective: a successful injection manifests as a dominance shift where the user request no longer provides decisive support for the agent's privileged action, while a particular untrusted segment, such as a retrieved document or tool output, provides disproportionate attributable influence. Based on this signature, we propose CausalArmor, a selective defense framework that (i) computes lightweight, leave-one-out ablation-based attributions at privileged decision points, and (ii) triggers targeted sanitization only when an untrusted segment dominates the user intent. Additionally, CausalArmor employs retroactive Chain-of-Thought masking to prevent the agent from acting on “poisoned” reasoning traces. We present a theoretical analysis showing that sanitization based on attribution margins conditionally yields an exponentially small upper bound on the probability of selecting malicious actions. Experiments on AgentDojo and DoomArena demonstrate that CausalArmor matches the security of aggressive defenses while improving explainability and preserving utility and latency of AI agents.

Causal discovery from time series is a fundamental task in machine learning. However, its widespread adoption is hindered by a reliance on untestable causal assumptions and by the lack of robustness-oriented evaluation in existing benchmarks. To address these challenges, we propose CausalCompass, a flexible and extensible benchmark framework designed to assess the robustness of time-series causal discovery (TSCD) methods under violations of modeling assumptions. To demonstrate the practical utility of CausalCompass, we conduct extensive benchmarking of representative TSCD algorithms across eight assumption-violation scenarios. Our experimental results indicate that no single method consistently attains optimal performance across all settings. Nevertheless, the methods exhibiting superior overall performance across diverse scenarios are almost invariably deep learning-based approaches. We further provide hyperparameter sensitivity analyses to deepen the understanding of these findings. We additionally conduct ablation experiments to explain the strong performance of deep learning-based methods under assumption violations. We also find, somewhat surprisingly, that NTS-NOTEARS relies heavily on standardized preprocessing in practice, performing poorly in the vanilla setting but exhibiting strong performance after standardization. Finally, our work aims to provide a comprehensive and systematic evaluation of TSCD methods under assumption violations, thereby facilitating their broader adoption in real-world applications. The user-friendly implementation, documentation and datasets are available at https://anonymous.4open.science/r/CausalCompass-anonymous-5B4F/.

We develop a unified $L$-statistic testing framework for high-dimensional regression coefficients that adapts to unknown sparsity. The proposed statistics rank coordinate-wise evidence measures and aggregate the top $k$ signals, bridging classical max-type and sum-type tests. We establish joint weak convergence of the extreme-value component and standardized $L$-statistics under mild conditions, yielding an asymptotic independence that justifies combining multiple $k$'s. An adaptive omnibus test is constructed via a Cauchy combination over a dyadic grid of $k$, and a wild bootstrap calibration is provided with theoretical guarantees. Simulations demonstrate accurate size and strong power across sparse and dense alternatives, including non-Gaussian designs.

Many inverse problems arising in engineering and applied sciences involve unknown quantities with pronounced spatial inhomogeneity, such as localized defects or spatially varying material properties, making reliable uncertainty quantification particularly challenging. While Bayesian inverse problem methodologies provide a principled framework for assessing reconstruction reliability, commonly used Gaussian priors, such as Whittle-Matern models, impose globally homogeneous assumptions that limit their ability to capture such structure in large-scale settings. We introduce a new class of inhomogeneous priors defined via convolution with white noise, yielding nonstationary Whittle-Matern-type random fields with a rigorous mathematical construction. These priors fit naturally within existing Bayesian well-posedness theory and enable efficient sampling by reducing prior realizations to the solution of a pseudo-differential equation, for which we develop numerical schemes with quantified approximation error. Numerical experiments in one-dimensional denoising and two-dimensional limited-angle X-ray tomography demonstrate significant improvements in reconstruction quality and uncertainty quantification, particularly in data-limited scenarios.

This study establishes a nontrivial upper bound on the out-of-sample $R^2$ ($R^2_{\text{OOS}}$) in return forecasting. In particular, we define a coin-flip oracle model that, under the same directional accuracy, theoretically outperforms practical models in terms of MSE. The $R^2_{\text{OOS}}$ of the oracle model, whose analytical expression is a quadratic function of directional accuracy, can therefore serve as a tractable upper bound on the actual $R^2_{\text{OOS}}$. Empirical analyses across multiple forecasting scenarios reveal that the $R^2_{\text{OOS}}$ values of common predictive models are fundamentally bounded by this quadratic function.

Causal decomposition analysis (CDA) is an approach for modeling the impact of hypothetical interventions to reduce disparities. It is useful for identifying foci that future interventions, including multilevel and multimodal interventions, could focus on to reduce disparities. Based within the potential outcomes framework, CDA has a causal interpretation when the identifying assumptions are met. CDA also allows an analyst to consider which covariates are allowable (i.e., fair) for defining the disparity in the outcome and in the point of intervention, so that its interpretation is also meaningful. While the incorporation of causal inference and allowability promotes robustness, transparency, and dialogue in disparities research, it can lead to challenges in estimation such as the need to correctly model densities. Also, how CDA differs from commonly used statistical decomposition estimators from the econometrics literature may not be clear, which may limit its uptake. To address these challenges, we provide a tour of estimation strategies for CDA, reviewing existing proposals and introducing novel estimators that overcome key estimation challenges. Among them we introduce what we call "bridging" estimators that avoid modeling any density, and sequential weighted regression estimators that are multiply robust. Additionally, we provide diagnostics to assess the quality of the nuisance density models and weighting functions they rely on. We formally establish the estimators' robustness to model mis-specification, demonstrate their performance through a simulation study based on real data, and apply them to study disparities in uncontrolled hypertension using electronic health records in a large healthcare system.

Amid China's ageing and digital shift, digital exclusion among older adults poses an urgent challenge. To unpack this phenomenon, this study disentangles age, period, and cohort effects on digital exclusion among middle-aged and older Chinese adults. Using three nationally representative surveys (CHARLS 2011-2020, CFPS 2010-2022, and CGSS 2010-2021), we fitted hierarchical age-period-cohort (HAPC) models weighted by cross-sectional survey weights and stabilized inverse probability weights for item response. We further assessed heterogeneity by urban-rural residence, region, multimorbidity, and cognitive risk, and evaluated robustness with APC bounding analyses. Across datasets, digital exclusion increased with age and displayed mild non-linearity, with a small midlife easing followed by a sharper rise at older ages. Period effects declined over the 2010s and early 2020s, although the pace of improvement differed across survey windows. Cohort deviations were present but less consistent than age and period patterns, with an additional excess risk concentrated among cohorts born in the 1950s. Rural and western residents, as well as adults with multimorbidity or cognitive risk, remained consistently more excluded. Over the study period, the urban-rural divide showed no evidence of narrowing, whereas the cognitive-risk gap widened. These findings highlight digital inclusion as a vital pathway for older adults to remain integral participants in an evolving digital society.

Contextual bandits are a core technology for personalized mobile health interventions, where decision-making requires adapting to complex, non-linear user behaviors. While Thompson Sampling (TS) is a preferred strategy for these problems, its performance hinges on the quality of the underlying reward model. Standard linear models suffer from high bias, while neural network approaches are often brittle and difficult to tune in online settings. Conversely, tree ensembles dominate tabular data prediction but typically rely on heuristic uncertainty quantification, lacking a principled probabilistic basis for TS. We propose Bayesian Forest Thompson Sampling (BFTS), the first contextual bandit algorithm to integrate Bayesian Additive Regression Trees (BART), a fully probabilistic sum-of-trees model, directly into the exploration loop. We prove that BFTS is theoretically sound, deriving an information-theoretic Bayesian regret bound of $\tilde{O}(\sqrt{T})$. As a complementary result, we establish frequentist minimax optimality for a "feel-good" variant, confirming the structural suitability of BART priors for non-parametric bandits. Empirically, BFTS achieves state-of-the-art regret on tabular benchmarks with near-nominal uncertainty calibration. Furthermore, in an offline policy evaluation on the Drink Less micro-randomized trial, BFTS improves engagement rates by over 30% compared to the deployed policy, demonstrating its practical effectiveness for behavioral interventions.

Circular biomarkers arise naturally in many biomedical applications, particularly in ophthalmology, where angular measurements such as astigmatism are routinely recorded. Similar directional variables also occur in the study of human body rotations, including movements of the hand, waist, neck, and lower limbs. Motivated by a clinical dataset comprising angular measurements of astigmatism induced by two cataract surgery procedures, we propose a novel two-sample testing framework for circular data grounded in hyperbolic geometry. Assuming von Mises distributions with either common or group-specific concentration parameters, we embed the corresponding parameter spaces into the Poincaré disk, an open unit disk endowed with the Poincaré metric.Under this construction, each von Mises distribution is mapped uniquely to a point in the Poincaré disk, yielding a continuous geometric representation that preserves the intrinsic structure of the parameter space. This embedding enables direct comparison of group distributions via hyperbolic distances, leading to natural and interpretable test statistics. We develop permutation-based tests for the common concentration case and bootstrap-based procedures for unequal concentrations. Extensive simulation studies demonstrate stable empirical size, strong consistency, and superior asymptotic power compared with existing competing methods. The proposed methodology is illustrated through a detailed analysis of the cataract surgery dataset, including a clinically informed restructuring of the original observations. The results highlight the practical advantages of incorporating hyperbolic geometry into the analysis of circular biomedical data and underscore the potential of geometry-aware inference for directional biomarkers.

We study generation in separable metric instance spaces. We extend the language generation framework from Kleinberg and Mullainathan [2024] beyond countable domains by defining novelty through metric separation and allowing asymmetric novelty parameters for the adversary and the generator. We introduce the $(\varepsilon,\varepsilon')$-closure dimension, a scale-sensitive analogue of closure dimension, which yields characterizations of uniform and non-uniform generatability and a sufficient condition for generation in the limit. Along the way, we identify a sharp geometric contrast. Namely, in doubling spaces, including all finite-dimensional normed spaces, generatability is stable across novelty scales and invariant under equivalent metrics. In general metric spaces, however, generatability can be highly scale-sensitive and metric-dependent; even in the natural infinite-dimensional Hilbert space $\ell^2$, all notions of generation may fail abruptly as the novelty parameters vary.

The analysis of multivariate discrete data is crucial in various scientific research areas, such as epidemiology, the social sciences, genomics, and environmental studies. As the availability of such data increases, developing robust analytical and data generation tools is necessary to understand the relationships among variables. This paper builds upon previous work on data generation frameworks for multivariate ordinal data with a prespecified correlation matrix. The proposed algorithm generates multivariate discrete data from marginal distributions that follow the generalized Poisson, negative binomial, and binomial distributions. A step-by-step algorithm is provided, and its performance is illustrated in four simulated data scenarios and three real-data scenarios. This technique has the potential to be applied in a wide range of settings involving the generation of correlated discrete data.

Many political surveys rely on post-stratification, raking, or related weighting adjustments to align respondents with the target population. But when respondents differ from nonrespondents on the outcome itself (nonignorable nonresponse), these adjustments can fail, introducing bias even into basic descriptives. We provide a practical method that corrects for nonignorable nonresponse by leveraging response-propensity proxies (e.g., a respondent's rating of the interview, or interviewer-coded cooperativeness) observed among respondents to extrapolate toward nonrespondents, while directly integrating observable covariates and retaining the benefits of post-stratification with known population shares. The method generalizes the variable-response-propensity (VRP) framework of Peress (2010) from binary to ordinal outcomes, which are widely used to measure trust, satisfaction, and policy attitudes. The resulting estimator is computed by maximum likelihood and implemented in a compact R routine that handles both ordinal and binary outcomes. Using the 2024 American National Election Study (ANES), we show that accounting for nonignorable nonresponse produces substantively meaningful shifts for life satisfaction (estimated latent correlation $\rho \approx 0.47$), while yielding only modest changes for retrospective economic evaluations ($\rho \approx 0.14$), highlighting when nonignorable nonresponse substantively affects survey estimates.

The impact of routine smaller outages on distribution system customers in terms of customer minutes interrupted can be tracked using conventional reliability indices. However, the customer minutes interrupted in large blackout events are extremely variable, and this makes it difficult to quantify the customer impact of these extreme events with resilience metrics. We solve this problem with the System Average Large Event Duration Index SALEDI that logarithmically transforms the customer minutes interrupted. We explain how this new resilience metric works, compare it with alternatives, quantify its statistical accuracy, and illustrate its practical use with standard outage data from five utilities.

Understanding how environmental drivers relate to vegetation condition motivates spatially varying regression models, but estimating a separate coefficient surface for every predictor can yield noisy patterns and poor interpretability when many predictors are irrelevant. Motivated by MODIS vegetation index studies, we examine predictors from spectral bands, productivity and energy fluxes, observation geometry, and land surface characteristics. Because these relationships vary with canopy structure, climate, land use, and measurement conditions, methods should both model spatially varying effects and identify where predictors matter. We propose a spatially varying coefficient model where each coefficient surface uses a tensor product B-spline basis and a Bayesian group lasso prior on the basis coefficients. This prior induces predictor level shrinkage, pushing negligible effects toward zero while preserving spatial structure. Posterior inference uses Markov chain Monte Carlo and provides uncertainty quantification for each effect surface. We summarize retained effects with spatial significance maps that mark locations where the 95 percent posterior credible interval excludes zero, and we define a spatial coverage probability as the proportion of locations where the credible interval excludes zero. Simulations recover sparsity and achieve prediction. A MODIS application yields a parsimonious subset of predictors whose effect maps clarify dominant controls across landscapes.

In this paper, we study the problem of detecting the presence of a planted perfect matching or spanning tree in an Erdős–Rényi random graph. More precisely, we study the hypothesis testing problem where the statistician observes a graph on $n$ vertices. Under the null hypothesis, the graph is a realization of an Erdős–Rényi random graph $G(n,q)$, while under the alternative hypothesis, the graph is the union of an Erdős–Rényi random graph and a random perfect matching (or random spanning tree). In order to avoid trivial detection by counting edges, we adjust the alternative hypothesis so that the expected number of edges under both distributions coincides. We prove that in both problems, when $q\gg n^{-1/2}$, no test can perform better than random guessing, while for $q\ll n^{-1/2}$, there exist computationally efficient tests that guess correctly with high probability.

Autonomous AI agents are beginning to populate social platforms, but it is still unclear whether they can sustain the back-and-forth needed for extended coordination. We study Moltbook, an AI-agent social network, using a first-week snapshot and introduce interaction half-life: how quickly a comment's chance of receiving a direct reply fades as the comment ages. Across tens of thousands of commented threads, Moltbook discussions are dominated by first-layer reactions rather than extended chains. Most comments never receive a direct reply, reciprocal back-and-forth is rare, and when replies do occur they arrive almost immediately – typically within seconds – implying persistence on the order of minutes rather than hours. Moltbook is often described as running on an approximately four-hour “heartbeat” check-in schedule; using aggregate spectral tests on the longest contiguous activity window, we do not detect a reliable four-hour rhythm in this snapshot, consistent with jittered or out-of-phase individual schedules. A contemporaneous Reddit baseline analyzed with the same estimators shows substantially deeper threads and much longer reply persistence. Overall, early agent social interaction on Moltbook fits a “fast response or silence” regime, suggesting that sustained multi-step coordination will likely require explicit memory, thread resurfacing, and re-entry scaffolds.

We extend classical analytic tools for finite-state statistical models to allow zero probabilities. Using methods from algebraic statistics and information geometry, we develop a framework in which a smooth statistical model could hit the boundary of the simplex, for example, in contingency tables with non-structural zeros. The central object of our approach is the vector bundle whose fibres are the $p$-contrasts associated to each probability distribution $p$. In this framework, Fisher score and other key statistical concepts, such as entropy for one-dimensional statistical models, admit an algebraic representation also on the boundary of the simplex.

Conformal prediction provides a distribution-free framework for uncertainty quantification via prediction sets with exact finite-sample coverage. In low dimensions these sets are easy to interpret, but in high-dimensional or structured output spaces they are difficult to represent and use, which can limit their ability to integrate with downstream tasks such as sampling and probabilistic forecasting. We show that any sufficiently regular differentiable nonconformity score induces a deterministic flow on the output space whose trajectories converge to the boundary of the corresponding conformal prediction set. This leads to a computationally efficient, training-free method for sampling conformal boundaries in arbitrary dimensions. Mixing across confidence levels yields conformal predictive distributions whose quantile regions coincide with the empirical conformal prediction sets. We provide an approximation bound decomposing CPD predictive error into score-induced distortion, base-measure quality, and gradient flow-induced distortion. We evaluate the approach on PDE inverse problems, precipitation downscaling, climate model debiasing, and hurricane trajectory forecasting.

In this work, we investigate the large-scale mean-field variational inference (MFVI) problem from a mini-batch primal-dual perspective. By reformulating MFVI as a constrained finite-sum problem, we develop a novel primal-dual algorithm based on an augmented Lagrangian formulation, termed primal-dual variational inference (PD-VI). PD-VI jointly updates global and local variational parameters in the evidence lower bound in a scalable manner. To further account for heterogeneous loss geometry across different variational parameter blocks, we introduce a block-preconditioned extension, P$^2$D-VI, which adapts the primal-dual updates to the geometry of each parameter block and improves both numerical robustness and practical efficiency. We establish convergence guarantees for both PD-VI and P$^2$D-VI under properly chosen constant step size, without relying on conjugacy assumptions or explicit bounded-variance conditions. In particular, we prove $O(1/T)$ convergence to a stationary point in general settings and linear convergence under strong convexity. Numerical experiments on synthetic data and a real large-scale spatial transcriptomics dataset demonstrate that our methods consistently outperform existing stochastic variational inference approaches in terms of convergence speed and solution quality.

We investigate the approximation capabilities of dense neural networks. While universal approximation theorems establish that sufficiently large architectures can approximate arbitrary continuous functions if there are no restrictions on the weight values, we show that dense neural networks do not possess this universality. Our argument is based on a model compression approach, combining the weak regularity lemma with an interpretation of feedforward networks as message passing graph neural networks. We consider ReLU neural networks subject to natural constraints on weights and input and output dimensions, which model a notion of dense connectivity. Within this setting, we demonstrate the existence of Lipschitz continuous functions that cannot be approximated by such networks. This highlights intrinsic limitations of neural networks with dense layers and motivates the use of sparse connectivity as a necessary ingredient for achieving true universality.

Balancing covariates is critical for credible and efficient randomized experiments. Rerandomization addresses this by repeatedly generating treatment assignments until covariate balance meets a prespecified threshold. By shrinking this threshold, it can achieve arbitrarily strong balance, with established results guaranteeing optimal estimation and valid inference in both finite-sample and asymptotic settings across diverse complex experimental settings. Despite its rigorous theoretical foundations, practical use is limited by the extreme inefficiency of rejection sampling, which becomes prohibitively slow under small thresholds and often forces practitioners to adopt suboptimal settings, leading to degraded performance. Existing work focusing on acceleration typically fail to maintain the uniformity over the acceptable assignment space, thus losing the theoretical grounds of classical rerandomization. Building upon a Metropolis-Hastings framework, we address this challenge by introducing an additional sampling-importance resampling step, which restores uniformity and preserves statistical guarantees. Our proposed algorithm, PSRSRR, achieves speedups ranging from 10 to 10,000 times while maintaining exact and asymptotic validity, as demonstrated by simulations and two real-data applications.

Modern benchmarks such as HELM MMLU account for multiple metrics like accuracy, robustness and efficiency. When trying to turn these metrics into a single ranking, natural aggregation procedures can become incoherent or unstable to changes in the model set. We formalize this aggregation as a social choice problem where each metric induces a preference ranking over models on each dataset, and a benchmark operator aggregates these votes across metrics. While prior work has focused on Arrow's impossibility result, we argue that the impossibility often originates from pathological examples and identify sufficient conditions under which these disappear, and meaningful multi-criteria benchmarking becomes possible. In particular, we deal with three restrictions on the combinations of rankings and prove that on single-peaked, group-separable and distance-restricted preferences, the benchmark operator allows for the construction of well-behaved rankings of the involved models. Empirically, we investigate several modern benchmark suites like HELM MMLU and verify which structural conditions are fulfilled on which benchmark problems.

Match-and-copy is a core retrieval primitive used at inference time by large language models to retrieve a matching token from the context then copy its successor. Yet, understanding how this behavior emerges on natural data is challenging because retrieval and memorization are entangled. To disentangle the two, we introduce Gaussian Match-and-Copy (GMC), a minimalist benchmark that isolates long-range retrieval through pure second-order correlation signals. Numerical investigations show that this task retains key qualitative aspects of how Transformers develop match-and-copy circuits in practice, and separates architectures by their retrieval capabilities. We also analyze the optimization dynamics in a simplified attention setting. Although many solutions are a priori possible under a regression objective, including ones that do not implement retrieval, we identify an implicit-bias regime in which gradient descent drives the parameters to diverge while their direction aligns with the max-margin separator, yielding hard match selection. We prove this max-margin alignment for GD trajectories that reach vanishing empirical loss under explicit technical conditions.

Despite the fact that experimental neural scaling laws have substantially guided empirical progress in large-scale machine learning, no existing theory can quantitatively predict the exponents of these important laws for any modern LLM trained on any natural language dataset. We provide the first such theory in the case of data-limited scaling laws. We isolate two key statistical properties of language that alone can predict neural scaling exponents: (i) the decay of pairwise token correlations with time separation between token pairs, and (ii) the decay of the next-token conditional entropy with the length of the conditioning context. We further derive a simple formula in terms of these statistics that predicts data-limited neural scaling exponents from first principles without any free parameters or synthetic data models. Our theory exhibits a remarkable match with experimentally measured neural scaling laws obtained from training GPT-2 and LLaMA style models from scratch on two qualitatively different benchmarks, TinyStories and WikiText.

It is a common practice in randomized clinical trials with the standard survival outcome to follow patients until a prespecified number of events have been observed, a type of trial known as the event-driven trial. The event-driven design ensures that the target power for a specified type 1 error rate is achieved to detect the target hazard ratio, regardless of the specification of other quantities. To understand the treatment effect for chronic conditions, the analysis of recurrent events has gained popularity in randomized controlled trials, particularly large-scale confirmatory trials. In the absence of within-subject correlation among multiple events, a similar event-driven design can be employed for recurrent event outcomes. On the other hand, in the presence of the within-subject correlation, one needs to model the correlation among recurrent events in evaluating power and setting the sample size. However, information useful in modeling the within-subject correlation is limited at the design stage. Failing to consider the correlation properly may lead to underpowered studies. We propose an event-driven type design for recurrent event outcomes. Our method ensures the target power for the target treatment effect, regardless of the specification of other quantities, by monitoring the robust variance under the marginal rates/means model in a blinded manner. We investigate the operating characteristics of the proposed monitoring procedure in simulation studies. The results of simulation studies showed that the proposed blinded monitoring procedure controlled the power well so that the test possessed the target power and did not lead to serious inflation of the type 1 error rate. Furthermore, we illustrate the proposed method using a real clinical trial dataset.

We prove weak convergence in a separable Hilbert space for estimators of high-dimensional regression coefficients, which yields asymptotic normality and enables direct use of standard asymptotic tools such as the continuous mapping theorem. The approach permits diverging sparsity with a growing number of small nonzero coefficients, while requiring that only finitely many have moderate magnitude. As applications, we develop a test for finitely many linear hypotheses and, via a Scheffé-type approach, simultaneous inference for infinitely many linear hypotheses, yielding both a global test and simultaneous confidence bands for the regression function. The limiting distributions are given by weighted sums of independent chi-squared variables, and plug-in critical values achieve asymptotically correct size.