Papers

We study the fundamental limits of convex-regularized estimation in high-dimensional linear regression with right-rotationally invariant design matrices. We show that the asymptotic risks of convex-penalized least squares estimators are lower bounded by the risk of an approximate message passing algorithm known as Bayes VAMP, and we further characterize when the lower bound is attainable. As a technical ingredient in the proof of our lower bound theorem, we characterize the asymptotic performance of the $\ell_2$-perturbed convex estimator for every fixed perturbation strength $\lambda>0$. This closes a gap in the literature, in which $\lambda$ was required to be sufficiently large or restrictions were imposed on the class of convex estimators. The benefit of our approach is that we can conduct a direct analysis of the spectrum's impact on the lower bound in the high-dimensional limit. In particular, we can show that the lower bound is monotone in an ordering on the limiting spectral distributions of the design matrix. These results isolate how the full singular-value distribution of the design—not merely the aspect ratio or average measurement strength—governs the limitations of convex regularization.

Learning disentangled representations is a key requirement for developing versatile, general-purpose, and sustainable models in multi-modal wearable computing. However, existing approaches do not operate as full-stack wearable processors, i.e., they do not simultaneously address task-specific classification performance, disentangled and interpretable representation learning, fusion, and generative modeling of highly heterogeneous multi-modal time series. To address this gap, we introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that efficiently learns multi-purpose representations in a unified and scalable manner from arbitrarily many modalities. OmniDecVAEs extend DecVAEs by learning modality-conditioned time-frequency latent subspaces through a multi-view self-supervised decomposition loss and a shared asymmetric autoencoder (AE) architecture. Results on a challenging omni-modal human activity recognition (HAR) setting with up to thirty modalities, demonstrate the ability of OmniDecVAEs to learn full-stack wearable representations. When compared to transformer-based and VAE-based methods, OmniDecVAEs full-stack disentangled representation properties lead to accuracy improvements of 1.01% and 6.75% in activity and identity recognition, respectively. Furthermore, OmniDecVAEs synthesize realistic omni-modal time-frequency data that manifest with enhanced reconstructions (mean absolute error improves by 76.84%) and distributional similarity between real and synthetic data (maximum mean discrepancy improves by 13.85%). Our results highlight OmniDecVAEs potential as a lightweight model suitable for intelligent edge wearables and clinical healthcare, unifying processing requirements and abilities in a single model, through its enhanced representational capacity, modality-invariant spatial complexity (4.1M parameters), and real-time latency.

This study investigated the behavior of Principal Component Analysis (PCA) when applied to datasets with extremely large numbers of observations. Although statistical theory suggests that sampling error diminishes and sample estimates converge toward their population values as sample size increases, relatively little empirical evidence exists regarding the behavior of PCA at scales measured in billions or trillions of observations. Three datasets were analyzed: a 10-billion observation random dataset, a 1-trillion observation random dataset, and a 10-billion observation engineered dataset designed to contain three latent factors. Results showed that the PCA solutions obtained from the 10BillionRandom and 1TrillionRandom datasets were nearly identical, indicating substantial stability of PCA at extremely large sample sizes. In contrast, the engineered dataset produced three dominant principal components that accounted for 99.996% of the total standardized variance and successfully recovered the intended latent-factor structure. These findings suggest that PCA solutions converge rapidly at very large sample sizes and suggest that PCA solutions may reach practical convergence well before sample sizes reach the trillions. These findings have implications for large-scale applications in fields such as remote sensing, digital mapping, environmental modeling, and other domains where datasets routinely contain millions or billions of observations.

Comparisons between AutoML systems at short time budgets – tens of seconds rather than hours – are common in tool READMEs and workshop papers, and they are easy to get wrong. We report a case study in which a simple AutoML engine, Orcetra, appeared to beat FLAML and AutoGluon on 513 OpenML datasets, winning 57.1% of them at a nominal 60-second budget and 78.4% of datasets against FLAML alone at 30 seconds. Both margins came from protocol defects that a results table cannot show. The search loop scored every candidate on the test split and reported the best, making the headline metric a maximum over dozens of noisy estimates while the baselines selected on training data and touched the test set once; and the budget was checked before launching a candidate but never enforced during one, so the system consumed a median of 120 s against a 60-second budget, 2.24x the wall-clock AutoGluon used. Re-running with selection moved to a validation split, the deadline enforced externally and every framework pinned to an equal share of the machine, Orcetra's win rate on the re-run subset falls from 59.4% to 34.3% and no pairwise difference against either competitor remains significant. Recording both estimands inside a single search lets us attribute the collapse: the selection rule accounts for 4.8 percentage points and unequal compute for most of the rest. The same traces give the selection bias as a function of budget, measured rather than assumed: it grows with $K$ but reaches only 0.27 accuracy points, about five times below the $\sigma\sqrt{2\ln K}$ bound a marginal-standard-error argument predicts, because candidates scored on shared test rows cancel most of the noise. We close with a checklist for short-budget comparisons. Code, per-dataset results and the scripts that regenerate every number and figure in the paper are released with it.

This paper investigates the asymptotic behavior of the out-of-sample prediction risk of the high-dimensional ridgeless least-squares estimator when the feature dimension $p$ and the sample size $n$ grow proportionally. We consider a generalized spiked population covariance model with multiple latent factors, where the number of spiked eigenvalues may remain finite or increase with $n$, and the spiked eigenvalues may be bounded or diverge at arbitrary rates. Beyond characterizing the impact of covariance spectra, we reveal a new mechanism underlying benign overfitting: the prediction behavior of ridgeless interpolation is fundamentally governed by the alignment between the regression coefficient $\boldsymbol\beta$ and the spiked eigenspaces of the population covariance matrix. In particular, we show that the signal energy distributed along latent spike directions determines whether interpolation leads to benign, tempered, or catastrophic overfitting. Our theoretical framework establishes sharp prediction risk limits under minimal moment conditions, requiring only finite fourth moments rather than Gaussianity. We characterize how the number, strength, and geometric structure of the spikes jointly influence the double-descent phenomenon. These results provide a unified understanding of when latent covariance structures facilitate or hinder generalization in overparameterized regression.

A growing literature argues that progress in human longevity is slowing, and that this slowdown may signal an approaching biological limit to lifespan. The argument rests on visible flattening of cumulative life expectancy records, evaluated through linear regressions or through comparisons of average gains across decades. These summaries, however, provide only weak evidence for slowing improvements, because they cannot distinguish a true from an apparent slowdown. An apparent slowdown may arise due to periods of low variance around unchanged mean annual gains. Applying a Bayesian change-point framework to annual gains, both at the best-practice frontier and separately for each record-holding country, we find no evidence that the expected annual gain has changed across nearly two centuries, but strong evidence that its variance has moved through distinct historical regimes, with progress most regular in recent decades. The recent flattening is consistent with a low-variance regime and provides little evidence for a decline in the underlying mean pace of improvement. Claims that the best-practice frontier signals a biological ceiling should therefore not rest on the recent bend in the cumulative record alone. The frontier is better read as an empirical benchmark, the highest country-specific survival achieved under observed historical conditions.

In numerous physical, chemical, and biological systems the dynamics can be reduced to the motion of state variables in a complex potential landscape. In case the manifold is known, the motion and response of the embedded object can be described deterministically up to stochastic effects usually associated with a noise. In contrast, if the manifold is unknown, the static and dynamic response of the state variable may be used as a spectroscopic tool to characterize the potential landscape. Inspired by a seminal work of L.\ Embon and co-workers, [Sci.\ Rep.\ 5, 7598 (2015)] we investigate the statistical properties of potential minima, in particular, their stability to thermal and quantum activation. For Gaussian random manifolds, we derive an algebraic expression to evaluate the statistical probability of the potential character (value, slope, curvature, ...). With this tool, we compute the expectation value for the rate of thermal and quantum activation and link these findings to the principal characteristics of the Gaussian potential, i.e., its Green's function. This link provides the opportunity to access information on the potential's Green's function by studying the activation behavior of an object in this manifold.

Hierarchical composite endpoints are commonly used in clinical trials when component out- comes differ in clinical importance. The win ratio compares patients across treatment groups according to a pre-specified order of clinical priority and has an intuitive interpretation. A strict hierarchical rule, however, uses a lower-priority endpoint only when all higher-priority endpoints are tied or non-informative. This may reduce power when the treatment effect is mainly ex- pressed through lower-priority outcomes. Recent threshold-based extensions relax this hierar- chy by allowing lower-priority outcomes to contribute when higher-priority outcomes are within pre-specified margins. We propose a continuous-weighted win ratio for hierarchical composite endpoints. The method replaces hard transitions between endpoint levels with smooth weights, so that a lower-priority endpoint can contribute gradually when the higher-priority endpoint is close to tied. The resulting estimator is a two-sample U-statistic. We derive its large-sample distribution using the Hoeffding projection and provide a consistent plug-in variance estimator. We also describe the role of the tuning parameter through local alternatives and local efficiency. The proposed class includes the strict win ratio and hard-threshold rules as special or limit- ing cases. Simulation studies show that relaxing the strict hierarchy can improve power when treatment effects are concentrated on lower-priority endpoints, but may reduce power when the highest-priority endpoint carries the main treatment signal.

The goal of this paper is to model node heterogeneity and link homophily for dynamic networks. The proposed framework brings new insights on how networks evolve over time. It also provides more sophisticated tools for the prediction of future networks with statistical guarantees. The new model accounts for the link homophily associated with both observed traits and latent traits. The joint modeling of node heterogeneity and both observed and latent homophily effects also poses the significant challenge in statistical inference, resulted from the large number of confounding parameters in the model. To overcome this, we propose a novel normalized squared loss, paving the way for efficient and stable estimation of parameters in a high-dimensional setting. We provide a rigorous theoretical analysis of the estimation method, and demonstrate its effectiveness through extensive simulations and the illustration with some real-world network data.

Understanding the relationship between home and away goal counts in football provides valuable insights into match-level dynamics. While the influence of home advantage is well-established, with historical records indicating roughly 50% of matches are won by home teams (versus about 30% by the away team), properly determining the joint goal distribution while accounting for key match factors remains under-explored. In this paper, we develop a Bayesian bivariate Conditional Poisson (BCP) regression model to explicitly capture the dependence between home and away goal counts, addressing a core limitation of traditional football scoring models that assume independence or only allow for positive correlation. The BCP regression is applied to the English Premier League (EPL) data spanning three seasons, incorporating stadium attendance and committed fouls by both teams as regressors. Inference is conducted within a Bayesian framework, enabling interpretable uncertainty quantification and model validation through posterior predictive checks. Our results reveal a negative correlation between home and away goal counts and highlight an asymmetry in how match attendance influences home versus away scoring.

Many economic and financial relationships may change gradually rather than abruptly. We study panel data models in which the coefficient vector is continuous and piecewise linear in calendar time, with a finite number of unknown kink dates at which its slope changes. We propose a penalised least squares estimator that applies adaptive weighted group penalties to the second differences of the coefficient path, and develop asymptotic theory showing that it recovers both the number and the locations of the kinks with probability approaching one. To our knowledge, this is the first panel framework to estimate an unknown number of common kink dates in a time-varying coefficient path under fixed effects. We establish that endpoint slopes converge at the usual cubic regime-length rate and interior slopes at rates determined by their own and adjacent regime lengths. We also develop a coefficient-by-coefficient extension allowing individual regressors to kink at different dates. Monte Carlo evidence supports the good finite sample properties, and we illustrate the method through an application in macro-finance, specifically the relationship between debt and growth.

We study Lambda-quantiles, a generalisation of classical quantiles in which the constant probability level $\lambda \in [0,1]$ is replaced by a functional parameter $\Lambda \colon \mathbb{R} \to [0,1]$. We consider the general case of non-monotone $\Lambda$, which arises naturally if closure properties of the class of corresponding Lambda-quantiles with respect to inf-aggregation or with respect to mixtures are required. As preliminary results, we characterise finiteness, constancy, and what we call the attainment property known from classical quantiles. We then consider the problem of reconstructing $\Lambda$ from the values of $\Lambda$-quantiles on a suitable family of simple distributions, showing its identifiability under mild assumptions. Next, we substantially refine several results obtained in the literature on weak upper and lower semicontinuity and on the property of convexity of the level sets with respect to mixtures, obtaining in both cases almost complete characterisations without any monotonicity assumption. We then move to the case in which $\Lambda$ has bounded variation, which enables us to prove a mixture representation result: any such $\Lambda$-quantile can be rewritten as a Lambda-quantile with an increasing functional parameter, evaluated at a mixture of the original distribution with a fixed reference distribution at a fixed weight, thus reducing the complexity of the parameter from bounded variation to monotone. Finally, we introduce and study the notion of the ordinal covariance group of a risk measure, showing that in the case of a $\Lambda$-quantile it coincides with the compositional invariance group of $\Lambda$ and with a certain group of measure-preserving transformations of the signed measure associated with $\Lambda$.

We consider the optimization of the Optimized Certainty Equivalent (OCE) risk, with applications including portfolio optimization in finance, and uncertainty quantification, classification, and regression in machine learning. Our contributions cover popular special cases of OCE, such as entropic risk, mean-variance risk, and smooth variants of Conditional Value-at-Risk. Our treatment sets out the conditions that facilitate the extension of OCE to unbounded r.v.s.. We provide a useful characterization of OCE that links OCE to utility-based shortfall risk (UBSR). Our characterization enables us to form an OCE estimator from the classic sample-average approximation (SAA) of UBSR. We derive mean-squared error (MSE) bounds for our proposed OCE estimator. For OCE optimization, we first derive an expression for the OCE gradient using the characterization linking OCE to UBSR. This expression serves as the basis for a gradient estimator for the OCE. We derive non-asymptotic bounds on the MSE for the proposed OCE gradient estimator. We incorporate the aforementioned gradient estimator into a stochastic gradient (SG) algorithm to optimize OCE and quantify its convergence rate using non-asymptotic bounds that we derive. Finally, we present three experiments that use our OCE optimization algorithm to solve portfolio optimization and uncertainty quantification problems.

This paper introduces Mixtures of Geodesic Factor Analyzers (MGFA) on Riemannian homogeneous spaces. MGFA uses a geodesic factor model within each mixture component, providing greater expressiveness than mixtures of Riemannian radial distributions and enabling clustering of manifold-valued data with anisotropic subpopulations. We establish root-$n$ consistency for the MGFA maximum likelihood estimator (MLE), thereby filling a theoretical gap for mixtures of Riemannian radial distributions as a special case. We also propose an iterative estimation algorithm and implement it on spheres, shape spaces, and hyperbolic spaces. Numerical experiments show that MGFA substantially outperforms competing methods in well-specified regimes while remaining robust under model misspecification. Finally, case studies on corpus callosum and left hippocampus shape datasets demonstrate MGFA's effectiveness for both 2D contour and 3D shape analysis.

Item response models usually assume a normal trait distribution, yet little is known about how often fitted distributions in real studies differ substantially from normality or which reported results are most affected. We analyzed 504 itemresponse data sets from 273 studies in the Item Response Warehouse, fitting each with the normal assumption and with a flexible distribution estimated from the responses, and we compared reliability, item estimates, predicted test responses, and person scores between the two calibrations. In more than half of the data sets the two fitted distributions differed by at least 10 percentage points of cumulative probability at some point on the trait scale, with a median maximum difference of about 11 points; about one-third reached 15 points, and nearly one-fifth reached 20 points. The estimated shapes included skewness, heavy tails, flat regions, and occasional multimodality. Differences were larger in several attitudinal, affective, and behavioral domains, although these contrasts weakened when data sets were compared within item-model families. Reliability usually changed little when item estimates were held fixed, whereas refitting the full model produced larger changes in some data sets, and item estimates, predicted test responses, and person scores did not change in parallel. Alternative flexible methods generally identified the same data sets as most unusual but disagreed somewhat about magnitude. Applied analyses should state the distributional assumption, examine a flexible alternative, and report sensitivity separately for each result used in interpretation or decision making.

Nearly every quantitative study in psychology reports a reliability coefficient, so the field knows a great deal about the reliability that authors choose to publish. It knows much less about the reliability of the data psychology actually produces, because published coefficients pass through decisions about what to compute and what to report. We therefore measure reliability directly, applying the same estimators under the same rules to 889 datasets from the Item Response Warehouse, a public collection of item-response data that spans cognitive tests, clinical screeners, personality inventories, and attitude scales. Three findings emerge. Low reliability is common: even under a lenient definition of reliability, 30% of datasets fall below the conventional .80 threshold. The variation across datasets is real rather than statistical, since estimation noise accounts for only about one percent of it. Finally, the answer depends on the definition itself: under a strict definition the share below .80 rises to 52%, a difference large enough to change what one concludes about the field. We conclude that a reliability report should say which definition it uses, attach a measure of uncertainty, and give a strict coefficient alongside a lenient one.

Covariate adjustment improves the efficiency of treatment-effect analyses in randomized clinical trials, provided the adjustment targets the correct quantity. For time-to-event endpoints, two marginal targets are of primary interest: the log-rank test for the presence of a treatment effect and the marginal hazard ratio for its magnitude. Existing covariate adjustment approaches reach these targets by different ways. Augmentation adjusts the log-rank score by regressing derived outcomes on the baseline covariates within each arm. Weighting instead reweights the two arms to balance the covariates before the survival comparison is formed: inverse probability weighting does so through a fitted propensity model, while calibration weighting solves directly for weights that match covariate means. In this manuscript, we first develop balancing weighting for time-to-event endpoints, covering both calibration weights (stable balancing weights and entropy balancing) and propensity score weights, and prove that any balancing-regular weighting is first-order equivalent to the augmented log-rank score and to the root of the marginal Cox score. All three routes therefore deliver the same estimator to first order, and calibration reaches it without fitting any model. The weighted procedures thereby inherit the validity and guaranteed efficiency gain of the augmentation approach. In addition, we show that the efficiency gain grows with the prognostic strength of the adjustment covariates, while the practical caveat lies in variance estimation, for which we give recommendations to guard against finite-sample Type I error inflation. We further confirm our results through simulation studies and an analysis of the REWIND cardiovascular trial.

Polypharmacy, commonly defined as the concurrent use of multiple medications, is increasingly prevalent in aging populations and is associated with adverse health outcomes. Motivated by longitudinal studies of aging people with HIV (PWH), we study how medication use evolves over time and characterizes multimorbidity patterns. Longitudinal medication data present substantial methodological challenges, including high dimensionality, sparsity, irregular observation times, and structured pharmacologic relationships among medications. Existing approaches are typically task-specific and lack a unified framework for learning general-purpose representations of medication trajectories. We propose LOPEL (LOngitudinal Polypharmacy Embedding Learning), a two-stage self-supervised framework for learning low-dimensional representations of longitudinal medication data. In the first stage, visit-level embeddings are learned using a Gaussian process model that incorporates pharmacologic similarity through the Anatomical Therapeutic Chemical hierarchy. In the second stage, subject-level embeddings are constructed by modeling trajectories over time and defining similarity through a Wasserstein-based representation capturing temporal dynamics and uncertainty. Simulation studies demonstrate that LOPEL accurately recovers latent structure under realistic conditions with high-dimensional sparsity and irregular sampling. In an application to aging cohorts of PWH, LOPEL identifies clinically meaningful subgroups that differ in the timing, composition, and progression of medication use, highlighting heterogeneity relevant for risk stratification and clinical management.

Medical imaging provides rich information for predicting Alzheimer's disease progression, but existing imaging-based methods typically focus on a single survival endpoint and treat transition times as exactly observed or right-censored. Motivated by the Alzheimer's Disease Neuroimaging Initiative (ADNI), we develop a predictive framework that represents disease progression as an intermittently observed multistate process with interval-censored transition times and predicts future progression from any current disease state. We incorporate imaging biomarkers as functional covariates in a semiparametric proportional intensity model and combine functional principal component analysis with nonparametric maximum pseudo likelihood estimation. We further develop a profile score test for assessing the overall association between the imaging covariate and the multistate process. We establish the asymptotic properties of the proposed estimators and test statistic, and simulation studies demonstrate satisfactory finite-sample performance. In the ADNI application, baseline lateral ventricular morphology is strongly associated with Alzheimer's disease progression. The proposed functional multistate model also achieves the best overall predictive performance among the competing methods.

Equitable scoring functions have a long history in meteorological forecast verification and have recently gained renewed prominence through the use of the Stable Equitable Error in Probability Space (SEEPS) score for evaluating precipitation forecasts from numerical and machine-learning weather prediction systems. This paper provides a systematic analysis of equitable scoring functions through the optimal forecasts they induce. We introduce the generalized diagonal score, which defines a broad class of equitable scoring functions that includes, up to equivalence, the SEEPS, Gerrity, Peirce, Barnston and diagonal scores. Within this framework, we show that optimal single-valued and categorical forecasts are characterized by crossing points between the predictive and climatological cumulative distribution functions. Moreover, we show that the generalized diagonal score, when evaluating predictive distributions, is proper but not strictly proper, and is insensitive to substantial forecast misspecification away from crossing points. The framework also yields scoring rules that are both proper and equitable for probabilistic forecasts with categorical outcomes. For single-valued and categorical forecasts, the crossing-point characterization shows that optimal forecasts can vary across climatologies even when the underlying predictive distribution is unchanged. Consequently, identical predictive distributions may lead to substantially different optimal forecasts under equitable scores, with important implications for assessing the suitability of such scores for any given application.

Latent space models represent network nodes as points in a geometric space, with connection probabilities determined by distances between latent positions under a fixed metric, typically Euclidean, spherical, or hyperbolic. We generalize the classical formulation by introducing nodal multiplicative effects motivated by a local deformation of the latent metric. This modification approximates a conformal deformation of the metric tensor while preserving the logistic predictor and the geometric interpretability of the model, thereby capturing additional structural heterogeneity without altering the global reference geometry. We study the generative behavior of the proposed model through simulation experiments and develop an optimization-based inference scheme derived from a hierarchical Bayesian formulation with parameter regularization. Applications to eight real networks show that the proposed approach increases the generative flexibility of classical latent space models and more accurately reproduces several topological properties observed in real networks.

We discuss Bayesian inference on a low-dimensional targeted parameter in the presence of possibly highly complex nuisance components within the semi-parametric inference framework using an estimating function approach. We obtain a posterior distribution using non-parametric Bayesian methods through the Dirichlet process and the Bayesian bootstrap. We relax the commonly deployed notion of stochastic equicontinuity and develop a framework leading to posterior inference with good frequentist properties, specifically we demonstrate that the posterior distribution is asymptotically Normal and concentrates at the true value of the parameter. We emphasize the specific assumptions that are required to obtain these results, and how relaxing any of them alters the conclusions. We verify the analytical results in simulation.

Statistical inference on individual activity networks has been a historically difficult task due to the lack of available data at the appropriate granularity and the complexity of modeling individual mobility patterns. The recent availability of GPS data from individual devices, combined with highly detailed demographic information, suggests that one of these challenges can now be addressed. We introduce a new model which we call the Symmetric Matrix-Variate Normal Mixture Model (STRUCTURED) to estimate how demographic traits influence changes in human activity networks, using sociomatrices that capture the probabilistic spatial overlap between individuals over time. We exploit the commutativity constraint inherent in the symmetric matrix-variate normal distribution to parameterize the column precision matrix as a polynomial of the row precision matrix, reducing the effective parameter space by an order of magnitude. We develop two variants of STRUCTURED: STRUCTURED-FP, which estimates the full polynomial, and STRUCTURED-RJ, which uses reversible-jump MCMC to select a reduced-order parameterization. Simulation studies demonstrate that STRUCTURED-RJ outperforms existing methods in sparse-data regimes, whereas STRUCTURED-FP is preferred when sample sizes are large. We apply the model to GPS-derived sociomatrices of 293 individuals in King County, WA, finding that local crime environments and youth employment density are the dominant demographic factors explaining variation in weekly activity overlap patterns.

Accurate solar irradiance forecasting is essential for the reliable integration of photovoltaic power into modern electricity grids. All-sky imagers (ASI) provide high-resolution observations of clouds, making them well suited for intra-hour forecasting. Recent deep learning approaches have substantially improved forecast accuracy but are often limited by deterministic predictions and a reduced capability to anticipate ramp events. This work proposes FarSky, a generative forecasting framework that leverages latent-space coupling to learn task-aware representations of sky images. A multi-task autoencoder first learns a shared latent representation for image reconstruction and irradiance estimation. A latent diffusion model then generates future latent states conditioned on recent observations, from which irradiance forecasts are directly decoded. Probabilistic forecasts are inherently obtained through stochastic sampling. The framework is developed using a multi-year ASI dataset acquired at the Plataforma Solar de Almería, Spain, and evaluated on two independent test datasets against persistence, state-of-the-art end-to-end, and generative forecasting approaches. FarSky achieves the best overall deterministic and probabilistic forecasting performance, improving forecast skill by up to 11 percentage points. Furthermore, it substantially improves ramp event detection over existing methods, achieving F1-scores above 60%. These results demonstrate the potential of combining generative models with task-aware latent-space coupling for solar forecasting.

High-throughput screening (HTS) assays are central to early-stage drug discovery but are often limited by extreme data sparsity, as primary screens typically use only a single replicate per test substance. This sparsity makes conventional machine-learning performance metrics, such as sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC), difficult to estimate empirically because they require adequately sized labeled samples. Here, we introduce a model-based framework that derives these classification metrics from the strictly standardized mean difference (SSMD), a well-established HTS effect-size parameter. Under a Gaussian equal-variance assumption, we derive closed-form relationships linking SSMD to Youden-optimal sensitivity and specificity, and sensitivity at a preset specificity, yielding explicit estimators and exact confidence intervals from the noncentral t-distribution, even under single-replicate designs. Unlike classical statistical power, which approaches 1 as sample size grows regardless of how small the true non-zero difference between group means is, the SSMD-derived sensitivity converges to a finite population value that reflects the true degree of separation between two groups, making it a more meaningful and stable performance measure for hit selection. We demonstrate the utility of this framework in a hepatitis C virus primary siRNA screen comprising approximately 22,000 single-replicate measurements, showing that SSMD, AUROC, and sensitivity-based thresholds yield equivalent and interpretable hit sets. This work bridges classical HTS statistics and machine-learning evaluation theory, providing a statistically principled, reproducible way to estimate classification performance in ultra-low-replication screening workflows.

Studying consequences following baseline exposures has become increasingly important for advancing comparative effectiveness research using real-world data. This case study evaluates the impact of hospital-acquired conditions (HAC) during hospitalization for hip fracture on post-discharge recovery trajectories among older adults living with Alzheimer Disease and Related Dementia, a population particularly vulnerable to high post-hospital mortality. To appropriately account for truncation of recovery trajectory due to death and to explore heterogeneity in effect modification by patient demographics, we introduce a novel pseudo data-based robust (PD-Robust) analysis strategy, accompanied by an R package and detailed usage guidance to inform real data analysis. Grounded in an interpretable estimand via principal stratification under principal ignorability and a structural working model, PD-Robust accommodates truncation by death, provides model diagnosis and robustness check against assumption violation, and facilitates the characterization of patient profiles among the principal stratum. Applied to Medicare claims data, where better recovery is defined as more days at home (DAH) over six months post-discharge, PD-Robust reveals heterogeneity in HAC effects, with males under the age of 85 years as a high-risk subgroup experiencing up to 23 fewer DAH, comparing HAC to no HAC. This exceeds the 8-day threshold regarded as clinically meaningful difference in DAH due to any exposure. Moreover, simulation studies further demonstrate that PD-Robust achieves low estimation bias and accurate statistical inference, supporting its utility in real-world data applications.

Athletic careers yield sparse, irregular longitudinal series: few seasons per athlete, incomplete paths, and selection into continued play. Scientific interest often centres on proximity to peak attainable performance-a ceiling-rather than on the mean trajectory, and on how that proximity co-varies with the rate of decline. Standard tools address only pieces of this problem. Linear mixed models describe average ageing; functional principal components describe dominant modes of variation; shape-invariant models register curves about a mean template; growth charts estimate population centiles but stop short of individual latent trajectories. We develop RACE (Relative Aging Curves via Envelopes), a two-stage framework that estimates a population performance envelope as an age-conditional high centile and then models each athlete as a low-dimensional geometric transformation of that envelope. STAR (Shape Translation And Rotation) is the Stage 2 mixed model, with parameters for level, timing, and tempo. Envelope geometry determines which of these parameters are identifiable when careers are short: near-linear envelopes identify level and tempo only, whereas curved envelopes identify all three. Embedding STAR in a nonlinear mixed-effects hierarchy makes the level-tempo association a parameter of the random-effect covariance rather than a post-hoc correlation of separate fits. Simulations ask whether that association is recoverable under sparsity, how geometry governs identifiability, and how sensitive results are to envelope misspecification. Applied to Major League Baseball Statcast sprint speed and bolt rate, the analysis demonstrates how the proposed embedding separates athletic level and ageing tempo in Functional Ageing Space, and shows why timing is identifiable for bolt rate but not for sprint speed.

Classical likelihood-ratio tests and $\Delta$AIC exacerbate the statistical significance crisis by scaling with sample size, often flagging negligible improvements as highly significant. While causal estimands like the average treatment effect (ATE) quantify practical magnitude, their reliance on the expectation operator ties them to the data's original coordinate scale. Furthermore, existing pseudo-$R^2$ metrics are inadequate: variance-based measures ignore higher-order distributional changes, and current formulations lack invariance to monotone transformations. We resolve these limitations by introducing Entropic Variance (EV) as a rigorous, scale-independent generalization of error variance in ordinary least squares. We define the population EV-based parameter, $\rho^2_V$, which projects unbounded cross-entropy onto a standardized $[0,1]$ scale, and establish that the EV-based $F_\text{V}$ statistic asymptotically follows an $F$-distribution. Building on these distributional properties, we propose two estimators: the empirical population $R^2_{\text{SV}}$ and the out-of-sample predictive $R^2_{\text{SVP}}$. Both are derived by exponentiating per-observation cross-entropy and incorporate a degrees-of-freedom correction for training optimism. Leveraging the $F_\text{V}$-distribution, we derive refined $p$-values and confidence intervals for $\rho^2_V$ without requiring intractable Fisher information matrices. Simulation studies and a Parkinson's disease microbiome application demonstrate the superiority of variable selection via these EV-$R^2$ metrics. Notably, evaluating the $R^2_{\text{SVP}}$ of a LASSO path via data-splitting reduced false discovery rates from 80% to 6% in simulations while fully preserving signal recall.

The Mahalanobis distance is a fundamental covariance-adapted metric for multivariate data and plays a central role in recovering latent geometry from nonlinear observations. We extend this principle from vector-valued data to probability measures by introducing a Wasserstein Mahalanobis distance. Our construction replaces Euclidean displacement vectors with optimal transport displacement fields and local covariance matrices with covariance operators defined on Wasserstein tangent spaces. We show that this construction inherits the geometry-recovery property underlying nonlinear independent component analysis. In particular, for Gaussian measures with common covariance transformed by a smooth nonlinear pushforward, the proposed Wasserstein Mahalanobis distance approximates the classical Mahalanobis distance between the transformed latent means. The correspondence is exact for affine transformations and holds up to controlled higher-order error terms for general smooth transformations. These results establish a distribution-valued analog of classical Mahalanobis geometry and provide theoretical support for covariance-adapted learning directly in Wasserstein space. Numerical experiments confirm the theoretical predictions and demonstrate accurate recovery of latent geometric structure.

Hidden Markov models (HMMs) are widely used probabilistic models for discrete sequential data but can be limited when hidden dynamics are complex. Hidden quantum Markov models (HQMMs) generalize HMMs by replacing probability vectors with density matrices and stochastic transitions with quantum operations, enabling richer latent representations. However, existing HQMM learning methods have not consistently outperformed Expectation–Maximization (EM)-trained HMMs on data not generated by quantum processes, limiting their practical applicability. We introduce NS-RIS, Newton–Schulz Retraction-based Inference on the Stiefel manifold, a scalable algorithm for learning trace-preserving HQMMs. NS-RIS uses Newton–Schulz orthogonalization to compute a polar-factor search direction while preserving Stiefel-manifold feasibility, avoiding costly matrix decompositions. We further establish a finite-time stationarity guarantee under standard assumptions on smoothness, stochastic gradients, and finite Newton–Schulz accuracy. Empirically, NS-RIS provides the first benchmark evidence that an HQMM can significantly outperform an EM-trained HMM on data not generated by a quantum model. On synthetic HMM-generated benchmarks, NS-RIS outperforms both EM and the state-of-the-art HQMM method COSM, improving the evaluation metric by an average of 38.5% and by up to 50.6%. On a synthetic HQMM benchmark, it improves the test metric over COSM by 18.9% while reducing runtime by 12.0%. On the real-world Splice classification benchmark, NS-RIS also surpasses both EM and COSM in higher-dimensional latent regimes, reducing mean classification error by 17.9% for latent dimension 6 and 14.9% for latent dimension 8 relative to COSM. These results move HQMMs beyond a theoretical generalization of HMMs and establish them as practical and expressive models for scientific sequence data.