Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Wednesday 7 October

Statistics Applications

Statistical Oceanography of Profiling Floats and Surface Drifters

The Global Ocean Observing System is key to understanding oceanic variability and climate change. The ocean is vast and often sparsely and irregularly sampled in time and space by various instruments and systems. Statistical models, especially spatio-temporal ones, are useful for enabling inferences, forecasts and decisions from sparse oceanic observations. This article focuses on the statistical treatment of two types of in situ observations in the Global Ocean Observing System: from profiling floats and surface drifters, focusing on the Argo and Global Drifter Programs. We describe the spatio-temporal models that have been developed in recent years for these data, to give a picture of the statistical challenges faced in modern physical oceanography. We will discuss in detail two types of reference frames when modeling such data: Eulerian and Lagrangian, the former of which is more appropriate for profiling floats and the latter for surface drifters.
Read the full abstract

Statistics Applications

Predicting Long-Term ADAS-Cog Change From Early Response After Repetitive Transcranial Magnetic Stimulation in Alzheimer Disease

Early cognitive response after repetitive transcranial magnetic stimulation (rTMS) may predict later cognitive change in Alzheimer disease, but the incremental value of domain-specific response is unclear. We analyzed data from 112 randomized participants; 101 entered a Week-5 landmark cohort and 91 had complete predictors and outcomes at 8, 16, and 24 weeks post-treatment. Nested predictor-set comparisons used ridge regression to evaluate baseline information, trial variables, early total-score response, and domain-specific response. Model-class comparisons included ridge, multi-task elastic net, restricted random forest, and longitudinal mixed effects. Validation used repeated treatment-site-stratified 5-fold cross-validation with five repetitions, nested hyperparameter tuning, and paired participant-level bootstrap comparisons with 10,000 resamples. Adding Week-3 and Week-5 ADAS-Cog total-score response improved prediction. The early total-response ridge achieved overall MAE 3.067, RMSE 3.885, and $R^2=0.384$. Replacing two total-response variables with 22 component-level changes worsened performance (MAE 3.363, RMSE 4.269, $R^2=0.257$). The paired MAE difference was -0.297 points (95% CI -0.485 to -0.115). Mixed effects was competitive (MAE 3.162), with an uncertain overall difference from total-response ridge. Sensitivity analyses using absolute future scores and excluding Week-3 response supported the main findings. In this sample, early aggregate ADAS-Cog response was more informative than a higher-dimensional domain-specific representation. The model is an internally validated early-response updating tool, not a pretreatment selection model or evidence of causal rTMS efficacy.
Read the full abstract

Statistics Applications

An unscented universal filter for joint input-state-parameter estimation of a partially measured system: an application to drive-by structural health monitoring

Drive-by structural health monitoring infers the condition of a bridge from the response of an instrumented vehicle, whose measurements are dominated by the road roughness and the vehicle dynamics. The bridge responses, contact forces and bridge parameters depend on one another through the vehicle-bridge interaction (VBI), so estimating them jointly is desirable. This paper proposes a novel filter in the Universal Filter family, the Unscented Universal Filter (UUF), for joint input-state-parameter estimation of nonlinear systems with unknown inputs. The unknown input is estimated by weighted least squares from the innovation, without any evolution model or statistics, and the system inversion imposes no rank condition on the feedforward matrix. The nonlinear propagation is linearised by the scaled unscented transform, and the linearisation residuals are propagated into the error covariances. For the acceleration-only configuration, auxiliary integrators form pseudo-measurements from the acceleration measurements themselves. The UUF is applied to the coupled VBI system, with the contact forces as the unknown input and the local stiffness of the bridge as the tracked parameter, from the responses of the vehicle body and at most a few bridge responses. In numerical case studies on a simply supported beam and the Old Ada Bridge, a steel truss bridge in Japan, the UUF is compared with classical and state-of-the-art joint estimation filters, all tuned by the same practical strategy. Across the damage severities, locations, road classes and noise levels examined, the UUF yields the lowest total error in most scenarios, and its error varies little with damage severity, location and road class. On the Old Ada Bridge, it identifies the stiffness of a damaged member from acceleration-only measurements, without the suspension parameters of the vehicle, and with a rank-deficient feedforward matrix.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Finite-Rank Logistic Gaussian Processes with Exact Likelihood for Conditional Density Estimation

Conditional density estimation describes how the entire distribution of a response changes with covariates, and in imaging studies also with location. Logistic Gaussian processes (LGP) give a flexible prior for such densities. However, the normalizing constant of an LGP has no closed form, so existing methods approximate or replace the likelihood. We propose the exact likelihood finite-rank LGP (ExFR-LGP), which writes the log density as the sum of two bivariate functions, one of the response and a location-varying linear index of the covariates, and one of the response and the location. Each function is assigned a finite-rank Gaussian process prior that is piecewise linear on a regular grid, enabling the normalizing constant, the conditional mean and the quantiles to enjoy closed form representations. Posterior samples are drawn from the exact posterior under this prior via a Gibbs sampler that updates the Gaussian components by elliptical slice sampling. When the true log density is the sum of two such bivariate functions, we show that the posterior contracts at the minimax rate of an $\alpha$-smooth bivariate density up to a logarithmic factor. Numerical experiments using synthetic data demonstrate the effectiveness of ExFR-LGP in conditional density estimation. Furthermore, application to fractional anisotropy responses along the corpus callosum in the Alzheimer's Disease Neuroimaging Initiative data yields covariate-adjusted percentile bands with uncertainty, elucidating how diagnosis changes the distribution along the tract.
Read the full abstract

All authors Jaehoan Kim, Indrajit Ghosh, Debdeep Pati, Dipankar Bandyopadhyay

Statistics Applications · Cross-listed from stat.ME

Sequential resetting procedures and false discovery rate

Data arrive sequentially, each associated with a null hypothesis. We develop testing procedures to locate intervals in which some null hypotheses fail, with false discovery rate (FDR) control. The new procedures are called sequential resetting procedures, and they are based on e-values and test supermartingales. We also develop a refined version of the procedures by dropping less informative data points before the block minimum of the test supermartingale in each rejection block. These procedures have explicit FDR bounds under two settings: a classic setting of independence and the more general setting of possible dependence across null data and non-null data. These FDR bounds are independent of the testing horizon, and the general one has anytime validity, but it has an extra logarithm factor compared with the standard FDR level. We present simulation studies and data experiments with applications of sequential resetting procedures to LLM watermark detection and financial backtesting.
Read the full abstract

Statistics Applications

A Sensitivity-based Framework for Calibrating Coupled Ordinary Differential Equations under Model Discrepancy

Coupled systems of ordinary differential equations (ODEs) are widely used to model complex physical and biological processes. In these applications, ODE model parameters must be calibrated to field data to enable prediction and parameter inference. In practice, the governing ODEs are an imperfect approximation to the true system, leading to non-negligible model discrepancy that must be incorporated to avoid biased parameter estimates. However, it is well known that highly flexible discrepancy models can confound discrepancy and simulator parameters, leading to poor identifiability. As a result, practitioners often must impose problem-specific prior constraints to adequately regularize the discrepancy, which can be challenging and cumbersome. In this work, we propose a novel calibration framework for coupled, multi-output ODE systems that enforces automatic, model-structural constraints on the discrepancy, improving identifiability without requiring application-specific discrepancy priors. Forward-model gradients are computed via sensitivity equations, solved jointly with the ODE system as a coupled initial value problem. Posterior inference is performed using an adaptive Metropolis-within-Gibbs sampler tailored to the resulting constrained posterior. We demonstrate the approach on three model systems: a mass-spring oscillator, a model of infectious disease spread, and a neuron firing model. We show that our method leads to improved parameter inference and predictive accuracy compared to standard calibration approaches.
Read the full abstract

Statistics Applications

Bayesian Optimization for Dose Finding with Two Agents: Participant Allocation and Final Selection

In two-agent dose-finding trials, the next cohort should help identify a combination for final selection. We studied a constrained knowledge-gradient (cKG) rule with one-cohort lookahead that updates independent Gaussian-process models of efficacy and continuous toxicity, reapplies a probability criterion for mean toxicity, and evaluates the resulting selection. We derived a deterministic calculation over a fixed set of dose combinations, holding fitted model parameters fixed during each hypothetical update. We compared cKG with constrained expected improvement (cEI) and two toxicity-only rules, targeted mean squared error (tMSE) and entropy, in four synthetic scenarios. In the primary obstructive sleep apnea (OSA)-derived scenario, averaged equally over strata and five probability cutoffs, cKG assigned fewer participants to combinations above the true mean-toxicity limit than tMSE (17.92% versus 27.08%), but selected such combinations more often at trial completion (18.80% versus 11.85%). Compared with cEI, cKG had higher mean simulated reduction in the 4%-desaturation apnea-hypopnea index (AHI4) at final selection (7.46 versus 6.72 events/hour), more above-limit final selections (18.80% versus 10.50%), and more above-limit assignments (17.92% versus 15.10%). Across scenarios, its efficacy advantage over cEI was smaller under stricter toxicity criteria. Continuous outcomes, uncalibrated toxicity limits, and a rule that still selects a combination when none meets the criterion limit clinical interpretation. Allocation and final-selection toxicity should be reported separately, alongside efficacy.
Read the full abstract

Tuesday 6 October

Statistics Applications · Cross-listed from stat.ME

Calibration-Free Robust Bayesian Inference for Discrete Unnormalized Models

Bayesian inference for discrete models with intractable normalizing constants is computationally challenging, particularly when the sample space is large or countably infinite. Although some likelihood-based Bayesian methods can circumvent the intractable normalizing constant, their uncertainty quantification may be poorly calibrated under model misspecification. Generalized Bayesian approaches can address this calibration issue, but typically require careful tuning of a learning rate. In this paper, we propose DFD-BETEL, a semiparametric Bayesian framework that uses the first-order optimality conditions of discrete Fisher divergence as moment conditions in Bayesian exponentially tilted empirical likelihood. The resulting posterior avoids evaluation of the normalizing constant and requires no learning-rate calibration. We establish consistency and asymptotic normality of the discrete Fisher divergence estimator and prove a Bernstein-von Mises theorem for the DFD-BETEL posterior, showing that its limiting covariance matches the sampling covariance of the estimator and yields asymptotically calibrated uncertainty quantification. Numerical studies show that DFD-BETEL remains competitive with likelihood-based Bayesian inference under correct specification and provides more reliable uncertainty quantification under model misspecification. An application to automobile-insurance claim data further illustrates the practical utility of DFD-BETEL.
Read the full abstract

Statistics Applications

Detecting Unseen Jailbreak Sources: A Multi-Source Conformal Detection Perspective

Jailbreak defenses for large language models are usually calibrated on a fixed set of known attack methods. However, new attack methods keep appearing after deployment, and existing detectors are observed to degrade under the new attacks (Piet et al., 2025). This paper studies how to detect, from a stream of attack prompts, that the prompts come from an attack source outside the known ones. We formulate it as a sequential test whose null hypothesis is that the stream comes from one of the known sources. Methodology-wise, we compute a conformal p-value against each known source, and reject the null only when the evidence against every known source is large. Using these source-wise p-values, we build Persist-3 and analyze its stationary false-alarm probability and detection power. The technical contribution of Persist-3 lies in a detection guarantee that uses only a finite window of recent observations. Our theoretical analysis reveals that its detection probability approaches one exponentially fast as the window grows, and thus a short window suffices to detect unseen sources that are well separated from all known sources. Empirically, we validate Persist-3 on top of two recent jailbreak defenses, MTK (Zhang et al., 2026) and Prompt Guard 2 (Chennabasappa et al., 2025), and observe a significant defense enhancement.
Read the full abstract

All authors Xunlei Qian, Yingqian Cui, Yuping Lin, Jun Wu, Yue Xing

Statistics Applications · Cross-listed from math.PR

When Lines Become Contagious: Analyzing the Spread of Infection in the M/M/1 Queue

In this paper, we construct and analyze a stochastic model for infectious disease spread in service systems. Our model considers a $M/M/1$ queue where a proportion, $p$, of the arriving population is infectious and the remaining population is susceptible. When an infectious person overlaps in the queue with a susceptible customer for more than $\ell$ units of time, then the susceptible person becomes infected. We provide both a personalized perspective of how an individual susceptible customer is infected and a community perspective of how one infectious customer can infect the others around them. From the personalized perspective, we derive the probability that a susceptible person becomes infected and the probability they are infected by at least $k$ different people. From the community perspective we compute the mean and higher moments of the number of susceptible customers that are infected by one infectious individual. Finally, we use stochastic simulation to verify our analytical results and show that our new formulas accurately describe the behavior our stochastic model of infection and to expand our model into the consideration of different arrival and service regimes.
Read the full abstract

Statistics Applications · Cross-listed from q-fin.RM

Learned Monotone Recurrent Features in Governed Credit Scoring: The Price of the Frame and the Necessity of Macro Conditioning

Regulated credit scoring requires scores monotone non-decreasing in every exposure input. Deployed pipelines – hand-crafted monotone aggregates feeding sign-constrained gradient boosting – already meet this by composition; the open question is what learned temporal aggregation is worth inside one. We answer on five production-scale credit datasets at matched admissibility (one priced baseline convention excepted), with a monotone recurrent architecture whose per-input guarantee we extend, with proofs, to vector-valued inputs and to exogenously macro-conditioned decay gates, severities, thresholds, and peak memory. Two findings result. First, a strictness ladder: the value of learned monotone features rises with governance-frame strictness – zero on unconstrained engineered panels, maximal in summaries-only frames – replicated across two datasets and an official temporal-stability metric, though unconditioned features degrade on externally adjudicated later weeks. Second, a conditioning-delivery asymmetry under regime shift. On a train-on-boom, test-on-crisis mortgage design, two public macroeconomic series hurt as input columns, yet conditioning the recurrence on them delivers the paper's only learned-block crisis-cohort uplifts. The confirmed effect: +0.006 to +0.013 AUC on an internally pre-registered Freddie Mac replication, at all five held-out seeds. The discovery estimate: +0.015 to +0.021 on Fannie Mae (three of five seeds post hoc), worth 10-27 basis points of defaulted balance at an 80% approval cutoff, and grows with early-prepaid loans excluded. A state-level test identifies the mechanism: between-cohort calibration transfer. A pandemic-band episode bounds scope: under forbearance-distorted labels the gain generalizes at a quarter to a third of crisis size on Fannie Mae, on Freddie Mac only against the capacity control.
Read the full abstract

Statistics Applications · Cross-listed from stat.CO

Scalable Regularized Vector Multiplicative Error Models for Positive-valued Financial Time Series

The logarithmic multiplicative error model (log-vMEM) has been useful in modeling and forecasting multivariate positive-valued financial time series. The number of parameters grow rapidly with the dimension of the system and the lag order, making estimation computationally demanding in high-dimensional settings. This paper describes regularized estimation via hierarchical lag structures (Nicholson et al., 2020) for log-vMEM models with multivariate gamma error distribution of Tsionas (2004). The parameter estimation is performed using a blockwise coordinate descent algorithm with a Gauss-Seidel-style update scheme (Wright, 2015). This enables an efficient computation strategy compared to traditional penalized maximum likelihood approaches. The competing models are juxtaposed against each other by combining three hierarchical lag structures (componentwise, elementwise, own-other) and four penalties(group-lasso, adaptive group-lasso, group-mcp, and group-scad). Extensive simulation runs have been performed to test the parameter recovery for both the unpenalized and the penalized models. We apply the proposed methods to model the joint dynamics of robust intraday realized volatility measures for Microsoft (NASDAQ: MSFT) for the competing models. The numerical integration step of the log-likelihood is identified to be the principal computational bottleneck. We address this issue by using GPU-accelerated quadrature integration thus improving computational scalability of the proposed models.
Read the full abstract

All authors Rohan Hemant Chhatre, Chiranjit Dutta, Nalini Ravishanker, Sumanta Basu

Statistics Applications · Cross-listed from stat.ME

Elastic kernel Ridge regression, with applications in phonetics

Predicting the shapes of entire curves requires accounting for nonlinear geometry and unknown alignments between curves. We develop elastic kernel ridge regression, a nonparametric method for planar curve responses with scalar, multivariate, or functional covariates. Using square-root velocity representations, we formulate penalized conditional Fréchet mean estimation in a shape space invariant to translation, rotation, scaling, and reparametrization. A vector-valued reproducing kernel Hilbert space provides a flexible nonlinear link through the spherical exponential map. We use an alternating algorithm to align each observed curve to its current fitted value, and updates the regression function. To facilitate this, we provide a new Euclidean quasi-Newton solver that exploits scale invariance of the reparametrization objective; this accelerates alignment while retaining accuracy in a numerical comparison. Simulations demonstrate the benefits of estimating alignment within the regression and respecting spherical geometry. Applied to vocal tract contours from a real-time magnetic resonance imaging recording, the method recovers missing frames and reconstructs shape trajectories from downsampled data. A speech inversion proof of concept on the same recording predicts tongue shapes from acoustic features, illustrating the method's potential. The method is implemented in the R package sphereg2.
Read the full abstract

All authors Beatrice Matteo, Almond Stoecker, Davide Pigoli, Shahin Tavakoli

Statistics Applications · Cross-listed from stat.ME

Bayesian heterogeneous copula mixtures with nonparametric margins: consistency, identifiability and tail asymmetry in physical fitness data

Finite mixtures of Clayton, Gumbel, Frank and Gaussian copulas can describe dependence that differs between the upper and lower tails. We study a two-stage Bayesian analysis in which ranks or kernel estimates replace the margins and the copula likelihood is evaluated at the resulting pseudo-observations. This pseudo-posterior is strongly consistent for the copula density whenever the log density admits a logarithmic boundary envelope. Both stages may use the same data, margins may be standardized within observed strata, and neither smoothness nor identifiability is required. The envelope holds for finite mixtures of Gaussian, Student, Clayton, Gumbel and Frank copulas in any fixed dimension; tail-dependence coefficients and conditional tail probabilities are therefore consistently estimated. We further prove that Clayton, Gumbel, Frank and Gaussian copulas are jointly finitely linearly independent, which makes mixture weights and components identifiable and consistently estimated. In simulations the pseudo-posterior matches multi-start maximum pseudo-likelihood in large samples. It is more stable in small samples and near independence (every component close to the independence copula), where tail coefficients are learned long before the weights and a marginal Metropolis sampler is up to twice as efficient as data augmentation. Two physical fitness datasets show mirror-image asymmetries. Among 8772 university students, sprint and jump performance are coupled mainly at the top. Among 5336 adults in a national health survey, low grip strength and low daily activity cluster together, increasingly with age, whereas high values do not. A Gaussian copula misses both patterns in held-out data.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

The Geometry of Existence and Uniqueness of Maximum Likelihood Estimation in Categorical Response Models

Nonexistence of the maximum likelihood estimate (MLE) under separation is treated as a solved problem for binary logistic regression and as a scattered collection of model-specific results everywhere else. We show that it is one phenomenon with one criterion. In a latent polyhedral categorical response model, every observed outcome corresponds to a polyhedral event in latent variables whose faces shift linearly with the parameter. Random-utility choice, cumulative-link, ranking, multivariate binary and ordinal, sequential, adjacent-category logit, and fixed-score stereotype models belong to this family. The likelihood sees each observed factor only through the columns of its threshold map, the structure vectors. A finite MLE exists if and only if the pooled structure vector set has overlap. Sufficiency requires only continuity, necessity requires only strictly threshold-increasing probabilities, and neither likelihood concavity, exchangeability, nor full design rank is needed. Positive strictly log-concave latent densities and threshold-identifiable polyhedra then yield uniqueness on the estimable span. A single linear program determines whether overlap holds, and convex cone geometry measures the dimension of separation. An application using cumulative-link models for willingness to share health data identifies observations and model terms associated with nonexistence and illustrates how the diagnostics inform model revision and sensitivity analysis.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

The Topp-Leone XLindley Distribution: Properties, Estimation and Applications to Lifetime Data

In this article, we use a family of distributions developed by Topp-Leone to create a novel lifetime two-parameter distribution. The Topp-Leone XLindley distribution is the name used for it. We study several types of statistical and mathematical characteristics of this distribution, such as reliability functions, moments, moment-generating function, quantile function, and the Renyi entropy. For the purpose of estimating parameters under the proposed distribution model, this simulation study is conducted to evaluate the performance of the maximum likelihood estimates for the parameters of the Topp-Leone XLindley distribution. A comprehensive simulation investigation is carried out to evaluate the performance of the proposed methods. Furthermore, a practical data set of bank customer waiting times has been investigated for the purpose of illustration.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

State-space representation of early-life child mortality dynamics: linking event history analysis and orbit-based models

Understanding early-life child mortality requires methods that capture both statistically significant risk factors and the structure of individual life trajectories. Event History Analysis (EHA) estimates associations between covariates and mortality risk but may obscure rare or high-dimensional configurations of risk. Orbit Theory (OT) represents individuals as exact trajectories in a combinatorial state space, preserving full multivariate structure. We analyse longitudinal data across 12 variables for 31,081 children from the Agincourt Health and Demographic Surveillance System (South Africa, 1998-2008) using both methods. We show three things EHA alone cannot. First, maternal migration is structurally pivotal in the transition dynamics leading to child death despite being statistically non-significant in EHA. Second, maternal refugee status, while statistically significant, generates no trajectory dynamics and is correctly excluded from any structural reduction. Third, a three-variable subsystem (child status, mother status, mother migration) recovers EHA's principal findings within a reduced state space of at most 48 states while exposing small high-risk clusters invisible to regression-based analysis. Maternal death is the dominant EHA predictor; maternal migration organises 87% of the observable sequential transitions into child-death states. These are not contradictory: maternal death rarely appears as a preceding state because its role is terminal rather than sequential. Early-life mortality can be read at two levels simultaneously - statistically, through effect estimates, and structurally, through observed transitions.
Read the full abstract

Statistics Applications · Cross-listed from stat.ML

Trustworthy Method Comparison with AI Judges: Estimation and Design under Order, Batch, and Aggregation Effects

Large language models (LLMs) are increasingly used as judges for automated AI evaluation. A common practice is to randomize prompt sequences and average the resulting scores, but its statistical validity remains unclear. We show that LLM evaluation mechanisms can be approximated by a class of Markov generalized linear mixed models (GLMMs), supported by out-of-sample predictions across three major commercial LLMs. Using a first-order Markov GLMM, we study leaderboard ranking and group comparison. For leaderboard ranking, randomize-and-average selection is consistent under a mild separation condition, and a Williams square design can improve efficiency when item qualities are close. For group comparison, naive averaging can yield inconsistent conclusions about differences in group-level quality because of the response model's nonlinearity. Empirical results further support the validity of the proposed model-based inference beyond the first-order theory, including settings with higher-order sequence memory. We illustrate the approach in an application where AI judges compare two graphical model estimation methods.
Read the full abstract

Monday 5 October

Statistics Applications · Cross-listed from cs.LG

Active Feature Acquisition for Cost-Efficient Temporal Prediction with Reduced Participant Burden

Accurate forecasting of pathological outcomes is a central problem in psychology. To do so, psychologists often collect intensive longitudinal data. However, in such studies, the desire to acquire a large number of variables for the sake of accurate prediction is often counteracted by the need to minimize participant burden. Acquiring more variables per occasion can yield better predictions, but having too many acquisitions increase the risk of non-response and attrition. Longitudinal Active Feature Acquisition (LAFA) is a principled approach to resolve this conundrum. Instead of requiring responses to every item at every acquisition occasion, LAFA produces a policy that seeks to optimally select dynamic subsets of items to be acquired at each timepoint while preserving our ability to forecast a specific outcome. However, existing LAFA methods are mostly based on Neural Networks (NN) that are difficult to interpret in practice. In this work, we introduce a tree distillation method for learning an interpretable policy from NN-based LAFA networks. We validated our method through both a simulation and an empirical EMA dataset on forecasting daily alcohol consumption. In both cases, we find that we can meaningfully reduce the number of items acquired at each occasion with minimal loss in accuracy. Networks (NN) that are difficult to interpret in practice. In this work, we introduce a tree distillation method for learning an interpretable policy from NN-based LAFA networks. We validated our method through both a simulation and an empirical EMA dataset on forecasting daily alcohol consumption. In both cases, we find that we can meaningfully reduce the number of items acquired at each occasion with minimal loss in accuracy.
Read the full abstract

All authors Yunni Qu, Bing Cai Kok, Whitney Ringwald, Grant King, Aidan Wright, Kathleen Gates, Junier Oliva

Statistics Applications

FireGen: Quantifying Wildfire Risk by Simulation

Wildfire regimes in Northern California exhibit strong spatio-temporal variability, heavy-tailed fire size distributions, and sensitivity to climatic conditions. We develop a hierarchical statistical framework to model wildfire occurrence, geometry, and burned area in Northern California from 1984-2023. Fire centroids are modeled as a spatio-temporal point process with covariate-driven intensity, extended by a multiplicative gamma-based stochastic shock to better capture the distribution of monthly fire counts. Conditional on fire location, burned area polygons are represented using a parametric ellipse model that separates scale, shape, and orientation, with ellipse parameters estimated from standardized fire perimeters. Total burned area is modeled using a heteroskedastic lognormal regression incorporating climatic and spatial covariates, including vapor pressure deficit anomalies. The framework enables Monte Carlo simulation of wildfire processes and provides a probabilistic tool for assessing wildfire risk and cumulative burned area under observed climate variability.
Read the full abstract

All authors Allyson Hineman, William Kleiber, Stephan Sain, Alexis Hoffman

Statistics Applications

DeepAJM: Deep Association Joint Model for Irregularly Sampled data

Joint Models simultaneously model longitudinal and survival outcomes, leveraging patterns in patients' longitudinal trajectory to improve the prediction of survival outcomes. The classical parametric joint models, however, rely on fixed parametric assumptions, making them susceptible to bias under model misspecification and smaller sample sizes. We propose a deep joint model, DeepAJM, that does not require any parametric assumptions, while retaining a partially interpretable, per-longitudinal-outcome association structure. The joint model uses an encoder-decoder (sequence-to-sequence) architecture to learn the latent structure in patients' time-varying covariate trajectories. The model links the longitudinal processes to the survival processes through a learned interpretable association structure, in which each longitudinal output from the decoder gets remodulated by baseline covariates before it contributes to the risk scores from the survival head of the architecture. The model was evaluated on three datasets ( a cardiovascular-disease EHR cohort, a primary biliary cirrhosis (PBC2) dataset, and a simulated dataset) against a classical parametric joint model, TransformerJM, DA-LSTM and a Cox-based survival-only model. All models were assessed using C-index, integrated brier score (IBS), time-dependent AUROC, and time-dependent AUPRC. Our model achieved the best discrimination in terms of the C-index, time-dependent AUROC, and AUPRC across all datasets.
Read the full abstract

Statistics Applications · Cross-listed from stat.ML

HyperNSDE: Personalized Neural SDEs for Joint Static-Longitudinal Clinical Data Generation

Synthetic patient data generation is a promising solution to the dual challenge of data scarcity and privacy constraints in healthcare machine learning. Realistic synthesis of patient-level clinical data requires jointly modeling heterogeneous static covariates, irregularly sampled longitudinal trajectories, and informative observation times - three tightly coupled components in practice yet rarely addressed together. We propose HyperNSDE, a continuous-time generative model that conditions a latent Neural SDE on static patient representations through a hypernetwork, allowing baseline characteristics to shape trajectory evolution beyond the initial condition without requiring a trajectory encoder, while stochastic latent dynamics capture realistic variability in generated paths. Observation times are modeled jointly through a latent-state-dependent intensity process, and training on irregular stochastic paths is stabilized via a deterministic-stochastic path decomposition with a non-adversarial signature-kernel objective. Experiments on simulated and real clinical datasets show improved observation-time fidelity and competitive performance, while matched-grid analyses reveal that forecasting and correlation metrics are affected by observation-grid regularity and trajectory smoothness.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

lifelines-hc: Higher Criticism testing for sparse non-proportional hazard departures in Python

The log-rank test is the standard tool for two-sample survival comparison and has good power against proportional-hazards alternatives, but it loses power against sparse hazard departures, in which the hazard difference is concentrated in a small number of time intervals whose locations along the follow-up are unknown a priori (Kipnis, Galili and Yakhini, Biometrika 2026). Weighted log-rank tests – Gehan-Wilcoxon, Tarone-Ware, Peto-Prentice, Fleming-Harrington – each impose a pre-specified temporal emphasis. Combination procedures such as MaxCombo and the Yang-Prentice short-term/long-term hazard-ratio model relax that choice, but still scan only a small dictionary of global temporal shapes and remain insensitive to sparse departures occurring outside them. We introduce lifelines-hc, a Python package extending the lifelines survival library with the HCHG test: Higher Criticism applied to per-interval hypergeometric p-values. HCHG applies to right-censored two-sample data and detects sparse hazard departures at unknown locations without committing to any temporal pattern. Across four clinical case studies – CheckMate 057 PFS (n=582), COMET-1 OS (n=1028), AZURE DFS (n=3359), and the Copenhagen Study Group for Liver Diseases cirrhosis trial (n=446, fully public individual patient data) – HCHG achieves p <= 0.014 in every dataset, while the log-rank test is non-significant throughout (p >= 0.26). MaxCombo and the Yang-Prentice adaptive log-rank test detect the delayed-benefit crossing in CheckMate 057 (p < 0.001) but are non-significant on the remaining three (p >= 0.24), where the departure is sparse or multi-window rather than a smooth short-term/long-term hazard-ratio pattern. lifelines-hc is freely available under the MIT license at https://github.com/alonkipnis/lifelines-hc and via pip install lifelines-hc.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

A Bayesian Multiscale Integrated Abundance Model for Estimating Latent Opioid Misuse Prevalence from Spatially Misaligned Data

Estimating small area prevalence of opioid misuse is critical for targeting public health interventions, yet direct measures are unavailable and related surveillance data are often reported on misaligned areal units. We propose a Bayesian Multiscale Integrated Abundance (MIA) model for estimating latent opioid misuse prevalence by jointly analyzing multiple indirect surveillance indicators observed on different geographic supports. The model extends existing integrated abundance models by representing all source geographies through a common set of atomic spatial units formed by the intersections of observed areal supports. Latent prevalence at the atomic level is modeled using a Fisher noncentral hypergeometric distribution, which preserves county-level prevalence totals while respecting local population constraints. To enable scalable inference, we develop a two-stage compositional Markov chain Monte Carlo algorithm that combines customized sampling strategies and parallel computing. Simulation studies show the proposed approach reduces bias and root mean squared error relative to common downscaling methods. We apply the model to Ohio data from 2010-2023, integrating state-level survey estimates, county-level counts of opioid overdose deaths and treatment admissions, and ZIP code tabulation area-level counts of emergency medical services naloxone administrations. Results reveal substantial within-county heterogeneity and identify localized areas of elevated opioid misuse prevalence that would be missed by county-level analyses.
Read the full abstract

All authors Staci A. Hepler, Brian N. White, Magdalena Cerda, William C. Miller, Lance A. Waller, David M. Kline

Statistics Applications

How Low Can You Go? Validating Administrative Union Records for Substate Estimates of US Union Membership

Unions are important economic and political actors at the local level, but the primary source on US union membership, the Current Population Survey (CPS), is unreliable below the state level. Researchers have therefore turned to union-reported membership in federal "LM" filings, geolocating unions and aggregating their reported membership to estimate union density for counties, commuting zones, and congressional districts. We ask whether this practice is sound. We develop and defend multilevel regression with poststratification (MrP) on the CPS as a benchmark, then validate LM-based estimates against it from the state level down. LM, CPS, and MrP are highly correlated at the state level, but this correlation is roughly halved at the congressional-district, commuting-zone, and county levels. The breakdown is driven in part by reporting "lumpiness," which produces improbable values, including union densities exceeding 100%. Missing public sector union members explains only a small part of this divergence. These issues are substantively consequential: in a standard commuting-zone "China shock" regression, swapping density measures changes the import-exposure coefficient enough to alter its magnitude, significance, and sign. We conclude that LM records remain a rich resource for studying unions as organizations, but aggregating LM-reported membership is not a reliable way to measure union density at fine geographic resolution.
Read the full abstract

Statistics Applications

Scale or speed? When do path signatures improve volatility-regime forecasts?

We study when path signatures, increasingly used as model-free features of return paths, improve out-of-sample forecasts of volatility regimes. In daily United States and developed-market equity returns from 2005 to 2025, machine-learning classifiers do not beat a heterogeneous autoregressive (HAR) regression of realized volatility, although signatures improve the classifiers' accuracy and probability forecasts in the United States. Adding raw signatures to the HAR regression itself improves United States forecasts of next-month volatility terciles, raising balanced accuracy by 2.7 percentage points and lowering the ranked probability score after multiple-testing correction, whereas adding conventional volatility features does not; outside the United States no gain survives correction. Controlled Heston experiments explain the pattern: signatures add modestly when regimes differ in variance level, which realized volatility already captures, and more when regimes share one stationary variance law but differ in mean-reversion speed. Better regime identification does not, however, deliver earlier regime-change signals, and no accuracy gain translates into a resolved Sharpe-ratio gain in a volatility-managed portfolio. Signatures thus add information about how volatility evolves, which realized-volatility predictors miss.
Read the full abstract

Statistics Applications

Evaluating LiDAR Data Sources, Predictor Resolution, and Spatial Random Effects in Bayesian Change-of-Support Models for Forest Inventory

Forest managers need timely stand-level information to support operational planning, particularly in mixed-species, structurally heterogeneous forests facing climate-related disturbance. Model-based estimation combines sparse field data with remotely sensed predictors to estimate growing stock volume (GSV) for small areas. While uncrewed aerial vehicle laser scanning (ULS) offers flexible, high-resolution LiDAR acquisition, its advantages over conventional airborne laser scanning (ALS) remain unclear. We compared publicly available ALS and newly acquired ULS data using Bayesian change-of-support models to estimate GSV in a mixed-species forest in north-eastern Germany. We evaluated distributional LiDAR metrics and spatial random effects. ULS consistently outperformed ALS, achieving cross-validated RMSPEs of 68.5 m^3/ha and 79.5 m3/ha, respectively-a 13.8 % reduction in prediction error. Distributional metrics improved ULS models more strongly, reducing RMSPE by up to 10.1 %; spatial effects provided only minor gains at substantially higher computational cost. ULS also produced lower uncertainty in latent stand-mean GSV estimates. The ULS advantage may reflect both finer-scale canopy information and closer temporal alignment with field measurements. Timely, information rich LiDAR may therefore be more valuable for stand-level GSV estimation than increasingly complex spatial models. Temporally matched ALS-ULS comparisons are needed to isolate platform effects.
Read the full abstract

All authors Arthur Spitta, Andrew O. Finley, Christoph Gollob, Ralf Kraßnitzer, Tim Ritter, Andreas Tockner, Arne Nothdurft

Statistics Applications · Cross-listed from eess.SP

A new MRI-based feature for quantifying the Diffuse Low-Grade Glioma brain infiltration and discriminating patterns of patients

Gliomas are the most common type of primary brain tumors. Diffuse low-grade gliomas (DLGGs) are slow-growing tumors that are often minimally symptomatic for a long period of time. They progress to a higher grade, resulting in the patient's death. Treatments include surgery, chemotherapy and radiation therapy to control tumor progression. Responses to treatments are highly variable among patients. Furthermore, the diffuse component of this tumor entity has been observed but is not yet well measured. Here, we propose a new variable to quantify glioma morphology, called ESVR (Extra Sphere Volume Ratio). We use then a machine learning approach to study the importance of different variables for discriminating patterns of patients: patient-specific variables, genetic alterations of tumor tissue, and some image-based variables like ESVR and the volume of the tumor, obtained by MRI analysis. Our machine learning approach shows that the patient's age and ESVR at diagnosis, as well as the pathology results seem to play an important role. Taking the highlighted variables into account could thus help to model the tumor behavior. In association with other classical biomarkers, our new ESVR feature could help the clinician to manage the treatment, as it carries significant information in terms of aggressivity of the DLGG.
Read the full abstract

All authors Jean-Marie Moureaux, Sophie Wantz-Mézières, Cyril Brzenczek, Marie Blonski, Luc Taillandier

Statistics Applications · Cross-listed from stat.ME

Calculating the Expected Value of Sample Information for Observational Studies affected by Confounding

Background: The Expected Value of Sample Information (EVSI) quantifies the value of collecting additional evidence to inform a health economic model. Existing EVSI methods typically assume idealized data collection mechanisms, most commonly randomized controlled trials (RCTs). However, in many realistic contexts, additional evidence comes from observational studies, which are subject to confounding or other bias. In this work, we define a methodology to calculate EVSI when additional data are observational and confounded. Methods: First, we define a simulation-based framework in which confounded observational data are generated through Inverse Target Trial Emulation (ITTE), a methodology that generates observational data with controlled levels of confounding starting from initial level data or prior information on population structure. Then, we apply inverse probability weighting (IPW) to obtain an adjusted summary statistic of the data targeting the corresponding randomized estimand. EVSI is finally computed using a regression based approach. Moreover, we propose a computationally efficient method to determine the sample size required to recover the EVSI achievable under an idealized randomized design. Results: We apply the methodology to two health economic models: a Normal Normal conjugate model and a chemotherapy treatment model combining a decision tree and Markov structure. We show that EVSI computed from observational data, even after adjustment, is lower than EVSI based on randomized data. The loss in EVSI increases with the level of induced confounding. Conclusions: This methodology extends EVSI when future evidence is expected to be observational and affected by confounding, enabling value-of-information analysis in more realistic and feasible data collection scenarios. It also provides a principled approach to sample size planning when observational evidence is anticipated.
Read the full abstract

Statistics Applications · Cross-listed from q-bio.QM

APOD: reasoning-guided agentic population ordinary differential equation discovery for pharmacological digital twins

Establishing ordinary differential equations (ODEs) describing population data is a fundamental part of mathematical modeling in pharmacology, crucial to developing digital twins. However, doing so from sparse, noisy data is a slow, expert-driven task. Existing automated methods either search a restricted model space or ignore population inter-individual variability. Here we introduce APOD (Agentic Population ODE Discovery), a language-model agent that iteratively reasons over biological knowledge and fit diagnostics in an open-ended search space to discover a population digital twin (PDT), i.e., a shared ODE system with between-subject variability. On synthetic pharmacokinetic and tumor-dynamics benchmarks, APOD recovered ground-truth structures in 94-100% of runs, 12-fold faster in median than an established library-based search. On real cohorts it converged to valid structures, and proposed a PDT of radioligand-therapy-induced platelet dynamics that predicts thrombocytopenia from first-cycle data and simulates alternative dosing schedules that lower the predicted risk of toxicity.
Read the full abstract

All authors Romain Ferrara, Martin Soucail, Victor Gertner, Adil Moussali, Joris Cocquebert, Sandrine Oziel-Taieb, Julien Nicolas, Florence Gattacceca, Mihaela van der Schaar, Sébastien Benzekry