Statistics Applications · Cross-listed from stat.ME
Estimating Climate Sensitivity Using Bayesian Model Averaging for CMIP Models
Jitong Jiang, Skylar Shi and Adrian E. Raftery
The Transient
Climate Response to cumulative CO2 Emissions (TCRE) is a key metric for linking greenhouse gas emissions to global temperature change and informing
climate policy. However, extant estimates of the TCRE often depend on subjective model selection or assumed sensitivity ranges, with limited validation against observed data. We develop a fully statistical data-driven approach using a
Bayesian Model Averaging (BMA) approach to estimate the TCRE. This uses 37
climate models from the Coupled Model Intercomparison Project phase 6 (CMIP6), weighted according to their consistency with observed temperature data. Compared to the Intergovernmental Panel on
Climate Change (IPCC)'s Sixth Assessment Report (AR6) TCRE estimate, our BMA approach yields a very likely range (90% interval) that overlaps substantially with that from the AR6, but with a higher mean and a lower standard deviation. The resulting projections of global temperature change to 2100 show somewhat higher warming and less uncertainty than some current methods. The use of statistical modeling methods makes it easier to validate the approach and to partition the uncertainty according to its sources. Out-of-sample predictive validation shows the method to be well calibrated. Variance decomposition shows model parameter uncertainty to be a main source of projection variance.
Read the full abstract ▾
Statistics Applications
Modelling temporal dynamics of suicidal ideation and behaviour across pre- to early adolescence using a Markov framework
Sieun Lee, Ben Cardoen, Marianne Etherson et al. (+10 more)
Understanding the dynamics of suicidal ideation and behaviour in youth and the factors associated with transitions from thoughts to behaviours is critical for early identification, monitoring, and prevention. Using longitudinal self-report data from the Adolescent
Brain Cognitive Development (ABCD) Study (n = 11,864) spanning ages 9 to 13 years, we developed a time-inhomogeneous discrete-time Markov chain framework to model transitions across eight states defined by suicidal ideation and behaviour and the co-reported presence or absence of non-suicidal self-injury (NSSI). The framework enables inference of year-to-year and multi-year transition probabilities, quantification of transition stability and uncertainty, and statistical comparison of transition likelihoods. We identified structured but developmentally changing transition patterns, including a generally high probability of recovery to a no-report state following reports of suicidal ideation or behaviour. Co-reported lifetime NSSI marked a distinct trajectory profile characterized by both greater risk and lower predictability: children with NSSI were more likely to transition to or persist in suicidal behaviour, were less likely to recover to a no-report state, and exhibited greater uncertainty in their transition likelihoods compared to those reporting suicidal ideation and/or behaviour alone. These findings suggest that NSSI marks not only elevated risk but also greater trajectory instability during pre- to early adolescence. By estimating probabilities of escalation, remission, persistence, and fluctuation in longitudinal cohort data, the framework provides a systematic, interpretable, and scalable approach to characterising suicidal trajectory dynamics in large, sparse mental health
datasets, with potential to inform future research on monitoring, early prevention, and time-varying risk in youth mental health.
Read the full abstract ▾
All authors Sieun Lee, Ben Cardoen, Marianne Etherson, Nitish Jawahar, Ellen Townsend, Kapil Sayal, Peter Fonagy, Aja Murray, Joanna Lockwood, Ayan Mahamud, Chris Hollis, Rory O'Connor, Dorothee Auer
Statistics Applications · Cross-listed from stat.ME
Testing for subgroup treatment effect consistency in the Cox model
Lukas Koletzko, Holger Dette, Björn Bornkamp et al. (+2 more)
An overall treatment effect in a clinical trial may inadequately represent particular patient subgroups, creating uncertainty about whether a population-level efficacy conclusion can legitimately be transferred to them. Conventional interaction tests only investigate whether subgroup-specific treatment effects are exactly equal or not, but cannot detect whether the differences are small enough to be clinically negligible. We develop a formal framework for assessing subgroup treatment effect consistency for time-to-event outcomes within the Cox proportional hazards model. Consistency is formulated as an equivalence problem based on the (weighted) treatment-by-subgroup interaction coefficient. We consider detecting consistency between two complementary subgroups and consistency of subgroup-specific treatment effects with the overall treatment effect. For each setting, we develop a conventional two one-sided tests procedure (TOST) and a new test motivated by optimal equivalence testing for normally distributed parameters. We prove the asymptotic validity of all procedures and show empirically that the new tests are more powerful than their TOST counterparts. Finally, we apply the new methodology to a case study motivated by the CANTOS cardiovascular outcomes trial.
Read the full abstract ▾
All authors Lukas Koletzko, Holger Dette, Björn Bornkamp, Frank Bretz, Zoe Kristin Lange
Statistics Applications · Cross-listed from stat.ME
Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients
Dayeon Lee and Gregory R. Hancock
In applied research, the decision to utilize multilevel modeling (MLM) is commonly guided by comparing a point estimate of the unconditional intraclass correlation coefficient (ICC) against some recommended threshold (e.g., 0.05). This naive approach, however, fails to account for sampling uncertainty and provides no formal inferential justification for a decision. The current work introduces negligible effect significance testing (NEST; or equivalence testing) for the ICC, proposing a framework that performs this test using a pivotal quantity of the ICC based on the F statistic. Furthermore, we offer guidance to help researchers define "negligible" that utilizes the design effect to characterize a tolerable level of variance inflation. R code illustrating the proposed procedure is provided.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CL
Can AI Agents Simulate A/B Test Outcomes? A Validation Framework for Agentic Experimentation
Stefan Hut and Lorenzo Masoero
A/B testing remains the standard for rolling out new features in the technology industry. Each experiment, however, consumes real traffic, engineering effort, and weeks of wall-clock time. Can AI
agents—conditioned on behavioral profiles and contextual descriptions of the intervention—simulate outcomes accurately enough to vet candidate treatments before committing live traffic? We formalize this question as a Simulated Randomized Controlled Trial (S-RCT) and derive a two-layer error decomposition that separates
agent approximation error from subsampling error, enabling targeted improvements to each. The framework is
agent-agnostic: any behavioral model—from a fine-tuned specialist to a general-purpose foundation model—can serve as the simulation engine. Validated on 67 historical marketing A/B tests, a baseline S-RCT using an off-the-shelf foundation model captures directional signal (sign overlap 0.70) but systematically overshoots effect magnitudes. A two-phase pre-period calibration protocol reduces the squared prediction error (after removing irreducible measurement noise) by ${\sim}77\times$; a within-subject design—where each
agent is exposed to both arms—reduces standard errors by ${\sim}2.4\times$. We discuss limitations of the current approach and identify applications where experimenters stand to benefit from agentic signals.
Read the full abstract ▾
Statistics Applications · Cross-listed from math.OC
Dynamic Traffic Allocation for Revenue Maximization on Creator Economy Platform
Zhengli Wang, Franklin Lin Feng and Zhixi Wan
Creator economy platforms face a strategic dilemma: allocating traffic to established
stars for immediate ad revenue versus nurturing emerging creators to build a follower base for future monetization. We develop a continuous-time dynamic optimization model to characterize the optimal traffic allocation policy for a platform managing heterogeneous creators with dual revenue streams (advertising and direct follower contributions). We characterize the optimal policy analytically, revealing a “most-valuable-creator-first" rule driven by a forward-looking activation set. Under Bass
diffusion dynamics, this policy exhibits a sophisticated “conditional reversal" strategy, where the platform temporarily prioritizes lagging creators to capitalize on word-of-mouth effects. Regarding the ecosystem structure, we find the optimal policy acts as a selective gatekeeper. Unlike myopic policies that lead to a harsh “winner-take-all"
market, or naive
fairness-driven heuristics that foster inefficient “indiscriminate growth," the optimal policy imposes a strict capability threshold that screens which creators receive platform traffic. Furthermore, it enforces disciplined growth for successful entrants, capping their follower bases at an optimal ceiling to prevent over-investment. Finally, we demonstrate that simple heuristics can lead to significant revenue losses (up to 25%) and propose a practical “follower-growth adjusted" heuristic that achieves near-optimal performance by leveraging observed growth momentum.
Read the full abstract ▾
Statistics Applications
Correlated frailty model for analysis of genetic association in family studies
Agnieszka Krol, Virginie Rondeau, Yun-Hee Choi et al. (+1 more)
Family-based study designs allow the investigation of gene mutation effects on a disease risk by considering related family members. Some methods have been developed for testing sets of genetic variants in family studies but only very few can handle right-censored time-to-event data. We propose here a correlated frailty model for the analysis of a survival outcome related to cancer in presence of familial correlations. These familial correlations are explained by a residual familial component specified by a kinship
matrix and a region- or gene-based specific correlation structure modeled via identical-by-descent (IBD) probability
matrix. The proposed approach is used to quantify and evaluate the association between a set of common single nucleotide polymorphism (SNPs) or rare variants (or both) from the same genomic region and a survival outcome, e.g. time to disease onset. The model's marginal likelihood is maximized using the Marquardt
algorithm. We evaluated the method by simulations under various scenarios where we varied the family size, the strength of genetic associations from multiple rare variants and the presence or not of residual familial correlation. The results indicate that the correlated frailty model can be valuable in family cancer studies, for example to identify genomic regions significantly associated with the time to cancer onset.
Read the full abstract ▾
All authors Agnieszka Krol, Virginie Rondeau, Yun-Hee Choi, Laurent Briollais
Statistics Applications
ALL-IN meta-analysis for flexibility and validity in prospective and retrospective evidence synthesis
Judith ter Schure, Alain Amstutz, Matthias Briel et al. (+2 more)
ALL-IN meta-analysis was developed and first applied during the COVID-19 pandemic. While this setting inspired its name, ALL-IN can also benefit non-pandemic circumstances. Conventional meta-analysis loses its coverage when updated repeatedly over time and when the decisions to initiate new trials and synthesize them depend on the results within the meta-analysis (accumulation bias). ALL-IN meta-analysis is anytime-valid. In its simplest form, ALL-IN meta-analysis stays familiar to run and read based on forest plots with confidence intervals that are wider than standard ones. Within collaborative prospective meta-analysis, the payoff is flexibility and speed, with fast sharing of individual participant data or harmonized aggregate data. Outside of this setting, the payoff is validity, when the decisions that shape the evidence (when to stop trials and whether new ones start) are typically outside the control of the meta-analyst. On the one hand, ALL-IN meta-analysis becomes inefficient when there is a maximum sample size or a stopping rule that the meta-analyist can control. On the other hand, it enables adaptations for any evidence synthesis to halfway become living, prospective or even real-time on interim trial results, without complicating the statistics.
Read the full abstract ▾
All authors Judith ter Schure, Alain Amstutz, Matthias Briel, Amir Aamodt Kazemi, Inge Christoffer Olsen
Statistics Applications · Cross-listed from stat.ME
Group-Level Inference with Micro-Assessments
Paul A. Jewsbury
When the target of inference is a group mean rather than an individual score, long assessments can be statistically inefficient. From the marginal score of the likelihood, we derive the asymptotic standard error of the marginal maximum likelihood
estimator of a latent IRT mean under random item sampling, and obtain a closed-form approximation via a Gaussian-convolution treatment of the item information. The formula makes the test-length–sample-size trade-off explicit and includes an inflation factor for jointly estimating the population variance when the item pool is mistargeted. We validate it against exact
benchmarks, Monte Carlo simulation under pool–population mismatch, and a NAEP mathematics resampling study; group means remain approximately unbiased even with single-item forms.
Read the full abstract ▾
Statistics Applications
Probabilistic Deep Learning for Drought Forecasting: Role of Internal Climate Variability
Henri Funk, Cornelia Gruber, Göran Kauermann et al. (+2 more)
Predicting drought risk is essential for anticipating impacts on water resources, agriculture, ecosystems, and
climate adaptation planning. Yet drought forecasts remain uncertain because variability can substantially alter regional precipitation and evaporative demand. Treating this variability as unstructured noise ignores the fact that internal variability has spatial, seasonal, and temporal structure and thus contains information that can be used to improve drought forecasting. We propose a deep-learning-based forecasting framework for European drought prediction and extend it with an uncertainty-aware drought bound that explicitly incorporates internal forecast variability from a large
climate model ensemble. This bound represents a physically plausible lower-tail trajectory of future drought conditions and marks how severe drought could plausibly become under an unfavourable realisation of internal variability, giving adaptation planning a conservative, risk-averse reference. We compare the proposed bound with a lower bound derived from reanalysis data only and show that our proposed ensemble-informed bound is better calibrated across most regions and seasons. This is specifically true during anomalously dry conditions, when historical reanalysis alone underestimates lower-tail drought risk. Our results show that internal variability should be treated as a forecast quantity in its own right. More broadly, large ensembles provide a practical way to transfer physically plausible
climate variability into machine-learning drought forecasts, yielding risk-aware bounds that are more informative for drought assessment under shifting
climate conditions.
Read the full abstract ▾
All authors Henri Funk, Cornelia Gruber, Göran Kauermann, Helmut Küchenhoff, Magdalena Mittermeier
Statistics Applications · Cross-listed from cs.SD
Comparative Validation of GPT-4o-mini and Teacher Mean Scores for Automated Scoring of Music Analysis Responses: Single-Pass Deployment, Repeatability, and Strategy-Specific Bias
Baicheng Lin, Lingxi Jin and Kyung-Seok Min
Scoring open-ended music analysis responses is time-consuming and requires nuanced judgments of harmonic knowledge and formal understanding. This study evaluates the validity and repeatability of GPT-4o-mini for rubric-based scoring of music analysis essays, using teacher mean scores as the
benchmark. A
dataset of 300 university-level student responses was scored by teachers on four dimensions: Harmony, Form,
Reasoning, and Terminology. GPT-4o-mini scored the same responses using three prompting strategies: few-shot prompting with chain-of-thought
reasoning (Fs+CoT),
retrieval-augmented generation (RAG), and self-consistency based on five internal generations per administration (SC). Each strategy was administered three times with the model,
prompt, rubric, and response held constant. Single-pass scores represented an operational scoring condition, whereas median aggregation across three runs was used to examine robustness. Agreement with teacher mean scores was evaluated using correlation, intraclass correlation, Krippendorff's alpha, quadratic weighted kappa, and scoring error indices. Fs+CoT showed the strongest agreement with teacher mean scores in both single-pass scoring and median aggregation. RAG showed systematic over-scoring, whereas SC produced highly repeatable scores but weaker individual-level agreement. Dimension-level analyses showed that scoring performance varied across rubric components, with Terminology generally showing weaker agreement than
Reasoning. These findings indicate that GPT-4o-mini can generate stable scores for complex music analysis responses, but prompting strategies produce distinct scoring profiles. Operational use therefore requires strategy-specific calibration, dimension-level validation, and continued human oversight.
Read the full abstract ▾
Statistics Applications · Cross-listed from stat.ME
Multivariate Multinomial Logit Model with ANOVA Decomposition for Correlated Categorical Outcomes
Sohyeon Kim, Luo Xiao, Xinming An et al. (+1 more)
In medical research, patients often have multiple interdependent outcomes, such as posttraumatic stress disorder (PTSD), depression, and pain among trauma survivors. Most existing research uses multinomial regression to analyze these interdependent outcomes separately, which ignores correlations between concurrent conditions. This omission may lead to loss of information and reduced predictive accuracy. Accounting for correlations between multiple categorical outcomes requires a high-dimensional parameter space, making model estimation challenging. In this paper, we propose a multivariate multinomial logit model that captures outcome correlations and uses the ANOVA decomposition of the parameter space to reduce the number of parameters. The ANOVA decomposition enables explicit conditional model formulations, which allow for a computationally much simpler composite likelihood for model estimation. We develop an efficient Minorization-Maximization (MM)
algorithm to maximize the composite likelihood, which also incorporates variable selection via a bridge penalty. Simulation studies are conducted to evaluate our method, demonstrating its accuracy in parameter estimation and variable selection. We further illustrate our method using data from the AURORA study.
Read the full abstract ▾
All authors Sohyeon Kim, Luo Xiao, Xinming An, Wenbin Lu
Statistics Applications · Cross-listed from stat.ME
Target Trial Emulation with the R Package TTE: A Tutorial and Methodological Guide
Hisashi Noma
Target trial emulation structures observational causal analyses around the protocol of an ideal randomized trial. By aligning eligibility, treatment assignment, time zero, and follow-up, it can reduce avoidable biases, but implementation still requires coordinated decisions about data construction, inverse probability weighting, diagnostics, outcome models, standardization, competing risks, and uncertainty estimation. This article provides a self-contained methodological guide and practical tutorial for TTE, an R package for target trial emulations with longitudinal observational data. We describe target trial protocols, intention-to-treat and per-protocol estimands, identification assumptions, baseline and person-period data structures, temporal ordering for longitudinal weights, stabilized treatment and censoring weights, weight truncation, balance and effective-sample-size diagnostics, weighted pooled discrete-time survival models, model-based standardization, competing-risk analysis, weighted Kaplan-Meier and Aalen-Johansen estimation, and cluster bootstrap at the original-individual level. Two fully synthetic examples illustrate end-to-end workflows: sodium-glucose cotransporter 2 inhibitor versus dipeptidyl peptidase-4 inhibitor initiation with all-cause death, and sequentially nested angiotensin receptor blocker versus calcium channel blocker trials with heart-failure hospitalization and competing death. The examples show how to obtain and diagnose estimates in R and interpret relative hazards, absolute risks, cumulative incidence, and differences between intention-to-treat and per-protocol effects.
Read the full abstract ▾