Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Friday 25 September

Statistics Applications

Feature-Based Likelihood Ratios for Forensic Science: Combining Neural Networks with Bayesian Probability Calculus

In forensic science, when crime-scene evidence (CSE) and suspect-related evidence (SRE) is present, it is customary to report on the value of this evidence in the form of a likelihood ratio (LR). The LR can be calculated as the probability of CSE given SRE divided by the probability of CSE given that it was generated by a randomly selected person from an alternative culprit population. In forensic science, this is known as a feature-based LR, and intuitively the LR contrasts "similarity" by "typicality". Since it is generally a problem for feature-based LRs to find appropriate models for the data, one either resorts to score-based LRs or to adjusting the feature-based output post-hoc to well-calibrated output. Either way, the above definition of the LR is broken and interpretation of the LR as similarity between CSE and SRE divided by typicality of CSE is destroyed. Here, we report on progress in obtaining instantly well-performing feature-based LRs using gradient descent in combination with Bayesian probability theory to train a two-level model, the main LR model in forensic science for describing distributions of continuous data. For a dataset of laser-ablation inductively-coupled-plasma mass-spectrometry measurements on glass fragments from forensic casework, we show that our best model on validation data yields much better calibrated feature-based LRs on the test set when compared to state-of-the-art feature-based LR systems trained on the same type of data, and that it improves a factor of 4.5 on average on $C_{\mathrm{llr}}$. For LRs interpretable in terms of "similarity" contrasting "typicality", this is a major advancement. However, a state-of-the-art LR system still performs a factor of 1.5 better on $C_{\mathrm{llr}}$ for this data. We also present future plans to close this remaining gap. In order to facilitate collaboration, we have put relevant code on GitHub.
Read the full abstract

Statistics Applications · Cross-listed from cs.AI

LLM Judge Validation Under Sparse Overlap: From Inference to Design

Validating an LLM-as-a-judge requires estimating its agreement with humans, yet annotation budgets rarely allow every item to be multiply labeled. We prove that this overlap sparsity is the first-order determinant of wrong deployment decisions: at 5% pairwise overlap, wrong-decision rates reach 25% and the probability of selecting the wrong best judge among ten candidates is 65%. The two actionable levers are overlap quantity and allocation. For quantity, we derive a minimum-overlap formula showing $\rho \geq 0.25$ suffices for non-borderline judges while borderline cases remain fundamentally hard. For allocation, a zero-cost stratified scheme halves false-rejection rates relative to random sampling when strata are informative. We validate on 10 LLM judges across four evaluation matrices spanning visual assessment, causal reasoning, and summarization.
Read the full abstract

Statistics Applications · Cross-listed from stat.ML

Uncertainty and Explainability in Deep Rough Volatility: A Neural Information-Theoretic Posterior Approach

Deep learning has substantially accelerated the calibration of complex stochastic-volatility models, but neural point calibration alone does not capture the uncertainty remaining after an implied-volatility (IV) surface has been observed. We develop a simulation-based inference framework for rough Heston (rHeston) calibration that learns the posterior distribution of the model parameters conditional on an IV surface. Using neural ratio estimation, we obtain calibrated posterior samples that can be propagated through heteroscedastic neural surrogate pricers for path-dependent exotic options. The resulting posterior-predictive distributions combine residual parameter uncertainty with conditional surrogate uncertainty and yield uncertainty-aware price intervals. We further introduce Hellinger-SHAP, an information-theoretic explainability method for posterior inference. Rather than attributing a single parameter point estimate, it applies local-background Kernel SHAP to a posterior-information functional measuring contraction from the prior to the posterior. This identifies maturity–moneyness regions associated with posterior information gain for individual rHeston parameters. In a simulation study, posterior-predictive intervals provide calibrated or conservative coverage across forward-start, barrier, and realized-variance claims, while point plug-in prices can be materially unreliable for selected contract regimes. Together, the UQ and XAI analyses provide a transparent framework for uncertainty-aware neural calibration and downstream exotic pricing under the specified prior-predictive model.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Statistical Foundations for a Google Play User-Review Sentiment Index: Signal Fusion, Shrinkage, Distributional Validation, and Dynamic Smoothing

The decision problem is whether valence among the users who write reviews of an app has changed enough to warrant investigation of a release, outage, or support issue. We propose a statistical specification for that decision, applied to written Google Play reviews collected under a declared locale, language, sort order, retrieval cap, and window rule. That endpoint returns a selected collection rather than a random sample, so the index describes review writers under the declared protocol and not all app users. What is new is not a sentiment classifier but an auditable interface: an ordinal threshold model is the preferred representation of star ratings, a calibrated two-signal generalized least-squares estimator is retained as an interpretable linear approximation when held-out labels support its conditional-unbiasedness assumption, the estimand is declared before the estimator, and every downstream module–uncertainty, diagnostics, and regularization–is tied to that declaration. The framework separates the latent average of a fixed collected set from the mean of a protocol-defined population of review writers, so measurement and between-review variation are not conflated, and treats helpfulness and recency weights as a different policy target. Shrinkage, cross-app empirical Bayes, histogram and dependence diagnostics, and dynamic smoothing are specified as separate modules with defaults and mandatory sensitivity analyses. A reference implementation, a declared 6000-review three-app collection, and a synthetic stress test exercise the displayed equations: fusion reduced error in clean and small-window scenarios, coordinated contamination caused severe bias and interval failure, and the collected sample reproduces the abstention rules. No real-review validation is claimed.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

A likelihood-based coefficient for biomedical independence testing: the binomial-cut composite likelihood ratio

The standard dependence summaries used in biomarker studies – Pearson's r, Spearman's rho, Kendall's tau – take values in [-1, 1] with 0 indicating no linear or monotone association. Zero does not distinguish independence from non-monotone dependence, so the scale cannot represent threshold effects, heteroscedasticity, and tail shifts common in biomarker practice. We formulate independence testing as a composite Bernoulli likelihood ratio: at each threshold t, comparing the Bernoulli laws of 1(Y <= t) conditionally on X versus marginally, aggregated over cut points. The resulting coefficient xi_cut lies on [0, 1] with 0 iff X and Y are independent (under continuity of Y) and 1 iff Y is a measurable function of X. Fisher weighting arises at second order from the Bernoulli likelihood, and xi_cut equals twice the threshold-averaged mutual information between X and 1(Y <= t), giving a distribution-free lower bound on I(X; Y). A second-order expansion recovers the Fisher-weighted Dette-Siburg-Stoimenov measure, which coincides under continuity with Chatterjee's rank correlation. Estimation uses a Nadaraya-Watson plug-in with a max-over-grid bandwidth; inference is by exact permutation. In biomarker-motivated simulations T_cut substantially outperforms rank-based coefficients on W-shaped non-monotone and heteroscedastic alternatives. We illustrate on the Seattle cohort (n=70, ages 21-88) of the aging plasma proteome dataset, screening all 1,305 proteins for age dependence: under Benjamini-Hochberg control at q<0.05, T_cut rejects on 70 proteins, six of which are missed by Pearson, Spearman, and Chatterjee at the same FDR level.
Read the full abstract

Statistics Applications

Modeling dependent degradation data considering inherent causal relationships for reliability analysis and remaining useful life prediction

Accurate modeling of dependent degradation processes is essential for credible reliability assessment and remaining useful life (RUL) prediction in complex systems. Existing dependent degradation models usually describe dependence using correlation-based methods with symmetric characteristics. However, they ignore inherent causal directionality between degradation paths, which may lead to biased reliability evaluation and RUL predictions when physical causality exists. To address this issue, this paper proposes a causality-driven framework for modeling dependent degradation data. Firstly, univariate degradation models with multi-source uncertainties are established for each performance indicator based on the Wiener process. Then, the stable Peter-Clark algorithm is employed to uncover inherent causal relationships between degradation processes, and an uncertainty-aware neural network is employed to quantify causal effects considering uncertainties. Next, univariate degradation predictions and causal predictions are integrated within a Bayesian framework to construct a causally dependent degradation model, and the corresponding loss function is derived for model training. Finally, system reliability and RUL predictions are derived via Monte Carlo simulation. The proposed methodology is validated on the C-MAPSS dataset. Results show that considering inherent causal directionality between degradation processes helps eliminate physically unrealistic degradation behaviors, yielding more accurate degradation and RUL predictions than independent and correlation-based dependent degradation models.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Factorial Multivariate Bayesian Causal Forests: heterogeneous main and interaction effects of multiple treatments on correlated outcomes

Many studies expose units to several binary treatments at once and record several correlated outcomes, yet analysts usually estimate one treatment's average effect on one outcome at a time – discarding how treatments interact and how their effects vary across units. We introduce the factorial multivariate Bayesian causal forest, a Bayesian nonparametric model that decomposes the factorial response surface over the treatment lattice into a prognostic sum-of-trees plus one sum-of-trees per main or interaction effect, each with correlated multivariate leaf parameters, estimated jointly with coherent uncertainty. A single indicator-weighted kernel samples every component – the prognostic term is the special case whose indicator is unity – so the two-treatment model, the single-treatment multivariate causal forest, and a general order-r truncation are one and the same sampler. We give the ANOVA/Möbius identification of each estimand, an efficient Rcpp engine, and an interpretability layer using value-suppressing uncertainty maps. In simulations the average-effect estimators are unbiased with nominal coverage, consistent, and robust under misspecification, failing only under unmeasured confounding, which we flag. We illustrate the method on clinical, agricultural and economic data and on a deeper application to the NHANES survey. Software is provided as an R package.
Read the full abstract

Statistics Applications · Cross-listed from cs.CR

Revisiting Certified Defense with Differential Privacy on Vision Transformers

Certified defenses that incorporate differential privacy have proven effective on Convolutional Neural Networks (CNNs), furnishing rigorous robustness guarantees against norm-bounded adversaries. However, the certified robustness behavior of Pixel Differential Privacy (PixelDP) remains largely unexplored with the self-attention architecture now dominating the deep-learning landscape. Given that the Transformer has a profound impact on our daily applications from the digital world to the physical world, it is crucial to study certified robustness through differential-privacy-style stability. To fill this research gap, we revisit this construction in Vision Transformers and identify a failure mode that is largely hidden in the convolutional setting. When noise is injected after the patch embedding, the Laplace mechanism with the inherited grouped $\ell_1$ sensitivity bound collapses to chance-level accuracy across noise scales, whereas the Gaussian mechanism remains trainable. This contrast isolates the source of failure: not the injected noise itself, but the geometry of the sensitivity constraint. We show that the attenuation induced by the inherited $\Delta_{1,1}$ projection increases with layer width and kernel size according to a random-matrix scale $C/(\sqrt{M}+\sqrt{N})$. Replacing the $\ell_1$-type constraint with a spectral-norm constraint eliminates the collapse across datasets and architectures, but creates a fundamental obstacle: the repaired models no longer satisfy the sensitivity condition required by the standard Laplace certificate. We resolve this mismatch by deriving a dimension-free $(\varepsilon,\ \delta)$-privacy guarantee for the Laplace mechanism under $\ell_2$ sensitivity through concentration of the privacy loss.
Read the full abstract

All authors Jun Yan, Weiquan Huang, Qixian Zhang, Yan Bai, Shutai Zhang

Statistics Applications · Cross-listed from cs.AI

DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration

Large language models (LLMs) as a judge enable scalable evaluation, but their judgments can be sensitive to response order and, even after removing such position effects, can still diverge systematically from human preferences.We introduce DIAL, a unified framework that combines abundant LLM comparisons with limited human comparisons to separate judge-specific position effects, learn shared structure in position-debiased LLM preferences, and adaptively calibrate that structure toward the human preference target. Theoretically, we study three aspects of DIAL: (i) identification of latent LLM preferences, position effects, and human calibration; (ii) adaptive estimation that balances LLM anchoring against limited human evidence; and (iii) fixed-weight uncertainty quantification for the calibrated human preference. Empirically, we evaluate position debiasing and human alignment separately in controlled simulations and on three human-preference benchmarks, showing that DIAL remains robust to unbalanced response order, achieves strong human-aligned rankings with limited labels, and adapts toward human evidence when LLM information is imperfect. Our real-data study collects over 410K judgments from 21 LLM judges in both display orders, providing a resource for future studies of LLM-judge bias, heterogeneity, and human alignment.
Read the full abstract

All authors Zesheng Cai, Yingqi Fan, Sichang Chen, Jin-Hong Du

Statistics Applications · Cross-listed from stat.ME

Dynamic factor and double PCA models for partially observed survival curves: Forecasting demand in short-term rental markets

This paper develops prediction models for population-level survival curves observed over time and sampled from a heterogeneous mix of populations. We consider a discrete-time setting where each curve is only partially observed and forecasts of the remaining trajectory are needed for downstream decision making. Our approach recasts cross-population heterogeneity into a multivariate sampling model. We propose two forecasting models for partially observed curves: a full factor analysis model that extends a general factor representation to incorporate the partially observed survival curve, and a double PCA model. The methodology is motivated by demand forecasting in short-term rental markets, where market-level occupancy paths can be viewed as survival curves over the booking horizon and where forecasts of future occupancy feed into dynamic pricing algorithms. We apply the models to the newly released Wheelhouse dataset, which contains time series of market occupancy curves for 500 markets from 2017 to 2022. Model performance is assessed using the integrated quadratic distance, and we compare the proposed PCA-based methods to Holt's linear trend model across multiple forecast horizons. The results show that the proposed models yield accurate and stable forecasts of the remaining survival trajectory and generally outperform Holt's method, particularly at longer horizons.
Read the full abstract

Statistics Applications · Cross-listed from cs.AI

Accounting for Bias Enables Sustainable LLM Evaluation

LLM-as-a-judge has become the de facto standard for scalable, subjective evaluation, yet current leaderboards compensate for systematic measurement bias by running ever more comparisons, an approach that is both statistically unsound and computationally wasteful. The root cause is an incomplete measurement model, treating LLM judges as neutral, interchangeable instruments ignores documented biases like position bias, verbosity bias, judge severity, and self-enhancement, that no volume of additional data can eliminate. We propose a unified latent variable framework that jointly models pairwise and ordinal data while explicitly correcting for these confounders, recovering reliable rankings from substantially fewer comparisons. Because fitting this model costs negligible compute relative to a single round of LLM inference, bias correction is not only more statistically rigorous but also a more sustainable approach to trustworthy evaluation.
Read the full abstract

All authors Harshita Katoch, David Antony Selby, Gerrit Großmann, Sebastian Vollmer

Statistics Applications · Cross-listed from cs.LG

Missingness-Aware Conformal Prediction Under Cross-Hospital Distribution Shift

Clinical measurements are recorded for some patients but not others, at rates that differ across hospitals, and marginal conformal coverage does not ensure coverage within groups defined by missingness. We propose a missingness-aware conformal calibration procedure for mortality prediction under cross-hospital distribution shift. It selects a measurement on an independent sample, groups patients by whether that measurement is recorded, and applies Mondrian calibration within each group, so no calibration outcome is reused. We evaluate the procedure across hospitals in eICU and across care units within one MIMIC-IV hospital, using three predictors. Relative to pooled calibration, it reduces the average worst-group coverage gap on its selected groups in all six settings, with a median reduction of 1.9 percentage points; paired site-bootstrap intervals exclude zero in five. These gains do not extend uniformly. Calibration by predicted risk achieves smaller gaps on a broader panel of missingness groups, and when eICU hospitals are evaluated separately, the gain shrinks for all three predictors and reverses in sign for one. We explain this discrepancy with a hospital-level decomposition. Pooling reweights hospitals through a covariance between group shares and coverage errors, and lets errors of opposite sign cancel: weighting explains the reversal, and cancellation accounts for most of the attenuation for the other two predictors. Constructed population distributions show that pooled and within-hospital evaluations can rank calibration methods oppositely even without sampling noise. Pooled improvement alone therefore cannot establish better coverage within hospitals, even when the calibration groups are fixed.
Read the full abstract

All authors Liang You, Dongwen Ou, Hengyu Shi, Siyuan Dai

Statistics Applications · Cross-listed from stat.ME

Design-Ignoring versus Design-Respecting World Models for Epidemiology

World models for epidemiology learn from records shaped by study designs, including assignment, sampling, measurement, and related processes. A model may therefore reconstruct observed trajectories while learning an intervention contrast that depends on how records were collected. We formalize study design as constraints on a world model latent-world interface, action mechanism, observation likelihood, and target readout. This distinguishes design-respecting models, which encode these constraints, from design-ignoring models, which fit selected records without relating assignment and observation to the intended intervention question. Using a large scale cluster-randomized test-negative trial, we hold the latent structure and fitting settings fixed and compare a design-ignoring case-count model with a design-respecting test-negative observation model. Both models achieve comparable factual reconstruction. Yet across 500 paired resampling experiments at different relative sampling intensity, median contrast changes are substantial for the design-ignoring model, versus merely marginal for the design-respecting model. The same qualitative separation holds when sampling also varies across clusters. Thus, factual reconstruction alone does not establish design alignment; testing whether fitted contrasts preserve design-implied observation-process invariances provides a sharper evaluation.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Amortized Bayesian Disease Mapping and Boundary Detection on Heterogeneous Spatial Graphs

Spatial disease maps help public-health researchers identify geographic inequalities, but standard Bayesian smoothing can obscure localized disparities when neighboring communities have sharply different socioeconomic or behavioral profiles. Analysts therefore need to determine where smoothing should be interrupted and repeat that analysis as maps, adjacency structures, and outcomes change. We develop a covariate-informed Bayesian boundary model and an amortized posterior approximation trained across heterogeneous areal graphs. The model distinguishes local interruptions in smoothing from broader residual spatial dependence; the trained approximation handles maps with different numbers of regions. Simulations examine posterior calibration, boundary-probability recovery, replicated-data behavior, and MCSE-controlled agreement with prior-matched MCMC. In contrast to traditional approaches that analyze these data separately, we demonstrate the effectiveness of using a single trained deep learning network to analyze respiratory hospitalizations in Greater Glasgow; lung cancer incidence in California; and tracheal, bronchial, and lung cancer mortality in South Korea, comprising 58 to 241 regions. Selected boundary density is greatest in Glasgow and lowest in South Korea despite substantial residual spatial dependence in both, showing that local interruption and broader spatial persistence need not vary together. Across all three applications, edge-level boundary probabilities agree substantially with dataset-specific analyses, although posterior spread and thresholded boundary sets differ. These results support reusable Bayesian boundary analysis across the evaluated disease-map class and identify the validation needed before deployment to new applications.
Read the full abstract

Thursday 24 September

Statistics Applications

Suitable Measures for the Potential Operational Utility of AI NWP Rainfall Forecasts Over Africa

Artificial intelligence (AI)-based weather prediction is approaching the skill of physical numerical weather prediction (NWP) systems at a fraction of the computational cost. This is particularly promising for Africa, where rainfall extremes are intensifying and many forecasting centres lack the infrastructure to run physical models at extended lead times. We present a calibrated comparison of GraphCast, GenCast and the Functional Generative Network (FGN) against the physical NWP model IFS for rainfall prediction across Africa. Deterministic and probabilistic forecasts are postprocessed using Isotonic Distributional Regression and evaluated with the Continuous Ranked Probability Score against IMERG, RFEv2 and CHIRPS across seasons, wet and dry regimes, elevation zones and lead times. All models retain skill beyond climatology across most seasons and at extended lead times. AI models generally outperform IFS in wet regions, whereas IFS performs better in dry, high-elevation areas, where its finer resolution better represents orographic controls on rainfall. Across observational datasets and seasons, AI models achieve a median improvement of approximately 5% over IFS. GraphCast achieves calibrated skill comparable to the ensemble-based FGN, although FGN provides greater significant skill at longer lead times. These results highlight the potential of calibrated AI weather prediction to provide accessible and computationally efficient rainfall forecasts across Africa, while demonstrating the continuing importance of spatial resolution, ensemble design and regional characteristics.
Read the full abstract

All authors Shruti Nath, Docko Sow, Koomi Toussaint Amoussouvi, Fenwick Cooper, Josiah Kiarie Kimani, John Bagiliko, Florian Pappenberger, Rendani Mbuvha

Statistics Applications · Cross-listed from stat.ME

When Do Surrogate Metrics Work? A Finite-Sample Comparison Under Realistic Failure Modes

Online experiments must often be evaluated before long-term outcomes mature. Under rolling enrollment, these outcomes are observed only for early enrollees, while short-term surrogates are available for everyone. We compare seven estimators across eleven data-generating processes, spanning partial mediation, drift, outcome sparsity, and enrollment-time labeling, with up to $R = 2,000$ replications over more than 500 method-by-scenario cells. We find a sharp robustness-efficiency tradeoff: the surrogate index delivers large efficiency gains when surrogacy holds but its coverage collapses under violations, while PPI-family methods stay asymptotically valid under random labeling at smaller gains. We give the finite-sample variance of PPI++ in the all-units parameterization for a fixed predictor, a joint asymptotic distribution for the two estimators under cross-fitting, and a Hausman-type estimator-disagreement diagnostic, then quantify the detection-damage gap: in the partial-mediation design, where we locate both edges, a band of violations destroys surrogate-index coverage yet is too small to detect on most datasets. On the 64,000-customer Hillstrom experiment the diagnostic rarely flags a violation that biases the surrogate index, and a Cauchy-kernel hybrid of the two estimators inherits 10.7% relative bias; on the 14-million-user Criteo experiment the violation is detected, and subsampling traces detection turning on with scale as damage persists. PPI++ has limits: at a rare-conversion $n = 30,000$ Criteo subsample its empirical coverage is 85.0%. We recommend prespecifying PPI++ with the exact variance as the primary analysis under random labeling and adequate labeled outcome counts, reading the diagnostic as a warning, not a certificate, and treating the surrogate index as a sensitivity analysis.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

An End-to-End Pipeline for Causal ML with Continuous Treatments: An Application to Financial Decision Making

This paper presents an end-to-end causal machine learning (ML) pipeline designed for real-world applications with continuous treatments. The proposed framework consists of six sequential steps: dimensionality reduction, causal identification, positivity assumption violation handling, estimation, refutation and evaluation, and policy optimization. We introduce practical contributions not currently available in existing causal ML toolkits, specifically: (1) a method for detecting and quantifying positivity violations in continuous treatment settings (2) a novel, scalable two-stage dimensionality reduction framework tailored for causal inference with high-dimensional data; (3) the adaptation of sensitivity analysis and estimation methods originally designed for binary treatments to the continuous treatment space and (4) an end-to-end integration of these components into a modular, reproducible workflow. These innovations address real-world challenges in causal inference that are often not covered in theoretical frameworks but frequently encountered in industrial applications. The methodology is validated with a synthetic dataset inspired in a real-world financial debt collection use case, however its design can be applied to analogous problems across different industries. Results demonstrate that the proposed methodology offers a more computationally efficient approach and produces less biased estimates compared to standard methods for problems with continuous treatment and high-dimensional data. A fully functional GitHub repository with documented code and numbered notebooks is made available ensuring reproducibility and practical implementation. The pipeline presented is intended to contribute to closing the gap between academic approaches and practical application in industry contexts where causal ML can be highly beneficial such as the financial sector.
Read the full abstract

Statistics Applications

Learning the Maximum Tolerated Dose for Continuous Toxicity via Monotone Bayesian Trees

Phase I cancer trials seek the maximum tolerated dose (MTD) while protecting patients from excessive toxicity. Dose assignments must therefore balance patient safety with learning the dose–toxicity relationship as data accrue. We model continuously measured toxicity outcomes using two forms of Bayesian additive regression trees (BART): isotonic BART projects posterior response curves onto nondecreasing functions, whereas monotone BART constrains the model. Joint curve and variance draws induce an MTD posterior that guides dose selection through escalation with overdose control (EWOC). We compare these methods with three parametric procedures across seven dose–toxicity curves in simulation. We assess dose-limiting toxicity (DLT) counts, above-MTD assignments, signed last-dose error, and relative absolute error. The tree procedures attained the lowest mean RAE on four nonlinear curves and jointly minimized mean DLT counts and above-MTD assignments on four curves. A Bayesian reinforcement learning perspective formulates these sequential decisions as a finite-horizon planning problem. Dose restrictions yield a lower bound on last-dose error; under exact posterior-predictive evaluation, an EWOC-based rollout policy has no greater expected weighted loss than its baseline. Dose Trial Lab, a desktop simulator for all five procedures, accompanies the supplementary material.
Read the full abstract

Statistics Applications

Bayesian joint modeling of longitudinal patient-reported outcomes and survival: an application to chronic obstructive pulmonary disease

Questionnaire-based patient-reported outcomes (PROs) are discrete, bounded and overdispersed, yet joint models relating them to survival may ignore these features or estimate both processes sequentially. We propose a Bayesian joint model combining a beta-binomial mixed-effects submodel with a Weibull proportional hazards submodel, linked through the subject-specific response probability. Simulations show that simultaneous estimation reduces bias in the longitudinal slope and yields practically unbiased association estimates, unlike two-stage estimation. In a cohort of 543 patients with chronic obstructive pulmonary disease, the model identified associations for all eight SF-36 dimensions and for two of three SGRQ dimensions, including several associations not detected by the two-stage approach, and provided dynamic survival predictions.
Read the full abstract

All authors Cristina Galán-Arcicollar, Danilo Alvares, Josu Najera-Zuloaga, Dae-Jin Lee

Statistics Applications · Cross-listed from cs.CL

Do Audio Language Models Hear and Read Distinctive Features Alike?

Audio language models pass speech and text through a single decoder. We ask whether that decoder represents a distinctive feature in the same direction when a phoneme is heard and when it is read. For minimal pairs of phonemes differing in one feature, we take the offset between the two members' mean representations. Averaging those offsets gives a direction for each stream, and we measure the cosine between the two. Because the two streams already agree about arbitrary phoneme pairs, we compare every measure against a reference built from random pairings rather than against zero. We apply this to 6 models, 7 features and 15 languages from 11 families. Only voicing in the two Qwen2.5-Omni models exceeds that reference after correction for multiple testing, and the reference varies by a factor of seven between models. In three of the six models, voicing has one direction in audio across the 14 languages with enough minimal pairs to measure it, and every language pair agrees in two of them. The model family, not the model size, predicts which stream represents a feature.
Read the full abstract

Statistics Applications

The risks of dichotomising ordinal outcomes: A spatial analysis of self-rated health in Western Europe

Survey responses are often measured using ordered response categories. In the European Social Survey, self-rated health is measured on a five-point scale from very good to very bad, yet analyses commonly dichotomise responses into binary categories of "good" and "poor" health. Binary indicators provide prevalence measures that are straightforward to communicate, but dichotomisation reduces information and may limit captured health variation, while the cut-off may influence estimates and substantive conclusions. Systematic evidence on these effects in spatial settings remains limited. We address this gap using European Social Survey round 11 (2023/2024) for Western Europe. We model self-rated health among male respondents by age, education and region. Bayesian spatial individual-level models with poststratification provide population-representative estimates. We compare an ordinal cumulative logit model for the five-category outcome with Bernoulli logistic regression models using two dichotomisations differing in the classification of "fair" health. Older age and lower education are consistently associated with worse self-rated health across specifications, suggesting relatively robust fundamental age and educational gradients. However, their magnitude and uncertainty, and some geographical conclusions, are sensitive to how the outcome is modelled. Assigning "fair" health to either side of a binary cut-off changes the regions identified as having above-average levels of less favourable health. The ordinal model retains category-specific information and can produce familiar binary prevalence estimates through aggregation. Binary indicators remain useful, particularly for monitoring and communication. Nevertheless, the selected cut-off should be justified and sensitivity to alternative cut-offs or an ordinal modelling approach should be considered, especially for geographical comparisons.
Read the full abstract

All authors Miguel Ángel Beltrán-Sánchez, Miguel Ángel Martínez-Beneito, Terje Eikemo, Sara Martino, Andrea Riebler

Statistics Applications

Hybrid Models for Short-Term Sea-Level Forecasting

Accurate tide forecasts are essential for coastal management, navigation, flood-risk reduction, and infrastructure protection. Observed sea level can be decomposed into astronomical and non-astronomical components, the latter mainly driven by meteorological effects. This study investigates a hybrid framework for hourly sea-level forecasting that combines harmonic analysis (HA) for the astronomical component with data-driven models for the non-astronomical contribution. The approach is evaluated at six tide-gauge stations with different tidal regimes: Venice, Trieste, Saint-Malo, Vardø, Nikiski, and Nagasaki. Four data-driven model classes are considered: (i) linear parametric models, represented by autoregressive models with exogenous variables; (ii) functional parametric models, based on functional autoregressive models with exogenous variables; (iii) semiparametric and nonlinear models, including generalized additive models and autoregressive neural networks; and (iv) a semi-functional non-standard k-nearest-neighbours approach combining similarity in recent non-astronomical trajectories and meteorological conditions. Results reveal that hybrid models reduce forecast errors by 52.9-54.9% on average relative to HA. The generalized additive model is the most competitive across locations, while k-nearest neighbours performs best at Saint-Malo and the autoregressive model with exogenous variables is favoured in Nagasaki. For Venice, an economic decision-making case study assesses the operational use of sea-level forecasts in managing the MoSE flood-barrier system.
Read the full abstract

Statistics Applications

Interpreting relative utility for probabilistic predictions

At a fixed threshold, relative utility (RU) measures the net-benefit gain of a prediction model over the better of treat-all and treat-none relative to the corresponding gain under perfect outcome classification. We illustrate that RU equal to 1 therefore represents perfect outcome classification at that fixed threshold, not perfect probabilistic prediction. However, even when every predicted probability equals the true probability, observed RU can equal 0. In a simple constant-risk setting, this occurs with probability approaching 1 as the sample size increases. Consequently, the distance from observed RU to 1 should not in general be interpreted as improvement achievable by a better prediction for binary probabilities.
Read the full abstract

Statistics Applications

GeoDose-CP: Graph-Local Conformal Inference for Continuous-Treatment Earth Observation

Reliable intervention-oriented uncertainty quantification from Earth observation (EO) remains challenging when continuous treatment shifts, spatial dependence, limited support, and satellite-outcome uncertainty must be addressed simultaneously. Existing causal, conformal, and spatial approaches address parts of this problem, but their direct combination does not generally recover the appropriate interventional reference law because candidate reassignment jointly alters treatment likelihood, standardized residuals, and graph-dependent residual likelihood. This study presents GeoDose-CP, a support-aware conformal framework for localized stochastic potential outcomes under continuous or mixed continuous-atomic treatment. Its central methodological contribution is a graph-local target-orbit law that jointly represents intervention-induced treatment shift, the inverse outcome-scale Jacobian, and spatial residual dependence. The framework further provides exact weighted candidate inversion, a scalable sparse approximation with explicit discrepancy accounting, and refusal under inadequate support. Evaluation used controlled known-truth experiments, MineDoseBench, treatment-density sensitivity analysis, external conformal comparators, and a multi-mine New South Wales (NSW) study. In MineDoseBench, GeoDose-CP achieved mean selective coverage of 0.9692 across 27 configurations and a minimum local q0.05 of 0.8951; exact-sparse auditing produced nine inclusion disagreements over 2,700 targets. In the NSW study, the absence of an auditable longitudinal rehabilitation treatment rendered treatment-dependent inference nonoperational rather than forcing inference through a proxy exposure.
Read the full abstract

All authors Md Khalid Hasan Sakib, Dristi Datta, Manoranjan Paul, Davina White

Statistics Applications · Cross-listed from stat.ME

Panel Conditioning in Fixed-Effects Models: Identification and Bias Propagation

Panel conditioning, the causal effect of prior survey participation on responses, can vary with tenure. Under an additive model of cell means in period, entry cohort, and tenure, we characterize which features of the conditioning path a staggered panel identifies on its observed support, and how the unidentified component affects common panel estimators. The identified set of the path is an affine translate of the tenure projection of the cell design's kernel, and a linear functional of the path is identified exactly when it annihilates that projection. It always contains an affine direction and, when the entry cohorts share a stride, periodic directions, which exhaust it under a connectivity condition on observed increments; second differences at that stride are then identified, and ordinary ones generally are not when the stride exceeds one. Under a recruitment condition, an interrupted schedule such as the four-eight-four rotation of the Current Population Survey (CPS) distinguishes a constant increment per interview from one per calendar month, which no equally spaced schedule can. We give support conditions for recovery under a plateau, entry-wave negative controls, or bounded cohort drift. A second set of results links identification to regression: two-way fixed effects absorb every unidentified direction, so the remaining conditioning bias is normalization-invariant and itself identified, and a two-way regression with tenure indicators corrects it under a residual-rank condition. When event time is aligned with tenure, conditioning shifts event-study coefficients by a known linear functional of the path, producing pre-trends without anticipation; bounds on identified curvature bound those shifts. Simulations verify the identities, a 19-wave Japanese panel illustrates the support calculations, and published CPS month-in-sample indices give a descriptive, not identifying, example.
Read the full abstract

Wednesday 23 September

Statistics Applications

Multivariate Continuous-Time Autoregressive Moving Average Processes for Astronomical Multiband Time Series

Large-scale astronomical surveys provide unprecedented volumes of multivariate time-series observations obtained through multiple optical filters. We develop a structured multivariate continuous-time autoregressive moving average (MCARMA) framework for multi-band time series with irregular sampling, heteroscedastic measurement errors, and partially observed bands. The framework allows band-specific stochastic dynamics while modeling cross-band dependence through correlated Brownian driving processes, with state-space and spectral representations enabling likelihood-based inference and interpretation of the fitted stochastic dynamics. We develop a two-stage estimation procedure in which a numerically stabilized preliminary fit initializes subsequent maximum likelihood estimation. Simulations show that higher-order stochastic structure can be recovered when its characteristic features are adequately resolved, but can become weakly identifiable because of limited temporal resolution or near pole–zero cancellation. Joint multivariate estimation improves parameter recovery in 23 of 27 settings and spectral recovery in 26 of 27 settings relative to separate single-band fits. Three Sloan Digital Sky Survey Stripe 82 quasars, respectively favoring MCARMA(1,0), MCARMA(2,0), and MCARMA(2,1), illustrate how joint multiband modeling uses cross-band dependence to inform marginal dynamics and can yield different model-order and spectral inference. The methodology is implemented in the Python package mcarma.
Read the full abstract

All authors Izak Schmidlkofer, Zhirui Hu, Lishan Shi, Weixiang Yu, Hyungsuk Tak

Statistics Applications

Statistical Methods for Estimating Probability of Detection in Structural Health Monitoring

There is much interest in the potential to use structural health monitoring (SHM) technology to augment traditional nondestructive inspection (NDI) methods to improve safety, increase asset availability, and reduce maintenance and inspection costs. SHM has the potential to be used in many applications, including critical components in aircraft and pipelines. Probability of detection (POD) plays a critical role in aircraft structural integrity programs, leading to increased interest in developing methods to assess POD in SHM applications. In contrast to traditional NDI laboratory experiments involving specimens with cracks, SHM sensors are fixed, and SHM data are acquired over time as cracks grow or otherwise evolve. Thus, traditional statistical methods for assessing POD must be replaced or extended to properly handle repeated-measures data. The purpose of this paper is to review the basic statistical concepts of POD and show how these concepts can be extended or adapted for SHM-POD applications. The paper presents statistical methods for modifying and extending existing POD methods, including a simple size-of-damage-at-detection (SoDaD) method and a random-parameter (RP) method for repeated-measures data. The methods are compared using three case studies involving Piezoelectric Transducer (PZT), Carbon Nanotube (CNT), and Comparative Vacuum Monitoring (CVM) sensor systems. Results show that the SoDaD method provides a simple approach for POD estimation with limited data, while the RP method offers enhanced modeling fidelity by utilizing repeated measurements. These methods are applicable when a scalar damage index or similar response is used to make a detection decision.
Read the full abstract

All authors Qizheng Xia, William Q. Meeker, Dennis Roach, Seth S. Kessler, Qing Li

Statistics Applications · Cross-listed from stat.ME

Long-Term Tail Modeling in Survival Analysis via Extended Generalized Pareto Distributions

We propose a class of extended generalized Pareto models for right-censored survival data, with particular emphasis on tail inference and long-term extrapolation. Our framework integrates extreme value theory and survival analysis, combining a generalized Pareto distribution with a flexible perturbation distribution on the unit interval. We consider three perturbation specifications: a parametric Beta model, a Bernstein polynomial estimator, and a histogram-based estimator. To facilitate direct comparison, all three models are fitted using a unified iterative procedure adapted to right censoring through Kaplan-Meier-based pseudo-observations. A Monte Carlo study evaluates finite-sample performance across different tail indices, censoring levels, sample sizes, and model complexities. The results reveal a trade-off between flexibility and stability: the Beta specification generally performs best for tail-index estimation, whereas the histogram estimator performs particularly well for scale estimation under low censoring. The Bernstein estimator shows intermediate performance and greater sensitivity to sample size and censoring. Applications to bladder cancer recurrence and heart-failure survival data show that models with very similar in-sample fits can nevertheless produce markedly different tail-index estimates and long-term extrapolations. These findings emphasize the importance of perturbation specification when extended generalized Pareto models are used for survival extrapolation under censoring.
Read the full abstract

Statistics Applications

Short-term rental market occupancy - daily time series for 2017-2022 on 500 markets worldwide

Short-term vacation rentals, as promoted by platforms such as Airbnb, Homeaway, Vrbo, etc., are a growing component of the travel industry. This paper provides a unique, large dataset on global market occupancy for the short-term rental market using data from the American company Wheelhouse. The dataset consists of data for $500$ markets around the world. For each market, a daily occupancy time series from January $2017$ to December $2022$ is provided, allowing for studies of local and global patterns in the evolution of the short-term rental market. Additionally, the dataset includes curves representing the booking trajectory of each market and stay date up to one year prior to the stay date. This large dataset comprises a unique combination of time series and survival analysis data, and is suitable as a methodological benchmark for both classical statistical and machine learning models.
Read the full abstract

All authors Marthe Elisabeth Aastveit, Andreas Buschermöhle, Alex Lenkoski, Thordis Thorarinsdottir

Statistics Applications

Bayesian calibration of adaptive-behavior SIR models for multi-wave COVID-19 incidence in New York City

Epidemic incidence reflects both transmission dynamics and adaptive human behavior, yet these mechanisms may be difficult to distinguish from aggregate case data alone. We calibrated four susceptible–infected–recovered (SIR) specifications to weekly confirmed COVID-19 incidence in New York City from June to December 2020, comparing a single continuous SIR trajectory, a wave-initialized SIR model, and two adaptive-behavior models with either shared or wave-specific transmission. Inference was performed using rejection Approximate Bayesian Computation (ABC), and in-sample reconstruction was assessed using root mean squared error (RMSE) and the weighted interval score (WIS). Reinitializing the epidemic state by wave produced the largest structural improvement over the continuous SIR trajectory, reducing mean-based RMSE by 48.6% and WIS by 17.6%. Adding delayed prevalence-dependent behavioral adaptation with shared transmission further reduced mean-based RMSE by 23.4%, but yielded essentially unchanged WIS relative to the wave-initialized SIR model. Allowing transmission to vary by wave did not provide a consistent additional advantage and produced strongly asymmetric posterior-simulation trajectories. Behavioral sensitivity, response midpoint, and delay remained only weakly to partially identified. The clearest posterior structure was a negative association between transmission intensity and the behavioral midpoint, indicating that higher transmission could be compensated by behavioral responses activated at lower prevalence. Sensitivity to ordered behavioral priors further showed that reconstruction and behavioral inference depend materially on structural prior assumptions. These results suggest that adaptive mechanisms can improve multi-wave incidence reconstruction, while aggregate incidence alone is insufficient to sharply separate transmission from behavioral adaptation.
Read the full abstract

All authors Luis A. Barboza, Carlos Pasquier, Baltazar Espinoza, Fabio Sanchez