Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Wednesday 12 August

Statistics Applications · Cross-listed from cs.CV

New Orthogonal Multiwavelet Filters Derived by Matrix Spectral Factorization

The paper considers the construction of two new orthogonal multiwavelets with supercompact support by using the Fast Bauer's method for matrix spectral factorization on the matrix product filter of the orthogonal CL multiwavelet filter. The new multiwavelets possess orthogonality, symmetry/antisymmetry, and one of them provides better coding and smoothness than other supercompact multiwavelets. The performance of the new multiwavelet filters in subband-based edge detection, grayscale and color image compression and 1D and 2D signal denoising is compared with the GHM, SA4, CL, Integer Haar and Alpert multifilters. The comparative analysis shows that new multiwavelets can provides better human visual measures, SSIM and MS-SSIM in image compression and denoising applications.
Read the full abstract

Tuesday 11 August

Statistics Applications · Cross-listed from stat.ME

An Exact Noise Floor for Contingency-Table Effect Sizes: A Per-Table Reporting Gate, and How Often It Would Change Reported Magnitudes

Cramer's V and the other chi-square-family effect sizes are positive under exact independence, so a reported value can be smaller than what the same statistic produces on a table with no association at all. The upward bias is classical (Tschuprow 1925; Bartlett 1937; corrected by Bergsma 2013), and reporting the boundary value, the critical effect size, beside the observed one has been recommended in general form (Perugini et al. 2025); we claim neither. Our contribution is two things a general recommendation does not supply. First, an exact per-table reporting gate: the noise floor V_0.95, the value the effect size reaches by chance 5% of the time under independence, computed from the exact margin-conditional law rather than the asymptotic critical value. The asymptotic floor miscalibrates where reporting is riskiest: on heterogeneous tables its false-alarm rate reaches 0.10 against a 0.05 target, while the exact floor holds nominal. Second, a measurement: across 4,129 real two-way tables from 291 public datasets, 21.8% of labeled effects (cluster-robust 95% CI 17 to 27%) sit on tables with no significant association. We recommend reporting V_0.95, in exact form on sparse or heterogeneous tables, beside every association measure and at the design stage.
Read the full abstract

2609.26128 · 11 Aug · 27 pages, 6 figures. Under review at Journal of Applied Statistics. Reproducible code, data, and the derivation companion are openly archived on Zenodo: https://doi.org/10.5281/zenodo.21664321. Part of a contingency-table effect-size series (companions on the point estimate, the confidence interval, and the exact conditional distribution engine)

Statistics Applications

Does a Structural Model Add Anything to the Closing Price? Calibrated forecasting, incremental information, and match leverage in the Italian Serie A

Studies of association-football forecasting routinely report three-way accuracy in the low fifties and present it as competitive with the betting market. Accuracy against a uniform benchmark answers the wrong question; the question worth asking is whether a model carries information a margin-free closing price has not already absorbed. We formalise that test as the fitted weight in a logarithmic opinion pool and apply it to nineteen complete Serie A seasons (7,220 matches). The answer is negative and stable. A Dixon-Coles model with tuned exponential decay attains 53.4% accuracy and a Ranked Probability Score of 0.1972 against the market's 0.1905; the paired difference is +0.0067 (95% CI [0.0046, 0.0088]) and the market wins in all seven test seasons. The fitted pooling weight on the structural model is 0.000, and the log-loss profile is monotone increasing in that weight on validation and test alike, so this is a boundary solution, not an optimisation artefact. Refitting the same machinery to shots on target yields a variant earning weight 0.35 against the goals model – it carries information the goals model lacks – and 0.000 against the market. Two structural signals, each informative about the other, both priced. The structural model is better calibrated than the market on the home-win margin (slope 0.995 versus 1.103) while clearly less sharp: the market's advantage is discrimination rather than honesty, which accuracy alone cannot distinguish. Value lies not in a better forecast but in what is built on a calibrated one. We define match leverage, the change in a club's probability of achieving a season objective between winning and losing a fixture, and compute it for ACF Fiorentina: an away fixture against a relegation rival carried 2.25x the leverage of hosting the eventual champions. The paper also documents and corrects errors in an earlier study of our own.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Goodness-of-Fit Tests and Calibration Machine-Learning Algorithms for Logistic Regression with Sparse Data

Assessing the goodness-of-fit of a logistic regression model is a critical prerequisite before the model is used for inference. However, goodness-of-fit (GOF) tests such as the chi-square and deviance tests often give invalid results when the data are "sparse" – a common issue with continuous predictors like age or weight, where the asymptotic distributional assumptions are not satisfied. This thesis studies classical GOF tests for binary logistic regression under both grouped and sparse data, comparing about 30 statistical tests and machine-learning calibration algorithms. These span the classical chi-square and Hosmer-Lemeshow variants, standardized Pearson statistics, covariate-space partitioning, smoothing-based methods, and contemporary calibration machine-learning and bootstrap procedures. At a fixed size, the GiViTI calibration test (2016), McCullagh (1989), Osius-Rojek (1992), le Cessie (1995) and Stute-Zhu (2002) proved empirically powerful, balancing correct identification of bad models (high empirical power) against not raising false alarms on good models (correct empirical Type I error). Relying on formal methods alone is insufficient: visual diagnostics such as calibration plots are a vital exploratory step for detecting model deficiencies that formal tests often overlook. An application to real data (the Low Birth Weight dataset) shows that many of these tests fail to give valid conclusions when exposed to the complexities of actual datasets. The main conclusion is that model assessment requires a combination of several powerful statistical tests alongside careful visual inspection of model calibration.
Read the full abstract

Statistics Applications

Early Pregnancy Treatment Decisions: Designing Perinatal Pharmacoepidemiology Studies using Real-World Data

Research on the use of medications during pregnancy has two primary goals: to detect signals that medications may be harmful to a pregnant individual or fetus, and to support better treatment of pregnant people who require pharmacotherapy. Target trial emulation has been proposed as an approach to estimate the effects of interventions in real world data, with recent extensions to the pregnancy setting. This approach focuses on aligning eligibility and treatment initiation with start of follow up, which is particularly desirable given methodological challenges specific to pregnancy, such as right and left censoring and truncation, competing events, differences in gestational length, and varying etiologically susceptible periods. While previous work on target trial emulation in pregnancy has focused on initiation versus non-initiation of point treatments such as vaccines or antibiotics, this paper focuses on research questions regarding changes to pregestational treatment regimes, and aims to highlight opportunities and approaches to designing studies that align with relevant time points during early pregnancy at which treatment decisions occur in clinical practice. Using the example of treatment for type 2 diabetes mellitus, we review methods for identifying pregnancy episodes in routinely collected healthcare data, introduce possible time zero candidates, and discuss analytic approaches that minimize potential bias due to selection and immortal person time.
Read the full abstract

All authors Mollie E. Wood, Robert W. Platt, Jennifer A. Hutcheon, Jacqueline M. Cohen, Chase D. Latour, Andrea V. Margulis, Lucia C. Petito, Sonia M. Grandi

Statistics Applications · Cross-listed from cs.SI

A Graph Approach to the Academic Publishing Network: A Heterogeneous Model and Structural Screening over OpenAlex Open Data

The academic publishing ecosystem is a vast, heterogeneous network of works, authors, institutions, journals, and topics. Traditional scientometrics reduces it to isolated tabular indicators (h-index, Impact Factor) that ignore topological context and are not designed to capture coordinated illegitimate practices. Building on our companion review, which proposed graph analysis of publishing integrity, this paper implements that approach. We define a heterogeneous multivariate graph model over OpenAlex open data (seven node types, seven edge types) and a methodology based on projections (citation and co-authorship networks), interpretable structural metrics, community detection, and three screening detectors of anomalous publishing patterns. We deliberately avoid binary classification: detectors return ranked candidates with explicit structural evidence for human assessment. On the institutional corpus of VSB - Technical University of Ostrava (2020-2025) with its one-hop citation neighbourhood, community detection recovers real research groups, centralities identify cross-disciplinary bridges, and the screenings flag dense co-authorship cliques, locally closed citation loops, and thematically isolated venues. On a second, venue-centric corpus with external ground truth (journals delisted by Scopus and DOAJ) and size-matched controls, a naive case-control design yields seemingly strong but spurious detectors (a prominence confound), whereas after matching the only robust signal is the breadth of disciplinary scope (AUC 0.70); an open graph-based prestige measure (PageRank over the journal citation network) tracks a JIF proxy while being an order of magnitude more resistant to citation gaming than count-based indicators. We release the method as the open-source library apnet with a reproducible CLI workflow and a web interface; the analysis runs on commodity hardware in minutes.
Read the full abstract

Statistics Applications

Evaluating the Role of Blockage Deficit Models in Robust Wind Farm Design

Uncertainty in wind farm layout optimization regarding model choice and the impact of global blockage always exists. This paper evaluates these interactions by extending a multi-objective approach that maximizes mean Annual Energy Production (AEP) from a model ensemble while minimizing their variance. By incorporating the Self Similar blockage model into an ensemble of five wake models, we assess the impact of blockage physics on layout robustness. Results from a linear mixed-effects model indicate that including blockage leads to a non-significant average AEP reduction of 0.0624 GWh ($p$-value = 0.323). Conversely, it caused a highly statistically significant increase in uncertainty, with model disagreement rising by 0.177 GWh ($p$-value < 0.001). Additionally, computational runtime increased nearly nine times. These findings highlight an accuracy vs. certainty paradox, where theoretically necessary physics can compromise model consensus if implemented without re-calibration. Ultimately, this work suggests that simple blockage couplings act as a diagnostic for model incompatibility, emphasizing the necessity of careful model tuning for wind farm design.
Read the full abstract

2608.10630 · 11 Aug · This is an author-formatted, editorially revised version of the paper published in Journal of Physics: Conference Series, Vol. 3224, Wind farms and wakes (2026), https://doi.org/10.1088/1742-6596/3224/3/032072. The results and conclusions are unchanged. Changes include LaTeX reformatting and minor editorial/structural revisions. Please consider citing this refined preprint

Statistics Applications

Probabilistic forecasting via post-processing prediction errors: In- or out-of-sample?

Many forecasting systems produce point forecasts even when decisions require information about uncertainty. We investigate whether post-processing methods can systematically improve upon traditional Gaussian predictive distributions constructed from in-sample residuals. We propose a hybrid framework that combines forecast error post-processing with model-specific scaling of forecast uncertainty across horizons. For a comprehensive evaluation, we apply historical simulation, conformal prediction, quantile regression, and GARCH-based post-processing to point forecasts generated by Theta, exponential smoothing, and ARIMA models. Using 14,407 monthly series from the M4 competition and forecast horizons of 1 to 12 months, we evaluate performance using the continuous ranked probability score and rank-based statistical comparisons. Averaged across horizons, all post-processing variants improve upon the benchmark predictive distributions, with gains of up to 4.6%. In-sample calibration outperforms its out-of-sample counterpart in 11 of the 12 model-method combinations, although the preferred post-processing method depends on the base model and forecast horizon. The advantage of in-sample calibration generally increases at longer horizons. Our results show that organisations can extend existing point-forecasting systems to provide useful uncertainty quantification without computationally intensive repeated model re-estimation.
Read the full abstract

All authors Piotr Zaborowski, Arkadiusz Lipiecki, Fotios Petropoulos, Rafał Weron

Statistics Applications · Cross-listed from cs.LG

A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes

Fair representation learning with a continuous sensitive attribute $S$ requires a representation $Z$ that is statistically independent of $S$. Existing criteria, including generalized demographic parity, the expectation of integral probability metrics (EIPM), and mutual information, enforce this independence by averaging a per-value discrepancy between the conditional law $P_{Z \mid S=s}$ and the marginal $P_Z$ over the law of $S$. This approach requires a nonparametric surrogate for the conditional law at each sensitive value. We propose evaluating independence through a single joint discrepancy $d\left(P_{Z, S}, P_Z \otimes P_S\right)$ between the joint law and the product of its marginals. We establish a disintegration identity; on decomposable witness classes it equals the conditional-integral functional that EIPM and generalized demographic parity instantiate. By reaching the same target without the conditional law, this discrepancy can be estimated directly from samples via a dependence statistic rather than conditional smoothing. We take the Hilbert-Schmidt independence criterion (HSIC) as an instance of the joint discrepancy $d$ to investigate the statistical efficiency of replacing the conditional formulation. The HSIC estimator is a closed-form $O\left(n^2\right)$ statistic that converges at the $O\left(n^{-1 / 2}\right)$ rate, in contrast to the nonparametric $O\left(n^{-2 / 5}\right)$ rate of the conditional-route estimators. We prove this instance is equivalent to the conditional maximum mean discrepancy (MMD) integral up to an explicit spectral tail. The corresponding algorithmic implementation, i.e., FRHSIC, attains fairness-accuracy tradeoffs comparable to conditional-route basel es while reducing per-epoch training time.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Expert-Guided g-computation with Large Language Models for Estimating Causal Effects on Timings: Applications to Hospital Quality Improvement

Hospital quality improvement (QI) programs routinely face multiple candidate interventions to optimize hospital flow, but existing methods struggle to estimate and rank the causal effects of such interventions. This work focuses on one of the most standard hospital metrics, the average length of stay (LOS), and its causal estimand, the average time saved. To characterize this causal effect, qualitative approaches rely on expert judgment to map patient trajectories, making them susceptible to cognitive biases; quantitative approaches rely on data-driven models, which fail when interventions are hypothetical with no historical data or have complex causal mechanisms that require clinical reasoning rather than data alone. We propose expert-guided g-computation, or egg-computation, which combines the complementary strengths of both approaches by connecting the Gantt charts commonly used to map patient trajectories with the causal DAG literature. We introduce a causal model over Gantt charts and establish identification using a variant of g-computation that seeks expert input only for components unidentifiable from data. To make egg-computation practical, we develop an LLM-assisted pipeline that reliably scales up expert reasoning. In simulations, egg-computation outperforms conventional causal inference methods when patients have diverse causal structures and intervention mechanisms. In a study of eleven candidate QI interventions at an urban safety-net hospital, the LLM pipeline generated graphs and time-saving estimates highly concordant with those of human experts. Beyond healthcare, egg-computation is a broadly applicable framework for estimating the average time saved for candidate interventions whose causal mechanisms can be represented using Gantt charts.
Read the full abstract

All authors Patrick Vossler, Jialin Ouyang, F. Richard Guo, Anran Huang, Ali Shojaie, Lucas Zier, Fan Xia, Jean Feng

Monday 10 August

Statistics Applications · Cross-listed from stat.ME

Joint return levels of maximum temperature and minimum relative humidity by combining copulas with an extreme value framework for bimodal data

Climate change has become a growing concern, particularly in regions experiencing increasingly frequent extreme events. In Brasília, the capital of Brazil, significant shifts in climate patterns have drawn attention, including episodes of intense heat, unusually cold weather, and prolonged dry periods, often accompanied by wildfires. These phenomena directly affect the population and local ecosystems, requiring detailed analyses to understand their causes and effects. In this work, the return period for the joint distribution of maximum air temperature and minimum relative humidity in Brasília is determined using a copula-based approach. First, the dependence structure between the variables is modeled through a copula, while the marginal distributions are fitted using the novel methodology for modelling extreme values in complex systems. Different copula families, including rotated versions, were evaluated using information criteria, leading to the selection of a model that adequately captured the dependence structure, including asymmetry and tail behavior. The selected model also reproduced the multimodal pattern observed in the joint density, a feature consistent with the possible presence of multiple climate regimes. In the context of climate data modelling, the bivariate return level was determined using a conditional approach to assess the occurrence of extreme scenarios characterized by high temperatures combined with low relative humidity. This analysis makes it possible to quantify the expected frequency of such events, providing valuable information for environmental risk monitoring, the planning of preventive measures, and the development of climate change adaptation strategies in the region.
Read the full abstract

All authors Beatriz G da Cruz Albernaz, Cira E G Otiniano, Carolyne Soares de Brito, Fidel E C Morales, Enzo Porto Brasil

Statistics Applications

Estimating Media Mix Models with Demand-Marketing Interactions: A Constrained Genetic Algorithm Approach

This paper proposes a novel extension to Media Mix Modeling (MMM) that introduces a multiplicative interaction between marketing activity and underlying consumer demand. Unlike standard MMM frameworks that assume additive and independent effects of media and baseline demand drivers, our specification allows marketing effectiveness to vary with prevailing demand conditions. However, the proposed structure introduces significant statistical challenges, particularly identifiability issues that can lead to unstable and biased parameter estimates. Using comprehensive simulation studies, we analyze the nature and severity of these identification issues through bias assessment and their implications for statistical inference. To address these issues, we develop a constrained genetic algorithm optimization approach which facilitates robust estimation that simultaneously mitigates biases arising from the aforementioned issues. The proposed approach also offers flexibility to incorporate economically meaningful parameter constraints. Finally, the proposed methodology is applied to real-world data to demonstrate its effectiveness in enhancing estimation accuracy while taking into account additional commercial constraints, facilitating budget allocation decisions.
Read the full abstract

All authors J. S. T. Wong, G. Hughes, Y. Bao, H. Dai, V. Giagos, H.M. Fernanda, H.O. Bakan, K. Passmore

Statistics Applications

Estimating Epidemic Rate Parameters: Adaptivity, Bias, and Convolution

Metrics like the case-fatality rate and reproduction number are key descriptors of epidemics from the COVID-19 pandemic to the seasonal flu. In retrospect, these quantities enrich our understanding of infectious disease outbreaks; in real-time, they are absolutely critical to informing public health response. Thus, an important question in epidemiology is how best to estimate such metrics, especially in real-time. This question is complicated by practical considerations like data availability, as well as the fact that the metrics themselves may change as the epidemic unfolds.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Covariate Informed Identification of Heterogeneity and Outliers in Longitudinal Data

We often observe heterogeneity in longitudinal data, where the mean and variance for certain profiles meaningfully differ from the rest. Some profiles may also exhibit outliers at a limited number of measurements. Using a standard mixed effects model, which assumes homogeneity, can lead to overestimating the residual variance and inefficient estimation. In this work, we identify and account for three sources of heterogeneity in longitudinal data: incompatible mean trajectories, increased residual variance, and outliers at individual measurements. Our Bayesian mixture model incorporates binary indicators of heterogeneity for each of these features, modeled through logistic regression using covariates. We perform statistical inference using Markov chain Monte Carlo and implement model selection to evaluate the inclusion of various heterogeneous components. Simulations demonstrate that our model can accurately identify heterogeneity and produce efficient estimates of the fixed effects parameters. We further validate our approach using the DHEAS hormone data from the SWAN study.
Read the full abstract

All authors Anish Mukherjee, Jeremy T. Gaskins, Jeffrey M. Albert, Tanujit Dey, Sounak Chakraborty

Statistics Applications

Status Association Does Not Reliably Predict Decision Leakage

Bias evaluations often move too quickly from evidence that a model encodes a social association to claims that the same association will alter consequential decisions. We test whether that inference is warranted using Chilean surnames as controlled socioeconomic probes. We evaluate eight frozen model-provider cells on 1,032 prompts each, yielding 8,256 verified primary responses. The design separates forced latent association from matched consequential decisions across academic selection, professional hiring, research fellowship selection, and legal-aid intake. Elite-coded surnames received higher forced high-status probability mass than common surnames in seven of eight models and higher mass than rare-frequency controls in all eight. Yet elite-minus-common decision effects were close to zero for most systems. Five models were statistically equivalent within a predeclared (Plus-Minus)0.10 standard-deviation margin, while the remaining three were imprecise or borderline, with no consistent elite advantage. Association strength did not reliably predict decision leakage across models (r = 0.201, p = 0.633) or across frozen surname-pair-by-model cells (r = 0.065, p = 0.565). The central result is a measurement dissociation: latent social association and consequential treatment are empirically distinct constructs. Evaluations should measure the transition from association to action directly.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

The Earth Moves, But So Does the Bias: Systematic Upward Bias of the Wasserstein (Earth Mover's) Distance and Permutation-Based Null Calibration

The Earth Mover's Distance (EMD) is gaining increasing interest among political scientists for assessing similarity in preference distributions. However, there remains a risk of finite-sample upward bias induced by sampling variation in empirical probability measures, which is under-recognized by existing studies. This problem is especially severe in high-dimensional or sparse settings, including conjoint distributions that serve as an illustrative example in this paper. As political scientists are broadening their use cases of EMD, this paper cautions against interpreting standard bootstrap uncertainty bounds as a correction for the upward bias of empirical EMD. It proposes a permutation-based null calibration framework for more robust hypothesis testing. As a non-parametric approach, it frees researchers from making directional or distributional shape assumptions. While alternative estimators require these rigid assumptions to correct for upward bias, political science data often fail to meet them in practice. Through four sets of Monte Carlo simulations, this paper demonstrates the utility of this framework. The proposed approach also applies more generally to empirical comparisons of two probability distributions defined on a common metric space, provided that the ground distance between support points is substantively meaningful.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Clustering Informed Inverse Probability Weighting Strategies for Causal Effect Estimation in Observational Studies

Inverse probability weighting (IPW) is widely used to estimate causal effects in observational studies but depends on adequate propensity-score specification. We compare three strategies for addressing treatment assignment heterogeneity: standard IPW, clustering augmented IPW with cluster specific propensity score models, and a global propensity score model including estimated cluster membership as a covariate. Through simulations with and without latent cluster structure and under correctly specified and omitted covariate propensity score models, we evaluate bias, mean squared error (MSE), and confidence interval coverage across sample sizes of 100 to 500. Both cluster informed strategies reduced bias and MSE from omitted covariate misspecification relative to standard IPW, but neither uniformly dominated: clustering augmented IPW achieved lower MSE when latent cluster structure was present, whereas the global model generally provided lower bias and better coverage at smaller sample sizes. We also apply the methods to 966 breast cancer patients treated with carboplatin, using generalized propensity scores to estimate the dose response relationship between treatment cycles and hypersensitivity reaction risk. Standard and clustered analyses produced similar pooled estimates, while clustering additionally provided subgroup specific estimates and diagnostic profiles. Overall, cluster informed strategies may improve robustness to propensity score misspecification, with relative performance depending on subgroup structure, sample size, and inferential priorities.
Read the full abstract

All authors Ruohui Chen, Scott Zuo, Whitney Stevens, Seth Pollack, Wenna Xi, Lucia Petito, Lihui Zhao, Hui Zhang

Statistics Applications

Longitudinal Bayesian networks for assessing team performance in the National Basketball Association

Assessing the performance of a basketball team requires the consideration of multiple sources of information. In recent years, the volume and the quality of data generated in sport has increased considerably, particularly in basketball. In this work, we propose a Bayesian graphical modelling framework for the longitudinal analysis of basketball team performance. The framework is based on Bayesian networks that explicitly represent the nodes, that is, the random variables of interest, as longitudinal stochastic processes. We propose three baseline longitudinal models: a static Bayesian network, a dynamic Bayesian network with an autoregressive structure between successive games, and a dynamic Bayesian network based on a hidden Markov structure. We illustrate the proposed framework through a real-world sports analytics case study involving the Philadelphia 76ers of the National Basketball Association (NBA) during the 2005–06 season. The analysis includes player participation, minutes played, fouls drawn, and one-, two-, and three-point shots attempted and made.
Read the full abstract

All authors Gabriel Calvo, Francisco Palmí-Perales, Carmen Armero, Virgilio Gómez-Rubio

Statistics Applications

Bayesian Node Edge Modeling of Road Crashes in Central Bogotá

Introduction: Crash counts on road segments and intersections exhibit differ- ent exposure and connectivity patterns that conventional analyses may obscure. Methodology: A Bayesian negative binomial node edge model was fitted to 8,169 road segments and 8,398 intersections in six central districts of Bogotá. Separate predictors represented road hierarchy, pavement, speed, signalization, intersection configuration, and land-use treatment. Model performance was examined through pre- dictive summaries and spatial diagnostics, while computational details are reported in the appendix. Results: Intersections with at least four incident segments and higher maximum incident speeds had higher expected crash counts. Land use treatment and signalized access intensity also showed posterior associations. Road hierarchy and signalization were the clearest segment-level factors; however, this component had weak raw scale predictive performance and numerical uncertainty for some pavement categories. Conclusion: Treating intersections and segments as distinct network elements provides an interpretable baseline for urban crash analysis, but the segment results require cautious interpretation and motivate spatially structured extensions.
Read the full abstract

Statistics Applications

Comparing Tobit and Two-Part Hurdle Models for Semi-Continuous Longitudinal Data with an Application to Clonal Hematopoiesis

Zero-inflated nonnegative continuous longitudinal data frequently arise in biomedical studies where outcomes consist of a mixture of excess zeros and positive continuous measurements. Two widely used approaches for analyzing such data are mixed-model versions of Tobit and the two-part hurdle models. The Tobit assumes a latent regression model that is censored below a specified threshold, while the hurdle separately models the continuous positive outcomes and the binary indicator of being positive. The choice between these models has rarely been systematically discussed, and inappropriate model choice may lead to biased estimation and misleading scientific interpretations. In this paper, we derive rigorous mathematical conditions under which the two models are equivalent and show that the Tobit can be viewed as a special case of the hurdle model when the link function for the binary process is probit. Based on simulation studies, we found that the hurdle is more flexible and robust than the Tobit model. On the other hand, if the assumptions of the Tobit are met, this model is easier to interpret since it does not require distinct inferences on both the continuous and binary processes. We therefore recommend that the Tobit model only be used when these assumptions are scientifically plausible and empirically supported; otherwise, the hurdle model is preferable. We applied both models to study the dynamics of somatic mosaicism using longitudinal clonal fraction measurements from the Prostate, Lung, Colorectal, and Ovarian study data while accounting for excess zero values. Estimates obtained from the Tobit and hurdle models were broadly consistent with those from a standard linear model that ignored zero inflation. These findings provide additional support for previously reported associations in studies of clonal hematopoiesis across different types of mosaic chromosomal alterations.
Read the full abstract

All authors Sumaja Bandreddi, Pei Zhang, Rebecca L. Kelly, Mitchell J. Machiela, Paul S. Albert

Statistics Applications

Amplitude-Phase Analysis of the COVID-19 Point Process and the Early Countermeasures

We investigate how governmental restrictions relate to the spread and temporal dynamics of COVID-19 early in the pandemic. We model daily infection data from each US state as realisations of a point process, taking the random intensity measure to be the latent object of interest and, crucially, allowing these realisations to vary not only in magnitude but also in the temporal dynamics. By non-parametrically separating these amplitude and phase variations, we examine how government restrictions relate to each source of variability, relating the infection curves to the Oxford Stringency Index, which we treat as a measure on the same time window. Employing Wasserstein PCA, we analyse the temporal variability of both the infections and the restrictions. We then use the resulting scores, together with the scalars representing the overall stringency budget and the total infection count, as inputs to a linear vector-on-vector regression model. Our findings suggest that, when considering the separate contributions of amplitude and phase variability, earlier implementation of restrictions is associated with flatter infection curves. By contrast, we do not find significant evidence of an association between stringency and total infection counts, nor between the overall stringency budget and the infection outcomes considered.
Read the full abstract

Statistics Applications

Beyond Zipf's Law: Equifinality and Mechanistic Inference from Scaling Laws

Scaling laws summarize complex systems through low-dimensional regularities, but the same marginal law can arise from different stochastic dynamics. We examine this ambiguity for Zipf rank–frequency scaling. An i.i.d. finite-Zipf process, a persistent Markov chain, and canonical sample-space reduction (SSR) are constructed to have exactly the same stationary marginal, $p_j=(jH_V)^{-1}$, while a latent-scale mixture produces a similar marginal through aggregation. The first three models therefore hold the population rank distribution fixed while changing temporal organization. Markov dependence shifts the finite-sample distribution of fitted exponents; at moderate persistence, a block-adjusted effective sample size reproduces most of this shift. Excess lag-1 mutual information detects serial dependence relative to a shuffle null, whereas transition direction separates reversible Markov persistence from the directional contraction of SSR. Conditioning on latent scale reveals the aggregation mechanism. Fit-window, alphabet, and sequence-boundary analyses quantify sensitivity to the observation design. These examples separate the population marginal, the finite-sample behavior of a fitted exponent, and temporal structure. Matching a scaling law is therefore a compatibility condition, not a mechanism identifier: discrimination requires observables on which candidate processes make different predictions.
Read the full abstract

Statistics Applications

From Risk Prediction to Risk Mechanisms: A Multi-Resolution Causal Representation for Road Safety and Motor Insurance

Operational risk models can estimate event frequencies precisely while leaving the underlying mechanisms weakly resolved. We study this resolution mismatch using motor insurance and road safety. A revisable DAG represents trip-level crash generation; a separate predictive layer links annual rating information to latent driving states; and an observation process maps crashes into recorded liability claims. Compatibility sets collect the structural and crash-to-claim laws that reproduce an observed annual contrast under stated restrictions. External studies enter only through explicit bridge assumptions and sensitivity bounds. The framework therefore distinguishes sampling uncertainty from uncertainty about structure, observation, and study-to-target correspondence. Two limited examples illustrate the gap. A sublinear mileage relation constrains an aggregate accident rate per unit distance but not its mechanism. In French motor-liability data, the 18-20 versus 40-49 claim-frequency relativity is 3.388 in a model including vehicle and geographic variables and 1.235 when the same model also conditions on a medium-resolution bonus-malus score. These are different predictive functionals, not successive causal adjustments. A Spanish culpability estimate is used only to show how a strong cross-study bridge would restrict a toy bookkeeping region. The framework does not estimate the full DAG; it makes explicit which assumptions are needed before an annual predictive contrast can support a mechanism-specific risk statement.
Read the full abstract

Statistics Applications · Cross-listed from cs.CV

Deep Learning based Detection of Fishing Vessels and Fishing Monitoring using Nightlight Images

The demand for maritime surveillance has given rise to the need for monitoring fishing vessel activities, particularly in addressing the challenge of "dark vessels" that operate without Automatic Identification System (AIS) transmission. This study presents a novel approach for detecting small-scale fishing vessels using nighttime light (NTL) imagery from the SDGSAT-1 satellite, combined with deep learning techniques to enhance fishing monitoring awareness along the western coast of India. A dual-branch YOLO11 architecture was developed to exploit both the 10-meter panchromatic and 40-meter RGB imagery from SDGSAT-1. The custom model architecture was specifically optimized for small object detection in NTL imagery, featuring parallel convolutional backbones that process both modalities before concatenation for enhanced feature extraction. The dual-branch YOLO11 model demonstrated optimal performance with a precision of 0.99, recall of 0.93, F1-score of 0.96, and mAP@50 of 0.96, significantly outperforming single-branch implementations of YOLOv5s, YOLOv8s, and standard YOLO11s architectures. When applied to the western coast of India, the model detected 31525 vessel instances across the temporal dataset spanning 2022-23. Cross-matching analysis with AIS data revealed that only 7146 (22.7%) of detected vessels had corresponding AIS transmissions, while 24379 (77.3%) were identified as potential dark vessels. Spatio-temporal analysis showed peak fishing activity during January-April, with a primary activity corridor parallel to the coastline within 50-100 km, corresponding to productive continental shelf areas. This research contributes to maritime surveillance capabilities by highlighting the effectiveness of nighttime lights satellite imagery for fishing vessel detection and provides valuable insights into fishing patterns and potential regulatory compliance issues in Indian waters.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Bayesian ACCESS for Understanding Latent Epidemic Trajectories from Publicly Released Suppressed Data: Application to U.S. Opioid-related Overdose Mortality

Publicly released health statistics play a central role in characterizing temporal trends and identifying structural changes in population health. However, disclosure limitation through suppression of small cell counts, as implemented in systems such as the Centers for Disease Control and Prevention Wide-ranging ONline Data for Epidemiologic Research (CDC WONDER), produces partially observed count data that complicate statistical inference. These challenges are particularly acute for rare outcomes and subgroup analyses, where suppression is widespread and varies across geographic regions, demographic populations, and time. We propose Bayesian ACCESS (Autoregressive Change-point and Clustering Estimation for Suppressed Count Series), a Bayesian hierarchical framework for inference on latent epidemic trajectories and their structural changes from disclosure-limited health statistics. The proposed model directly represents suppressed count data through a suppression-aware observation model, jointly infers multiple temporal change points and latent trajectories, and borrows information across related geographic and demographic populations through Bayesian nonparametric clustering while preserving meaningful heterogeneity. We apply Bayesian ACCESS to opioid-related overdose mortality data from CDC WONDER for U.S. states from 1999 to 2024. The analysis identifies distinct subgroup-specific epidemic trajectories and structural changes that would be difficult to characterize using publicly released health statistics without explicitly accounting for data suppression.
Read the full abstract

Statistics Applications

Extreme Value Alpha and Crash Risk: Separating Structural Tails from Lottery Tails with LLM-Extracted Disclosure Networks

A heavy upper tail in a stock's returns is ambiguous: it can be a lottery tail, transient jump risk that investors overpay for (the MAX discount), or a structural tail, the statistical shadow of an economic reconfiguration that precedes extreme winners. Returns alone cannot separate them, so tail heat alone is not an alpha signal. Our discriminator is the firm's disclosure-measured network: a directed, span-grounded graph from 10-K filings via an auditable LLM pipeline, whose rewiring decomposes into edge birth, death, and drift. The central sign pattern: tail heat with network death is the crash side; tail heat with an intact or forming network is where structural tails and historical winners live. Pilot evidence from 24 technology firms (2014-2025) supports the crash side: upper-tail heat interacted with death mass predicts negative forward abnormal returns (monthly t = -2.9; firm-vintage t = -3.9; wild-cluster p = 0.04; robust to two-way clustering and controls), and the same configuration preceded NVIDIA's 2018 and 2022 drawdowns. This death-side signal is immediately useful as a risk-monitoring danger flag. The alpha side is directionally positive but not significant in the pilot and awaits the confirmatory test. A pre-registered replication on 50 random S&P 500 firms failed, defining the boundary: outside coherent ecosystems the disclosure graph nearly vanishes (83% of firm-vintages have zero death mass), so the discriminator exists only where firms densely document counterparties. The confirmatory design is fully pre-specified, with a gatekept primary pair, archived power simulations, positive-only winner labels, selection-corrected benchmarks, and a frozen ecosystem-coherent universe with a density gate; it activates only if the gate passes. If confirmed, tail heat becomes a conditional signal separating crash risk from structural winners.
Read the full abstract

Statistics Applications

When Can Text Embeddings Replace Item Calibration? A Geometric Diagnostic for Semantic Loadings in Multidimensional Adaptive Testing

Multidimensional item response theory relies on calibrated item parameters, such as discrimination and category threshold values, which are usually estimated from large samples of human test responses. This study investigates whether the directional loadings of these parameters can be recovered directly from item text using pre-trained sentence embeddings, avoiding the need for initial item calibration. Using the open-source IPIP Big-Five dataset ($n=19{,}719$; 50 items), we built a multidimensional computerized adaptive testing (CAT) simulation using D-optimal item selection. We compared three item loading sources: fitted graded response model parameters, semantic text embeddings, and a lexical baseline. In simulation, semantic embeddings recovered latent trait profiles almost as accurately as fitted parameters (correlation $0.825$ vs. $0.857$), performing noticeably better than simple word overlap ($0.752$). However, the embedding-based model produced inflated posterior variance, showing nearly four times higher measurement uncertainty despite accurate point estimates. We attribute this to collinearity across dimensions, as embedding-derived loadings pointed in similar directions across traits (condition number $137$ vs. $1.0$; mean trait cosine $0.90$). This outcome reflects the shared vocabulary common in personality items. We propose a simple diagnostic metric based on the loading matrix condition number to evaluate whether an item bank is suitable for text-derived loadings prior to testing.
Read the full abstract

Statistics Applications

Forecasting in the Fog: Real-Time versus Revised-Data Evidence on Machine Learning's Edge over the Phillips Curve

ML inflation forecasts are almost universally trained on fully revised data, even though real-time forecasters never have such data, and reported feature importances are typically computed in-sample, conflating predictive relevance with retrospective fit. This paper asks whether the ML advantage over the Phillips curve documented in Agyekum (2026) survives when models are trained and evaluated on real-time (ALFRED) vintages rather than revised series, and whether SHAP feature-importance rankings are an artifact of in-sample estimation. Using 2000-2026 U.S. data on unemployment, CPI and PCE inflation, payrolls, real GDP, and the 10-year-2-year Treasury spread, vintage-consistent panels are built for four traditional models (random walk, AR(1), Phillips curve, ADL-OLS) and four ML models (Random Forest, Gradient Boosting, Elastic Net, SVR), re-estimated recursively at 3-, 6-, and 12-month horizons (208, 206, 204 forecasts). Real-time/revised accuracy differences are small and, apart from one exception at 6 months (Gradient Boosting vs. Phillips curve, DM = -1.671, p = 0.097), indistinguishable under Diebold-Mariano tests; Gradient Boosting alone shows consistent positive skill at longer horizons. The random walk remains a strong short-horizon benchmark, consistent with the puzzle in Agyekum et al. (2026) for exchange rates. Using walk-forward, out-of-sample SHAP, a Random Forest on revised data assigns dominant importance to PCE inflation (mean |SHAP| = 0.778, rank 1 of 9), while on real-time data it assigns PCE negligible importance (0.039, rank 6), relying instead on current CPI (0.834 vs. 0.223). This twenty-fold swing, larger than the in-sample estimate, is invisible to point-forecast metrics and shows the model's PCE reliance is substantially a hindsight artifact. An RSI summarizes the accuracy gap by model and horizon, with implications for auditing ML inflation forecasts.
Read the full abstract

Sunday 9 August

Statistics Applications · Cross-listed from physics.ao-ph

Do AI Forecast Ensembles Sample the Correct Conditional Distribution?

Ensemble forecasting aims to sample the conditional distribution of outcomes; whether AI forecast ensembles do this correctly in a joint sense remains largely untested. We train a diffusion model for probabilistic subseasonal coastal sea level forecasts at eight US East Coast tide gauge stations, with sea level derived from reanalysis, and find that marginal and joint forecast quality decouple: positive skill at every station and lead time marginally, while joint spatial structure is worse than climatological draws. A shuffle-based permutation decomposition reveals this failure is invisible to the energy score but detected by the variogram score. Lorenz-96 experiments across 0.7-170 equivalent years show the gap persists regardless of training volume and is reproduced by a linear baseline, indicating structural inadequacy of the learned distribution. A dynamical ensemble does not replicate the failure while a deterministic emulator does, suggesting it is specific to learned emulators rather than ensemble forecasting generally.
Read the full abstract

Statistics Applications

Modeling Disease Transmission Dynamics with Population-Weighted Spatial Inference for Wastewater-Based Epidemiology

Wastewater-based epidemiology (WBE) is an effective, noninvasive tool for tracking community-level circulation of respiratory viruses, yet standard renewal models treat each wastewater treatment plant (WWTP) in isolation, even though respiratory syncytial virus (RSV) spreads through contact networks that cross service-area boundaries. Using weekly data from $m = 15$ Houston WWTPs, we extend a single-plant Bayesian renewal model by coupling neighboring WWTPs via a spatial weight matrix built using a novel Population Extended Hausdorff Distance, defined as a directional distance between WWTPs. This distance measures how far a WWTP must reach to access a meaningful share of a neighboring WWTP's residents. A single parameter $\rho$ controls spatial mixing in the plant-level growth rate, allowing us to recover the effective reproduction number $R_{it}$. We find that $\rho$ is estimated well above zero, indicating substantial cross-WWTP transmission that the independent-plant model cannot capture. Spatial coupling reshapes plant-level $R_{it}$ relative to the independent-plant baseline ($\rho = 0$) while leaving inferred infection trajectories essentially unchanged.
Read the full abstract

All authors Jose R. Palacio, Katherine B. Ensor, Julia Schedler, Rebecca Schneider, Kaavya Domakonda, Loren Hopkins, Lauren B. Stadler, Sharmila Bhandari