Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Monday 31 August

Statistics Applications

A Tool for Reconstructing Transit Vehicle Trajectories: A Case Study at IndyGo

Automatic vehicle location (AVL) data produced by transit vehicles is invaluable in performance studies, but turning raw AVL points into a detailed view of vehicle stop-and-gos is burdensome: the datasets are sparse, noisy, and prone to blunders. While recent research has explored methods of reconstructing trajectories describing the position of vehicles over time, the common techniques can be complex, and no open-source tools exist to help practitioners process the raw AVL. We fill this gap by proposing a thorough methodology for cleaning transit AVL data and providing an open-source R package, built on open data standards, to implement the workflow and reconstruct vehicle trajectories, allowing practitioners to easily formulate custom microscopic performance metrics. Using a large AVL dataset with over 3,000 trips from Indianapolis, Indiana, we demonstrate the workflow, evaluate the package's efficiency, and demonstrate the utility of the trajectories by estimating various traffic signal performance metrics. Finally, we use cross-validation to quantify the error in reconstructed trajectories at various polling frequencies. We find that the proposed data processing methodology and tool are efficient, requiring roughly 3 minutes of processing time on the large dataset, and error in position estimates is low (root mean square error under 10 meters at polling frequencies of 15 seconds). The estimated signal performance metrics can inform future evaluations of signal programming along the corridor. In sum, this paper serves as a practical guide and complementary toolbox for practitioners seeking to use transit AVL data to build a detailed view of vehicle stop-and-go cycles.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Selection-Aware Stress Testing for Interactive Agents

Agent evaluations often use one benchmark to choose a workflow and then search for task types where its advantage weakens, so both conclusions are selected from the same data. We introduce Selection-Aware Semantic Stress Testing (\SASST{}), which learns a task reweighting from pre-execution features on discovery tasks and evaluates the same paired comparison on separate confirmation tasks. The protocol checks support and stability, uses joint bounds for all planned claims, and can return no claim. We prove conditional asymptotic validity under stated cluster assumptions. A forty-cluster audit finds Gaussian undercoverage and conservative Bonferroni $t$ bounds. In one 480-episode $\tau$-bench study, a $3.75$ point discovery gain vanished on confirmation. A second-model study likewise confirmed neither a workflow benefit nor a stable stress rule.
Read the full abstract

All authors Yang Xu, Chenang Li, Jiefu Zhang, Haixiang Sun, Zhou Li, Vaneet Aggarwal

Statistics Applications · Cross-listed from astro-ph.SR

Extremes of solar spectral irradiance in the SORCE/XPS record

Extreme and rare changes in space mission solar irradiance records are scientifically relevant but difficult to quantify because these records are finite, instrument dependent, and affected by observational gaps and time varying measurement quality. We evaluated extreme daily logarithmic changes in the band integrated 0.1-7.0 nm irradiance measured by photodiode 7 of the Solar Radiation and Climate Experiment/X-Ray Photometer System (SORCE/XPS) from 2005 to 2019. After constructing a regular daily series by linear interpolation, we analyzed daily logarithmic changes in irradiance and fitted stationary Generalized Extreme Value models to 60 day block maxima and transformed 15 day block minima. Block lengths were selected using Ljung-Box diagnostics and sample autocorrelation functions. Maximum likelihood estimation was used as the primary inferential method, with probability weighted moments as a sensitivity check. The maximum likelihood GEV shape estimates were 0.1661 for maxima and 0.2928 for transformed minima, with closely aligned estimates under the two fitting methods. These positive point estimates are compatible with Fréchet type tails under the selected block constructions. Annualized return levels provide interpretable summaries of extreme relative increases and reductions, but estimates for long return periods remain strongly dependent on extrapolation beyond the 15 year record. Reported measurement precision and absolute uncertainty were used to qualify the interpretation of the fitted tails and were not propagated through the likelihood. By combining EVT based tail modeling with explicit consideration of measurement precision, absolute uncertainty, interpolation, and mission data gaps, the analysis provides an uncertainty aware astrostatistical baseline for extreme value inference from processed solar mission records.
Read the full abstract

All authors Enzo Brasil, Cira E. G. Otiniano, Carolyne Brito, Beatriz Albernaz, Fidel Morales

Statistics Applications

Long-Memory Estimation and Fractionally Integrated Modeling of White Maize Prices in Togo

Agricultural commodity prices often exhibit strong temporal persistence, which may limit the performance of conventional time series models. This study investigates long memory in logarithmic monthly white maize prices from six major markets in Togo between January 2001 and June 2022. Long memory is examined using the Geweke–Porter–Hudak, Local Whittle, Exact Local Whittle, and wavelet log-regression estimators. SARIMA, ARFIMA, and SARFIMA models are subsequently compared using the Bayesian Information Criterion and residual diagnostics. Long-range dependence is found across all markets. Fractionally integrated models provide the best fit for most markets, although SARIMA remains preferable for some. The results demonstrate that evidence of long memory does not necessarily imply that a fractionally integrated model provides the best empirical fit, emphasizing the importance of data-driven model selection.
Read the full abstract

Sunday 30 August

Statistics Applications · Cross-listed from stat.ME

Robust K-means Clustering using the Density Power Divergence Measure

We introduce a robust clustering method, MK-means DPD, that estimates cluster centers and covariance matrices using density power divergence (DPD) measures combined with Mahalanobis distance, making it resistant to outliers and adaptable to heterogeneous, elliptical clusters, unlike the classical K-means algorithm. Since Mahalanobis distance-based K-means lacks a general convergence guarantee, we further introduce a convergent variant, Density-Consistent MK-means DPD (DC-MK-means DPD), which redefines the cluster assignment step in terms of a pointwise DPD loss. We prove a formal theorem establishing that the resulting algorithm converges in a finite number of steps. We also propose two new robust internal evaluation indices, a Median Davies-Bouldin Index and a Trimmed Calinski-Harabasz Index, to ensure that performance comparisons are not themselves distorted by outliers. The efficacy of the proposed methods is demonstrated on simulated data, showing superiority over existing methods, and on two real datasets: Iris data, to identify similar species, and COVID-19 case fatality rate and infection rate data for countries worldwide, examining the resulting clusters' geographic and socio-economic patterns.
Read the full abstract

Statistics Applications

Nonlinear trajectories of lung function recovery in patients with pulmonary disease: empirical evaluation of longitudinal modeling approaches

Introduction: Longitudinal lung function recovery after pulmonary disease commonly follows nonlinear trajectories, and failure to adequately model these trajectories can lead to biased or misleading estimates of treatment effects. However, an important methodological gap remains as there is limited assessment of statistical methods for modeling nonlinear lung function trajectories. Methods: We compared several longitudinal modeling approaches for characterizing recovery in percent predicted forced expiratory volume in one second (FEV1p) using data from a phase 2 randomized trial for pulmonary tuberculosis (TB). We estimate the differences in repeated mean FEV1p between each treatment arm and control arm over a 180-days follow-up period. We compared 6 different statistical models: (1) linear mixed-effects model, (2) a piecewise linear mixed-effects model, (3) quadratic and (4) natural cubic spline mixed-effects models, (5) a nonlinear mixed-effects model with exponential recovery function, and (6) a generalized additive mixed model. We discussed the assumptions, clinical interpretations, and resulting treatment-effect estimates across these approaches. Results: The results from the TB trial analyses showed that the conventional linear mixed-effects model provided limited evidence of treatment differences over follow-up, whereas several flexible models identified significant differences during specific periods of recovery. Conclusion: Flexible longitudinal models can complement conventional linear mixed-effects models by revealing treatment differences at certain periods of follow-up that may be obscured by assuming a single linear trend. The choice of nonlinear modeling strategy should be guided by the scientific objective, available data, and the desired balance between clinical interpretability and flexibility.
Read the full abstract

All authors Zongyue Teng, Ningkun Zhou, Xinyu Zhang, Robert Wallis, Qingyan Xiang

Statistics Applications · Cross-listed from stat.ME

Design of Experiment in Complex Systems based on Computational Taxonomy

Via Computational Taxonomy (CT), we develop Design of Experiment(DoE) based on rigorously redefined constituting ingredients of complex system dynamics: randomness, nonlinearity and even class, through a data-driven constructed Taxonomic Hierarchy. As an opposite quest of Classification without man-made assumptions and structures, we illustrate this new perspective of DoE through a civil engineering complex system: Concrete Compressive Strength (CCS). CT begins by building a Taxonomic Hierarchy as a heterogeneity-vs-homogeneity map framed with a tree geometry to represent CCS-system dynamics. At each internal node of this hierarchy, a heatmap is computed via Scientific Data Analysis (SDA) to reveal locality-embraced heterogeneity through block structured covariate homogeneity annotated with response's locality-split. Only arriving at each ending-node of this hierarchy, coherence of homogeneity is achieved on both response and covariate sides. As such a class of finite sample nature is computationally recognized and confirmed. In contrast, nonlinearity is evidently observed as incoherence of response-vs-covariate homogeneity when comparing two classes located on two distinct branches. This hierarchy explicitly maps out system's randomness and nonlinearity to serve as a scientific basis for any DoE quest. Design of Experiment (DoE) for any designated class is redefined as a search for a covariate subspace that embraces the class-representative randomness and at the same time avoids potential nonlinearities with respect to the rest of classes. This is a brand-new theme of DoE.
Read the full abstract

Statistics Applications

Decarbonising price formation: unit-level evidence on battery storage and the imbalance price in the GB Balancing Mechanism

Renewables now dominate Great Britain's generation mix but rarely occupy the marginal price-setting position, which raises the question of which flexible technologies translate a renewable-rich system into real-time price formation. This study reconstructs the price-ranked edge of the eligible bid or offer stack in the GB Balancing Mechanism across 50,684 half-hourly Settlement Periods from 2023 to 2025 and attributes it to individual Balancing Mechanism Units, separating long-system bid-active from short-system offer-active conditions and retaining co-marginal ties. Batteries rose from 0.8% to 36.2% of bid-active and from 3.2% to 26.9% of offer-active marginality, displacing combined-cycle gas on the bid side and pumped storage on the offer side. Capacity-normalised marginal capture rose on both sides, and unit-quarter fixed effects show that an additional 100 MW of Balancing Mechanism active capacity was associated with a 0.55 and 0.48 percentage-point increase in quarterly bid- and offer-side marginal share. By 2025, battery actions were around {\pounds}9 to 10/MWh more favourable than volume-matched non-battery alternatives drawn from the same ladder, yet batteries were under-represented by 12.9 percentage points in the highest-priced 5% of short-system periods. Individual batteries remain consistent with price-taking, while the fleet has become endogenous to routine balancing price formation.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

A Unified Approach to Interpretable Causal Root Cause Attribution

Understanding why a target metric changes is a fundamental problem in data-driven decision making, beyond anomaly detection alone. We study root cause attribution for metric changes in complex e-commerce systems, focusing on trade-offs between interpretability, efficiency, and causal validity. As a starting point, we extend a metric-decomposition method into a recursive metric-tree framework for multi-level root cause analysis, but this relies on independence and decomposability assumptions that miss complex causal dependencies. In contrast, graphical causal models (GCMs) relax these assumptions and improve causal validity, at the cost of interpretability, higher computational and data demands, and potential attribution target misalignment. Through real-world applications, mathematical proofs, and simulations, we characterize the fundamental sources of these trade-offs. Guided by these insights, we propose a unified, causally informed attribution approach that integrates structural causal information into the metric-tree decomposition framework and corrects key sources of misalignment in GCM-based causal attributions, substantially improving causal validity while preserving interpretability and fast computation. Analytical proofs and simulations demonstrate that the proposed approach produces more accurate root cause attributions, and we also present a real-world application.
Read the full abstract

All authors Jing Zhou, Dominik Janzing, Sepp Tsang, Patrick Blöbaum, Marco Visentini Scarzanella

Statistics Applications

Text Data Analysis and Classification Methods - Insights from Customer Letters in Life Insurance

The business of life insurance companies is characterized by long-term contracts. For this reason, data describing customers is of immense value. A portion of the data provided to the customer is rarely or not at all analyzed. This includes customer letters of any kind. This work focuses on classifying customer letters as cancellations and identifying the respective reason, if available. The outlined approach can also be applied to other business transactions and reasons. We discuss data acquisition and preparation, present alternatives, and explain the reasons for the chosen approach. A successful implementation of such a tool can lead to a better understanding of customer cancellation behavior by the insurer, enabling more targeted actions in certain situations.
Read the full abstract

Statistics Applications

Scalable, Likelihood-Free Calibration of Ice-Sheet Models with Deep Diffusion Emulators and Feature Matching

The Antarctic ice sheet is a major source of uncertainty in future sea-level projections, and physical simulators such as the PSU3D-ICE model are essential for studying its evolution. Calibrating them against observations is challenging: the simulator outputs and observed ice-thickness fields are high-dimensional, spatially dependent, and semi-continuous, with a large point mass at zero denoting ice-free regions. These features make conventional Gaussian-process emulation and likelihood-based calibration ill-suited and computationally infeasible at full resolution. We propose the sequential calibration method guided by a diffusion model and a Siamese network (SC-DS), a fully neural framework. For emulation, we develop a single conditional diffusion model that jointly generates the binary ice presence–absence pattern and the continuous thickness field, using global and local conditioning to represent how the input parameters shape the output. Because the emulator induces an intractable likelihood, our likelihood-free calibration replaces the hand-chosen distance and tolerance of approximate Bayesian computation with a probabilistic acceptance rule learned by an iteratively retrained Siamese network, together with a data–model discrepancy adjustment. Applied to the West Antarctic Ice Sheet, SC-DS matches the accuracy of state-of-the-art Gaussian-process calibration at a fraction of its computational cost and scales to the full-resolution domain, where existing methods become intractable.
Read the full abstract

All authors Kanghyun Wi, Jaewoo Park, Saumya Bhatnagar, Won Chang

Saturday 29 August

Statistics Applications

Rescheduled, not redefined: The moving plateau of old-age mortality

Whether the risk of death keeps climbing at extreme ages or levels off has divided researchers for a century. We show this conflict reflects a moving target. Using cohort data from twelve low-mortality populations, we find that mortality deceleration and plateau onset shift steadily later across cohorts born from the mid-19th to the early-20th century. The data support a plateau across the full cohort range, though evidence weakens for the youngest, incompletely observed cohorts. In these younger cohorts, deceleration begins near age 100, and the fitted plateau begins beyond age 108. Because past studies focused on different cohorts and fixed age ranges, they sampled different phases of the exact same shift. The mortality plateau has no fixed age: it has been rescheduled.
Read the full abstract

Statistics Applications · Cross-listed from cs.IR

Content Exploration Beyond the Feed: Creator Supply and the Shared Corpus

Industrial recommenders give new content initial views through budgeted exploration, then use early performance to decide further delivery. On many short-video platforms, exploration is the primary way new videos reach viewers. Viewer-side tests measure consumption; the published budget objectives we review omit creator response. We analyze four experiments on a major short-video platform. An eight-month creator ablation finds production exploration raises videos posted per creator by 8.55% and creators posting at least once by 7.10% relative to a minimal floor. A budget-matched reallocation raises creator participation with no detectable short-run viewer-side change. A year-long viewer ablation finds 1.74% more video views but 2.13% less view time. A delivered view creates immediate feed value, can trigger organic take-up, and can induce creator supply. Take-up and supply replenish a shared corpus, creating two measurement limits. Viewer-side A/B tests cancel the corpus effect when both arms consume the same corpus. Giving each arm its own corpus avoids cancellation, but turnover still controls the horizon. If the corpus turns over at rate w per posting cycle, a t-cycle experiment expresses at most wt of the eventual corpus effect. More users reduce noise but do not speed turnover. Before the corpus path visibly bends, data cannot distinguish a modest fast effect from an arbitrarily large slow one, so a valid confidence interval may lack a finite upper endpoint. As predicted, the three-week co-diverted experiment cannot determine the sign of the eventual corpus effect. Within the window, it identifies the direct feed effect, and an exploratory cohort analysis detects organic lift after exploration ends. The experiments establish a positive creator response, measure the gross corpus flow visible within three weeks, and show the design and duration needed to identify total value.
Read the full abstract

All authors Yuanyuan Shen, Yiren Yan, Wenjie Li, Chunhui Zhu

Statistics Applications

Burn-in-Free Simulation of VARMA Time Series

Varmapack is a software package for efficient, exact simulation of VARMA time series without a burn-in period. For stationary models, Varmapack can generate initial states and innovations from their joint stationary distribution. Alternatively, the user can supply initial states, in which case innovations are generated from their conditional distribution. The core C library has interfaces to R, Python, and Matlab. It also offers VARMAX simulation and computation of autocovariances and correlations, spectral radii, and impulse responses. Varmapack automatically selects between vector-Yule-Walker and state-space methods for covariance computation and uses level-3 BLAS operations for efficient generation of multiple replicates. Benchmarks on several platforms show substantial performance gains over existing simulation software, ranging from several-fold to more than three orders of magnitude for the packages and models considered. Varmapack is open source and publicly available through GitHub.
Read the full abstract

Friday 28 August

Statistics Applications · Cross-listed from stat.ME

Bayesian model averaging of risk set probabilities using a geometric representation of multivariate extremes

Modeling multivariate extremes using a geometric perspective leverages the shape of the multivariate point cloud to make inference on joint tail probabilities. While the original statistical framework for geometric extremes was fully parametric, relying on a gauge function that uniquely defines the shape for a given density, newer methods have introduced semi- and non-parametric alternatives to increase flexibility. We propose a modeling approach that retains the simplicity of the parametric framework but adds flexibility by using Bayesian model averaging (BMA) to improve prediction of tail risk probabilities. In contrast to previous works that use a truncated radial likelihood, we propose using a censored likelihood, which we find has consistently better performance in terms of predictive mean squared error, particularly in small-sample settings, although at the expense of increased bias. To generate predictions, we use a simple importance sampling scheme that seems to match the accuracy of more bespoke methods. Finally, we apply our approach to two fire weather indices, which are designed to capture somewhat orthogonal aspects of fire risk, to illustrate our method's practical utility in environmental applications.
Read the full abstract

Statistics Applications

Physics-Informed Basis Functions for Nonlinear Response Curve Decomposition: A Parsimonious Alternative to Splines

Curve fitting for physical, biological, and engineering data typically forces a choice between interpretable but rigid parametric forms and flexible but physically opaque smoothers. This paper introduces the Growth-Decay Curve (GDC), a physics-informed basis derived as the product of a lognormal growth cumulative distribution function and an exponential decay term, each traceable to a governing differential equation. GDC's parameters correspond directly to growth and decay timescales, yielding dimensionless ratios and shape descriptors that connect the fit back to the underlying dynamics. Across simulated differential-equation solutions and applications spanning physical, biological, and economic data, GDC matches the fit quality of splines, generalized additive models, radial basis function networks, and Fourier regression, while remaining substantially more parsimonious and physically interpretable.
Read the full abstract

Statistics Applications

Comparison of statistical methods for high-dimensional compositional data from flow cytometry: A critical perspective on log-ratio transformation and differential abundance testing

Flow cytometry generates inherently compositional count data: observed cell population counts are constrained to sum to the total number of acquired events, which precludes direct inference about absolute cellular abundance. Despite this constraint, most differential abundance (DA) analyses in cytometry rely on methods originally developed for RNA sequencing, such as edgeR with trimmed mean of M-values (TMM) normalization, without fully acknowledging the conditional nature of the resulting estimates. Here, we offer a critical perspective on the statistical implications of compositionality for flow cytometry DA analysis. We compare the theoretical foundations and interpretational scope of three per-population DA testing methods (edgeR/TMM), centered log-ratio transformation with linear modeling (CLR + limma), and analysis of composition of microbiomes with bias correction-2 (ANCOM-BC2), and evaluate their performance through a comprehensive simulation study spanning seven scenarios at two sample sizes (n = 50 and n = 200 per group). We additionally evaluate compositional data analysis using kernels (CODAK) as a global, omnibus screening test for overall compositional differences, implemented via PERMANOVA on Aitchison distances. Our results show that, considering sample size, zero-inflation, dispersion, and correlation among cell populations in addition to false discovery rate (FDR) and sensitivity, the methods developed for microbiome data analysis, ANCOM-BC2 and CLR + limma, emerge as better-suited methods for per-population DA analysis of flow cytometry data than edgeR/TMM, while CODAK/PERMANOVA offers a robust first-stage screen for overall compositional shifts.
Read the full abstract

Statistics Applications · Cross-listed from cs.CR

Optimal Adversarial Testing: Extracting Honest Test Results from Dishonest Test Takers

In applications, it is often required to test objects or people to determine their qualities in terms of certain metrics. However, besides being naturally noisy, the test results can be corrupted by adversarial behaviors of objects or people being tested (test takers). For example, dishonest test takers can cheat in the exams to distort the test results. With the development of AI technologies, such distortions driven by cheating using AI technologies are becoming more commonplace and severe. In this paper, we propose optimal testing strategies which can still recover needed test results even if there are cheaters polluting the results. The proposed testing strategies will optimally re-test selected group of test takers using different testing security measures. We determine the optimal testing strategies using a dynamic programming method.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Response Propensity Estimation and Cross-Fitting

This paper investigates whether five fold cross fitting improves nonresponse adjustment in survey estimation when flexible machine learning methods are used to estimate response propensities. We conduct a finite population Monte Carlo simulation with 90 experimental configurations and 2,000 replications per configuration, varying sample size, response rate, and the structure of the response mechanism. Logistic regression is used as a conventional parametric benchmark, while Random Forest and Gradient Boosting Machine are considered as flexible nonparametric models.
Read the full abstract

Thursday 27 August

Statistics Applications · Cross-listed from cs.LG

Diffusion Distillation for Efficient Weather Ensembles

Diffusion models generate skillful weather ensembles but require costly iterative sampling. We introduce a supervised energy-distance distillation method that compresses a multi-step diffusion teacher into a single-step student by aligning student forecasts with teacher samples and ground-truth observations. Experiments on global forecasting and typhoon-track prediction show that our student outperforms existing distillation methods and preserves skill for extreme events. It matches or surpasses the teacher across key metrics using only one neural function evaluation per autoregressive step.
Read the full abstract

All authors Yiming Yang, Valentin Brekke, James Briant, Serge Guillas

Statistics Applications

How exceptional was the Big Three era? Extremes and persistence in men's professional tennis

Three players won 66 of the 81 Grand Slam titles contested between 2003 and 2023, and their era is widely held to be the most dominant in the history of tennis. Assessing it means comparing players who never met, so that every comparison passes through the opponents each did face. We measure dominance by how far a player stands above the field of his own day, how many stand there at once, and how long they remain, the last through averages over fixed windows rather than through unbroken runs, which are unstable when the strengths beneath them are themselves estimated. Strengths come from a Bayesian dynamic Bradley-Terry state space model, and dominance is characterized through exceedances of a high threshold. Applied to 197,926 matches played between 1968 and 2025, only Djokovic separates from the field on peak strength, Federer and Nadal being indistinguishable from Borg, McEnroe and Lendl. What distinguishes the recent period is not how high its best players stood but how long three of them stood there together. At its height three players held a top-three position in at least four fifths of all weeks, where in the strongest comparable decade of the past those positions were shared among four. The sharper anomaly lies between: for twelve years from 1990 the upper tail was not merely unoccupied but compressed.
Read the full abstract

Statistics Applications

A spatio-temporal block aggregation model for latent log Gaussian outcomes: application on modelling wastewater virus concentration in Wales

Wastewater-based epidemiology has emerged as a valuable tool for monitoring community-level infectious disease dynamics, providing population-wide signals that complement clinical surveillance. However, wastewater measurements are often observed as aggregated values over irregular spatial units. This work develops an approach to link an underlying spatially continuous processes and an aggregated outcome. We propose a spatio-temporal model for latent log-Gaussian outcomes that provides a coherent framework for inference and prediction, allowing the process to be integrated over arbitrary spatial configurations. This framework can also be used for subsequent analyses, such as linking wastewater signal to health outcomes at administrative areas. We use a Bayesian framework for inference via the linearised integrated nested Laplace approximation (INLA) approach. We apply the proposed methodology to model SARS-CoV-2 N1 gene copies in wastewater across 47 catchment areas in Wales from the beginning of August 2022 to the end of July 2023. The results demonstrate that the model captures spatial and temporal patterns and has good predictive performance. Results also show that estimated viral gene copies are strongly linked to positivity rates from COVID-19 PCR tests at the local authority level. Our findings highlight the importance of explicitly modelling block aggregation when analysing wastewater surveillance data. The proposed framework provides a flexible and principled approach for integrating environmental surveillance data into public health monitoring systems.
Read the full abstract

All authors Stephen Jun Villejo, Peter Diggle, Guangquan Li, Ella White, Matthew Wade, Christopher Williams, Davey L. Jones, Alisha Davies, Marta Blangiardo

Statistics Applications

The Wasserstein Distance for Mortality Comparisons: Absolute versus Net Differences in Survival

It is well known that the gap in life expectancy at birth can be seen as the net difference between two survivorship functions. When calculating the absolute difference between the two survivorship functions instead, we derive a distributional inequality measure which is called the Wasserstein distance. The measure quantifies how far apart two probability distributions are by the minimal cost of transporting one into the other. This paper relates the Wasserstein distance to the difference in life expectancy at birth. Both measures correspond to each other, whenever the survivorship functions do not cross, i.e., the net difference equals the absolute difference in the no crossing case. Whether the curves cross is driven by the accumulated difference in age-specific death rates. Crossing occurs in situations where an accumulated survival advantage for given populations is reversed because elevated death rates at later ages outweigh the earlier survival advantage. These cases are particularly interesting because the net difference in survival (or the life expectancy gap) may be small even though the two populations show very different mortality schedules. In our empirical analysis, we search for these cases using life table data from Human Mortality Database life tables for the period 1990 to 2020. Across 69 188 population pairs, survivorship functions cross in 59.8 % of comparisons. Yet, the magnitude of the reversal in survivorship is usually very small. We therefore conclude that in most cases, the gap in life expectancy at birth is not only a comparison of means but quantifies the overall difference between the two mortality regimes
Read the full abstract

Statistics Applications

Modeling Spatially Obfuscated Street-Crime Data using Log-Gaussian Cox Processes on Metric Graphs

We develop a log-Gaussian Cox process framework for modelling street-level crime data observed on a road network when the released event locations are spatially obfuscated. Motivated by UK Police street-level crime data, where published coordinates are anonymised proxy locations rather than exact event locations, we address the resulting support mismatch by representing each observation through an aggregated support on the street network. The latent log-intensity is modelled as a Whittle–Matérn Gaussian field defined on a metric graph through an SPDE representation, allowing the crime intensity to vary continuously along streets while respecting the geometry of the road network. We compare the proposed metric-graph aggregated model with two alternatives: a planar point model that treats the released locations as exact points, and a planar aggregated model that accounts for spatial aggregation but ignores the network support. In a simulation study where data are generated on a street network, the metric-graph model provides more accurate parameter recovery and better overall fit than the planar alternative. In the City of London application, the metric-graph model also achieves the best fit across several crime types, including theft from the person, robbery, drugs, and bicycle theft. The results further suggest that the relationship between environmental amenities and crime risk varies by crime type, with supermarkets showing the most consistent positive associations. The proposed framework provides a principled approach for analysing network-constrained spatial event data with privacy-protected and imprecise locations.
Read the full abstract

Statistics Applications

Climate Physics Dynamic Matching

Deep generative models such as flow matching and diffusion models have shown potential for learning complex dynamical systems, but typically act as black boxes that neglect underlying physical structure, while physics-based models governed by partial differential equations are often incomplete due to missing source terms, or uncertain parametrisations. We present Climate Physics Dynamic Matching (ClimPhyDM), a variational simulation-free dynamics informed framework for weather forecasting that combines an advection-type physics prior with data-driven components in a variational framework. % to capture the stochasticity and multi-modality of unresolved atmospheric dynamics. On the ERA5 benchmark at hourly (42-hour) and monthly (5-month) resolutions, ClimPhyDM outperforms ClimODE, and GB-DM, keeping the lower error at extended horizon, indicating improved temporal stability and resistance to error accumulation, while its simulation-free paradigm also enables training on a single modest 12 GB consumer GPU.
Read the full abstract

Statistics Applications

Integrating Network Psychometrics and LLMs: The Ising-Embeddings-Model applied to Reliability Auditing

Scoring consistency for constructed-response items in large-scale assessments is typically estimated through double-scoring, which uses small samples and assumes independence among responses. We present an integrated framework combining network psychometrics with the Linguistic-Integrated Reliability Audit (LiRA) via a modified Ising model. The model defines a joint distribution over binary correctness labels with pairwise interactions set to the cosine similarity of sentence embeddings and a global bias parameter for item difficulty. LiRA's weighted majority voting over semantic neighborhoods is shown to approximate the conditional logistic distributions of this Ising model. The parametric framework supports benchmark score generation, uncertainty quantification, and missing label imputation; parameters are estimated by maximum pseudo-likelihood. The approach uses the full dataset without requiring extensive double-scoring, accounts for semantic dependencies, and provides diagnostics for rater inconsistencies. The integration of LiRA's scalable methodology with a probabilistic graphical model offers a comprehensive tool for reliability assessment in international assessments such as PIRLS, PISA, and TIMSS.
Read the full abstract

Wednesday 26 August

Statistics Applications · Cross-listed from physics.med-ph

Impact of Multiple Non-Invasive Biosignals on Cardiovascular Biomarker Estimation via Simulation-Based Inference

As the population ages, the number of patients with cardiovascular diseases continues to increase, highlighting the need for early detection before progression to severe and irreversible functional decline. Consequently, estimating cardiovascular biomarkers from non-invasive biosignals, such as photoplethysmography (PPG) and arterial pressure wave (APW) signals, has attracted increasing attention. These signals can be measured using wearable and cuff-type devices. Previous studies have used PPG and APW signals to estimate cardiovascular biomarkers. However, these signals exhibit strong similarities in both the temporal and frequency domains and primarily reflect peripheral and arterial pulse waveforms. Therefore, they may provide limited information about cardiac mechanical function. In contrast, the quantitative impact of additional biosignals, such as ballistocardiography (BCG), which reflect the body's minute mechanical responses to cardiac ejection, remains unclear. In this study, we generated synthetic PPG, APW, and BCG signals from a unified whole-body cardiovascular circulation model and evaluated the complementary contribution of BCG to probabilistic cardiovascular biomarker estimation. We estimated posterior distributions of cardiovascular biomarkers using neural posterior estimation and simulation-based inference. The results showed that adding BCG signals significantly improved estimation performance. Furthermore, even in ill-posed cases where PPG and APW alone produced multimodal posterior distributions, adding BCG yielded unimodal posterior distributions. These findings provide fundamental insights into signal selection for estimating cardiovascular dynamics.
Read the full abstract

All authors Shusaku Maeda, Masahiro Nakano, Tomoharu Iwata, Kenji Komiya, Ryo Nishikimi, Kunio Kashino

Statistics Applications · Cross-listed from cs.DL

Gender and the Production of Research Impact

Evaluating the impact of scientific research beyond academia — on policy, health, the economy, and cultural life — has become a cornerstone of science policy and research-funding allocation worldwide. Yet which researchers produce the research underpinning this impact, and how this production is shaped by gender, remains poorly understood. We combine structured and unstructured records from the United Kingdom's latest Research Excellence Framework, the largest national research assessment currently in operation, with large-scale bibliometric data to quantify gender differences among the researchers underpinning documented impact. Women account for 38.16% of these contributors: underrepresented overall, but with a consistently higher share than in research-output authorships (33.63%), both overall and across all four REF panels. Impact production is also strongly gendered across domains: women are better represented in case studies concerning education, health, cultural, and civil-society impact, whereas those concerning commercialisation pathways such as patenting and manufacturing remain dominated by men. These findings reveal critical inequalities across the pathways that connect research to impact beyond academia, offering crucial evidence for policymakers and academic institutions aiming to build more equitable and representative systems for evaluating scientific contributions.
Read the full abstract

Statistics Applications

FLEX-CP-DT: A Flexible Conditional Power Framework for Interim Futility Analysis in Clinical Trials with Count Endpoints and Temporal Trends

Many large-scale phase III trials with recurrent event endpoints include a pre-planned interim analysis to evaluate early futility. Conditional power (CP), which quantifies the probability of achieving statistical significance at the final analysis given the interim data, is a commonly used tool to support such decisions. The standard negative binomial model with an offset term, widely adopted for analyzing recurrent events, implicitly assumes that event rates and treatment effects remain constant over the study period. At the interim analysis, however, a substantial proportion of patients have incomplete follow-up, and when the treatment effect is delayed in onset or diminishes over time, the constant-rate assumption introduces systematic bias into the interim estimate and can lead to incorrect futility decisions. In this paper, we propose FLEX-CP-DT, a piecewise negative binomial framework that captures temporal trends in both event rates and treatment effects without imposing the constant-rate assumption. The framework yields a formula-based conditional power calculation that does not require resampling or trial simulation at the interim stage. Through extensive simulations spanning constant-effect and delayed-onset scenarios, we demonstrate that FLEX-CP-DT performs comparably to the standard approach when the constant-rate assumption holds and improves interim futility decision-making when it is violated. A case study calibrated to a published phase 3 bronchiectasis trial further illustrates the practical advantage of the proposed method in reducing the probability of falsely terminating an efficacious drug with delayed treatment onset.
Read the full abstract

All authors Yanzhao Wang, Dateng Li, Ningya Wang, Haitao Gao, Chenguang Wang

Statistics Applications · Cross-listed from physics.soc-ph

District-Level Food Environment Indicators and Social Vulnerability in São Paulo

Urban food environments may reflect broader socioeconomic inequalities, but district-level evidence remains limited in Brazilian cities. This study examined whether indicators of food retail and street-market availability discriminate between levels of social vulnerability across the 96 districts of São Paulo. We conducted an exploratory cross-sectional ecological analysis integrating the São Paulo Social Vulnerability Index (IPVS), establishment records from the Relação Anual de Informações Sociais (RAIS), and street-market data from CAISAN. Census-sector information was aggregated at the district level. Twenty districts without an IPVS classification were excluded, resulting in 76 observations. The outcome distinguished districts classified as IPVS level 1 from those classified as levels 2–7. Predictors described the densities of healthy and unhealthy food establishments, the number of street markets, and the availability of establishments selling fresh or in natura food. Eight conventional machine-learning classifiers were evaluated using leave-one-out cross-validation. Reported mean F-scores ranged from 0.62 to 0.75, with XGBoost obtaining the highest value. In the Random Forest model, the densities of healthy and unhealthy food establishments jointly accounted for approximately 60% of the total impurity-based feature importance. These findings indicate that publicly available food-environment indicators contain information associated with the district-level distribution of social vulnerability. However, the small ecological sample, class imbalance, outcome binarization, and cross-sectional design limit predictive generalization and preclude causal or household-level interpretations.
Read the full abstract

All authors Pedro Lemes Sixel Lobo, Eric Tokuda, Kuruvilla Joseph Abraham, Roberto Fray, Dirce Maria Marchioni, Alexandre Cláudio Botazzo Delbem, Rogerio Salvini