Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Monday 5 October

Statistics Applications · Cross-listed from stat.ME

Which modelling choices shape your interpolation map? An Optimal Transport sensitivity analysis for groundwater sulfate concentration

Spatial prediction maps are crucial for environmental management yet highly sensitive to subjective modelling choices such as, e.g., covariates, spatial dependence structures, or mesh resolution. We develop a workflow that attributes the resulting map uncertainty to these choices and is applicable to any spatial model, since it operates only on an ensemble of maps produced by propagating the choices through spatial prediction. This ensemble is then filtered to retain only models with acceptable predictive performance, reduced in dimension by principal component (PC) analysis, and the sensitivity of the resulting scores to each choice is quantified by global sensitivity analysis based on optimal transport (OT) metrics. We demonstrate the framework with Bayesian geostatistics applied to the spatial mapping of sulphate concentrations in groundwater of Paris Basin (France), which is key for assessing groundwater chemical quality and establishing the natural geochemical baseline of aquifers. For this application, the covariates, mesh resolution, covariance models, and priors are combined into 18000 candidate configurations. These are filtered by cross-validated root mean square error (RMSE) and 95% coverage of the prediction interval, and the retained configurations are then inferred on a high-resolution grid and reduced to its three PCs. The OT analysis identifies covariate selection (under 95% coverage) and mesh resolution (under RMSE) as the dominant sources of uncertainty to the PCs, and a negligible one for the choice of the covariance model.
Read the full abstract

All authors Corinna Perchtold, Jérémy Rohmer, Augustin Thomas, Julie Lions

Statistics Applications · Cross-listed from q-fin.RM

Comparing two approaches for modelling the loss given default of credit cards: Run-off triangles vs regression

The use of run-off triangles (ROTs) is a common industry practice in estimating the loss given default (LGD) risk parameter when predicting credit losses in banking. We benchmark this industry practice using credit card data against a more sophisticated (though classical) regression-based approach, which is able to leverage various types of input variables in producing loan-level LGD-estimates. This regression-based approach can demonstrably recover the typical characteristics of the 'U-shaped' empirical LGD-distribution, which the ROT-based approach cannot do. First, we critically review the ROT-based approach and identify multiple demerits using data-driven diagnostics. We then estimate a two-stage regression-based LGD-model and favourably assess the model performance of each component (or 'stage'). Finally, we aggregate the LGD-estimates produced by each approach over time, and compare each time series to the mean empirical loss rate over time. The ROT-based aggregates diverge substantially from the empirical rate over most time periods, whilst the regression-based aggregates follow the empirical trends much closer. These results underscore the greater prediction accuracy of the regression-based LGD-model, relative to the ROT-based one. By implication, the former approach is probably better than the latter ROT-based approach when estimating the LGD under the IFRS 9 accounting framework, which prioritises accuracy.
Read the full abstract

All authors Arno Botha, Henko Crewe, Marcel Muller, Janette Larney

Statistics Applications

Spatio-Temporal Stochastic Interventions for Causal Inference in Climate Science

Estimating causal effects in climate science, such as the effect of anthropogenic warming on crop loss, is challenging because of complex spatio-temporal dependence and the high-dimensional nature of the treatment. To address this dependence and the resulting poor overlap between observed and counterfactual scenarios, we develop a spatio-temporal stochastic-intervention framework for estimating causal effects from climate observations. We introduce a regularized estimator of the stochastic-intervention treatment effect that trades a controlled bias for a reduction in the weight variance caused by poor overlap. Simulation studies show that this estimator attains lower mean squared error than alternative weighting estimators and removes the confounding bias of an unadjusted estimator. We apply the framework to estimate the effect of historical warming on vapor-pressure deficit, a driver of crop stress, adjusting for precipitation, which confounds the effect by affecting both temperature and humidity. In GISS-E2-1-G climate-model simulations, the global effect is distinguishable from zero in every year from 1995 onward, and omitting the precipitation adjustment inflates the global estimate by 47%. Adjustment reverses the sign of the estimate over 8% of global cropland (125 million hectares), where an unadjusted analysis could misdirect adaptation between heat-focused and moisture-focused measures.
Read the full abstract

Statistics Applications · Cross-listed from cs.LG

Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data

Classification problem in the context of highly imbalanced data is a major challenge in many real-world applications (e.g., FinTech, healthcare, etc.). In these cases, the vast majority of instances belong to a single class and a small fraction represent the minority class (often the most critical class). Recently, diffusion models have emerged as powerful approaches to reduce the degree of “imbalanced-ness” in the dataset; they work by generating synthetic data by capturing complex data distributions using iterative transformations. However, standard diffusion models are not inherently suited to highly skewed or heavy-tailed data, due to inbuilt quadratic error loss, which lacks the structural sensitivity to capture rare, extreme values, and minority-class nuances. We propose a novel approach, namely, Quantile-TabDDPM, based on a quantile-regularized denoising objective that combines the standard quadratic error loss with a quantile loss term to explicitly capture rare events while preserving the theoretical grounding of the original denoising objective. We extensively evaluated our approach on a real-world credit card transaction dataset characterized by extreme class imbalance. The results demonstrate that the integration of diffusion-based synthetic data generation with a quantile-regularized denoising objective provides a robust and effective framework for fraud detection in highly imbalanced datasets.
Read the full abstract

Statistics Applications · Cross-listed from cs.DL

Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based Prompts

Large language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliographic information in environmental science. Therefore, we quantitatively compared the errors of widely used LLM platforms in retrieving references related to original articles from five leading environmental science journals (Energy and Environmental Science, Nature Sustainability, Nature Climate Change, Lancet Planetary Health, and Environmental Science and Technology) published in 2024 to 2025. Claude, ChatGPT, Grok, DeepSeek, Perplexity, and Gemini were used as the LLM platforms. LLMs retrieved 10 references for each of the 50 randomly selected original article using either the article's abstract or its full-text as prompt. The retrieved references were subject to a multimetric score ratio combining validity of bibliographic data, Google Scholar link, digital object identifier, Scopus Electronic Identifier and relevance score (cited by or being the index paper), and the proportion of complete fabrication that failed all metrics. Abstract-only prompt yielded significantly higher accuracy than full-text one. This advantage was confirmed in multilevel mixed-effect multivariable regression after adjusting for journal, platform, and output order. Source journal and the position of a reference within the output list were also independently associated with retrieval accuracy, with lower-listed references associated with lower accuracy. These findings suggest that LLM assisted literature retrieval in environmental science remains moderately accurate and overall inconsistent, varying significantly by platform, journal, prompt type, and output position. Abstract-based prompting, as task-aligned information compression, may outperform full-text one in literature retrieval. Caution should be used when generalizing our findings.
Read the full abstract

Sunday 4 October

Statistics Applications · Cross-listed from cs.LG

A Data-Driven Framework for Unsupervised Monitoring of Transmission Systems Using End-of-Line Testing Data: A Case Study at Ford Motor Company

Sensing technologies have advanced rapidly across industries ranging from energy to automotive manufacturing. These systems generate high-dimensional (HD) data characterized by complex nonlinear patterns and strong temporal dependencies. Traditional statistical monitoring methods are often limited in their ability to capture such nonlinear structure. Likewise, many analytical approaches used in End-of-Line testing rely on predefined thresholds and heuristic rules, which restrict their ability to detect informative anomaly signatures in HD temporal data. In contrast, while modern deep learning and generative AI models offer strong predictive capabilities, they are often unsuitable in applications where data are costly to collect and where the monitoring system must remain interpretable, low-latency, computationally efficient, and usable by non-technical practitioners. To overcome these limitations, we propose an advanced multivariate monitoring framework for HD data. The framework operates in two stages. In the first stage, the data are preprocessed to remove incomplete and non-informative samples and to temporally align time series data. In the second stage, nonlinear dimensionality reduction is performed, followed by anomaly detection through a control chart based phase I monitoring procedure. The framework can be used in both unsupervised and supervised settings, depending on the availability of ground truth labels during training. Moreover, its flexible and modular structure allows practitioners to adapt its components to different domains and operational requirements. We evaluate the proposed framework on real production data from an automotive manufacturing environment at Ford Motor Company. The proposed method achieves higher accuracy, recall, and F1 score than the company's existing model, improving these metrics from 0.50, 0.30, and 0.429 to 0.625, 1.00, and 0.769, respectively.
Read the full abstract

All authors Mohammad N. Bisheh, Mehrdad Moradi, Parinaz Farajiparvar, Colin Brady, Rajesh Gupta, Xueling Li, Javad Navaei, Milad Parvaneh, Kamran Paynabar

Statistics Applications

Does the AI Productivity Dividend Diminish? Does the AI Productivity Dividend Diminish? A Validated Framework for Identifying the Shape of Firm-Level Returns to Artificial Intelligence

Forecasts of the productivity dividend from artificial intelligence (AI) differ by an order of magnitude, partly because they extrapolate average gains observed among early, highly exposed adopters. Whether those gains scale linearly, flatten, or are competed away is an empirical question about the shape of the return to AI, not its average. This paper develops and validates a design-based framework for measuring that shape with linked firm-worker data. The framework combines a pre-determined, occupation-based measure of firm exposure to generative AI with a continuous-treatment difference-in-differences design around the release of ChatGPT in November 2022, three pre-specified tests of concavity, a decomposition of exposure effects into an adoption margin and a per-adopter return, a doubly robust analysis of use intensity, a staggered-adoption analysis of earlier AI adopters, and a production-function estimator that places AI in the law of motion of productivity. We validate every component on a calibrated synthetic panel of 20,000 enterprises that reproduces the structure of Statistics Canada's linked business microdata and embeds a known concave effect. The estimators recover the true dose-response curve and detect its concavity (slope difference -0.010, standard error 0.003, one-sided p = 0.002); Monte Carlo coverage of the main confidence interval is 0.97, and the concavity test has power of 0.93. The validation also exposes a practical pitfall: pre-trend tests clustered at roughly 40 industries over-reject a true null. The framework, released as open code, is ready for application to Canadian enterprise microdata, where it will inform estimates of potential output and the persistence of Canada's productivity gap.
Read the full abstract

Statistics Applications

Within-fire estimates of biomass loss across national forests of the US West Coast: inventory-informed inference with uncertainty

Wildfire is reshaping forests in the western United States (US), yet estimates of wildfire impacts often rely on burned-area summaries or satellite-derived severity metrics that do not directly quantify changes in forest biomass or their uncertainty. We combine US Forest Inventory and Analysis plot measurements with annual 30 m tree canopy cover data in a Bayesian spatio-temporal modeling framework to estimate within-fire changes in live aboveground biomass (AGB) and forest mortality area across National Forest System lands in Washington, Oregon, and California. The model represents both live forest presence or absence and AGB conditional on live forest, allowing posterior predictive estimates of pre- and post-fire conditions at the pixel scale. Applied to 6,239 fires from 2000 to 2022, the analysis estimates 95.3 million Mg of wildfire-associated AGB loss and 2.39 million ha of forest mortality area, with the largest aggregate impacts occurring in 2020. Pixel-level estimates reveal substantial spatial heterogeneity within fire perimeters, including large differences between total burned area, forest mortality area, and associated biomass loss. This inventory-informed, uncertainty-aware approach provides a scalable framework for quantifying forest carbon impacts of disturbance across broad regions while retaining within-fire spatial detail.
Read the full abstract

All authors Elliot S. Shannon, Andrew O. Finley, Paul B. May, Hans-Erik Andersen, Harold Zald, Grant M. Domke, George C. Gaines III

Statistics Applications

LLM Benchmarking via Representation Multi-task Learning

Quantifying and evaluating the capabilities of Large Language Models (LLMs) remains a fundamental challenge in modern data science and artificial intelligence. In this paper, we consider LLM evaluation based on their performance across items in multiple benchmark domains (e.g., mathematical reasoning and coding) within a leaderboard framework. Our goal is to address two core questions: (1) How do we derive more accurate domain-specific scores by borrowing information across domains? and (2) How do we define and estimate an overall score that aggregates performance across multiple domains? To solve these problems, we propose a novel statistical framework based on representation Multi-task Learning (MTL) and an item response theory model. Specifically, we define overall and domain-specific LLM traits through an Item Response Theory (IRT) model, and propose an MTL approach to estimate these traits from item-level response data. We develop a computationally efficient estimator and establish its minimax optimality under certain asymptotic regimes. This framework provides a rigorous measurement foundation for systematic LLM evaluation. We conduct extensive simulations, demonstrating the superior performance of the proposed method over competing methods. Crucially for the Applications and Case Studies section, we apply the proposed framework to MMLU response data from the Hugging Face Open LLM Leaderboard, covering 4,272 LLMs and 13,232 items across 56 subjects. The empirical analysis reveals substantial heterogeneity in domain size and difficulty, together with strong positive cross-domain dependence, highlighting the practical value and substantive insights generated by our approach.
Read the full abstract

All authors Yuqing Xie, Yuxuan Xu, Yang Feng, Yunxiao Chen

Statistics Applications

From Counting to Continuous Natural Exponential Families

Natural exponential families (NEFs) are characterized by their variance functions (VFs), namely by the pair \((V,M)\), where \(V\) expresses the variance as a function of the mean and \(M\) is the mean domain. This characterization naturally raises the inverse question: when is a given function \(V\) the VF of an NEF, and what structural properties of the family can be read directly from \(V\)? Bar-Lev (1987), as part of a more general result for absolutely monotone functions, showed that every nonzero polynomial \[ V(m)=\sum_{j=1}^{r} a_j m^j,\qquad a_j\ge 0, \] is the VF of an infinitely divisible NEF on a positive mean domain. This polynomial class splits exhaustively into the cases \(a_1>0\) and \(a_1=0\). Bar-Lev, Letac and Ridder (2024) proved that, after scaling so that \(a_1=1\), the first case yields a counting NEF supported on \(\mathbb N_0\). We establish the complementary result: when \(a_1=0\), the corresponding NEF is absolutely continuous with respect to Lebesgue measure on the positive half-line. More generally, we prove absolute continuity whenever \[ V(m)=m^2G(m), \] where \(G\) is a nonzero power series with nonnegative coefficients. When \(G\) is a polynomial, the mean domain is the entire positive half-line. We also record several consequences for the cumulant structure and for statistical inference.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

PKComb-BOIN12: A Pharmacokinetically Guided Bayesian Design for Dose Optimisation in Oncology Drug-Combination Trials

Dose optimisation of two-agent combinations in early-phase oncology trials must jointly balance toxicity, efficacy, and pharmacologically meaningful drug exposure, yet existing model-assisted combination designs use only binary toxicity–efficacy outcomes and leave pharmacokinetic (PK) data unused for dosing decisions. We propose PKComb-BOIN12, a utility-based Bayesian optimal interval design embedding a continuous PK exposure metric into two-agent dose optimisation via a PK-guided escalation rule and a PK-sufficiency constraint on the final admissible set. Two contributions distinguish this design from a direct combination extension of existing single-agent PK-informed designs. First, PK-based elimination uses a coordinatewise order-ideal (down-set) removal rule, so that eliminating a combination for insufficient exposure is licensed exactly by the assumed monotonicity of exposure in each agent's dose, rather than an unjustified scalar summary of combined dose intensity. Second, the pre-specified target exposure is treated as transportable-but-verifiable: a hierarchical calibration mechanism updates the assumed monotherapy-derived target using accruing combination-arm PK data, with a formal bound on the resulting selection bias under transportability violations. A time-to-event extension, TITE-PKComb-BOIN12, accommodates late-onset outcomes. A factorial simulation study spanning realistic scenarios, including one calibrated to a published combination trial, decomposes the source of PKComb-BOIN12's gains over its non-PK predecessor and evaluates robustness to non-monotone exposure, PK-variance misspecification, and target-transportability violations. An R Shiny application supports design evaluation and real-time conduct.
Read the full abstract

Statistics Applications

How Long Is the Journey to Work? One-Way Commuting Time in the United States from American Community Survey Data, 2006-2024, with Travel-Diary, European and English Comparisons

The mean travel time to work is one of the most cited statistics of the American Community Survey (ACS), yet its distribution, its dependence on mode and departure time and its relation to other measurement bases are rarely set out together. Using ACS 1-year tables, we report that US workers aged 16 and over who did not work from home averaged 27.2 minutes one way in 2024, against 25.0 minutes in 2006, a peak of 27.6 in 2019 and 25.6 in 2021. Dividing published aggregate minutes by workers, commuter rail, long-distance train and ferry riders averaged 71.0 minutes, 2.68 times the 26.5 minutes of solo drivers, and all public transportation 49.5 minutes. In 2024, 9.3% of commuters, about 13.3 million people, travelled 60 minutes or more each way and 3.0% 90 minutes or more; commuters leaving between 5:00 and 5:29 a.m. averaged 35.1 minutes. State means ranged from 33.2 minutes in New York to 17.5 in North Dakota, and big-metro means from 36.5 minutes in New York to 21.5 in Buffalo. Working from home rose from 5.7% of workers in 2019 to 17.9% in 2021 and stood at 13.3% in 2024. The 2022 National Household Travel Survey recorded an average commute trip of 13.43 miles at 25.51 mph. On different definitions, the 2019 EU labour force survey module gave an EU average of 25 minutes and 30 minutes for the United Kingdom, and England's travel diary a commuting trip of 31.5 minutes in 2025 against 26.5 in 2002. The three bases are reported side by side and never pooled. All inputs are public.
Read the full abstract

Statistics Applications

Bayesian Markov Chain Monte Carlo-Based Simultaneous Speed Prediction Model for Heterogeneous Traffic on Urban Primary Roads

Objectives: The study designs a simultaneous speed prediction model based on Bayesian Markov Chain Monte Carlo (MCMC) methods for heterogeneous, non-lane-based traffic on urban primary roads, with the aim of determining the effects of vehicle-specific traffic densities on the operating speeds of the six vehicle types. Methods: The traffic data were obtained from videos recorded by unmanned aerial vehicles on five main urban roads in Dhaka, Bangladesh, and were then processed by a deep learning approach in order to derive information on vehicle trajectories, traffic volumes, densities, and space-mean speeds. Six simultaneous equations for speed prediction were developed using a Bayesian MCMC framework, covering standard cars, utility vehicles, heavy vehicles, three-wheelers, two-wheelers, and non-motorized vehicles. The performance of the models was assessed by means of goodness-of-fit measures. Findings: The coefficients of determination for the models varied from 0.922 to 0.956 and the root mean square errors were between 0.50 and 1.56 km/h for the six types of vehicle. Operating speeds decreased steadily as traffic volume increased, although traffic composition had a significant effect on speed when the demand was moderate or high. Operating speeds increased and congestion was delayed when non-motorized vehicles were excluded. The transferability study showed that the models had a reliable ability to make predictions on separate road sections.
Read the full abstract

All authors S.M Towhidul Alam, Md. Muhtashim Shahrier, Nazmul Haque, Md Asif Raihan, Md. Hadiuzzaman

Statistics Applications

Reproducibility and Leakage-Controlled Evaluation of Machine-Learning Models for Chronic Kidney Disease Classification

Near-perfect classification of small clinical datasets can hinge on preprocessing, data leakage, missing-data handling, and the evaluation sample. We reproduced a published chronic kidney disease (CKD) classification study on the UCI CKD dataset (400 records, 24 predictors) under explicit leakage controls. A stratified 70:30 development-holdout partition was fixed before any learned preprocessing; imputation, encoding, scaling, supervised feature selection, and PCA were fitted only within training folds. Nine classifiers, three feature representations, and three missing-data strategies were compared by 5x3 repeated stratified cross-validation and ranked on training cross-validation only (ties reported); a frozen set of finalists was evaluated once on the holdout (discrimination, calibration, Brier score), treated as a locked historical internal sample. Ablation, grouped SHAP values, three further seeds, and post hoc sensitivity analyses examined robustness. The top-ranked selected-feature pipeline (KNN imputation, Extra Trees) reached a repeated cross-validation accuracy of 0.9869 and a holdout accuracy of 0.9833 (exact 95% interval 0.9411-0.9980). PCA (27 components, 95.96% of variance) gave only a marginal, seed-dependent advantage over interpretable original-feature representations, and several pipelines were statistically indistinguishable. Nested resampling of the selection procedure gave an outer accuracy of 0.9845, 0.0095 below the top-ranked pipeline on the same folds; missingness indicators alone yielded a ROC area of 0.85-0.88, and removing renal, urinary, and hematologic fields cut accuracy to 0.89. High internal performance persisted after leakage controls but appears driven largely by clinically proximal laboratory and urinalysis variables and CKD-associated missingness; it is not clinical validation, and external, prospective evaluation remains necessary.
Read the full abstract

Saturday 3 October

Statistics Applications · Cross-listed from math.OC

Economic resources, sporting efficiency and uncertainty: a stochastic optimization model of football performance in Africa

We propose a stochastic optimization framework in which football performance is modeled as the result of an efficiency-driven production process under uncertainty. The model incorporates economic inputs, a latent efficiency variable evolving according to a stochastic differential equation, and a control variable representing investment in sports development. We derive the associated discounted Hamilton–Jacobi–Bellman (HJB) equation, characterize the optimal investment policy, prove a verification theorem under explicit growth conditions, and show that a bounded investment budget is required for the control problem to be well posed. We relate our efficiency measure to the stochastic frontier and data envelopment analysis (SFA/DEA) traditions used elsewhere in sports economics, and clarify what a dynamic control formulation adds relative to those static approaches. The theoretical results are illustrated with a numerical solution of the HJB equation – validated by a grid-refinement convergence study – and Monte Carlo simulations. A cross-sectional analysis of the 24 AFCON 2025 national teams, based on GDP, population, and FIFA ranking data, estimates country-specific conditional log-efficiency residuals using heteroskedasticity-robust inference. We show that these residuals are mechanically and strongly correlated with FIFA points by construction, quantify this dependence, and interpret the results accordingly. Teams such as Senegal and Morocco consistently outperform their GDP-implied potential, whereas others, including South Africa and Tanzania, tend to underperform. While these patterns are broadly consistent with observed AFCON outcomes, the estimated residual also captures unobserved factors, such as institutional quality, diaspora talent, and demographic structure, that are not included in the model.
Read the full abstract

Statistics Applications · Cross-listed from stat.ML

Low-Rank and Structured Sparse Tensor Decomposition for Anomaly Detection in Multivariate Functional Data

Multivariate functional data arise in many modern manufacturing systems, where multiple sensors record densely sampled process trajectories. Monitoring such data is challenging because nominal variation is strongly correlated across samples, sensors, and time, while faults may appear either as isolated deviations or as structured departures concentrated within a limited number of sensor-specific temporal trajectories. We propose two unsupervised sparse tensor decomposition methods that preserve this multimode structure. Entrywise Sparse CP Decomposition (ES-CP) uses an entrywise \(\ell_1\) penalty to identify localized anomalies, whereas Fiberwise Sparse-Group Lasso CP Decomposition (FG-Lasso) combines entrywise and fiberwise penalties to detect both localized deviations and anomalies concentrated within temporal fibers. Both methods represent nominal process behavior through a low-rank CP decomposition and are estimated using alternating optimization with closed-form sparse-component updates. Two simulation studies evaluate performance under different fault structures, signal severities, noise levels, and missing observations. FG-Lasso attains or ties the highest macro F$_1$ score in almost all settings in the first study and achieves the highest macro F$_1$ score. In a multichannel forging-process case study, FG-Lasso and ES-CP obtain macro F$_1$ scores of 0.85 and 0.82, respectively, compared with 0.69 or lower for TRPCA and PCA-based anomaly detectors. The results demonstrate that explicitly matching the sparse penalty to the anticipated fault structure improves both anomaly detection and fault localization in high-dimensional functional processes.
Read the full abstract

Statistics Applications

A Markov Chain Approach to Modeling Rugby Union

Rugby Union's continuous play, variable-length possessions, and multiple scoring outcomes make it difficult to model with standard approaches. We present a Markov Chain framework that represents Rugby Union as a finite state space over score differential, remaining time, and pitch location, with transition probabilities estimated from historical match data. The resulting model supports simulation and analysis of in-game trajectories, including win probability and expected point estimates from any given state. We validate the model through calibration analysis and comparison to models that predict only final outcomes. Our state-space approach produces well-calibrated, competitive estimates while additionally enabling analysis of how games evolve between states.
Read the full abstract

Statistics Applications · Cross-listed from cs.CL

Stance Drift: How AI-mediated Communication Distorts Our Message

Large language models (LLMs) increasingly mediate human communication, from drafting emails to summarizing scientific reports, yet whether they faithfully preserve a speaker's position remains largely untested. We model AI-mediated communication as a two-step generation-extraction pipeline: one LLM produces an argument from a specified stance, and a second LLM extracts the stance from that argument. We represent the pipeline as a probabilistic state transition over five Likert-type stance categories and define the stance preservation rate (SPR) as the average probability that the extracted stance matches the initial stance. Across 112 debate propositions, none of the nine LLMs tested exceeded an SPR of 0.7 under the default configuration. Three drift patterns accounted for most of the drift: polarization, deviation from neutrality, and flipping. Among the mitigation strategies tested, including in-context learning, multiple extraction with shuffled options, assertion, and reflection, only adding medium reasoning effort to a reflection prompt for GPT-5.4 substantially improved the SPR, to 0.775, yet polarization remained the largest pattern, with 0.119 of the transition mass. An exploratory comparison with human extraction on a single proposition suggests that drift arises at both the generation and the extraction stage. These results point to a fidelity gap in AI-mediated communication, with implications for journalism, policy deliberation, scientific communication, and other domains where opinion-laden messages pass through language models.
Read the full abstract

All authors Lingchong Liu, Yanfei Zhou, Jacob Bien, Y.X. Rachel Wang, Lucy Xia, Xin Tong

Friday 2 October

Statistics Applications

Multi-Target Tracking of Cooperating Swarms by an Ensemble Gaussian Mixture Filter

Per-target labeled random-finite-set filters, such as the state of the art $\delta$-generalized labeled multi-Bernoulli filter, cannot represent cooperating, swarm-like, motion between targets. In this work, we present a new filter that can. The new particle filter couples the joint particle filter with ensemble Gaussian mixture filter such that a measurement updates the density at every particle instead of merely reweighting. We prove that the new particle filter converges to the joint particle filter in the limit of particle number and thus to the true Bayesian posterior, under certain assumptions. Experimental results on two coupled systems, the Vicsek model and coupled Brownian motion provide evidence on the efficacy of the filter.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Network-Constrained Matching and Randomization-Based Inference for Causal Effects

Matching is widely used to mimic randomized experiments by forming matched sets in which treated and control units differ only randomly with respect to observed covariates. However, when the study population consists of interconnected units from a single network or a small number of networks, matching solely on observed covariates may produce matched units that are more closely connected in the network than would occur by chance. This increased network proximity within matched sets can induce spurious associations between treatment and outcome when both variables exhibit similar autocorrelation patterns on the network. To reduce spurious associations while preserving the validity of causal comparisons, we propose a new matching method that matches units with similar covariates subject to additional network-proximity constraints. For post-matching inference, we propose a randomization-based procedure for testing the sharp null hypothesis of no causal effect. The inference uses the asymptotic normal approximation and accommodates statistical dependence among test statistics obtained from each matched set without requiring explicit specifications of their correlation structures. We demonstrate the validity and utility of the proposed methods through simulation studies and apply them to the Framingham Heart Study. The matching method and subsequent inference procedure are implemented in the R package netmatchRI.
Read the full abstract

Statistics Applications

Noise Emissions from Hyperscale AI Data Centers: A Poorly Characterized Exposure at the Doorstep of Community Health Research

The rapid build-out of hyperscale computing for artificial-intelligence workloads has made data-center noise a recurring source of community concern in the United States, yet calibrated, standards-grade acoustic data remain scarce. This narrative review synthesizes direct measurements, regulatory and legal records, investigative journalism, and relevant engineering literature to assess what is currently known and identify the measurements needed for credible health inference. Available sources report heterogeneous acoustic values at residences, property lines, and near-source positions, with individual readings from 39 to above 70 dB(A). Because location, metric, duration, instrumentation, and provenance differ, these values are not directly comparable community exposures, do not define a pooled distribution, and do not support an aggregate comparison with nighttime limits. Among cases with spectral detail, all but one reported continuous tonal noise associated with complaints, most often low-frequency content near 70-140 Hz, that A-weighted metrics alone may under-represent. Engineering evidence further suggests that cooling architecture may be a major determinant of facility sound emission; the cited 15-30 dB span is an equipment-level sound-power estimate, not a measured facility- or residential-receptor difference, and remains a hypothesis pending comparative field measurements. We propose a standards-based field-measurement and modeling protocol designed to generate frequency-resolved exposure data at named facilities. Such exposure characterization is a necessary foundation for credible assessment of community health effects and environmental justice implications.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Community Detection from Aggregated Relational Data: Identifiability and Spectral Recovery

Aggregated relational data replaces a fully observed network with a compressed version, where survey respondents are asked questions such as “How many of your friends belong to group $G$?”. We consequently only observe a matrix $R = OA$ of the adjacency matrix $A$ with a fixed design $O$. In this paper we understand the effects of this compression on community detection under degree-corrected stochastic block models. First, we give guarantees for when it is possible to recover the full network parameters, and then only just the communities under conditions on a diagonal corrected version of $\mathbb{E}[R]$, which can be used to inform experimental design. Secondly, we show that spectral clustering with single linkage exactly recovers communities when $n\rho_n\gtrsim K^2+K\log n$ when there are $K$ balanced communities under “uniformly spread” or near-orthogonal designs, and characterize more generally how the design influences rates of convergence. We also introduce second-order “friends-of-friends” queries and develop the corresponding theory.
Read the full abstract

Statistics Applications · Cross-listed from econ.EM

A Path-Dependent Agent-Based Microsimulation of Crime and Violence Reduction Policy Portfolios in Bolivia: Survey Calibration, Adaptive Emulator Ensembles, Global Sensitivity, Tempered MCMC, and Multi-Objective Policy Analysis

This study develops a survey-calibrated, path-dependent agent-based microsimulation for evaluating crime and violence reduction policies in Bolivia. A synthetic population is constructed from the 2025 Household Survey and the fourth-quarter 2025 Continuous Employment Survey using official expansion weights and cross-survey donor matching. The model combines heterogeneous agents, place-based risk, criminal adaptation, institutional capacity, community organization, and multiple policy channels. Uncertainty is propagated through adaptive Gaussian-process and tree-ensemble emulators, global sensitivity analysis, tempered ensemble MCMC, and predictive meta-calibration. Simulated portfolios include focused policing, community control, Guardabosques-type incentives, social prevention, environmental interventions, justice reforms, investigation, rehabilitation, and financial-network disruption. Results are interpreted as structural policy scenarios rather than causal forecasts, with explicit attention to displacement, distributional effects, implementation capacity, and multi-objective trade-offs.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Bayesian Analysis of Covariate-Driven Hawkes Processes with Application in Plant Epidemiology

Hawkes processes are widely used to model temporal point patterns exhibiting self-exciting behavior. In this work, we consider a Hawkes model with a baseline intensity depending on a set of dynamic covariates and an exponential triggering function. A Bayesian approach is applied to variable selection in this context. Spike-and-slab priors are proposed to enable a parsimonious choice of relevant variables. We also provide practical simulation procedures for the fitted covariate-driven Hawkes model, which support posterior predictive checks and probabilistic forecasting. This approach is illustrated by the example of spore release of Venturia inaequalis, the fungus responsible for apple scab disease. We study the dependence of primary spore releases on environmental covariates and highlight the self-exciting behavior of subsequent releases. The Bayesian framework also provides a natural basis to forecast future spore releases. By delivering interpretable posterior inclusion probabilities alongside uncertainty-aware predictive risk bands, this new methodology provides a rigorous foundation for managing temporal event risks driven by both external environmental forcing and internal contagion.
Read the full abstract

All authors Katarzyna Adamczyk-Chauvat, Madalina Deaconu, Sylwester Masny, Guillaume Kon Kam King

Statistics Applications · Cross-listed from stat.ME

Cellwise, Blockwise, and Casewise Robust Multiblock PCA for Sustainable and Inclusive Wellbeing in the EU

Gross Domestic Product (GDP) is widely used to guide economic and social decision-making, but it provides only a partial view of wellbeing. For this reason, the European Union (EU) has launched initiatives to monitor sustainable and inclusive wellbeing beyond GDP, developing indicator frameworks that cover dimensions such as health, education, environment, and social inclusion. These indicators are naturally grouped into thematic areas, and policymakers are interested in understanding how these areas contribute to global wellbeing and which indicators explain differences across countries. However, such data are high-dimensional, contain missing values, and may include anomalies affecting entire observations, specific thematic areas, or individual indicators. We introduce blockwise outliers and propose bloccPCA, a robust multiblock PCA method that simultaneously handles casewise, blockwise, and cellwise outliers, as well as missing values. The method provides robust global components to summarize the overall structure of wellbeing, while preserving thematic-area contributions through robust blockcomponents. It also yields diagnostic tools to identify whether anomalies arise at the case, block, or cell level. Monte Carlo simulations and an application to the EU wellbeing dataset show that bloccPCA provides valuable insights into sustainable and inclusive wellbeing.
Read the full abstract

Statistics Applications · Cross-listed from cs.AI

CreateScore: Domain-Theory-Informed Bayesian Routing for LLM-Based CV Screening

Large language models (LLMs) can support rubric-based screening of CVs, but applying a high-capability model to every candidate and criterion is costly. We present CreateScore, a domain-theory-informed Bayesian network for criterion-level LLM routing. A hand-specified directed acyclic graph with Dirichlet-multinomial conditional probability tables converts CV evidence into posterior uncertainty; low-uncertainty decisions are resolved by a local 8B model and uncertain ones are escalated to a 120B reference model. The graph is causally motivated, but the system performs standard Bayesian conditioning, not causal inference. The escalation threshold is calibrated on a training fold (target: 70% resolved locally) and then frozen. On 200 synthetic Data Science CVs (139 training and 61 test candidates, five criteria), 77.7% of criterion decisions were resolved locally (237 of 305). Relative to a reference condition in which the 120B model adjudicated every criterion, routed escalation reduced token use by 65.2% and raised exact score agreement from 32.8% (8B alone) to 42.6% (95% CI 31.0-55.1%); at n = 61 the gain was not statistically distinguishable. The uncertainty signal did not, however, identify the decisions on which the 8B model erred: disagreement with the reference was 16.2% among escalated and 19.4% among locally resolved decisions (AUROC 0.47, 95% CI 0.39-0.56), no better than random selection. We also document how an earlier evaluation was invalidated when truncated reasoning-model outputs were silently replaced by local labels, and we recommend safeguards for cascade evaluation. CreateScore is supported as an auditable cost-reduction mechanism, not yet as a targeted error detector, and is not an autonomous hiring system.
Read the full abstract

Thursday 1 October

Statistics Applications · Cross-listed from physics.soc-ph

The efficacy-competitiveness frontier in the Swiss system

An ideal tournament is both efficacious and attractive: it selects the best player as the winner, but only at the end of the tournament in order to maintain suspense as long as possible. We identify and explore the trade-off between these criteria in Swiss-system chess team tournaments via simulations. Organising more rounds is shown to improve efficacy but increase the probability of a stakeless last round. The 2008 reform in the Chess Olympiad ranking is found to be a questionable move, as the previous rule based on board points determines the efficacy-competitiveness frontier, regardless of whether two or three points are awarded for winning a match. Combining board points and match points is better than using match points only. Our results can help optimise the design of the Swiss system, which, besides chess, is also widely used in e-sports.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Identifying Panel Conditioning with Refreshment Samples: Sharp Bounds and Design Assumptions

Refreshment samples are the standard remedy for panel attrition, and the identification results behind them maintain that participation does not change measurement. We characterize what a refreshment sample identifies about panel conditioning, modelled as a deterministic monotone map at reinterview, when attrition is unrestricted. A candidate map is consistent with the data if and only if the retention-scaled distribution of the stayers' implied latent outcomes is setwise dominated by the refreshment distribution; every such map is rationalized by an explicit attrition process. Without attrition the map is identified on the latent-outcome support; with attrition, a density-ratio condition governs the identified set, and tail behaviour alone does not determine it. For an unrestricted map the survivors' mean effect has the familiar trimming bounds; for an item with all categories reported, the model reduces to a test of no conditioning. Within a cohort, other waves, dropout patterns and entry-wave items leave the set unchanged unless restrictions link selection across waves. Under explicit selection restrictions, survival matching, symmetric matching and entry-wave correction identify survivor effects. We derive their biases and give rank conditions under which refreshment schedules identify curvature in the conditioning path. A Japanese panel illustrates the results.
Read the full abstract

Statistics Applications

On relations between the Cochran–Mantel–Haenszel test and Augmented Inverse Probability Weighting

This note revisits the Cochran-Mantel-Haenszel (CMH) test in light of the increasing use of causal inference methods in clinical trials. It shows that the CMH test statistic can be interpreted as a standardized version of the augmented inverse probability weighted (AIPW) estimate of the causal average treatment effect. Consequently, it is not restricted to an interpretation as a test of the common-odds ratio in a stratified logistic regression model. The note then discusses conditional and unconditional variances for this estimator as well as variance estimation and its implications for testing. The results clarify the relationship between the classical conditional CMH test and Wald-type AIPW tests, and show that, under equal treatment allocation, the CMH test is asymptotically conservative for the weak null hypothesis of zero average treatment effect. %This note revisits the Cochran–Mantel–Haenszel (CMH) test in light of the increasing use of causal inference methods in clinical trials. For stratified randomized trials with binary endpoints and a common treatment allocation fraction across strata, we show that the numerator of the CMH statistic is proportional to an augmented inverse probability weighted estimator of the population-average causal risk difference. This provides an interpretation of the CMH test that does not depend on the common-odds-ratio model often used to motivate it. We then compare conditional and unconditional variance concepts for this estimator and discuss their implications for testing. The results clarify the relationship between the classical conditional CMH test and Wald-type AIPW tests, and show that, under equal treatment allocation, the CMH test is asymptotically conservative for the weak null hypothesis of zero average treatment effect.
Read the full abstract

Statistics Applications

Efficient Convex Optimization Methods for State-Space Heartbeat Dynamics Models with Gamma Generalized Linear Models

Objective: This work proposes a dynamic state-space statistical model of heartbeat dynamics whose optimal state estimate can be solved efficiently with convex optimization and used to assess autonomic and cardiovascular health. Methods: This state-space model combines a Gauss-Markov process prior with gamma emissions using a log-link function. Parameters are estimated using a static version of the model, and alternating direction method of multipliers is used to efficiently find the optimal maximum a priori latent state estimate. Model selection is performed by finding the tradeoff between model size and model fit using the partial autocorrelation function. Results: We demonstrate in synthetically generated heartbeats that this model is able to closely recover the underlying latent weights. Then, we show using a tilt table dataset that this statistical model is able to better capture sharp transients in heart rate patterns than traditional local averaging methods. In addition, combining heart beat timings with information from other organs such as the respiratory phase, we show that our statistical modeling framework enables the extraction of information-theoretic measures to quantify inter-organ coupling such as respiratory sinus arrythmia. Conclusion: This statistical framework for modeling heartbeat dynamics provides an efficient framework that can provide physiologically meaningful insights. Significance: This work broadens potential tools that can be used to assess heartbeat dynamics.
Read the full abstract