Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Other Statistics

stat.OT Watch Other Statistics (sign in to use)

Wednesday 14 October · still arriving

Other Statistics

Differentially Private Learning of Markov Chains: Near-Optimal Sample Complexity for Item and User Level Privacy

Sequential data such as documents, conversations, and trajectories are naturally partitioned across users, motivating privacy guarantees that protect each user's entire trajectory. We initiate the study of privately estimating the transition matrix of a discrete ergodic Markov chain from N independent trajectories of length T, with arbitrary initial distributions, under maximum row-wise ell 1 error. Besides N, T, and the number of states, our bounds depend on the minimum stationary probability and the pseudo-spectral gap, which govern state coverage and mixing. Under item-level privacy, we give upper and lower bounds that separate the costs of sampling, state occupancy and mixing, and privacy. Under user-level privacy, we reduce the problem for sufficiently long trajectories to private product distribution estimation. Along the way, we establish optimal user-level private bounds for estimating products of discrete distributions, which may be of independent interest. In the user level setting, our bounds are nearly tight in the large T regime, whereas in the item-level setting, our bounds as tight up-to a non-private term.
Read the full abstract

Sunday 11 October

Other Statistics

AI-Guided Assignments in Practice: Students' Experiences in a Large Introductory Statistics Course

Large language models (LLMs) raise new questions for instructors, among them how to integrate LLMs productively into coursework and what to evaluate in students' work when an LLM can produce much of it directly. LLM-guided homework assignments offer one answer. Instructors configure the LLM to guide students in ways that achieve specific pedagogical goals, such as recalling material, discovering new concepts, or critically evaluating deliberately flawed reasoning, and grade the full student–LLM conversation, and with it the student's reasoning, rather than only the final answer. In this paper, we report on the large-scale deployment of such assignments, implemented through LLteacher, an open-source tool that supports recall, discovery, and critical-thinking assignments, in a large introductory statistics course serving mostly pre-major undergraduates. Results on 134 students show meaningful gains in self-reported confidence, with over 80% agreeing that the assignments improved their comprehension of course material and their ability to interpret statistical results; gains were largest among students who started with lower confidence, suggesting this approach may help narrow gaps in preparation. Interaction patterns varied widely, from long single messages to rapid iterative exchanges, and hallucinations, when present, were rarely disruptive to learning.
Read the full abstract

All authors Emanuela Furfaro, Katherine Elizabeth Lovelace, Yuhan Zhang, Simone Mosciatti

Wednesday 7 October

Other Statistics

Generative AI in Publishing: An Editors' Panel on Ethics and Policies

Generative artificial intelligence (GenAI) is reshaping scholarly research faster than journals have developed stable norms for its use. This article presents an edited thematic account of a 2026 Joint Statistical Meetings panel that brought together editorial perspectives from mathematical statistics, data science, biomedical statistics, and general statistical scholarship. The discussion examines journal policies, disclosure, authorship and research integrity, peer-review confidentiality, researcher training, editorial workload, access, and possible future models of scholarly publishing. Panelists shared commitments to human accountability, the protection of confidential submissions, and disclosure of consequential assistance, while offering different recommendations on assistance with research ideas and proofs, disclosure requirements, automated review, and policy enforcement. By distinguishing shared principles from unresolved implementation questions, the article clarifies the choices facing statistical publishing and outlines an agenda for evaluating policies and practices as GenAI evolves. The account seeks to foster continued discussion of GenAI in scientific communication and encourage statistical organizations to develop more robust operational standards.
Read the full abstract

All authors Jun Yan, Mark Glickman, Paul S. Albert, Yulin Hswen, Lan Wang, Minge Xie

Tuesday 29 September

Other Statistics

Evidence Over Ideology: A Quantitative Assessment of LTNs and Blanket 20 mph Regimes in London

We evaluate the road safety implications of Low Traffic Neighbourhoods (LTNs) and blanket 20 mph speed limits in London using injury-collision records (2020-2024). Because London implemented 20 mph limits primarily via sign-only orders without physical traffic calming, this work evaluates the blanket statutory model rather than lower urban speeds in principle. For LTNs, we apply a causal pipeline combining naive pre/post comparisons, Empirical Bayes shrinkage for Regression to the Mean (RTM), and spatial Difference-in-Differences. Naive comparisons suggest substantial crash reductions, but Empirical Bayes adjustment reveals much of this reflects mean reversion. After correction, only one zone exhibits a statistically significant inside-area reduction. We find no consistent evidence of boundary-road crash displacement nor uniform inside safety gains. Given the modest corrected benefits, small journey-time increases on boundary roads (approximately 10 seconds per vehicle) monetarily negate safety gains. For 20 mph versus 30 mph roads, five associational methods across 14 contexts show that the aggregate difference in the fatal or serious (KSI) collision share is negligible (0.6 percentage points). Controlling the false discovery rate, four contexts display robustly higher KSI shares on 20 mph roads: A-roads, junctions, pedestrian crossings, and single carriageways, with no context showing a robust reduction. In movement corridors where road geometry cues higher speeds, sign-only limits fail to self-enforce. These findings highlight a structural "wrong-road problem." Indiscriminate sign-only limits impose economic drag without proportionate safety returns. Transport authorities should move away from blanket statutory defaults and return to targeted physical engineering on high-harm corridors.
Read the full abstract

Sunday 27 September

Other Statistics · Cross-listed from stat.AP

Timing of First Alzheimer's Neuroimaging: A Multicenter Analysis of Genetic, Cognitive, and Social Factors

Background. Magnetic resonance imaging (MRI) is central to Alzheimer's disease research and clinical evaluation, yet timing of first MRI after cohort entry varies substantially. In aging cohorts, many participants die before imaging, and treating death as noninformative censoring can distort inference on neuroimaging access and conceal disparities. Objective. To quantify individual-level determinants and center-level heterogeneity in timing of first neuroimaging while accounting for the competing risk of death. Methods. We analyzed National Alzheimer's Coordinating Center Uniform Data Set v3 linked to MRI records from 2015 through March 2025. The study included 20,867 participants aged 50-95 years without dementia at baseline and with known APOE {\epsilon}4 status. Time from baseline to first post-baseline MRI was the primary event, with death before MRI treated as a competing event. We fit Bayesian multilevel cause-specific accelerated failure time models with Alzheimer's Disease Center random intercepts, using Weibull and log-logistic models for MRI timing and death, respectively. Frequentist accelerated failure time and Cox models were used for sensitivity analyses. Results. APOE {\epsilon}4 was not credibly associated with timing of first MRI. Older age, lower educational attainment, and non-White race were associated with delayed MRI acquisition. Center-level variation in MRI timing was substantial. Conversely, APOE {\epsilon}4 homozygosity was associated with earlier death prior to MRI, alongside strong effects of age and cognitive impairment. Conclusions. Timing of first MRI in Alzheimer's research cohorts is driven primarily by demographic, social, and center-level factors rather than genetic risk. Accounting for competing mortality and site heterogeneity is necessary to avoid biased neuroimaging samples and support equitable Alzheimer's research.
Read the full abstract

2609.33293 · 27 Sep · 27 pages, 8 figures, and 4 tables. This work was presented at the Alzheimer's Association International Conference (AAIC) 2026 in London, United Kingdom; the conference abstract was published in an online supplement to Alzheimer's & Dementia: The Journal of the Alzheimer's Association

Friday 25 September

Other Statistics · Cross-listed from cs.SI

Network Analysis in Communication Research: Research Topics, Knowledge Organization, and Research Practices

Communication network research explains access to information, patterns of participation, and the organization of public meaning through different relationships and observations. This integrative review connects research topics, knowledge organization, and research practices, using bibliographic analysis of a Web of Science candidate pool to guide selective reading. Three judgments emerge from the comparisons. First, the relevance of a connection depends on the task and the criterion of value: team members anticipate consulting colleagues whose expertise they recognize, while journalists distinguish monitoring sources, using their information, and citing them. Second, commonality at one level can coexist with differentiation at another: audiences share outlets while selecting different articles, and shared issue agendas accommodate different evaluations. Third, some differences remain unresolved: contrasting media-use influence findings cannot be explained simply by whether models include selection and content co-nomination. Reading the uses of homophily, transactive memory, sourcing, agenda-setting, and framing resources clarifies which expectations and observations support these judgments. Selected citation contexts also show how conceptual and measurement resources enter the same argument. The resulting agenda calls for comparisons of relationship types across group stages, source use across reporting tasks, and encountered content with recipients' interpretations. These purposive comparisons establish specific connections among literatures without estimating their prevalence or demonstrating a common causal mechanism.
Read the full abstract

Saturday 19 September

Other Statistics

See You at the Posterior Line: Learning Bayesian Modeling Through a Car Racing Game

We present an interactive classroom activity designed to address a central challenge in teaching introductory Bayesian statistics: how to formalize subjective knowledge and available information into prior distributions and then update them with empirical data. Role-playing as data analysts for a racing team, students evaluate candidate tires by converting qualitative engineering reports into prior distributions, collecting primary data via a virtual racing game, and using a Beta-Binomial model to inform team strategy. This discovery-based exercise allows small groups to observe directly how different prior choices and sample data jointly shape posterior inference. Student feedback ($n=32$) highlights high enjoyment, engagement and improved conceptual clarity. Open-access materials to implement the activity are provided, alongside recommendations for adapting it to other teaching contexts.
Read the full abstract

Thursday 17 September

Other Statistics

Exact analysis of a split–merge queue with latent Erlang-factor dependent subtask times

This paper studies a two-server split–merge queue with positively dependent subtask service times modeled through a latent-factor bivariate Erlang construction. An exact characterization of the split–merge completion time is obtained, including explicit formulas for its first two moments and the resulting mean waiting time. Under fixed marginal service-time distributions, independence is shown to stochastically increase the completion time and hence overestimate mean waiting time. Numerical illustrations show that this benchmark gap can be substantial.
Read the full abstract

Tuesday 15 September

Other Statistics

When Can World Models Recover Physical Laws?

Accurate prediction does not establish that a world model has recovered a physical law: distinct dynamics can generate identical records under the same observation protocol. We formulate law recovery on a fixed physical domain under an explicit catalog of experiments, sensor uncertainty, and an acquisition budget. A rate–distortion converse separates the information needed to describe a law from the information the apparatus can reveal. Its constructive counterpart gives a finite response codebook and an explicit decoding budget. On compact world classes, uniform recovery is possible exactly when every pair of different laws is experimentally distinguishable; equivalently, the apparatus can recover all the entropy of every finite law source. An inverse response modulus quantifies stability. For Lipschitz fields on a $d$-dimensional state–action domain, noisy full-state readouts after resets require minimax budget $\Theta(\varepsilon^{-(d+4)/2})$ for squared law error $\varepsilon$, compared with $\Theta(\varepsilon^{-(d+2)/2})$ for direct field observations. Exact crossing-time symmetries establish the lower bound under adaptive experiment selection and arbitrary durations with constant inputs. Reproducible synthetic cases illustrate the separate roles of intervention, calibration, and repeated measurement. Together, the results identify which evidence supports a claim of physical-law recovery and the cost of acquiring it.
Read the full abstract

Other Statistics · Cross-listed from cs.LG

Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation

Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.
Read the full abstract

Tuesday 8 September

Other Statistics

The Design and Implementation of a Virtual Statistical Computing Lab to Teach R Coding to Introductory Statistics Students

Motivated by national calls for computationally enriched, data-centric instruction across the statistics curriculum, this study investigates the design, implementation, and impact of a Virtual Statistical Computing Lab (VSCL) integrated into an introductory statistics course at a medium-sized minority-serving university in the USA. The redesigned course embedded R-based coding through two virtual lab formats: Design I (a static Posit Cloud environment) and Design II (an interactive learnr-based interface). Using a quasi-experimental design across three instructional formats, traditional (no lab), Design I, and Design II, we evaluated students' conceptual learning gains, levels of data science (DS) readiness, and DS aspirations. The results indicated significant learning gains across all groups, with the highest gains observed in Design II. Students in both VSCL formats achieved greater gains in DS readiness than the traditional group, with Design II again yielding the largest gains across the demographic subgroups. Conversely, DS aspirations remained low or declined, suggesting a gap between skill acquisition and long-term interest. These findings highlight the value of structured, interactive computing environments in supporting statistical reasoning and building confidence in modern data tools. They also point to the need for intentional curricular bridges and career mentoring to help students translate early computing exposure into sustained academic and professional pathways in statistics and data science.
Read the full abstract

All authors Sayed A Mostafa, Tamer M Elbayoumi, Seongtae Kim, Oluwatobi Akinbode

Saturday 5 September

Other Statistics · Cross-listed from econ.GN

Housing Price Appreciation, Housing Affordability, and the Spatial Restructuring of Toronto Commuting

This study investigates the spatial relationships among housing-market appreciation, housing affordability, and automobile commuting to Toronto across 23 census metropolitan areas and municipalities in southern Ontario from 2016 to 2021. Utilizing Statistics Canada Census data, the analysis employed linear and quadratic ordinary least-squares regression, Global and Local Moran's I, and residual spatial diagnostics to assess geographic patterns in average dwelling values, owner shelter-cost burden, and lone-driver commuting. The study found that average dwelling values increased significantly throughout the region, maintaining a strong nonlinear relationship with distance from Toronto, which indicates a persistent metropolitan accessibility gradient. Commuter volumes also demonstrated pronounced nonlinear distance gradients; however, housing appreciation was not significantly correlated with percentage changes in commuting ($R^2$ = .045, p = .331), suggesting that the expansion of the housing market did not result in proportional growth in Toronto-bound automobile travel. The shelter-cost burden exhibited weaker distance relationships and greater residual spatial dependence, indicating localized affordability influences not fully explained by distance or dwelling values. Overall, the findings suggest an increasingly extensive regional housing market with spatially heterogeneous affordability and commuting outcomes, highlighting the importance of integrating housing, transportation, and regional planning.
Read the full abstract

Thursday 3 September

Other Statistics

Statistical Theory in the Age of Machine-Assisted Mathematics: Rethinking How Theory Is Made and Taught

The computational revolution is advancing at an unprecedented pace. The combination of proof-assistant technologies and generative AI tools has recently enabled the solution of complex problems in pure mathematics at a scale that seemed unattainable only a few years ago. However, these technologies have not yet become standard tools in the development of statistical theory. In this paper, we do not present new theoretical results. Instead, we discuss five case studies involving classical problems in statistics and describe how they can be analyzed using a machine proof-checking. Our goal is not to propose a definitive workflow, but to stimulate reflection on how these technologies may transform theoretical research and advanced statistical education. We focus on two main aspects. First, statistical theory often compresses substantial mathematical content into expressions such as "under the usual regularity conditions". Formalization in a machine-verifiable language forces each assumption to be explicit, reveal hidden dependencies, and provide a deeper understanding of the formalized objects. Second, we argue that the statistical community could benefit from a collaborative effort to build repositories of formalized axioms, definitions, and theorems, supporting more precise and reliable theoretical developments. Finally, we discuss the role of these tools in graduate education. Just as high-level programming languages revolutionized empirical research by enabling rapid experimentation and prototyping, machine-assisted formalization may introduce a new paradigm for the development, verification, and communication of statistical theory.
Read the full abstract

Tuesday 1 September

Other Statistics · Cross-listed from stat.ME

Scalable Inversion of Contests with Correlated Performances, Including Softmax and Multinomial Probit

Multinomial probit choice probabilities over n alternatives are Gaussian orthant integrals, computed by simulation for thirty years, one expensive integral per alternative. Inversion, which is to say determining item attractiveness consistent with a prescribed choice probability vector, is even more difficult and has been considered impractical for correlated contests when n is large. Yet here, for families lying within a grammar including factor, block and hierarchical covariance structures, we exhibit a calibration tested at n = 1,000,000 reproducing probabilities to very high accuracy, even in the extreme tail. We must return to much smaller problems for any performance comparison to be possible due to limitations of the prior art. The Geweke-Hajivassiliou-Keane simulator is the standard (and still appropriate for high rank) but is two hundred times slower already at n = 200, and its measured cost grows roughly as n^2.8 while ours is linear. Furthermore our approach applies to any continuous performance distributions within reason: the Thurstone-Mosteller model families thereby become a practical alternative to logit at modern scale.
Read the full abstract

Sunday 30 August

Other Statistics

Statistical Leadership of What? Statistics After AI

Statisticians have spent over a century arguing that we are more than calculators, usually by pointing to what else we know. AI is making that defense harder, since the list of what only statisticians can do grows shorter with each model release. AI makes claims cheap to generate and may eventually make the statistics behind them cheap too. However, a model cannot be answerable in the way that statistical practice requires. Statistical leadership then becomes a question of which claims we are there to answer for, including the ones we answer for in advance by building judgment into systems.
Read the full abstract

Friday 28 August

Other Statistics

When to take one box, when to take two: A concrete analysis of Newcomb's problem

Fifty-seven years of debate have not led to a consensus on Newcomb's problem. I argue that this failure comes from not viewing the problem in concrete terms, for which common-sense intuitions are clearly applicable. If near-perfect predictions are not required - only predictions good enough that Evidential Decision Theory (EDT) recommends taking only one box - Newcomb scenarios without any fantastic elements are actually feasible. I give a detailed account of such concrete scenarios, in which the decision-maker has varying amounts of information about the prediction process. I argue that in these scenarios it is rational to take both boxes, as Causal Decision Theory (CDT) recommends. To make this more convincing, I introduce a simpler impersonal Newcomb problem, also with a concrete realization, to help bridge causal intuitions from everyday decisions to the standard Newcomb scenario. I also argue that Newcomb's problem with nearly perfect predictions can only be realized by a (currently fantastic) process in which the Predictor accurately simulates the thoughts of a Participant. In a functionalist view of consciousness, this simulation will be a conscious entity, whose experience will not be distinguishable from that of a “real” person. In this scenario, CDT recommends taking one box. I conclude that when Newcomb's problem is properly analysed in concrete terms, CDT gives the correct answer in all scenarios.
Read the full abstract

Thursday 27 August

Other Statistics

How many cards, until the first ace: variations, extensions, lachrymae, confidence, Dirichlets

From a deck of cards, how many cards do I need to draw, until the first ace? I identify the distribution for this waiting time $T$, and its satisfyingly nice expected value ${\rm E}\,T=(N+1)/(n+1)$, with $N$ the number of cards and $n$ the number of aces; hence $53/5=10.6$ for the standard setup. After having solved this Question One I go on to certain alternative solutions and extensions, involving e.g. Beta approximations. I also consider the distributions and means for the 2nd, the 3rd, the 4th occurrences of aces, with generalisations, where there is a Dirichlet distribution in wait for us, with further links to order statistics for the uniform. Furthermore, an apparatus is developed for obtaining estimators and full confidence distributions for applications where one knows the number $n$ of aces, but not the deck size $N$; and correspondingly for inference about the unknown population size $N$ when $n$ is known. If you have 1000 people in a room, and need to interview 11 of them until you've found the first left-handed person, how may left-handed are there in the room – here we need both an estimate and a clear measure of uncertainty.
Read the full abstract

Other Statistics · Cross-listed from eess.SP

Stagewise Anomaly Detection for E-Transaxle Quality Monitoring Using Wavelet and STFT Features

This paper presents two interpretable machine-learning frameworks for quality screening of e-transaxle assemblies in electric vehicles: a Stagewise Wavelet Isolation Forest (SWIF) framework and a short-time Fourier transform (STFT)-based diagnostic framework. High-dimensional vibration signals acquired from front and back accelerometers are analyzed across multiple operating stages to capture stage-dependent vibration behavior. In the SWIF framework, signals are decomposed using a five-level Daubechies-4 discrete wavelet transform, and blockwise mean-squared coefficients are extracted from the selected wavelet detail level to obtain compact multiscale features. In the STFT-based framework, dominant-frequency trends are extracted from time-frequency representations and summarized through regression coefficients with respect to instantaneous motor speed. Anomaly detection models are trained using accepted production units under the assumption that only a small fraction of accepted assemblies contain latent defects, and their performance is evaluated using road-tested units with validated quality outcomes. Experiments on production and road-tested e-transaxle units show that both approaches provide interpretable diagnostic information, while SWIF achieves the most favorable balance between defect detection and false-positive control. Compared with the STFT-based method and alternative anomaly detectors, SWIF combined with Isolation Forest yields lower anomaly rates within the Accept population while identifying high-risk units from the Reject population. The stagewise structure further localizes anomalous behavior to specific operating conditions, supporting root-cause analysis and targeted process improvement.
Read the full abstract

All authors Mohammad N. Bisheh, Rajesh Gupta, Qian Wang, Mohammad Babakmehr, Colin Brady, Parinaz Farajiparvar, Saurabh Singh, Kamran Payanabar

Other Statistics

Algorithms for optimizing model-based incomplete block designs

Because of time limitations or participation burden, the treatments in an experimental design can be too large for a single subject. Instead of addressing this using combinatorial incomplete block designs, we propose a model-based approach that optimizes model parameters. This offers distinct advantages: it incorporates subject-specific covariates to tailor treatment allocation to individual characteristics, allows for varying block sizes, and eliminates the equal-treatment replication requirement. Despite these benefits, model-based approaches are limited by a lack of software and prohibitively large search spaces, making exact optimization computationally intractable. Therefore, we present local search heuristic algorithms and compare them to existing methods. We evaluate first- and best-improvement algorithms, simulated annealing (SA), threshold accepting (TA), and two novel algorithms utilizing directional derivatives (dd) to guide exchanges. Serving as discrete versions of continuous gradient-based methods, these dd algorithms take smaller steps and avoid flat regions by prioritizing large-difference dd exchanges. Our broadly applicable approach uses item calibration in achievement tests as a comparative example to evaluate objective values and computational times. Results demonstrate that for larger problems, the dd algorithms achieve near-optimal solutions significantly faster than SA and TA. Due to computational efficiency, our algorithms offer a highly appealing approach for practical applications.
Read the full abstract

Monday 24 August

Other Statistics

Data Assimilation: Addressing Spurious Correlations and Scalability Issues

Data assimilation in the form of the ensemble Kalman filter (EnKF) is commonly used to combine large-scale physics-based models and real-world observations of a quantity of interest. However, the EnKF is known to be adversely affected by spurious correlations when applied to Numerical Weather Prediction (NWP) models with high-dimensional state spaces. To mitigate the effect of spurious correlations, various localization methods have been proposed in the NWP literature. Unfortunately, the efficacy of and need for these methods in the setting of general PDE-based forward models with large spatial mesh sizes is not well-understood. We conducted a numerical data assimilation experiment on an example PDE model to demonstrate the presence of spurious correlations and need for localization methods outside of the context of NWP. In addition, we compared the computational efficiency of various EnKF implementations in high-dimensional settings for which theoretical complexity bounds are available but empirical costs are unclear.
Read the full abstract

Other Statistics

Becoming Good Stewards of Information: A framework for integrating ethical, civic, and professional formation throughout the statistics and data science curriculum

Recent recommendations in statistics and data science education emphasize goals that extend beyond content mastery, including statistical literacy, evidence-based decision-making, communication, ethics, responsible use of data, and civic responsibility. We argue these goals can all be understood through a common lens: helping students become better stewards of information. Stewardship of information provides a unifying framework for the formation of students as statisticians, data scientists, and citizens who are thoughtful producers and consumers of data and evidence, communicate findings with integrity and transparency, engage critically with data ethics, recognize the human beings behind the data, and harness data to make informed decisions, serve communities, and advance the common good. We present stewardship of information as a developmental model that can be introduced early and revisited with increasing sophistication across a curriculum, including in introductory, advanced, applied, and theoretical courses. To illustrate this approach, we provide concrete examples of course activities and curated weekly readings as a low-barrier implementation strategy and present preliminary evidence of student engagement with the framework. By offering both a conceptual foundation and practical implementation strategies, this paper contributes a coherent approach for integrating ethical, professional, and civic formation across the statistics and data science curriculum.
Read the full abstract

Sunday 23 August

Other Statistics

Data Quality Assessments: A Theoretically Structured Overview of Approaches and Methods

The quality of data is crucial for both practice and academia, and this holds for descriptive statistics, AI and advanced analytics alike. This study addresses an apparent gap in the literature and as such presents a full and theoretically grounded overview of data quality assessment approaches and methods, as well as their defining characteristics and inter-relationships. For this purpose a broad typology is introduced that employs two theoretical dimensions, namely the evaluation logic (formal versus informal) and the assessment driver (norms versus data). This yields four high-level approaches and 15 methods for assessing data quality. The research results are relevant for academia, as they provide a theory-based overview and definition of the ways that data quality can be evaluated. The study is also relevant for practice because it allows professionals to make informed decisions on using these methods, e.g. as part of an audit or broad data quality assessment strategy.
Read the full abstract

Saturday 22 August

Other Statistics

Inferential Validity of Digital Health Measures

Digital health measures increasingly inform treatment evaluation, risk classification, clinical monitoring, and regulatory decisions. Existing validity concepts concern a measure's technical soundness, clinical meaningfulness, usability, and scalability. Inferential validity concerns how well a measure supports a scientific, clinical, or regulatory conclusion. We propose assessing inferential validity as a downstream evaluation layer that links measurement system evidence to trustworthy conclusions.
Read the full abstract

All authors Carmen D. Tekwe, Mercy Oladuti, Yuanyuan Luan, Andi Mai, Orfeu M. Buxton, See Ling Loy, Jeffrey S. Gonzalez, Roger S. Zoh

Thursday 20 August

Other Statistics

Statistics in the Age of AI

Artificial intelligence (AI) can automate programming, model fitting, visualization, simulation, literature synthesis, and increasingly sophisticated methodological tasks, but it cannot remove the logical conditions under which data support scientific claims or consequential decisions. We formalize these conditions through statistical warrant, which connects data to a claim through the target, observation regime, assumptions, procedure, uncertainty assessment, validation criterion, loss structure, governance and accountability. No algorithm can consistently recover a target that is not identified by the observation regime without additional information or assumptions. From this principle, we organize the argument around five statements. Questions and targets are integral to statistical methods. Data acquire evidential meaning only through design, provenance, and assumptions. Description, prediction, causal inference, and decision are mathematically distinct tasks. Analytical abundance requires accounting for how analyses are selected, uncertainty across the analytical system, and deployment validation. The statistician's fundamental role is therefore to construct, criticize, and safeguard statistical warrant, including by developing new methodology when existing theory is inadequate. This role requires statistical reasoning and attributable human and institutional responsibility.
Read the full abstract

Other Statistics · Cross-listed from stat.AP

Copula-Based Reconstruction and Clustering of Coccidioides Minimum Inhibitory Concentration Profiles

Coccidioidomycosis is a fungal lung infection endemic to parts of the Pacific Northwest and southwestern United States, Mexico, Central America, and South America. A 2017 study by Thompson et al. reported that many analyzed Coccidioides isolates had elevated minimum inhibitory concentration values for fluconazole but comparatively low minimum inhibitory concentration values for other triazole drugs. We constructed 172 synthetic joint minimum inhibitory concentration profiles from the published drug-specific marginal frequency tables. Values sampled from each marginal distribution were paired across drugs using a Gaussian copula, and potential cross-drug patterns were examined using k-means clustering, hierarchical clustering, and Gaussian mixture models. On the selected reconstruction, a forced four-cluster partition consistently identified an upper caspofungin minimum-inhibitory-concentration tail. Across the evaluated dependence scenarios, the gap statistic favored a single cluster in 96% to 100% of nested reconstructions, providing no evidence that the reconstructed multivariate data supported a broader multicluster structure. A separate simulation study examined recovery of an upper-severity-score category defined from prespecified quantiles of a composite minimum-inhibitory-concentration score. The study included 4,000 replications across 20 prevalence and measurement-noise conditions. The multiclass adjusted Rand index ranged from moderate to high depending on the method and prevalence, whereas binary partition agreement for the upper-severity-score category approached zero at 1% prevalence even though recall remained near 1.0. Clustering conclusions therefore depended on the assumed cross-drug dependence structure and on the realized number of observations in the target category.
Read the full abstract

Monday 17 August

Other Statistics · Cross-listed from stat.ME

Identifying Model Quality Effects on User Engagement: A Within-Version Causal Estimator with Synthetic Data Validation

Every team building Large Language Models (LLMs) faces a core challenge: offline benchmarks show performance gains and user engagement rises post deployment, but isolating cause from effect remains difficult. Simultaneous marketing, media coverage, and seasonal demand obscure whether model updates truly drive engagement gains. This paper presents a novel causal estimation approach that leverages non uniform quality improvements across capabilities within a single model version. Because capabilities improve unevenly (e.g., strong gains in coding versus modest gains in writing), users experience varied quality depending on their task distribution. This variation in experienced quality provides causal signal for estimation. We validate this approach on synthetic data with known ground truth. The raw estimator recovers 81% to 88% of the true effect, with the remainder lost to measurement error in usage estimates. Applying an errors in variables disattenuation adjustment (using a test retest reliability ratio mean correlation of 0.861) corrects the estimate to 1.017 (bootstrapped 95% CI: 0.915 to 1.109). By contrast, naive methods fail significantly, recovering only 63% without version controls and 62% using real time rather than frozen usage patterns. Permutation tests confirm the framework distinguishes true causal effects from noise. Sensitivity analysis indicates recovery improves monotonically from 78% to 87% as pre period length increases from 3 to 7 weeks, highlighting a clear tradeoff between sample duration and estimator precision. Seed sensitivity tests further confirm stability across random draws.
Read the full abstract

Saturday 15 August

Other Statistics · Cross-listed from stat.ME

Quantile-stratified sampling for multivariate normal simulations and other multivariate distributions

In this paper we show how to extend quantile-stratified sampling to produce simulations from various multivariate distributions. These simulations have desirable space-filling and coverage properties relative to simulation using IID sampling. We examine the coverage performance of these simulations against IID sampling by looking at plots of ordered log-density values from the simulations.
Read the full abstract

Friday 14 August

Other Statistics

A Tale of Two Pathways to Gompertz Mortality: Reliability and Vitality from an Actuarial Perspective

This paper studies two mechanistic explanations for human mortality by examining reliability theory and vitality modelling through a unified actuarial perspective. While the two approaches arise from different ageing mechanisms, we show that both can naturally generate the Gompertz law under suitable assumptions and can be extended to produce the Makeham law and late-life mortality plateaus. Using Canadian mortality data, we investigate the empirical behaviour of each approach and highlight the roles of heterogeneity, extrinsic risk, and stochastic randomness. Furthermore, we develop parallel definitions of biological age under both approaches and analyse how subjective survival beliefs emerge from misspecified parameters. Our comparison of these two approaches provides actuarial insights into the natural foundations of Gompertz mortality and the interpretation of ageing, frailty and death.
Read the full abstract

Thursday 13 August

Other Statistics

A Copula-Based Framework for Multivariate Zero-Inflated Mixed Poisson Models

Multivariate count data often contain overdispersion, excess zeros, and complex dependence. Existing multivariate zero-inflated count models usually use a single dependence structure to jointly model structural zeros and count outcomes. This makes the two sources of dependence difficult to interpret separately and often leads to slow computation or intractable likelihoods. This paper proposes a general framework for multivariate zero-inflated mixed Poisson models that explicitly separates these two sources of dependence within a unified likelihood-based formulation. The proposed hierarchical model combines zero-inflated mixed Poisson marginals with separate dependence models for the structural-zero and latent-intensity components. Dependence among structural zeros is modeled by standard parametric copulas, while dependence among latent mixing variables is modeled by checkerboard copulas. Likelihood-based inference is carried out using an inference-functions-for-margins procedure, and the proposed likelihood also supports full maximum likelihood estimation when computationally feasible. Simulation studies show accurate parameter estimation and good finite-sample performance under different dependence structures and latent mixing distributions. Applications to benchmark datasets from existing statistical software and a healthcare utilization dataset demonstrate the flexibility, computational efficiency, and practical usefulness of the proposed framework.
Read the full abstract

Wednesday 12 August

Other Statistics · Cross-listed from cs.LG

Interpretable Causal Discovery via Causal-Effect Constraints

Causal discovery aims to uncover the underlying causal relationships given data generated from a system. The goal, however, is not merely to predict causal edges given data, but also to be able to interpret and explain either observed or hypothesized phenomena, such as a particularly large causal effect. We consider this task of conditional causal discovery and cast it as a Bayesian inference problem, in which we target the posterior over causal graphs and parameters conditional on an event such as a causal-effect constraint. Unfortunately, this poses a computational challenge: existing approaches to Bayesian causal discovery struggle when the event has small posterior mass. To address this, we adapt rare-event estimation techniques to perform inference the joint graph-parameter space. Our method gradually drives a particle population toward the constrained region while maintaining samples that approximate the conditional posterior. Empirical evaluation on synthetic graphs validates the accuracy of our approach at small and large scales, and we show in a case study on the Sachs protein dataset how our method can be used to aid scientific exploration by providing pathway-level summaries.
Read the full abstract