Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics Statistics Applications

stat.AP Watch Statistics Applications (sign in to use)

Friday 9 October

Statistics Applications

Probabilistic Forecasting of Electric Vehicle Charging in a Time-varying Hierarchical Infrastructure

Uncertainty quantification for electric vehicle (EV) charging demand is essential to turn the increasing pressure of EV adoption on power systems into a source of flexibility. We study a large-scale dataset of public charging stations in Scotland, where demand exhibits a hierarchical structure, organized across multiple aggregation levels, from individual charging points to regional and national scales. The charging infrastructure is continuously evolving, with new stations deployed and existing ones occasionally removed, which calls for forecasting and uncertainty quantification methods tailored to this evolving structure. To address these challenges, we combine hierarchical forecasting with recent advances in conformal prediction with missing data. The conformal procedures considered produce valid prediction intervals at every node of the hierarchy and the algorithm CP-MNR-Nested successfully accounts for the time-varying hierarchical structure of the data.
Read the full abstract

All authors Guillaume Principato, Yvenn Amara-Ouali, Yannig Goude, Bachir Hamrouche, Jean-Michel Poggi

Statistics Applications

Warning of infrastructure deformation anomaly through probabilistic analysis of InSAR observations

Interferometric Synthetic Aperture Radar (InSAR) offers a low-cost, wide-area solution for near continuous infrastructure monitoring. However, translating InSAR observations into automated anomaly warnings remains challenging because displacements are relative, affected by uncertainty, and acquired along line of sight (LOS). We introduced a probabilistic warning framework that accounts for these factors. LOS displacements were modelled through a Gaussian distribution. A limit state function was defined to describe the displacement exceedance, and the time-dependent probability of exceedance was evaluated. Instead of a fixed threshold, warnings were identified through abrupt increases in exceedance probability. The framework was validated using three case studies of quay walls in Amsterdam, providing timely anomaly warnings and issued no warnings for the non-collapse case. Results demonstrate the potential of a probabilistic interpretation of InSAR time series for deformation anomaly warning.
Read the full abstract

All authors Hao Kuai, Na Hao, Pantelis Karamitopoulos, Alice Cicirello, Giorgia Giardina

Statistics Applications

A functional mixed-effects model for the joint analysis of amplitude and phase variations in function-valued traits

Quantitative genetics is the study of continuous traits and the statistical analysis of phenotypic variation, that is, the observable differences in physical traits among individuals within a population. If family relationships between individuals are known, phenotypic variation can be partitioned into genetic and environmental components. Some traits, such as growth trajectories, can be described as a function of a continuous index and are known as function-valued traits. However, function-valued traits from different individuals often vary not only in their shape (amplitude) but also in the speed at which the biological process develops (phase). In this work, we present and compare two approaches to account for amplitude and phase variations in estimating the additive genetic and environmental components. The first one follows from a conventional framework where warping functions remove phase variations, and then the aligned curves are analysed. The second approach adopts the perspective of looking at the aligned curve and the warping function as a joint functional object. In both cases, we propose a functional mixed-effects framework that uses functional principal components as the data-driven basis for genetically correlated data. The proposed procedure is applied to simulations and a real-case study of Tribolium castaneum growth trajectories.
Read the full abstract

All authors Yilin Chen, Davide Pigoli, John A.D. Aston, Patrick A. Carter

Statistics Applications · Cross-listed from econ.EM

Realized Tail-Risk Conditional Autoregressive Expectile Models: Accurate and Stable Forecasting Performance

A realized tail-risk conditional autoregressive expectile (RTCARE) model is proposed for joint forecasting of value-at-risk (VaR) and expected shortfall (ES). Realized tail measures such as the realized quantile and the realized ES, constructed from intraday high-frequency returns, are incorporated into the dynamics and measurement equations of the conditional expectile, allowing information about lower-tail risk to enter the forecasts. The parameters are estimated by Markov chain Monte Carlo, and we evaluate the resulting VaR-ES forecasts in numerical examples and in an empirical application using U.S. stock-return data. Across the considered tail levels, the RTCARE models deliver competitive and often superior forecasting performance relative to existing alternatives, with stable results across Monte Carlo replications and rolling forecast evaluations.
Read the full abstract

Statistics Applications · Cross-listed from eess.IV

Radiomics-Based Thyroid Nodule Classification with Label-Conditional Conformal Prediction

Thyroid nodule malignancy assessment relies on ultrasound imaging, yet current risk stratification systems suffer from substantial interobserver variability, contributing to a high rate of unnecessary biopsies. We propose a radiomics-based framework combining quantitative feature extraction via PyRadiomics with a Generalized Additive Model (GAM) classifier, selected among six candidates for its favorable trade-off between discriminative performance and interpretability. To complement point predictions with uncertainty estimates, Cross-Conformal Prediction with label-conditional guarantees is applied, flagging ambiguous cases for clinical review rather than forcing a binary decision. Experiments are conducted on TN5000, a publicly available ultrasound dataset of thyroid nodules. After Spearman correlation filtering, 36 features are retained and the GAM achieves an AUC of $0.89$, a sensitivity of $0.83$, and a specificity of $0.82$ on the validation set. At a significance level $\epsilon=0.09$, selected to ensure empirical coverage guarantees on both classes, 63% of nodules on the test set receive a certain (i.e., single-label) prediction, with conditional precision of $0.94$ for certain malignant and conditional Negative Predictive Value (NPV) of $0.73$ for certain benign cases, while the remaining 37% are referred to the clinician.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Design-based effect sizes and their synthesis in randomized multiple-baseline experiments

Effect sizes for single-case experimental designs usually rely on models for the baseline trend and the serial dependence, which are challenging to check with few sessions. We develop design-based effect sizes for multiple-baseline experiments with a randomized intervention onset. We define case-level and pooled standardized effects with potential outcomes and propose weighted estimators that are unbiased or asymptotically unbiased without an outcome model. The usual difference between phase means is generally biased when the untreated outcome changes over sessions. For research synthesis, we provide variance estimators and asymptotic confidence intervals that require no model for serial dependence, and variance formulas for planning. Simulations and three randomized studies of reading and writing interventions illustrate the methods.
Read the full abstract

Statistics Applications · Cross-listed from stat.ME

Sequential Conformal Calibration under Recursive Selection

Conformal prediction provides finite-sample, distribution-free coverage guarantees that can support decision-making in specific settings. In multistage problems, earlier decisions determine the populations relevant to later calibration. When later scores are refit using earlier calibration results, reusing those cases for ranking need not preserve exchangeability with a surviving test case. We study sequential conformal calibration with a fresh fold at each stage: conditional on earlier folds, the selected fold scores and the surviving test score are exchangeable. This yields finite-sample coverage guarantees for every prefix of the decision sequence. For atomless survivor score distributions, we derive a universal Binomial-thinned Beta chain for the realized coverage process. Its law depends only on fold sizes and target levels, even under predictable refitting, and yields exact coverage slack, abstention probabilities, and calibration-conditional guarantees. We further characterize the tradeoff between freshness and data reuse. For fixed scores, strict-survivor reuse is exactly valid. For adaptive scores refit using past calibration data, reusing survivors can incur worst-case rank loss, while blocked designs provide an intermediate option between full reuse and stagewise fresh folds. As a case study, we apply the framework to serial multi-echelon inventory problem and propose a sequential conditional-quantile formulation. Each stock level is learned as a context-dependent quantile of cumulative demand among scenarios satisfying earlier no-backlog constraints. We connect these targets to common-shift stationarity and derive decision-level implications for the resulting calibrated policies. In a three-echelon simulation, fresh-fold refitting achieves 81.98% final-prefix coverage against an 81.82% target while reducing mean cost by 5.5% relative to the Bonferroni baseline.
Read the full abstract

All authors Yuning Yang, Zexuan Liu, Tianci Wang, Danlin Li

Thursday 8 October

Statistics Applications · Cross-listed from cs.SI

Credit Attribution in Timestamped Soccer Networks Using Multivariate Hawkes Processes

The two main traditional player statistics used in soccer are goals scored and assists, with an assist credited when a player's pass immediately leads to a teammate scoring. Attributing credit solely to the player who makes the final pass before a goal fails to account for the contributions of multiple players in the buildup to the goal. Consequently, traditional statistics provide only a limited view of player involvement. In this paper, we use multivariate Hawkes processes (MHPs) to model timestamped sequences of passes and shots, which form continuous-time networks. MHPs provide a branching mechanism where an event (a pass or shot) can be generated as a child of a previous event, which allows us to quantify each pass's contribution to a subsequent pass or shot. This approach distributes credit across multiple passes within an attacking sequence, making it applicable across a wide range of situations. Building on this framework, we propose expected goal involvements (xGI), a metric that measures a player's involvement in creating goal scoring opportunities based on their passes. This metric identifies players who play key roles in creating goal scoring opportunities even if they do not make the final pass, offering a more comprehensive measure of attacking involvement and player performance.
Read the full abstract

Statistics Applications · Cross-listed from physics.soc-ph

Learning Social Influence with Unobserved Attention

Social influence, the tendency of our social environment to sway our beliefs and behaviors, is often cited as a driver of population-level trends in opinions or beliefs. Many quantitative theories of social influence assume that an agent interacts with its entire social environment. This assumption is often unrealistic. For example, users of online platforms typically see posts from only a small subset of the accounts they follow at any given time. We introduce a simple probabilistic agent-based model of social influence that explicitly models unobserved attention mechanisms. The first of these latent mechanisms is selection: each agent interacts with only one other agent at a time from its social environment. The second is retention: the agent chooses whether or not to ignore the opinion of the other. Our model admits an efficient maximum-likelihood algorithm which allows us to fit it to opinion traces even when these mechanisms are unobserved. On synthetic data, omitting either latent mechanism reduces the overall strength of social influence governed by the inferred interaction kernel, which hides the evidence for the underlying nonlinearity of the social influence. We study a selection of previously analyzed empirical datasets and curate several new ones. We show that models that include the latent mechanisms are often more predictive on these datasets; indeed, they typically improve prediction far more than the choice of interaction kernel does. Modeling latent attention increases held-out predictive likelihood on 14 of these 15 datasets. Each mechanism changes the size of the inferred kernel in a predictable direction, while its effect on the kernel's shape depends on the data.
Read the full abstract

All authors William Thompson, Moyi Tian, Linh Huynh, Alexander Wiedemann, Vicki Modisette, Philip S. Chodrow, Heather Z. Brooks

Statistics Applications

Mixup Barcodes for Topology-Aware Financial Decision Making

We present a topology-aware system based on mixup barcodes and persistent homology for financial decision making. The suggested approach uses topological summaries obtained from Takens delay embeddings of a univariate price series to quantify structural changes between reference and current market regimes. The 1-Wasserstein distance between persistence diagrams, a mixup barcode disruption index, and persistence entropy divergence are combined to provide a novel stress score. A Golden Cross trading method is dynamically modulated by this score, which transforms a binary buy/sell signal into a continuous position-sizing process. At embedding settings chosen by maximizing in-sample Sharpe over a 72-point grid, in-sample assessment on the S&P 500 and Bitcoin yields Sharpe ratios of 1.116 and 1.238, respectively, with much lower maximum drawdown compared with both Buy-and-Hold and the standard Golden Cross baseline. Our findings imply that topological summaries of market geometry include useful information that goes beyond traditional trend markers.
Read the full abstract

All authors Buddha Nath Sharma, Joe Opitz, Adam Moser, Sayam Palrecha, Sushovan Majhi, Atish Mitra

Statistics Applications · Cross-listed from stat.ME

Formalization and consistency of clone-censor-weight

Target trial emulation has become a standard framework for causal inference from observational data. Within this paradigm, the clone-censor-weight (CCW) methodology provides a practical way to deal with complex treatment and adherence definitions when treatment regimes are not distinguishable at baseline. Despite its increasing use, the statistical interpretation of CCW remains limited. In this work, we formalize the CCW methodology from a statistical viewpoint based on admissible sets and stochastic interventions. We characterize the causal estimands targeted by CCW, establish consistency of CCW estimators under standard identification assumptions, and provide theoretical guarantees for bootstrap-based inference methods.
Read the full abstract

All authors Iqraa Meah, Charlotte Voinot, Stefan Michiels, Noémie Simon-Tillaux, Claire Jouffroy-Zeller, Inès Vaz-Luis, Clément Berenfeld, François Petit

Statistics Applications

Regime-Aware Estimation of Large Equity Correlation Matrices: Tail-Pairwise Dependence and Dynamic Conditional Correlation Decomposition for Water and Energy Sectors

Equity correlation matrices are foundational to quantitative risk, yet the operational default of a single Gaussian-copula structure fit to the full sample has a structural defect: it forces asymptotic tail-independence and so cannot represent the joint moves that drive risk in stress episodes. We argue that the dependence of equity returns is not one object but three distinct under lower-tail, central, and upper-tail conditions, and that any estimator aimed at tail-sensitive applications should be regime-aware. We develop a decomposition that fits a dedicated dependence matrix to each regime, using a tail-dependence estimator grounded in multivariate regular variation in the two tails and a time-varying conditional-correlation model in the central regime, and recombines the three at the unconditional regime probabilities to form a steady-state matrix. The construction identifies tail-regime dependence that the Gaussian-copula benchmark structurally suppresses. Demonstrated on water- and energy-sector Exchange Traded Funds (ETF) constituents, the tail regimes reorganize the commodity partition produced by the central regime, identifying the pairs where the Gaussian-copula default systematically fails to identify joint extreme co-movement.
Read the full abstract

All authors Katherine B. Ensor, Quadri Popoola, Meredith Kruse, Daniel S. Cooley

Statistics Applications · Cross-listed from stat.ME

Design-Based Validation Method of EHR Algorithms to Address Partial Verification Bias

Electronic health record (EHR) phenotyping algorithms support clinical prediction, observational research, and quality improvement, but manual chart review for gold-standard validation is costly. For algorithms targeting rare outcomes, algorithm-negative patients often greatly outnumber algorithm-positive patients. We propose a design-based validation framework that fully verifies algorithm-positive patients and reviews a simple random sample (SRS) of algorithm-negative patients. Horvitz-Thompson estimators of false-negative (FN) and true-negative totals yield estimates of sensitivity, specificity, negative predictive value, positive predictive value, and accuracy. We derive finite-population-corrected variance estimators and confidence intervals (CIs), with sample-size planning for a prespecified CI half-width. An optional Phase 2 extension uses pilot data to assess whether risk-guided Neyman allocation may improve precision. In simulations, the design-based approach substantially reduced bias relative to naive verified-sample analyses and achieved near-nominal CI coverage. Adaptive allocation reduced root mean squared error and CI width when FNs were concentrated across risk strata, but offered little benefit when the risk signal was weak. We illustrate the framework using a breast cancer recurrence algorithm in 3,336 patients, with endpoint-specific algorithm-negative review budgets of 125, 150, and 175 charts. The framework combines SRS-based validation and precision planning with optional pilot-guided adaptive allocation.
Read the full abstract

All authors Rushi Tang, Maya Blasingame, Lillian Zheng, Joan Ac-Lumor, Saher Mubarek, Jennifer K. Plichta, Samuel I. Berchuck

Statistics Applications

How Much Does Submitting a Burst of Jobs Impact Waiting Time? An Analysis of the Jean Zay Workload Trace

This paper examines job waiting times on the Jean Zay supercomputer at IDRIS (Institut du Développement et des Ressources en Informatique Scientifique http://www.idris.fr/), driven by the need to improve scheduling strategies for crossfacility scientific workflows. Drawing on large workload traces, we show that job waiting times are not independent of one another but are instead correlated through job bursts-batches of identical jobs submitted together, also called “flurries”. We introduce a method for quantifying the effect of these bursts on waiting times, and find a strong Pearson correlation between waiting time and a job's rank within its burst. This correlation is above 0.8 for more than 40% of the jobs. We further derive an exact formula for the Slurm fairshare factor based on a convolution product, showing that the burst penalty operates on a much shorter timescale than fairshare itself-pointing to a distinct throttling mechanism specifically targeting bulk submissions. Once burst effects are accounted for, we use Welch's t-test to find that partition occupancy and number of requested cores have a significant effect on waiting time, though the effect is not significant for all partitions.
Read the full abstract

Statistics Applications · Cross-listed from econ.TH

Singular Equilibrium and Selection of Narratives

An agent learns from data that many parameters of her model explain equally well. Berk's theorem states that her belief concentrates on the parameters that best fit the data, but does not select between them. We show that, with a continuum of parameters, the posterior concentrates on the best-fitting parameters with the smallest local learning coefficient. We propose a definition of narratives, subsets of the best-fitting parameters on which the learning coefficient is constant. Narratives are characterized by the number of coincidences they require (complexity) and how exactly each must hold (robustness). When the agent's actions affect her data, every long-run action is a singular equilibrium, a best reply to a belief supported on the least complex and most robust narratives. Singular equilibrium refines Berk-Nash and self-confirming equilibrium by restricting overly complex or knife-edge off-path beliefs, and its strict uniform version characterizes uniformly stable actions.
Read the full abstract

Statistics Applications

Automated Detection of Match Phases in Football from Spatio-Temporal Tracking Data Using Graph Neural Networks

Spatio-temporal tracking data has opened new possibilities for detecting complex tactical patterns in football, yet modeling the interactive movements of multiple players remains challenging. This paper proposes a framework combining graph neural networks (GNNs) with a sequential model to classify match phases on a second-by-second basis across a seven-class taxonomy. Match phase classification is tactically meaningful, and the availability of rule-based labels across 203 matches makes it a suitable testbed for a systematic comparison of adjacency constructions and message-passing layers, a question that has received limited attention in existing research. Our selected GNN-LSTM model outperforms all aggregated-feature baselines, including XGBoost and a Long Short-Term Memory (LSTM) network, as the strongest baseline scores 4.6% lower in macro F1. Graph representations using a domain-informed Delaunay triangulation that approximates passing lanes, paired with a custom Spatial Edge-Augmented Convolution (SEAConv) layer that injects edge attributes directly into messages, achieve the best performance by capturing spatial dependencies while limiting uninformative messages from redundant edges. Integrated Gradients attributions indicate the model uses spatial player configurations, particularly horizontal positioning, in combination with possession and ball-status indicators. This work offers an automated solution for fine-grained tactical analysis, reducing the need for manual tagging and providing deeper insight into dynamic team behavior.
Read the full abstract

All authors Vincent Renner, Nils Koster, Pascal Bauer, Melanie Schienle

Statistics Applications · Cross-listed from cs.AI

TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models

System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generative interface. Software can act on these probabilities through thresholds, cost-weighted choices, and escalation rules. Consequently, if these probabilities are miscalibrated or wording-sensitive, the software ma take unintended actions leaving human operators with no textual rationale to inspect. Current evaluations largely report accuracy and calibration on public classification datasets without a clear reference. We present TypedBench, a benchmark for typed decision models like Jev built from seven policy-labelled generators and nine evaluation suites. We report accuracy as median and range across paraphrases, and calibration error relative to the finite-sample noise floor of a matched, perfectly calibrated predictor. We assess probability quality through selective prediction, ordinal proper scoring rules, and realised cost under asymmetric cost matrices. We evaluate a hosted model, an open encoder, and a family of open decoders spanning 0.8B-9B parameters on identical items. The hosted model follows the stated policy but is wording-sensitive and systematically underconfident; under asymmetric costs, using its probabilities can be worse than taking its top answer. Decoders route exactly and are slow as options or questions are added. The decoder is least accurate on policy questions and degrades with more options. Overall, typed decision models must be evaluated jointly on policy adherence, wording robustness, probability quality, and induced decision outcomes.
Read the full abstract

All authors Rahul Sharma, Andrew B. Duncan, Gaétan Marceau Caron, Sebastian J. Vollmer

Statistics Applications

Event-Aware Spatiotemporal Precipitation Forecasting with Geographic Context and Physics-Guided Regularization

Hourly precipitation forecasting involves several distinct statistical challenges, including spatially varying predictor-precipitation relationships, a strongly imbalanced precipitation distribution, and progressive degradation of precipitation event skill with increasing lead time. We develop a spatiotemporal forecasting framework that addresses these challenges through three complementary components: explicit geographic representation, event-aware learning for imbalanced precipitation, and weak asymmetric regularization derived from the atmospheric water budget. The physical information is treated as an asymmetric constraint rather than as an additional prediction target, designed to discourage physically unsupported precipitation attenuation without replacing the data-driven forecast. Experiments using ERA5 data across regional, enlarged domain, and spatial subset settings show that explicit geographic information improves spatial field prediction, while event-aware learning provides the most consistent gains in detecting moderate and heavy precipitation events. Physical regularization has a more selective effect, mainly reducing systematic underprediction over the enlarged domain while improving longer lead precipitation event prediction in the spatial subset experiment. These results indicate that the benefit of physical guidance depends on the available data regime and becomes most apparent when data-driven precipitation information deteriorates with increasing lead time.
Read the full abstract

Statistics Applications

BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments

AI evaluation pipelines often produce conflicting judgments rather than clean labels. In pairwise LLM evaluation, this conflict is especially visible: disagreement can arise from ambiguous items, underspecified rubrics, heterogeneous or unstable human raters, or an LLM judge whose verdict changes when the response order is swapped. We propose BayesJudge, an online Bayesian meta-evaluation layer for conflicting human-LLM judgment streams. For each comparison, BayesJudge estimates a panel-relative posterior verdict distribution over the two responses, with a tie or ambiguity state when such labels are available. At the same time, it estimates rater-specific human confusion matrices and LLM presentation-order bias. The method uses tie-open labels to keep ambiguity observable and paired order-swapped judge calls to separate response quality from presentation effects. We formulate the exact online posterior recursion and use a scalable Rao-Blackwellized assumed-density SMC approximation for streaming inference. Controlled synthetic experiments demonstrate recovery of prespecified evaluator parameters and illustrate two protocol-level identifiability mechanisms: tie-open labels expose ambiguity mass, and order-swapped paired judgments separate item preference from position bias. On a real-world SummEval dataset, BayesJudge successfully detects systematic presentation-order effects in LLM judge outputs, infers distinct expert and crowdworker behavior signatures without rater metadata, and produces posterior uncertainty estimates that correlate with human disagreement. Our code is available at https://anonymous.4open.science/r/BayesJudge-0879.
Read the full abstract

All authors Jiahao Zhang, Pengbin Feng, Chunlei Meng, Hang He, Tianyong Hao

Wednesday 7 October

Statistics Applications

Tree statistics improve neural simulation-based inference of epidemiological parameters over case counts alone

During an infectious disease outbreak, accurate estimates of epidemiological parameters such as the reproduction number and infection duration can inform the public health response. Recently, neural simulation-based inference (NSBI) methods have become popular for estimating these parameters due to lower computational costs and more input data flexibility than traditional methods. Input data modalities that have been evaluated independently in an NSBI framework include case counts and transmission trees. Here, we evaluate whether combining information from these two data types outperforms using information from each individually with NSBI. To do so, we simulated stochastic susceptible-infectious-recovered (SIR) outbreaks across a range of basic reproduction numbers (R0) and average infection durations (D) to generate training, validation, and testing datasets. We then calculated, for each simulated outbreak, summary statistics from case count trajectories and from transmission trees. Using these features, individually and in combination, we performed a type of NSBI called neural ratio estimation (NRE) to estimate R0 and D. We also calculated maximum likelihood estimates of the parameters by fitting a deterministic SIR model using the case count trajectory. All three NRE models outperformed the baseline method. Furthermore, incorporating tree summary statistics improved the estimation of both R0 and D over case count summaries alone. This suggests that trees provide valuable additional information for estimating epidemiological parameters above and beyond case counts alone.
Read the full abstract

All authors Zena Lapp, Emma E. Goldberg, John R. Tipton, Thea Sukianto, Dave Osthus, Lauren A. Castro

Statistics Applications

Estimating Optimal Dynamic Treatment Regimes for Personalized Education: A Tutorial and Applications with Machine Learning

Optimal dynamic treatment regimes (ODTRs) are increasingly used to generate sequences of personalized recommendations that maximize a final outcome of interest. Although ODTRs are widely used in biostatistics and precision medicine, their applications in education remain limited, especially for longitudinal decision-making. This study translates ODTR methods for single- and multi-stage designs into individualized high school math course recommendations, providing step-by-step guidelines and demonstrations with data from the High School Longitudinal Study of 2009 (HSLS:09). We use Targeted Maximum Likelihood Estimation with ensemble machine learning to estimate ODTRs that maximize students' math achievement and college enrollment, imposing feasibility constraints based on domain knowledge and propensity-score thresholds to ensure recommendations are practically implementable. Results show that unconstrained regimes yield higher estimated values but infeasible recommendations, while feasible regimes produce realistic pathways with stronger empirical and expert-informed support. Finally, we discuss practical considerations for designing ODTRs in education.
Read the full abstract

All authors Chenguang Pan, Yuxuan Li, Thomas Harris, Youmi Suk

Statistics Applications · Cross-listed from cs.SI

Some Twitch chats exhibit multifractal characteristics resembling stream-of-consciousness narrative

Live-streaming platforms generate high-frequency records of collective human activity in which many loosely coordinated users react in real time to shared external stimuli and to one another. We investigate whether the temporal organization of Twitch chat messages exhibits multifractal signatures characteristic of complex dynamical systems. Selected high-activity chats were converted into time series by measuring the length of each consecutive message in words, with platform emotes treated as individual tokens. The resulting signals display heterogeneous fluctuations, bursty organization, and heavy-tailed distributions that, in several cases, are better captured by q-Weibull forms than by conventional light-tailed models. Autocorrelation analysis reveals persistent temporal dependence over broad ranges of message lags, indicating that the variability is not produced by independent message generation alone. MFDFA shows that some chats possess well-developed multifractal spectra and generalized Hurst exponents strongly dependent on moment order, whereas others exhibit weak scaling, approximately monofractal behavior, or no well-defined scaling regime. Surrogate tests indicate that temporal correlations are essential for the observed multifractality, while heavy tails enhance its apparent strength. Under particular dynamical conditions, Twitch chat may therefore form an emergent collective signal whose fragmented, reactive, associative, and multiscale organization is structurally reminiscent of stream-of-consciousness narrative. This expression is used strictly as a structural and dynamical analogy, not as a claim of a unified collective mind or of semantic, cognitive, or phenomenological equivalence with literary stream-of-consciousness writing. The results position live-stream chat dynamics within the broader physics of complex, intermittent, human-generated signals.
Read the full abstract

All authors Marcin Ból, Stanisław Drożdż, Robert Kluszczyński, Jarosław Kwapień, Marcin Wątorek

Statistics Applications

On the Benefit of Blocking for Online Experiments with Skewed Data

This paper advocates for using blocked (stratified) assignment in online A/B tests to handle highly skewed population data. While blocking yields only modest precision gains (5-10%) due to the limits of discretization, the authors demonstrate it offers two crucial structural benefits over post-hoc statistical adjustments. First, fixed-weight blocking correctly targets the true average treatment effect, avoiding the severe bias introduced by efficiency-weighted alternatives. Second, blocking localizes extreme outliers, enabling targeted within-block winsorization that effectively controls noise without destroying the tail-end treatment effect.
Read the full abstract

All authors Abbas Zaidi, Matthew Reimherr, Rina Friedberg, Fabian Martinez, Richard Mudd

Statistics Applications

Estimating Uncoded Crash Factors with Tabular Foundation and System One Models: Kumo Tabular and Jev

Road safety programs count the coded fields of police crash records, while the officer's narrative, which often records factors the fields omit, is rarely read. A safety office thus cannot tell how much its counts miss or where to review. This study develops and evaluates a system that joins both views of the 5,601,890 Texas crashes from 2017 to 2025 into population estimates with stated validity. An in-context tabular foundation model, Kumo Tabular, reads the coded record of every crash, a calibrated System One model, Jev, reads the narratives of two probability samples, and human judgments recalibrate its probabilities. A multiwave predict-then-debias estimator joins the three tiers, and a second human tier drawn with recorded probabilities checks the estimates by design. For hydroplaning, medical episodes, fatigue, animals, and phone use, the narrative documents more injury crashes than the coded field, 15,074 against 7,340 for phone use, and the human check agrees with all fifteen estimates within its margin. A re-read list ranked by Kumo Tabular finds confirmed discordance 7 to 58 times as often as random reading. At the planning cost of human coding, one further round of human judgments would cut the root mean square relative half-width from 22.0 to 16.2 percent, against 21.2 for reading every narrative. Two calibrated readers of different views, joined by a sampling design, give a safety office counts, a discordance map, a validated re-read list, and a reading budget, with Kumo Tabular reading the table at 15 times the speed of TabPFN 3.5.
Read the full abstract

Statistics Applications

Neutral Is Not Free: Evaluating Downside Risk in Neutral Launches

Evaluating "neutral launches" (e.g., infrastructure upgrades) using traditional confidence interval overlap is flawed: it is dangerously permissive with scarce data and excessively restrictive with abundant data. To resolve this, this paper introduces Expected Bayesian Loss (EBL), a continuous metric that quantifies both the probability and expected severity of metric degradation. Computable directly from standard frequentist estimates, EBL explicitly penalizes empirical noise and high-variance experiments. Validated against expert decisions, EBL provides experimentation platforms with a rigorous, tunable guardrail that aligns statistical safety with institutional risk appetite.
Read the full abstract

All authors Pablo Alcain, Jason Kang, Mia Garrard, Marine Veits, Houssam Nassif, Abbas Zaidi

Statistics Applications · Cross-listed from stat.ML

Conformal Prediction for Spatially Dependent Data via Sequential Whitening

Split conformal prediction uses prediction errors on held-out (calibration) data to determine how wide the prediction intervals should be. It guarantees distribution-free finite-sample coverage when these errors and the error at the target site are exchangeable. This assumption may fail under spatial dependence and nonrandom sampling geometry. Existing spatial methods use fitting residuals to remove the predictable part of spatial variation from calibration and target errors. However, the spatial variation that only the calibration residuals can predict remains in both the target and calibration errors, reducing the efficiency and stability of the interval. We address this by additionally conditioning on the calibration residuals sequentially, which scales to large networks through nearest-neighbour approximations. Under a correct working covariance and an elliptical residual law, the resulting interval has exact finite-sample coverage under any spatial design, and under further conditions it is asymptotically oracle efficient. We also bound coverage loss under covariance misspecification and develop a diagnostic that identifies regions at risk of undercoverage. In simulated data, our method produces narrower and more stable intervals than global and localized state-of-the-art alternatives. In a national PM2.5 application, it produces narrower intervals within the network and identifies regions at risk of coverage failure.
Read the full abstract

Statistics Applications

Multivariate LSTM Modeling for Wastewater Quality Prediction

The proposed approach relies on high-frequency multivariate time series collected at the Seine aval (SIAAP 1 ) treatment plant, including pH, temperature, conductivity, and total suspended solids, as well as exogenous variables related to precipitation and measurements from an upstream plant. The objective is to forecast wastewater quality over a 24-hour horizon. A multivariate Long Short-Term Memory (LSTM) recurrent neural network is implemented to capture complex temporal dependencies and nonlinear patterns in the data. The model is trained on one year of data and validated on four months of data (one per season), and is compared with a persistence model and SARIMA models. The evaluation shows an overall superiority of the LSTM model, particularly for variables exhibiting high levels of noise and nonlinearity.
Read the full abstract

Statistics Applications · Cross-listed from stat.ML

Extreme Binary Classification: Extreme Value Theory for Extreme Constraint on False Negative

While binary classification is one of the most extensively studied problems in machine learning, the regime in which the goal is to learn a classifier with an almost zero false negative rate remains largely unexplored. In this paper, we introduce the Extreme Binary Classification problem, where the objective is to learn a classifier whose false negative rate $\alpha$ is constrained by $\epsilon_{N_1}=o_{N_1\to\infty}(1/N_1)$, with $N_1$ denoting the number of positive examples in the training set. To address this problem, we propose a threshold adaptation method theoretically grounded in guarantees derived from Extreme Value Theory, together with a feature selection procedure based on a permutation test applied to sample maxima. Experimental results on four real-world datasets of varying sizes demonstrate that our approach compares favorably with state-of-the-art methods. In addition, we illustrate its interpretability through an application to a cancer screening dataset.
Read the full abstract

Statistics Applications · Cross-listed from stat.CO

Overlapped We Stand, Separated We Fall: Software Approaches to Diagnosing Separation Phenomena in Categorical Response Models

In statistical models for categorical responses, including binary, nominal and ordinal responses, a phenomenon called “separation” may occur which can render maximum likelihood estimate nonexistent and/or nonunique. Based on Sablica et al. (2026)'s general characterization of separation phenomena via structure vectors, in this article we discuss computational, numerical and practical aspects of separation phenomena in different software packages, including consequences, remedies and ways of diagnosing them. We further introduce the R package divoRce which provides the most comprehensive software suite to diagnose separation in existence. It allows to check for separation, to distinguish between types of separation, characterize the recession cone and identify variables and observations contributing to separation for almost any categorical response model in the literature, including models with baseline-category link, cumulative link, adjacent-category link and sequential link specifications. Diagnostics are based on methods for computational geometry and linear programming and can be done in floating-point or rational arithmetic with different solvers. The functionality is easily extendible by wrappers and S3 methods to accommodate implementations in third-party packages or of new categorical response models. We discuss our implementation in detail and show our software in action in a wide range of applications. We recommend to incorporate our checks for separation and existence of the MLE as a diagnostic routine in every categorical data analysis both in third-party software implementations and as part of a standard model diagnostic workflow.
Read the full abstract

Statistics Applications

Learning joint probabilistic weather forecasts from station observations alone

Assessing compound weather risks requires forecasts representing dependence between variables. CLARA (Calibrated Advection-Routing Attention) learns joint Gaussian predictive distributions of five surface variables from station observations alone, without numerical weather prediction or reanalysis; the approximately 28,000-parameter model supports CPU training and prediction. Across six multi-year folds on 96 stations, its lead-mean energy score is 4.9% lower than that of a learned comparator with matched temporal inputs (4.7% with a similar parameter count) and 11-65% lower than those of statistical baselines. Holding marginal variances fixed, removing learned correlations worsens joint negative log-likelihood by 1.0-2.8 nats per station. A covariance-scale estimator, proved consistent under stated assumptions, improves short-lead calibration but over-corrects at long leads. Synthetic interventions show an attention-bias coefficient alone does not measure forecast influence. Retrained in ten regions on six continents, CLARA outperforms persistence in all 60 multi-year region-lead comparisons and a similarly sized learned model in 57 of 60.
Read the full abstract