Machine Learning (Statistics)
Finding the Sweet Spot: Trading Quality, Cost, and Speed During Inference-Time LLM Reflection
Jack Butler, Nikita Kozodoi, Zainab Afolabi et al. (+2 more)
As Large
Language Models (LLMs) continue to evolve, practitioners face increasing options for enhancing inference-time performance without model retraining, including budget tuning and multi-step techniques like self-reflection. While these methods improve output quality, they create complex trade-offs among accuracy, cost, and latency that remain poorly understood across different domains. This paper systematically compares self-reflection and budget tuning across mathematical
reasoning and translation tasks. We evaluate prominent LLMs, including Anthropic Claude, Amazon Nova, and Mistral families, along with other models under varying reflection depths and compute budgets to derive Pareto optimal performance frontiers. Our analysis reveals substantial domain dependent variation in self-reflection effectiveness, with performance gains up to 220% in mathematical
reasoning. We further investigate how reflection round depth and feedback mechanism quality influence performance across model families. To validate our findings in a real-world setting, we deploy a self-reflection enhanced marketing content localisation system at Lounge by Zalando, where it shows
market-dependent effectiveness, reinforcing the importance of domain specific evaluation when deploying these techniques. Our results provide actionable guidance for selecting optimal inference strategies given specific domains and resource constraints. We open source our self-reflection implementation for reproducibility at https://github.com/aws-samples/sample-genai-reflection-for-bedrock.
Read the full abstract ▾
All authors Jack Butler, Nikita Kozodoi, Zainab Afolabi, Brian Tyacke, Gaiar Baimuratov
Statistical Computation
Approximating evidence via bounded harmonic means
Dana Naderi, Christian P Robert, Kaniav Kamary et al. (+1 more)
Efficient
Bayesian model selection relies on the model evidence or marginal likelihood, whose computation often requires evaluating an intractable integral. The harmonic mean
estimator (HME) has long been a standard method of approximating the evidence. While computationally simple, the version introduced by Newton and Raftery (1994) potentially suffers from infinite variance. To overcome this issue,Gelfand and Dey (1994) defined a standardized representation of the
estimator based on an instrumental function and Robert and Wraith (2009) later proposed to use higher posterior density (HPD) indicators as instrumental functions. Following this approach, a practical method is proposed, based on an elliptical covering of the HPD region with non-overlapping ellipsoids. The resulting
estimator, called the Elliptical Covering Marginal Likelihood
Estimator (ECMLE), not only eliminates the infinite-variance issue of the original HME and allows exact volume computations, but is also able to be used in
multimodal settings. Through several examples, we illustrate that ECMLE outperforms other recent methods such as THAMES and its improved version (Metodiev et al 2024, 2025). Moreover, ECMLE demonstrates lower variance, a key challenge that subsequent HME variants have sought to address, and provides more stable evidence approximations, even in challenging settings.
Read the full abstract ▾
All authors Dana Naderi, Christian P Robert, Kaniav Kamary, Darren Wraith
Machine Learning (Statistics)
Diffusion Autoencoders with Perceivers for Long, Irregular and Multimodal Astronomical Sequences
Yunyi Shen and Alexander Gagliano
Self-supervised learning has become a central strategy for representation learning, but the majority of architectures used for encoding data have only been validated on regularly-sampled inputs such as images, audios. and
videos. In many scientific domains, data instead arrive as long, irregular, and
multimodal sequences. To extract semantic information from these data, we introduce the
Diffusion Autoencoder with Perceivers (daep). daep tokenizes heterogeneous measurements, compresses them with a Perceiver encoder, and reconstructs them with a Perceiver-IO
diffusion decoder, enabling scalable learning in diverse data settings. To
benchmark the daep architecture, we adapt the masked autoencoder to a Perceiver encoder/decoder design, and establish a strong baseline (maep) in the same architectural family as daep. Across diverse spectroscopic and photometric astronomical
datasets, daep achieves lower reconstruction errors, produces more discriminative latent spaces, and better preserves fine-scale structure than both VAE and maep baselines. These results establish daep as an effective framework for scientific domains where data arrives as irregular, heterogeneous sequences.
Read the full abstract ▾
Statistical Methodology · Cross-listed from eess.SP
Time-series Random Process Complexity Ranking Using a Bound on Conditional Differential Entropy
Jacob Ayers, Richard Hahnloser, Julia Ulrich et al. (+5 more)
Conditional differential
entropy provides an intuitive measure for relatively ranking time-series complexity by quantifying uncertainty in future observations given past context. However, its direct computation for high-dimensional processes from unknown distributions is often intractable. This paper builds on the information theoretic prediction error bounds established by Fang et al. \cite{fang2019generic}, which demonstrate that the conditional differential
entropy \textbf{$h(X_k \mid X_{k-1},...,X_{k-m})$} is upper bounded by a function of the determinant of the covariance
matrix of next-step prediction errors for any next step prediction model. We add to this theoretical framework by further increasing this bound by leveraging Hadamard's inequality and the positive semi-definite property of covariance
matrices. To see if these bounds can be used to rank the complexity of time series, we conducted two synthetic experiments: (1) controlled linear autoregressive processes with additive Gaussian noise, where we compare ordinary least squares prediction error
entropy proxies to the true
entropies of various additive noises, and (2) a complexity ranking task of bio-inspired synthetic audio data with unknown
entropy, where neural network prediction errors are used to recover the known complexity ordering. This framework provides a computationally tractable method for time-series complexity ranking using prediction errors from next-step prediction models, that maintains a theoretical foundation in information theory.
Read the full abstract ▾
All authors Jacob Ayers, Richard Hahnloser, Julia Ulrich, Lothar Sebastian Krapp, Remo Nitschke, Sabine Stoll, Balthasar Bickel, Reinhard Furrer
Statistics Theory
Bootstrap Consistency for Empirical Likelihood in Density Ratio Models
Weiwei Zhuang, Weiqi Yang and Jiahua Chen
We establish the validity of bootstrap methods for empirical likelihood (EL) inference under the density ratio model (DRM). In particular, we prove that the bootstrap maximum EL
estimators share the same limiting distribution as their population counterparts, both at the parameter level and for distribution functionals. Our results extend existing pointwise convergence theory to weak convergence of processes, which in turn justifies bootstrap inference for
quantiles and dominance indices within the DRM framework. These theoretical guarantees close an important gap in the literature, providing rigorous foundations for resampling-based confidence intervals and hypothesis tests. Simulation studies further demonstrate the accuracy and practical value of the proposed approach.
Read the full abstract ▾
Statistics Theory
SCMD: A Kernel-Based Distance for Structural Causal Models to Quantify Transferability Across Environments
Théotime Le Goff and Émilie Devijver
Out-of-distribution generalization is key to building models that remain reliable across diverse environments. Recent causality-based methods address this challenge by learning invariant causal relationships in the underlying data-generating process. Yet, measuring how causal structures differ across environments, and the resulting generalization difficulty, remains difficult. To tackle this challenge, we propose the Structural Causal Model Distance (SCMD), a principled metric that quantifies discrepancies between two SCMs by combining (i) kernel-based distances for nonparametric comparison of distributions and (ii) pairwise interventional comparisons to capture differences in causal effects. We show that SCMD is a proper metric and provide a consistent
estimator with theoretical guarantees. Experiments on synthetic and real-world
datasets demonstrate that SCMD effectively captures both structural and distributional differences between SCMs, providing a practical tool to assess causal transferability and generalization difficulty. Given two joint distributions P 1 (V j ) and P 2 (V j ), the Maximum Mean Discrepancy (MMD, Gretton et al. which defines a metric between distributions for characteristic kernels (Fukumizu et al., 2007). Kernel conditional mean
embeddings (Park and Muandet, 2020) represent conditional expectation operators in an RKHS H Vj , which allows us to define the Maximum Conditional Mean Discrepancy between two
Read the full abstract ▾
Machine Learning (Statistics)
Concentration and excess risk bounds for imbalanced classification with synthetic oversampling
Touqeer Ahmad, Mohammadreza M. Kalan, François Portier et al. (+1 more)
Synthetic oversampling of minority examples using SMOTE and its variants is a leading strategy for addressing imbalanced classification problems. Despite the success of this approach in practice, its theoretical foundations remain underexplored. We develop a theoretical framework to analyze the behavior of SMOTE and related methods when classifiers are trained on synthetic data. We first derive a uniform concentration bound on the discrepancy between the empirical risk over synthetic minority samples and the population risk on the true minority distribution. We then provide a nonparametric excess risk guarantee for kernel-based classifiers trained using such synthetic data. These results lead to practical guidelines for better parameter tuning of both SMOTE and the downstream learning
algorithm. Numerical experiments are provided to illustrate and support the theoretical findings
Read the full abstract ▾
All authors Touqeer Ahmad, Mohammadreza M. Kalan, François Portier, Gilles Stupfler
Statistics Applications · Cross-listed from cs.CL
Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models
Christian Hobelsberger, Theresa Winner, Andreas Nawroth et al. (+2 more)
Large
language models (LLMs) produce outputs with varying levels of uncertainty, and, just as often, varying levels of correctness; making their practical reliability far from guaranteed. To quantify this uncertainty, we systematically evaluate four approaches for confidence estimation in LLM outputs: VCE, MSP, Sample Consistency, and CoCoA (Vashurin et al., 2025). For the evaluation of the approaches, we conduct experiments on four question-answering tasks using a state-of-the-art open-source LLM. Our results show that each uncertainty metric captures a different facet of model confidence and that the hybrid CoCoA approach yields the best reliability overall, improving both calibration and discrimination of correct answers. We discuss the trade-offs of each method and provide recommendations for selecting uncertainty measures in LLM applications.
Read the full abstract ▾
All authors Christian Hobelsberger, Theresa Winner, Andreas Nawroth, Oliver Mitevski, Anna-Carolina Haensch
Statistical Methodology
On Multiple Robustness of Proximal Dynamic Treatment Regimes
Yuanshan Gao, Yang Bai and Yifan Cui
Dynamic treatment regimes are sequential decision rules that adapt treatment according to individual time-varying characteristics and outcomes to achieve optimal effects, with applications in precision medicine, personalized recommendations, and dynamic marketing. Estimating optimal dynamic treatment regimes via sequential randomized trials might face costly and ethical hurdles, often necessitating the use of historical observational data. In this work, we utilize proximal causal inference framework for learning optimal dynamic treatment regimes when the unconfoundedness assumption fails. Our contributions are four-fold: (i) we propose three nonparametric identification methods for optimal dynamic treatment regimes; (ii) we establish the semiparametric efficiency bound for the value function of a given regime; (iii) we propose a (K+1)-robust method for learning optimal dynamic treatment regimes, where K is the number of stages; (iv) as a by-product for marginal structural models, we establish identification and estimation of counterfactual means under a static regime. Numerical experiments validate the efficiency and multiple robustness of our proposed methods.
Read the full abstract ▾
Machine Learning (Statistics)
Learning Decentralized Routing Policies via Graph Attention-based Multi-Agent Reinforcement Learning in Lunar Delay-Tolerant Networks
Federico Lozano-Cuadra, Beatriz Soret, Marc Sanchez Net et al. (+2 more)
We present a fully decentralized routing framework for multi-
robot exploration missions operating under the constraints of a Lunar Delay-Tolerant Network (LDTN). In this setting, autonomous rovers must relay collected data to a lander under intermittent connectivity and unknown mobility patterns. We formulate the problem as a Partially Observable Markov Decision Problem (POMDP) and propose a Graph
Attention-based Multi-
Agent Reinforcement Learning (GAT-MARL) policy that performs Centralized Training, Decentralized Execution (CTDE). Our method relies only on local observations and does not require global
topology updates or packet replication, unlike classical approaches such as shortest path and controlled flooding-based
algorithms. Through Monte Carlo simulations in randomized exploration environments, GAT-MARL provides higher delivery rates, no duplications, and fewer packet losses, and is able to leverage short-term mobility forecasts; offering a scalable solution for future space robotic systems for planetary exploration, as demonstrated by successful generalization to larger rover teams.
Read the full abstract ▾
All authors Federico Lozano-Cuadra, Beatriz Soret, Marc Sanchez Net, Abhishek Cauligi, Federico Rossi
Statistical Methodology
Clustering of multivariate tail dependence using conditional methods
Patrick O'Toole, Christian Rohrbeck and Jordan Richards
The conditional extremes (CE) framework has proven useful for analysing the joint tail behaviour of random vectors. However, when applied across many locations or variables, it can be difficult to interpret or compare the resulting extremal dependence structures, particularly for high dimensional vectors. To address this, we propose a novel clustering method for multivariate extremes using the CE framework. Our approach introduces a closed-form, computationally efficient dissimilarity measure for multivariate tails, based on the skew-
geometric Jensen-Shannon divergence, and is applicable in arbitrary dimensions. Applying standard clustering
algorithms to a
matrix of pairwise distances, we obtain interpretable groups of random vectors with homogeneous tail dependence. Simulation studies demonstrate that our method outperforms existing approaches for clustering bivariate extremes, and uniquely extends to the multivariate setting. In our application to Irish meteorological data, our clustering identifies spatially coherent regions with similar extremal dependence between precipitation and wind speeds.
Read the full abstract ▾
Statistical Methodology
Identification and Debiased Learning of Causal Effects with General Instrumental Variables
Shuyuan Chen, Peng Zhang and Yifan Cui
Instrumental variable methods are fundamental to causal inference when treatment assignment is confounded by unobserved variables. In this article, we develop a general nonparametric causal framework for identification and learning with multi-categorical or continuous instrumental variables. Specifically, the mean potential outcomes and the average treatment effect can be identified via a regular weighting function derived from the proposed framework. Leveraging semiparametric theory, we derive efficient influence functions and construct two consistent, asymptotically normal
estimators via debiased machine learning. The first
estimator uses a prespecified weighting function, while the second
estimator selects the optimal weighting function adaptively. Extensions to longitudinal data, dynamic treatment regimes, and multiplicative instrumental variables are further developed. We demonstrate the proposed method by employing simulation studies and analyzing real data from the Job Training Partnership Act program.
Read the full abstract ▾
Machine Learning (Statistics)
Testing Most Influential Sets
Lucas D. Konrad and Nikolas Kuschnig
Small influential data subsets can dramatically impact model conclusions, with a few data points overturning key findings. While recent work identifies these most influential sets, there is no formal way to tell when maximum influence is excessive rather than expected under natural random sampling variation. We address this gap by developing a principled framework for most influential sets. Focusing on linear least-squares, we derive a convenient exact influence formula and identify the extreme value distributions of maximal influence - the heavy-tailed Fréchet for constant-size sets and heavy-tailed data, and the well-behaved Gumbel for growing sets or light tails. This allows us to conduct rigorous hypothesis tests for excessive influence. We demonstrate through applications across economics, biology, and machine learning
benchmarks, resolving contested findings and replacing ad-hoc heuristics with rigorous inference.
Read the full abstract ▾
Machine Learning (Statistics)
Neural Networks for Censored Expectile Regression Based on Data Augmentation
Wei Cao and Shanshan Wang
Expectile regression neural networks (ERNNs) are powerful tools for capturing heterogeneity and complex nonlinear structures in data. However, most existing research has primarily focused on fully observed data, with limited
attention paid to scenarios involving censored observations. In this paper, we propose a data augmentation based ERNNs
algorithm, termed DAERNN, for modeling heterogeneous censored data. The proposed DAERNN is fully data driven, requires minimal assumptions, and offers substantial flexibility. Simulation studies and real data applications demonstrate that DAERNN outperforms existing censored ERNNs methods and achieves predictive performance comparable to models trained on fully observed data. Moreover, the
algorithm provides a unified framework for handling various censoring mechanisms without requiring explicit parametric model specification, thereby enhancing its applicability to practical censored data analysis.
Read the full abstract ▾
Statistics Applications
Reorienting Age-Friendly Frameworks for Rural Contexts: A Spatial Competence-Press Framework for Aging in Chinese Villages
Ziyuan Gao
While frameworks such as the WHO Age-Friendly Cities have advanced urban aging policy, rural contexts demand fundamentally different analytical approaches. The spatial dispersion, terrain variability, and agricultural labor dependencies that characterize rural aging experiences require moving beyond service-domain frameworks toward spatial stress assessment models. Current research on rural aging in China exhibits methodological gaps, systematically underrepresenting the spatial stressors that older adults face daily, including terrain barriers, infrastructure limitations,
climate exposure, and agricultural labor burdens. Existing rural revitalization policies emphasize standardized interventions while inadequately addressing spatial heterogeneity and the spatially-differentiated needs of aging populations. This study developed a GIS-based spatial stress analysis framework that applies Lawton and Nahemow's competence-press model to quantify aging-related stressors and classify rural villages by intervention needs. Using data from 27 villages in Mamuchi Township, Shandong Province, we established four spatial stress indicators: slope gradient index (SGI),
solar radiation exposure index (SREI), walkability index (WI), and agricultural intensity index (AII). Analysis of variance and hierarchical clustering revealed significant variation in spatial pressures across villages and identified distinct typologies that require targeted intervention strategies. The framework produces both quantitative stress measurements for individual villages and a classification system that groups villages with similar stress patterns, providing planners and policymakers with practical tools for designing spatially-targeted age-friendly interventions in rural China and similar contexts.
Read the full abstract ▾
Statistics Applications · Cross-listed from physics.app-ph
Multi-Task Deep Learning for Surface Metrology
D. Kucharski, A. Gaska, T. Kowaluk et al. (+9 more)
A reproducible deep learning framework is presented for surface metrology to predict surface texture parameters together with their reported standard uncertainties. Using a multi-instrument
dataset spanning tactile and optical systems, measurement system type classification is addressed alongside coordinated regression of Ra, Rz, RONt and their uncertainty targets (Ra_uncert, Rz_uncert, RONt_uncert). Uncertainty is modelled via
quantile and heteroscedastic heads with post-hoc conformal calibration to yield calibrated intervals. On a held-out set, high fidelity was achieved by single-target regressors (R2: Ra 0.9824, Rz 0.9847, RONt 0.9918), with two uncertainty targets also well modelled (Ra_uncert 0.9899, Rz_uncert 0.9955); RONt_uncert remained difficult (R2 0.4934). The classifier reached 92.85% accuracy and probability calibration was essentially unchanged after temperature scaling (ECE 0.00504 -> 0.00503 on the test split). Negative transfer was observed for naive multi-output trunks, with single-target models performing better. These results provide calibrated predictions suitable to inform instrument selection and acceptance decisions in metrological workflows.
Read the full abstract ▾
All authors D. Kucharski, A. Gaska, T. Kowaluk, K. Stepien, M. Repalska, B. Gapinski, M. Wieczorowski, M. Nawotka, P. Sobecki, P. Sosinowski, J. Tomasik, A. Wojtowicz
Statistical Methodology
Unifying Boxplots: A Multiple Testing Perspective
Bowen Gang, Hongmei Lin and Tiejun Tong
Tukey's boxplot is a foundational tool for exploratory data analysis, but its classic outlier-flagging rule does not account for the sample size, and subsequent modifications have often been presented as separate, heuristic adjustments. In this paper, we propose a unifying framework that recasts the boxplot and its variants as graphical implementations of multiple testing procedures. We demonstrate that Tukey's original method is equivalent to an unadjusted procedure, while existing sample-size-aware modifications correspond to controlling the Family-Wise Error Rate (FWER) or the Per-Family Error Rate (PFER). This perspective not only systematizes existing methods but also naturally leads to new, more adaptive constructions. We introduce a boxplot motivated by the False Discovery Rate (FDR), and show how our framework provides a flexible pipeline for integrating state-of-the-art robust estimation techniques directly into the boxplot's graphical format. By connecting a classic graphical tool to the principles of multiple testing, our work provides a principled
language for comparing, critiquing, and extending outlier detection rules for modern exploratory analysis.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
What Does It Take to Build a Performant Selective Classifier?
Stephan Rabanser and Nicolas Papernot
Selective classifiers improve model reliability by abstaining on inputs the model deems uncertain. However, few practical approaches achieve the gold-standard performance of a perfect-ordering oracle that accepts examples exactly in order of correctness. Our work formalizes this shortfall as the selective-classification gap and present the first finite-sample decomposition of this gap to five distinct sources of looseness: Bayes noise, approximation error, ranking error, statistical noise, and implementation- or shift-induced slack. Crucially, our analysis reveals that monotone post-hoc calibration – often believed to strengthen selective classifiers – has limited impact on closing this gap, since it rarely alters the model's underlying score ranking. Bridging the gap therefore requires scoring mechanisms that can effectively reorder predictions rather than merely rescale them. We validate our decomposition on synthetic two-moons data and on real-world
vision and
language benchmarks, isolating each error component through controlled experiments. Our results confirm that (i) Bayes noise and limited model capacity can account for substantial gaps, (ii) only richer, feature-aware calibrators meaningfully improve score ordering, and (iii) data shift introduces a separate slack that demands distributionally robust training. Together, our decomposition yields a quantitative error budget as well as actionable design guidelines that practitioners can use to build selective classifiers which approximate ideal oracle behavior more closely.
Read the full abstract ▾
Statistical Methodology
Bias-Variance Tradeoff of Matching Prior to Difference-in-Differences When Parallel Trends is Violated
Mingxuan Ge and Dae Woong Ham
Quasi-experimental causal inference methods have become central in empirical operations management for guiding managerial decisions. Among these, empiricists utilize the Difference-in-Differences (DiD)
estimator, which relies on the parallel trends assumption. To improve its plausibility, researchers often match treated and control units before applying DiD, with the intuition that matched groups are more likely to evolve similarly absent treatment. Existing work that analyzes this practice, however, has focused solely on bias. In this work, we not only generalize earlier bias results under weaker assumptions but also analyze properties of variance and mean squared error (MSE), a practically relevant metric for decision making. Under a linear structural model with unobserved time-varying confounders, we show that variance results contrast with established bias insights: matching on observed
covariates prior to DiD is not always recommended over the classic (unmatched) DiD due to a sample size tradeoff; furthermore, matching additionally on pre-treatment outcomes is always beneficial as such tradeoff no longer exists once matching is performed. We therefore advocate MSE as an additional metric if applied researchers weigh bias and variance equally and further give practitioner-friendly guidelines with theoretical guarantees on when and on what variables they should match. As an illustration, we apply these guidelines to re-examine a recent empirical study that matches prior to DiD to study how the introduction of monetary incentives by a knowledge-sharing platform affects general engagement on the platform. Our results show that the authors' decision was both warranted and critical to produce a credible causal estimate.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
Topics in Probability, Parametric Estimation and Stochastic Calculus
Levi Lopes de Lima
We begin our journey by recalling the fundamentals of Probability Theory that underlie one of its most significant applications to real-world problems: Parametric Estimation. Throughout the text, we systematically develop this theme by presenting and discussing the main tools it encompasses (concentration inequalities, limit
theorems, confidence intervals, maximum likelihood, least squares, and hypothesis testing) always with an eye toward both their theoretical underpinnings and practical relevance. While our approach follows the broad contours of conventional expositions, we depart from tradition by consistently exploring the
geometric aspects of probability, particularly the invariance properties of normally distributed random vectors. This
geometric perspective is taken further in an extended appendix, where we introduce the rudiments of Brownian motion and the corresponding
stochastic calculus, culminating in Itô's celebrated change-of-variables formula. To highlight its scope and elegance, we present some of its most striking applications: the sharp Gaussian concentration inequality (a central example of the "concentration of measure phenomenon"), the Feynman-Kac formula (used to derive a path integral representation for the
Laplacian heat kernel), and, as a concluding delicacy, the Black-Scholes strategy in Finance.
Read the full abstract ▾
Statistical Methodology
Asynchronous Distributed ECME Algorithm for Matrix Variate Non-Gaussian Responses
Qingyang Liu, Sanvesh Srivastava and Dipankar Bandyopadhyay
We propose a regression model with
matrix-variate skew-t response (REGMVST) for analyzing irregular longitudinal data with skewness,
symmetry, or heavy tails. REGMVST models
matrix-variate responses and predictors, with rows indexing longitudinal measurements per subject. It uses the
matrix-variate skew-t (MVST) distribution to handle skewness and heavy tails, a damped exponential correlation (DEC) structure for row-wise dependencies across irregular time profiles, and leaves the column covariance unstructured. For estimation, we initially develop an ECME
algorithm for parameter estimation and further mitigate its computational bottleneck via an asynchronous and distributed ECME (ADECME) extension. ADECME accelerates the E-step through parallelization, and retains the simplicity of the conditional M-step, enabling scalable inference. Simulations using synthetic data and a case study exploring
matrix-variate periodontal disease endpoints derived from electronic health records demonstrate ADECME's superiority in efficiency and convergence, over the alternatives. We also provide theoretical support for our empirical observations and identify regularity assumptions for ADECME's optimal performance. An accompanying R package is available at https://github.com/rh8liuqy/STMATREG.
Read the full abstract ▾
Machine Learning (Statistics)
Compositional Generation for Long-Horizon Coupled PDEs
Somayajulu L. N. Dhulipala, Deep Ray and Nicholas Forman
Simulating coupled PDE systems is computationally intensive, and prior efforts have largely focused on training surrogates on the joint (coupled) data, which requires a large amount of data. In the paper, we study compositional
diffusion approaches where
diffusion models are only trained on the decoupled PDE data and are composed at inference time to recover the coupled field. Specifically, we investigate whether the compositional strategy can be feasible under long time horizons involving a large number of time steps. In addition, we compare a baseline
diffusion model with that trained using the v-parameterization strategy. We also introduce a symmetric compositional scheme for the coupled fields based on the Euler scheme. We evaluate on Reaction-
Diffusion and modified Burgers with longer time grids, and
benchmark against a
Fourier Neural Operator trained on coupled data. Despite seeing only decoupled training data, the compositional
diffusion models recover coupled trajectories with low error. v-parameterization can improve accuracy over a baseline
diffusion model, while the neural operator surrogate remains strongest given that it is trained on the coupled data. These results show that compositional
diffusion is a viable strategy towards efficient, long-horizon modeling of coupled PDEs.
Read the full abstract ▾
Other Statistics
Factors Associated with Unit-Specific Failure in a University-Level Statistics Course
Biviana Marcela Suarez Sierra
This study investigates the factors associated with failure in each of the four thematic units of a General Statistics course offered at a private university in Colombia. Unlike traditional analyses that treat performance as a single outcome, this research disaggregates results by unit: Exploratory Data Analysis, Probability and Random Variables, Statistical Inference, and Linear Regression – highlighting distinct challenges across content areas. Based on a sample of 186 undergraduate students from Engineering, Geology, and Interactive Design programs, the study combines exam performance data with self-perceived preparedness surveys to develop unit-specific logistic regression models. The findings reveal consistent structural disadvantages for students from non-engineering programs, especially in concept-heavy units such as Inference and Regression. Academic stage and perception of competence also emerged as important predictors, though their effects varied across units. The results align with prior research on statistical thinking and self-efficacy, and support the need for targeted pedagogical interventions and curricular
alignment. This disaggregated approach offers a more nuanced understanding of academic vulnerability in statistics education and contributes to the design of evidence-based, context-sensitive strategies to reduce failure and improve learning outcomes.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.PR
On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers
Krishnakumar Balasubramanian, Sayan Banerjee and Philippe Rigollet
We study stationary solutions of McKean-Vlasov equations on the circle. We establish an exact equivalence between solutions of the stationary McKean-Vlasov equation and an infinite-dimensional quadratic system of equations over
Fourier coefficients, allowing explicit characterization of stationary states in a sequence space rather than a function space. This framework provides a transparent description of local bifurcations, their periodicity and resonance structures, while accommodating singular potentials. We derive analytic expressions for the emergence, form and shape (supercritical, critical, subcritical or transcritical) of bifurcations involving possibly multiple
Fourier modes and connect them with discontinuous phase transitions. We also characterize, under suitable assumptions, the detailed structure of the stationary bifurcating solutions through leading small-amplitude coefficients for arbitrarily many fixed
Fourier modes. At the global level, we establish regularity and concavity properties of the free energy landscape, proving existence, compactness, and coexistence of globally minimizing stationary measures, and identifying discontinuous phase transitions with points of non-differentiability of the minimum free energy map. We apply the theory to the Noisy Mean-Field
Transformer model, where we show how changing the inverse temperature $\beta$ affects the geometry of the infinitely many bifurcations from the uniform measure. We also discuss how increasing $\beta$ motivates the study of approximate multi-mode stationary profiles as candidates for a separate metastability analysis; persistence times require additional dynamical estimates. Moreover, we determine the exact threshold $\beta_0\simeq2.447$ separating continuous and discontinuous phase transitions in this model. The same threshold marks the change of the first bifurcation from supercritical to subcritical.
Read the full abstract ▾