Statistical Methodology · Cross-listed from cs.LG
A new classification method based on Minimum Spanning Trees
Julio González-Díaz, Beatriz Pateiro-López and Iria Rodríguez-Acevedo
Minimum Spanning Trees have been used in unsupervised learning, particularly in clustering tasks, due to their ability to recognize clusters by removing edges that are considered inconsistent in defining those clusters. This paper aims to study the use of Minimum Spanning Trees in supervised learning. Specifically, we propose a classification
algorithm based on Minimum Spanning Trees. To improve its performance, we introduce a robust version of the method that is also computationally more efficient. We evaluate the effectiveness of our proposed method through an extensive simulation study. We also apply the proposed methodology to a real-world case study involving aircraft trajectories.
Read the full abstract ▾
Statistics Theory
Second order mixed moment inequalities based on Gram matrices
Sergio Scarlatti
Recently [LT;
Theorem 3.1] showed an extension of Walker's inequality [W] based on $N=3$ random variables. In this note we prove that extension is just a particular three-dimensional instance of a general family of second order mixed moment inequalities based on Gram
matrices of arbitrary random vectors. We also discuss some implications of these inequalities on Cramer-Rao lower bound for biased
estimators.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
A Transport-Based Geometry of Belief-Cost
Laurent Caraffa
A finite
agent, a machine's digital twin or any bounded reasoner, infers a fixed and noisy world through finite sensors, so its coherent output is a belief: a probability density over states (the Bayes posterior). Such an
agent stops short of certainty, and revising a belief carries a cost. We propose a framework for belief costs based on optimal transport, motivated by these facts. We pose two postulates. P0 (the arena): a revision cost is a scalar price on optimal transport, so beliefs live in Wasserstein space. P1 (uniform pricing): one nat of knowledge costs the same metric length everywhere, the eikonal condition. Among conceivable pricing rules we study this one. Under P0 and P1 the cost metric is optimal transport conformally reweighted by Fisher information, $\tilde g_{e,U}=2(e+U)\,g_{W_2}$, and the Fisher family is a characterization: among continuous reliefs, uniform pricing is equivalent to $U=cJ$. Two consequences follow on the conformal class. Certainty sits at infinite cost-distance once the relief dominates the Fisher information, so a well-posed inference has a cost floor diverging at certainty (necessity conjectural beyond power laws). On location-scale leaves the geometry is
hyperbolic, and the Stam bound places the Gaussian as the most curved one (at $e=0$). The results are
geometric, in nats, and hold up to units: a change of cost unit rescales all distances and preserves every conclusion (boundary, eikonal family, hyperbolicity, Gaussian extremum), a gauge
theorem; a global change of state units at $e=0$ is an isometry; the content lies in signs, rankings and ratios. Via Landauer (one nat worth $k_BT$) the cost floor becomes an energy floor: revising toward certainty would demand unbounded energy. Physics anchors the unit and enters no
theorem. Removing either postulate leaves the selection open.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
What's the Magic Formula Instrument?
Peizan Sheng and Alexander Torgovitsky
Two recent papers by Borusyak and Hull (2023, 2026) propose using known formulas to adjust linear instrumental variable
estimators for confounding
covariates. Implementing this "formula instrument" approach requires making a parametric assumption on the distribution of the unobserved shocks that generated the instrument. We develop a method for systematically evaluating the sensitivity of formula instrument estimates to this parametric assumption. The method is straightforward to implement using our companion R package formulaiv. We use our method to reanalyze the applications in both Borusyak and Hull (2023) and Borusyak and Hull (2026). In both applications, we find that a variety of estimates of different signs and magnitudes can be recovered by slightly changing the shock distribution.
Read the full abstract ▾
Statistics Theory
Reformulation Invariance and the Axiomatic Foundations of Inference
Raphaël Trésor, Thijs van de Laar and Bert de Vries
Maximum
entropy,
Bayesian updating, and exponential-family estimation are all instances of a common inference principle: selecting the measure or distribution that minimizes a divergence subject to the available constraints. Which divergence to use is usually decided by analytic convenience, by empirical performance, or by a set of axioms chosen to single it out, leaving open a basic question: why one divergence and not another? We answer it from a single requirement: an inference method should return the same answer whenever the same problem is presented in an equivalent form, for instance, after simply renaming its parts. This requirement alone forces inference to be the minimisation of a classical divergence, and each further reformulation it must respect tightens the admissible family one notch, narrowing the broad f-divergences to the {\alpha}-divergences and finally to the single Kullback-Leibler (KL) divergence. Mathematically, inference is recast from minimising a numerical functional to selecting a least element under a preorder on positive measures, a divergence being merely one numerical scale that reproduces that preorder. The reformulations are the morphisms of a category of inference problems, and the invariance requirement says the inference operator is a covariant functor into the category of statistical models of Cencov, mirroring his characterisation of the Fisher metric. The representation is proved on finite spaces and lifted to general measurable spaces by an elementary closure, covering discrete and continuous spaces alike. Earlier axiomatisations, such as those of Shore-Johnson and Csiszar, postulate their consistency axioms directly and only on finite alphabets; here the axioms follow from reformulation invariance alone.
Read the full abstract ▾
Statistical Methodology
Refining Effect-Size Measures and Classification for Differential Item Functioning: Toward Unified Guidelines Across Methods
Michaela Cichrová, Adéla Hladká and Patrícia Martinková
Differential Item Functioning (DIF) analysis is used to identify potentially biased items in multi-item measurements. In addition to testing the statistical significance, it is essential to evaluate the practical significance of DIF through effect-size measures. We review existing DIF effect-size measures and cut-off values used to classify the effect-size magnitudes for the Mantel-Haenszel test, SIBTEST, and model-based methods for binary items, and introduce a refinement of area-based effect-size measures. A simulation study is conducted to investigate the properties of these effect-size measures and existing classification guidelines, and to assess their comparative performance. The results indicate that some commonly used effect-size measures exhibit undesirable properties, including inconsistent classifications, systematic underestimation of the magnitude of the underlying DIF, and strong dependence on design factors. To address these issues, we introduce usage restrictions for some effect-size measures, revise cut-off values that unify results across different methods, and propose new cut-off values for area-based effect-size measures. The methods are demonstrated using two real data examples. Implementation is provided in the R software.
Read the full abstract ▾
Statistical Methodology
A Censored Transformed Model for Proportional Outcomes with Boundary Mass and an Application to Loss Given Default Modeling
Yuan Christopher Qiang and Fabio Sigrist
We introduce the zero-one censored transformed normal (ZOC-TN) model for proportional responses with potential probability mass at the boundaries 0 and 1. The model combines a censored Gaussian variable with a two-parameter affine-logit transformation on the interior (0,1). We characterize the transformation parameters, establish large-sample properties, and relate the affine-logit specification to broader classes of interior distributions. Theoretical and experimental results demonstrate that the proposed model can capture a wider range of qualitative density shapes than several
benchmark models while remaining parsimonious, computationally efficient, and numerically stable. Furthermore, the ZOC-TN model can be extended (i) to account for nonlinearities and interactions in a tree-boosting machine learning framework and (ii) to explicitly model residual spatio-temporal variability. We apply the ZOC-TN model to loss given default (LGD) modeling for a large
dataset of U.S. residential mortgages and compare it to multiple
benchmark models. We find that a tree-boosted ZOC-TN model with a spatio-temporal frailty Gaussian process delivers the strongest out-of-sample performance, indicating that mortgage losses are shaped by nonlinear
covariate effects and by unaccounted-for space-time variation.
Read the full abstract ▾
Statistical Methodology
Bayesian model selection of vine copulas: a loss-based perspective
Rosario Barone, Luciana Dalla Valle, Fabrizio Leisen et al. (+1 more)
The growing popularity of vine copulas in multivariate statistical analysis is largely driven by their ability to capture complex dependence structures. However, this flexibility comes at a cost, as the number of possible vine models grows rapidly and becomes intractable even in moderately low-dimensional settings. These limitations affect the practical applicability of current
Bayesian inference and model selection approaches, effectively restricting it to problems of relatively small-dimension due to their high computational cost. This paper addresses the still open challenge of efficient model selection and estimation in
Bayesian vine methodology. We propose a novel framework for
Bayesian vine copula model selection that combines loss-based model priors with the shotgun
stochastic search strategy. The strength of the proposed approach is twofold: it promotes sparsity and enables fast and effective structure selection. Furthermore, our comprehensive framework jointly identifies the vine structure, selects the copula families, and estimates the model parameters. The power of the proposed approach is demonstrated via simulation studies and an application to a real
dataset of EFT portfolio asset returns.
Read the full abstract ▾
All authors Rosario Barone, Luciana Dalla Valle, Fabrizio Leisen, Cristiano Villa
Statistical Methodology
Maximum Likelihood Estimation for Network Models with Latent Geometry under Snowball Sampling
Nurzhan Sapargali, Sergio Buttazzo and Göran Kauermann
Snowball sampling is a widely used design for collecting network data from large or hard-to-reach populations, yet naive inference that ignores the sampling mechanism produces systematically biased parameter estimates. We derive the exact likelihood of a multi-wave snowball sample for the class of continuous latent space (CLS) models, in which edges form independently conditional on latent vertex-level quantities, and show that conditional edge independence reduces the marginalization over unobserved network configurations to a closed-form expression portable across the entire CLS class. We develop a
stochastic Expectation-Maximization
algorithm for the Euclidean latent distance model as a concrete implementation, and apply the framework to the large-scale co-inventor network of German
semiconductor patent applicants by drawing multiple snowball samples. We find that the naive procedure severely underestimates latent space variance, produces networks with nearly twice the observed edge count, and achieves a spectral goodness-of-fit nine times worse than the corrected model, which directly affects the quantitative interpretation of
covariate effects.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
Central limit theorem for the averaged Adam optimizer
Steffen Dereich and Arnulf Jentzen
In this article, we analyse convergence of the averaged Adam optimizer to an attracting zero of the Adam vector field. We provide a central limit
theorem that, in particular, quantifies exactly the speed of convergence. The order of convergence is $n^{-1/2}$ in the number of steps of the
algorithm which coincides with the order observed for classical
stochastic approximation
algorithms. The covariance in the central limit
theorem is given in terms of properties of the Adam
algorithm in the state of the attractor.
Read the full abstract ▾
Machine Learning (Statistics)
Subsampling for supervised learning in reproducing kernel Hilbert spaces
Eyal Vayness and Maxime Sangnier
In the era of big data, subsampling became a common practice in statistical learning. By selecting a subgroup of individuals based on which the learner is trained, subsampling aims at reducing the computational cost and time of the estimation step, and ideally leads to a decrease of its energy consumption and carbon footprint. This work focuses on a nonparametric setting, in which the hypotheses set lies in a reproducing kernel
Hilbert space, and the
estimator is a minimizer of an empirical risk reweighted à la Horvitz-Thompson. By studying the asymptotic properties of this
estimator, we reveal an optimal subsampling scheme (regarding the trace of the covariance operator) and show that it can be used via plug-in. A numerical study on synthetic and real-world
datasets shows the practicability and the benefit of the proposed approach.
Read the full abstract ▾
Statistical Methodology
Multi-Source Prediction-Powered Inference
Wenhui Li, Fen Jiang and Xinyu Zhang
Prediction-powered inference integrates a small gold-standard
dataset with large pseudo-labeled data, whose labels are generated by machine learning methods, to enhance statistical inference. In modern applications, multiple data sources and diverse machine learning methods often give rise to multiple pseudo-labeled
datasets, each encoding potentially different aspects of the underlying information. However, how to optimally combine multiple data sources and machine learning methods for statistical inference remains unclear. To address this problem, we propose a multi-source prediction-powered inference method by aggregating multiple pseudo-labeled
datasets together, where the aggregation weights are estimated by minimizing the asymptotic volume of the resulting confidence region. We study both homogeneous settings, where the source and target distributions coincide, and heterogeneous settings, where distributional discrepancies arise between source and target distributions, including
covariate shift and domain shift. Theoretically, we establish the asymptotic normality of the proposed
estimator and show that the resulting confidence-region volume is asymptotically equivalent to the oracle optimal volume within the proposed weighting class. We further characterize when our method yields smaller confidence regions compared with both classical target-only inference and single-source prediction-powered inference. Simulation studies and a real-data application on dual-energy X-ray absorptiometry measured high body fat prevalence show that MPPI can reduce confidence-region volume while maintaining inferential validity in the settings considered.
Read the full abstract ▾
Machine Learning (Statistics)
Orthogonal Discrepancy Kernels for Learning with Partial Physics
Swapnil Manna, Timothy J. Rogers and Lawrence Bull
We introduce a semi-parametric framework for nonlinear system identification, which decouples discrepancy functions from physics-based components. Orthogonal Gaussian process regression balances sparse parameter selection (the white box) with discrepancy learning (the black box) to produce interpretable models from incomplete physics.
Read the full abstract ▾
Machine Learning (Statistics)
Two Layers of Instability in Causal Estimation
Alexis Bellot
There is a precise sense in which drawing causal inferences from observational data is hard, even when identifiability is assumed. In particular, Robins and Ritov (1997) and Robins et al. (2003) showed that causal effects can be discontinuous as a function of the data distribution: two arbitrarily close data distributions might correspond to different causal effects. This is a fact independent of the choice of
estimator; however, not all
estimators are equally unstable. Our contribution is to surface a second layer of instability that depends on the choice of
estimator. We show that many standard point estimates can be read as point summaries of
multimodal distributions over the space of structural causal models. As such,
estimators can jump discontinuously in the data distribution. This defines a taxonomy of
estimators that admits a decision-theoretic reading: stability depends on whether the implicit loss function an
estimator optimizes is aligned with the causal effect itself. Specifically, inverse propensity weighted
estimators and regression
estimators are examples of discontinuous summaries, while explicit posterior means and medians are shown to be continuous.
Read the full abstract ▾
Statistics Applications
Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS
Tobias Biegert, Sam Allen, Annika Alber et al. (+1 more)
We study whether deterministic AI weather prediction (AIWP) models issue more informative forecasts for extreme weather events than deterministic numerical weather prediction (NWP) models. The deterministic model output is subjected to statistical post-processing via isotonic distributional regression (IDR), or EasyUQ, before the resulting probabilistic forecasts are assessed using weighted versions of the continuous ranked probability score (CRPS). This extends the Potential CRPS (PCRPS) measure proposed by Gneiting et al. (2026) to focus on extreme outcomes. Since IDR exhibits optimality properties with respect to weighted versions of the CRPS, the proposed approach inherits desirable properties of the PCRPS, and, in particular, facilitates fair comparisons between data-driven and physics-based models when
forecasting extreme weather events. We apply this evaluation framework to forecasts in the WeatherBench 2
dataset issued by the AIWP models GraphCast, Pangu-Weather, and FuXi, with the ECMWF's high-resolution NWP model serving as a physics-based reference. The forecast models are compared when predicting mean sea level pressure, temperature, wind speed, and precipitation extremes, defined as exceedances or non-exceedances of thresholds obtained from historical observation data. We additionally study forecast performance when predicting record-breaking events, though the ordering of the different methods is largely insensitive to the thresholds on which emphasis is placed. We find that AIWP models, particularly FuXi, result in the most informative forecasts for extreme weather events across most settings, suggesting that AIWP models have the potential to outperform NWP models when
forecasting extremes.
Read the full abstract ▾
All authors Tobias Biegert, Sam Allen, Annika Alber, Sebastian Lerch
Machine Learning (Statistics) · Cross-listed from cs.LG
Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity
Tejas Pradeep Shirodkar and P. J. Narayanan
Singular learning theory characterises the complexity of a deep network through the geometry of its loss singularities. The local learning coefficient (LLC), the standard
estimator of Watanabe's real log canonical threshold (RLCT, $\lambda$), reads this geometry as an integrated
Bayesian scalar through SGLD, which needs per-task calibration and $10^4$-$10^6$ forward-backward passes per checkpoint. We introduce Dead-Direction Signatures (DDS), a family of cheap closed-form spectral readings of singular structure: each reads a network's activation
matrix or per-sample-gradient Fisher-Gram at a chosen layer, replacing the SGLD posterior chain with spectral linear
algebra. The readings rest on a dead-direction framework that predicts a structural correlation between activation- and Fisher-side spectra at any singular minimum, and a rank-multiplicative volume identity that single-
eigenvalue monitors cannot produce: the active-volume $\log\det^{+}(G)$ slope counts the dead directions, tracking the rank-deficit $r$ across $r \in \{1,2,3,4\}$ (slope ratios $2.0, 3.1, 4.0$ at $r{=}2,3,4$ against the predicted $2,3,4$), where the smallest
eigenvalue is rank-blind. On reduced-rank regression with closed-form $\lambda$, calibrated LLC recovers $\lambda$ at $99\%$ mean and the DDS observables rank-track it at the framework-predicted sign; on a non-linear modular-addition
transformer DDS separates $d_{\mathrm{model}}$ across eighteen orders of magnitude where calibrated LLC at the protocol budget is rank-flat. Complementary to LLC's integrated posterior reading, DDS gives a directional, layer-local handle on a network's dead directions, read in closed form from its activation and gradient spectra.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CY
Virginia Tech Transportation Safety Index (VTTSI)
Jason Cusati and Cheng-Shun Chuang
The Virginia Tech Transportation Safety Index (VTTSI) is a real-time, cloud-native framework for quantifying intersection safety using
multimodal connected-vehicle telemetry and multi-year VDOT crash history. Traditional crash-based methods rely on lagged, aggregated data and cannot reflect rapidly changing operational conditions. VTTSI addresses this gap through a hybrid modeling approach that fuses Empirical Bayes (EB) crash stabilization, uplift factors derived from speed and conflict behavior, and a CRITIC-weighted multi-criteria decision-making (MCDM) module combining SAW, EDAS, and CODAS. The system produces interpretable, exposure-adjusted safety scores on a 0–100 scale every 15 minutes. A cloud-deployed architecture built on FastAPI, PostgreSQL, PostGIS, and Streamlit supports interactive visualization of traffic volumes, VRU exposure, speed variance, and real-time incident activity. Validation across intersections demonstrates coherent diurnal patterns, consistency among MCDM methods, and sensitivity to observable operational
turbulence. Sensitivity analysis further shows that the RT–SI is robust to parameter perturbations, with deviations typically remaining below one point on the 0–100 scale. By integrating long-term crash risk with short-term behavioral dynamics, VTTSI provides a transparent, adaptive, and proactive safety-monitoring framework suitable for transportation agencies, traffic management centers, fleet operators, and autonomous vehicle systems.%~\cite{persaud2007, montella2020systemic, schultz2025_surrogate, Amraji2025CombinedSafetyIndex}.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility
Yuren Hao, Xiang Wan and ChengXiang Zhai
Despite the success of deep learning, training deep networks in biologically plausible and hardware-efficient ways remains an open challenge. Feedback
alignment (FA) methods address this by replacing backpropagation's symmetric backward weights with fixed random
matrices, but their effectiveness depends critically on whether they can be accurately evaluated. The standard evaluation relies on two quantities: task accuracy and cosine similarity between the method's credit signal and the backpropagation gradient. We show that this reporting pair is insufficient by identifying two independent failure modes, both silent under current reporting: (1) measurement degeneracy, where the BP reference gradient collapses to the numerical floor in terminal-LayerNorm residual architectures, rendering cosine uninterpretable; and (2) aggregation collapse, where the aggregate cosine masks layerwise heterogeneity that concentrates credit at one end of the network. To address these limitations, we propose a diagnostic evaluation protocol based on three checks – scale stability, reference validity, and depth utility – together with per-layer rather than aggregate cosine reporting. Across multiple architectures and methods, the standard reporting pair gives no signal of failure in any audited case, while our protocol identifies all failures with wide calibration margins. The two failure modes are causally independent: a per-block scale penalty alleviates Mode 1 (residual scale explosion driving reference collapse) without affecting Mode 2 (cosine ranking that contradicts every functional metric we measured). Identifying these silent failures prevents researchers from building on non-functional credit assignment and provides actionable guidance for developing FA methods that genuinely train deep layers.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Enhancing Differentially Private Mechanisms via Empirical Bayes
Minwoo Kim, Junyong Park and Sungkyu Jung
Differential
privacy (DP) has become the gold standard for ensuring the
privacy protection of machine learning and statistical
algorithms in recent decades. A plethora of
algorithms and methods have been developed to enhance the utility of DP
algorithms while maintaining the same level of DP. However, these are often overly complex or computationally ineffective. We propose a novel approach focusing on denoising the output of the simple additive Gaussian mechanism by adopting the idea of empirical Bayes estimation. We highlight that the empirical Bayes approach can reduce the mean-squared error solely by taking the output of the Gaussian mechanism as input. Our numerical studies show that this simple yet powerful approach can be applied to improve upon various statistical problems, including histogram release, principal component analysis, and linear regression, often outperforming existing private
algorithms.
Read the full abstract ▾
Machine Learning (Statistics)
Regional summaries of Bayesian model uncertainty
Hanqing Li, Robert Nguyen and Xuewen Lu
Bayesian posterior uncertainty can be spread across many models that share the same substantive pattern. Lists of individual models can obscure this structure, while marginal summaries do not describe which configurations are jointly supported. We study how to represent joint model uncertainty through a compact collection of interpretable regions, each grouping models that share a specified pattern. We formulate this task as minimizing representation complexity subject to retaining a prescribed posterior mass within an analyst-supplied family. To address unknown regional probabilities, we develop Adaptive Regional Selection (ARS), which combines adaptive exploration and regional weight estimation with conservative and optimistic selection. These complementary problems identify a summary meeting the mass target and bound how much further its complexity could improve. Under valid regional bounds, we establish guarantees for retained mass, total-variation error, and suboptimality, together with conditions for certifying optimality. We also show that, under bounded within-region weight variation, sufficient sample counts for regional estimation need not grow with the number of represented models, and construct bounds that remain valid under adaptive sampling and stopping. Empirical studies illustrate regional summaries in
Bayesian model averaging and
Bayesian networks, examining retained mass, representation complexity, and certification.
Read the full abstract ▾
Statistics Applications
Triage Score: A Counterfactual Risk Assessment Instrument
Kosuke Imai, Sooahn Shin, D. James Greiner et al. (+1 more)
Risk assessment instruments, also known as "risk scores," are widely used in high-stakes decision-making settings such as medicine and the criminal justice system. A risk score predicts the likelihood of an undesired outcome if no intervention is made. Thus, a sufficiently high score is often interpreted as a recommendation to intervene. However, risk scores fail to account for what would happen if a decision-maker does intervene. This failure is problematic because effective decision making requires consideration of both or multiple potential outcomes. We propose "triage scores," which are based on additive counterfactual utilities and include risk scores as a special case. Unlike risk scores, triage scores can incorporate counterfactual outcomes under alternative decisions, enabling decision makers to incorporate a wide range of ethical and practical factors. We illustrate the use of triage scores with an application to our own randomized controlled trial evaluating a pretrial risk score. Our analysis demonstrates that triage scores are able to capture rich utility structures and yield substantively distinct results regarding policy evaluation and learning.
Read the full abstract ▾
All authors Kosuke Imai, Sooahn Shin, D. James Greiner, Ryan Halen
Statistical Methodology
De-meaning Simulation Studies
Thomas Lumley, Brian Williamson and Pamela Shaw
In simulation studies evaluating asymptotic approximations it is common practice to report averages and standard deviations over repeated simulations. We argue that
quantile-based summaries are more appropriate from both a theoretical and practical point of view. Theoretically, convergence of moments – or even existence of moments – is not guaranteed by convergence in distribution, so sample moments are not ideal for assessing the accuracy of a distributional approximation. In practice, means and variances are not good summaries of approximately-Normal distributions that may have occasional outliers. We suggest the median and median absolute deviation, and empirical confidence interval coverage, as better general summaries, and argue that moments should be reserved for simulation settings where they are of substantive interest.
Read the full abstract ▾
Machine Learning (Statistics)
Diffusion-Driven State Space Models
Jack Ruder and Michael Wojnowicz
In many domains, practitioners seek models that produce accurate forecasts while faithfully capturing latent system dynamics. Existing approaches typically sacrifice one of these goals: deep state space models often assume Gaussian latent transitions, limiting fit and
forecasting, while
diffusion models are highly expressive but lack principled inference for the underlying dynamics. To combine the strengths of both, we introduce the
Diffusion-Driven State Space Model (DDSSM), which replaces the conventional Gaussian transition distribution with a
diffusion model. Our DDSSM resolves the open problem of how to jointly train an autoencoder and a
diffusion model on sequential data, thereby extending the literature on latent
diffusion models for time series. Moreover, we find that the DDSSM empirically outperforms a state-of-the-art deep SSM at fitting and
forecasting a simulated time series with
multimodal transitions.
Read the full abstract ▾
Other Statistics
Movement synchronization in complex and dynamic team coordination: Relationship with mutual anticipation
Jun Ichikawa, Mizuki Yokoyama, Soma Ishida et al. (+1 more)
Mutual anticipation of teammates' actions enables efficient interactions in team coordination that achieves a common goal and high performance. In team sports involving direct competition, such implicit and non-verbal interactions within short periods are required. If players begin moving only after observing their teammates, gaps may emerge, allowing opponents to interfere. When mutual anticipation functions properly, players' interactions are smooth without gaps, and their movements are expected to become synchronized. Synchronization represents a temporally stable structure in interactions and its mechanisms have been examined in previous studies. However, few studies have investigated synchronization in real-world coordination involving heterogeneous roles and interactions evolving over time, or quantitatively examined how temporally stable structures differ from a baseline. In our approach, we utilized team sports and introduced a statistical method to probabilistically examine these differences. The purpose of this study was to extract the temporal components of movement using 3-on-3 basketball. We calculated the relative phases in which players approached or moved away from their teammates during mini-games in a field experiment that investigated the effects of advice on offensive coordination. These frequency distributions were estimated using
Bayesian inference and were compared before and after advice. The results showed that the probability of a synchronization trend among the offensive players after advice compared with before advice reached 70% or higher. This may be a typical case that is related to mutual anticipation based on the team strategy established through coaching. These findings contribute to a quantitative understanding of coordination processes.
Read the full abstract ▾
All authors Jun Ichikawa, Mizuki Yokoyama, Soma Ishida, Genki Ichinose
Statistical Computation
Data-Adaptive Multi-Bin Rarefying for Alpha Diversity Association Analysis under Heterogeneous Library Sizes
Mo Li
Alpha diversity association analysis is sensitive to heterogeneous library sizes, and overall rarefying to a common depth can either discard many reads from high-depth samples or exclude low-depth samples. We propose MBRarefy, a data-adaptive framework for selecting multi-bin rarefying designs. The method chooses library size cutpoints by minimizing an objective that combines residual within-bin library size dependence with a fragmentation penalty, subject to a minimum bin size constraint. The framework supports both fixed-K and varying-K cutpoint selection. In T-
cell receptor (TCR) motivated simulations, the primary fixed-K procedure controlled depth-induced false positives and improved power relative to overall rarefying when group and depth confounding was absent. Applications to TCR immunosequencing and wild baboon gut microbiome data illustrate its ability to reduce residual library size dependence while improving read and sample retention.
Read the full abstract ▾