Machine Learning (Statistics) · Cross-listed from cs.LG
Path-Based Gradient Boosting for Graph-Level Prediction
Claudio Meggio, Johan Pensar and Riccardo De Bin
We propose PathBoost, a gradient tree boosting method for graph-level classification and regression that learns discriminative path-based features directly from the input graph structure. Building on a previous work, which was tailored to a specific chemistry application, PathBoost introduces three key extensions: (i) adaptation to binary classification through gradient boosting with a logistic loss, (ii) incorporation of multiple node and edge attributes into the path feature space via a prefix-based decomposition, and (iii) automatic anchor node selection based on categorical attribute diversity, eliminating the need for the user to specify the starting point of the considered path features. We compared PathBoost to graph neural networks and graph kernel approaches on several
benchmark datasets, obtaining better results in half of them, and comparable results in the rest. PathBoost shows better performances on graphs with larger average node counts. Overall, the results demonstrate that path-based boosting methods can be competitive with more complex black-box approaches.
Read the full abstract ▾
Statistical Computation · Cross-listed from cs.NE
MAEO: Multiobjective Animorphic Ensemble Optimization for Scalable Large-scale Engineering Applications
Omer F. Erdem, Dean Price, Paul Seurin et al. (+1 more)
Multiobjective optimization remains challenging for many scientific and engineering problems due to the need to balance convergence, diversity, and computational efficiency across high-dimensional objective landscapes. This work presents the Multiobjective Animorphic Ensemble Optimization (MAEO) framework, a parallelizable ensemble strategy that unifies state-of-the-art evolutionary
algorithms within an island-based architecture, overcoming the limitations of relying on a single optimizer, as implied by the No Free Lunch
theorem. MAEO uses a parameter-free hypervolume indicator for island performance assessment and a strict Pareto-rank-based individual scoring formulation that incorporates crowding distance and nadir-point proximity to ensure consistent selection pressure within each front. The framework is initiated using four
algorithms (NSGA-III, CTAEA, AGEMOEA2, SPEA2) and evaluated through extensive benchmarking on 12 DTLZ/ZDT functions under 36 dimensionality settings using Wilcoxon signed-rank tests with both hypervolume and inverse generational distance metrics. Results show that MAEO achieves balanced convergence-diversity performance, outperforming or matching some of the leading multiobjective optimization
algorithms across different
benchmark problems. To demonstrate practical applicability, MAEO is applied to the
equilibrium-cycle optimization of a small modular
nuclear reactor. Eight discrete design variables (and three objectives (levelized cost of electricity, peak soluble boron concentration, fuel cycle length) are optimized under two safety constraints. The
algorithm carried out roughly 40000 evaluations using computer simulations. MAEO identifies core designs that lower both the levelized cost of electricity and the peak boron concentration, while preserving fuel cycle length and meeting all safety constraints.
Read the full abstract ▾
All authors Omer F. Erdem, Dean Price, Paul Seurin, Majdi I. Radaideh
Machine Learning (Statistics) · Cross-listed from cs.LG
Gromov-Wasserstein Methods for Multi-View Relational Embedding and Clustering
Rafael Pereira Eufrazio, Eduardo Fernandes Montesuma and Charles Casimiro Cavalcante
Learning low-dimensional representations from multi-view relational data is challenging when underlying geometries differ across views. We propose Bary-GWMDS, a Gromov-Wasserstein-based method that operates directly on distance
matrices to learn a consensus
embedding preserving shared relational structure. By leveraging intrinsic distances, the approach naturally handles nonlinear distortions across views. We also introduce Mean-GWMDS-C, a clustering-oriented formulation that averages distance
matrices and learns reduced-support representations via a consensus Gromov-Wasserstein transport. Experiments on synthetic and real-world
datasets show that the proposed framework yields stable and geometrically meaningful
embeddings.
Read the full abstract ▾
Statistical Methodology
Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities
Yichen Xu
Synthetic tabular data are often evaluated by distributional similarity,
privacy distance, or train-on-synthetic-test-on-real predictive performance, but these criteria do not ensure validity for causal inference. We show that fully generative tabular synthesizers, including GAN- and LLM-based models, can preserve predictive utility while distorting average treatment effect (ATE) estimates. The failure is structural: ATE preservation requires both a realistic
covariate law and an accurate treatment-effect contrast, whereas prediction loss penalizes treatment-effect error only through an overlap-weighted term. Thus, under imbalance or limited overlap, a generator may reproduce dominant observed outcomes while underlearning intervention-relevant contrasts. We formalize this mismatch through sensitivity and loss-decomposition results. Motivated by this causal analysis and intuition, we propose a hybrid synthetic-data framework for causal inference that generates
covariates while modeling treatment and outcome mechanisms separately. We evaluate the framework in three settings: ATE preservation under fully generative versus hybrid synthesis, augmentation for practical positivity problems, and diagnostic simulation engines for comparing OR, IPW, AIPW, and TMLE before real-data analysis. We also stress-test the hybrid construction across settings that vary overlap,
covariate dimension, seed sample size, and treatment-effect complexity, including a logistic outcome-model misspecification check. Across controlled simulation experiments, hybrid synthesis improves causal fidelity relative to fully generative baselines; the ACTG application shows improved predictive fidelity and potential for finite-sample
estimator benchmarking. LLM-based hybrid synthesis is often more faithful than CTGAN in settings where causal fidelity can be assessed.
Read the full abstract ▾
Statistical Methodology
A Review of Methods and Practices for Missing Data in Sequential Multiple Assignment Randomized Trials (SMARTs): An Ancillary Study of a Scoping Review
Nikki L. B. Freeman, Chenyao Yu, Margaret Hoch et al. (+5 more)
Background: Missing data poses an acute threat to sequential multiple assignment randomized trial (SMART) analyses because of the sequential treatment structure and response-dependent re-randomization. Objectives: This study aimed to (1) review the current statistical methods for handling missing data in SMARTs, and (2) characterize how missing data is reported and handled in published SMARTs. Methods: We conducted a narrative review of statistical methods developed for missing data in SMARTs. Additionally, we conducted a pre-specified secondary extraction of a previously published scoping review of SMARTs focused on missing data. Extraction captured attrition rates, methods for handling missingness, and planned versus performed missing data analyses. Results: Seven methodological papers were identified; nearly all assume missing at random (MAR), and only one addresses the full set of SMART-specific missingness types. Across 30 published SMARTs, median overall attrition was 18.1% (range 0.6%-56.5%). Methods used to address missing data were described in 80% of the manuscripts; mixed-model methods were most common (30%). Among 14 studies with paired protocols, sensitivity analyses were pre-specified in 2 (14%). Conclusions: SMART-specific methodology for missing data is limited, and a substantial gap exists between available methodology and current SMART practice.
Read the full abstract ▾
All authors Nikki L. B. Freeman, Chenyao Yu, Margaret Hoch, Sydney Browder, Bradley G. Hammill, Avi Kenny, Kevin J. Anstrom, Michael R. Kosorok
Machine Learning (Statistics) · Cross-listed from cs.LG
Inverting Foundation Models of Brain Function with Simulation-Based Inference
Niels Leif Bracher, Xavier Intes and Stefan T. Radev
Foundation models of
brain activity promise a new frontier for in silico neuroscience by emulating neural responses to complex stimuli across tasks and modalities. A natural next step is to ask whether these models can also be used in reverse. Can we recover a stimulus or its properties from synthetic
brain activity? We study this question in a proof-of-concept setting using TRIBEv2. We pair the
brain emulator with large
language models (LLMs) that generate news headlines from linguistic parameters such as valence, arousal, and dominance. We then use simulation-based inference to learn a probabilistic mapping from
brain maps to latent stimulus parameters. Our results show that these parameters can be recovered from predicted
brain maps, demonstrating that the emulator's synthetic neural encodings preserve information about the controlled stimulus dimensions. They also show that LLMs can serve as controllable stimulus generators for simulated experiments. Together, these findings provide a step toward decoding and inverse design with foundation
brain models.
Read the full abstract ▾
Statistical Methodology
Bayesian change-plane regression
Yuki Ohnishi and Fan Li
Change-plane regression represents treatment effect heterogeneity through an interpretable rule that assigns patients to two groups according to whether a linear score of baseline
covariates crosses a threshold. Although easy to communicate, likelihood-based inference for the hard threshold is nonregular, invalidating standard large-sample approximations. We develop a
Bayesian framework that replaces the sharp indicator with a smooth probit gate at a declared smoothing scale and treats the smoothed subgroup rule as the reported estimand. At a fixed smoothing scale, the posterior satisfies a misspecified Bernstein–von Mises
theorem centered at the smoothed rule. Under a vanishing smoothing schedule, the posterior learns the boundary faster than the parametric rate, while within an explicit schedule window the regression block and treatment-effect contrast satisfy a Bernstein–von Mises
theorem centered at the hard-threshold values with known-subgroup oracle efficiency. Thus, effect inference pays no first-order price for regularization. A decision-theoretic reporting protocol with consistency guarantees separates evidence of heterogeneity from reporting a boundary. Computation uses latent-variable augmentation, a great-circle elliptical slice sampler, and a normalized horseshoe prior. Simulations and an application to the PREMIER trial illustrate the method.
Read the full abstract ▾
Statistical Methodology
Privacy-preserving Meta-analysis through Low-Rank Basis Hunting
Wenqi Shi, Kosuke Imai and Yi Zhang
A central challenge of meta-analysis is that the populations underlying existing studies often differ from the target population in unknown ways. We study the problem of predicting function-valued quantities, such as regression and conditional average treatment effect functions, for a new target population using only study-level
covariates and study-specific function estimates. We propose MetaHunt, a new meta-analysis methodology based on a shared low-rank structure, in which the true function from each study lies within the convex hull of a small set of latent basis functions. To recover these basis functions, we extend a vertex-hunting procedure called the Successive Projection
Algorithm to the functional setting, incorporating a denoised basis-hunting step. We establish consistency of the recovered basis functions under mild regularity conditions. We then model the relationship between study-level
covariates and the corresponding mixing weights using flexible semi-parametric or non-parametric methods. MetaHunt is
privacy-preserving and enables meta-analytic prediction based on study-level information alone, even when individual-level data are unavailable to analysts. In addition, for each study, functions of interest can be estimated using possibly different machine learning
algorithms. For uncertainty quantification, we construct prediction intervals via conformal prediction. We show that, under exchangeability and mild estimation-error conditions, these intervals achieve asymptotically valid marginal coverage. We demonstrate the effectiveness of MetaHunt through both simulation studies and empirical applications. An open-source software package is available for implementing MetaHunt.
Read the full abstract ▾
Statistical Methodology
Beyond the mean: Sequence analysis methods for clustering ordinal EMA data
Tianyi Wang, Anna L. Smith, Jillian R. Silva-Jones et al. (+2 more)
Ecological momentary assessment (EMA) ratings are widely used in studies of behavioral and psychological phenomena to capture real-time data in subjects' real-world environments. Because the data are collected repeatedly over the study period, they provide rich longitudinal rating profiles for each individual. However, the number of observations per subject is often large, while both sample size and sampling intensity can vary substantially across individuals, which complicates the analysis. In some settings, simplified summaries of individual profiles, such as averages computed across the study period, are used for downstream analyses, including regression-style modeling. Although such summaries can be convenient, they may fail to fully capture dynamic temporal patterns present in the complete longitudinal profiles. To address this, we borrow measures from sequence analysis that capture individual-level patterns over time and then applied principal component analysis (PCA) followed by $K$-means clustering to identify unobserved latent groups of individuals with similar profiles. We test our approach using simulated data from a categorical functional regression model and compare its performance with two commonly used methods for detecting unobserved group structures: latent class analysis (LCA), and latent transition analysis (LTA). Using EMA stress observations from a large sample of U.S. adults (Newman et al., 2024, 2025), we identify distinct latent stress profile groups and show that they improve characterization of the impact on cognitive performance.
Read the full abstract ▾
All authors Tianyi Wang, Anna L. Smith, Jillian R. Silva-Jones, Wendy Berry Mendes, Lauren N. Whitehurst
Statistics Theory · Cross-listed from math.PR
Kac's walk on rotation matrices mixes in $n^2 \log n$ steps
Natesh S. Pillai and Aaron Smith
Kac's walk on the rotation group, introduced by Hastings in 1970, is an important high-dimensional Markov chain with applications in statistical physics, statistics, cryptography, and computational science. Despite its simple transition rules, determining its total-variation mixing time has remained a challenging problem for decades. A key obstacle is that the walk is not conjugation-invariant, placing it beyond the reach of classical
Fourier-analytic techniques that apply to many related random walks on compact groups. We prove that Kac's walk mixes in total variation in \(O(n^2 \log n)\) steps, matching the conjectured mixing time up to constants. The proof is based on a refined two-stage coupling. Building on earlier work, the first stage contracts two copies of the chain to a small neighborhood via a Wasserstein coupling. Our main contribution is a new framework for analyzing the second-stage coupling. It can be viewed as a discrete analogue of Malliavin calculus for Markov chains. We represent the law of the chain as the pushforward of high-dimensional noise and prove quantitative non-degeneracy of the associated linearization using
matrix martingale methods. This yields an approximately Gaussian distribution in the Lie
algebra with well-conditioned covariance, allowing small group translations to be absorbed at negligible cost in total variation. Our approach provides a general framework for studying mixing in high-dimensional Markov chains in continuous state spaces with singular transition kernels.
Read the full abstract ▾
Statistical Computation
Building a GPU-Accelerated Multivariate Statistics Platform
Mike Crowhurst
Classical multivariate statistical methods such as covariance estimation and principal component analysis are well understood mathematically, yet their application at extreme data scales remains challenging. When the number of observations reaches billions, performance is limited by data movement, input-output bottlenecks, and numerical stability rather than arithmetic complexity. This work presents a case study of scaling classical multivariate statistics on a single multi-GPU node. Using C++ and CUDA, a GPU-accelerated workflow was developed to compute sufficient statistics in a single pass over a 10-billion-row
dataset. Column sums and cross-product
matrices are used to enable downstream computation of means, covariance, correlation, and principal component analysis without revisiting the raw data. The results highlight the importance of data representation, validation using known invariants, and careful numerical treatment when applying established statistical methods at large scale.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Causal Representation Learning from General Environments under Nonparametric Mixing
Ignavier Ng, Shaoan Xie, Xinshuai Dong et al. (+2 more)
Causal representation learning aims to recover the latent causal variables and their causal relations, typically represented by directed acyclic graphs (DAGs), from low-level observations such as image pixels. A prevailing line of research exploits multiple environments, which assume how data distributions change, including single-node interventions, coupled interventions, or hard interventions, or parametric constraints on the mixing function or the latent causal model, such as linearity. Despite the novelty and elegance of the results, they are often violated in real problems. Accordingly, we formalize a set of desiderata for causal representation learning that applies to a broader class of environments, referred to as general environments. Interestingly, we show that one can fully recover the latent DAG and identify the latent variables up to minor indeterminacies under a nonparametric mixing function and nonlinear latent causal models, such as additive (Gaussian) noise models or heteroscedastic noise models, by properly leveraging sufficient change conditions on the causal mechanisms up to third-order derivatives. These represent, to our knowledge, the first results to fully recover the latent DAG from general environments under nonparametric mixing. Notably, our results match or improve upon many existing works, but require less restrictive assumptions about changing environments.
Read the full abstract ▾
All authors Ignavier Ng, Shaoan Xie, Xinshuai Dong, Peter Spirtes, Kun Zhang
Other Statistics · Cross-listed from physics.optics
From Random Fringes to Deterministic Response: Statistical Foundations of Time-Reversed Young Interferometry
Jianming Wen
Young interference is usually read as the gradual statistical accumulation of random detection events. Here we show that a time-reversed Young (TRY) geometry has a different statistical character: the fringe is not a marginal distribution of detector positions, but a conditional response indexed by a programmed source coordinate. With a fixed detector and a scanned source basis, the observable is an operational hybrid correlator between detector signal and source label. The resulting interference is deterministic at the response-function level, while noise enters only through estimation precision. We formulate this distinction using Fisher information,
estimator variance, and noise scaling, clarifying why TRY naturally supports calibration, lock-in readout, null-fringe sensing, and source-plane superresolution.
Read the full abstract ▾
Statistics Applications · Cross-listed from astro-ph.IM
Beyond the Final Label: Exploiting the Untapped Potential of Classification Histories in Astronomical Light Curve Analysis
Zhuoyang Zhou, Alex I. Malz, Chad M. Schafer et al. (+3 more)
The Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate a massive collection of time series (light curves) of the measured flux of transient and variable astronomical objects. With each new flux observation, light curve classifiers need to generate updated probability distributions over candidate classes, which will then be shared with the global community for the purpose of identifying interesting targets for follow-up observations as well as less time-sensitive analysis applications. Using the synthetic light curves and classification results of participating classifiers from the Extended LSST Astronomical Time-series Classification Challenge (ELAsTiCC), we investigate a novel framework to enhance existing light curve classifications by incorporating their classification histories and the temporal
evolution of these histories. To demonstrate the potential of this approach, we introduce a model that combines a recurrent neural network and an additive
attention module, which shows improved classification accuracy and more balanced precision-recall performance compared to existing classifiers from the challenge. Furthermore, at this stage, most, if not all, of the existing classifiers are evaluated by their final classification results on complete light curves; we propose new metrics that evaluate the stability, accuracy, and early classification performance of a classifier's predictions when using limited data by considering the Wasserstein distance between the temporally evolving classification probability distributions. Our metrics offer a more comprehensive perspective for model assessment by supplementing classical methods such as the confusion
matrix and precision-recall.
Read the full abstract ▾
All authors Zhuoyang Zhou, Alex I. Malz, Chad M. Schafer, Konstantin Malanchev, Guillermo Cabrera-Vives, Christopher Hernández
Machine Learning (Statistics) · Cross-listed from cs.LG
A General Representation-Based Approach to Multi-Source Domain Adaptation
Ignavier Ng, Yan Li, Zijian Li et al. (+3 more)
A central problem in unsupervised domain adaptation is determining what to transfer from labeled source domains to an unlabeled target domain. To handle high-dimensional observations (e.g., images), a line of approaches use deep learning to learn latent representations of the observations, which facilitate knowledge transfer in the latent space. However, existing approaches often rely on restrictive assumptions to establish identifiability of the joint distribution in the target domain, such as independent latent variables or invariant label distributions, limiting their real-world applicability. In this work, we propose a general domain adaptation framework that learns compact latent representations to capture distribution shifts relative to the prediction task and address the fundamental question of what representations should be learned and transferred. Notably, we first demonstrate that learning representations based on all the predictive information, i.e., the label's Markov blanket in terms of the learned representations, is often underspecified in general settings. Instead, we show that, interestingly, general domain adaptation can be achieved by partitioning the representations of Markov blanket into those of the label's parents, children, and spouses. Moreover, its identifiability guarantee can be established. Building on these theoretical insights, we develop a practical, nonparametric approach for domain adaptation in a general setting, which can handle different types of distribution shifts.
Read the full abstract ▾
All authors Ignavier Ng, Yan Li, Zijian Li, Yujia Zheng, Guangyi Chen, Kun Zhang
Machine Learning (Statistics) · Cross-listed from econ.EM
Bootstrapping with AI/ML-generated labels
Timothy Christensen, Silvia Goncalves and Benoit Perron
AI/ML methods are increasingly used in economics to generate binary variables (or labels) via classification
algorithms. When these generated variables are included as
covariates in regressions, even small misclassification errors can induce large biases in OLS
estimators and invalidate standard inference. We study whether the bootstrap can correct this bias and deliver valid inference. We first show that a seemingly natural fixed-label bootstrap, which generates data using estimated labels but relies on a corrupted version in estimation, is generally invalid unless a strong independence condition between the latent true labels and other
covariates holds. We then propose a coupled-label bootstrap that jointly resamples the true and imputed labels, and show it is valid without this condition. Two finite-sample adjustments further improve coverage: a variance correction for uncertainty in estimated misclassification rates and a Hessian rotation for near-singular designs. We illustrate the methods in simulations and apply them to investigate the relationship between wages and remote work status.
Read the full abstract ▾
Statistics Applications
Sparse Reduced-rank Regression Methods for Spatially Misaligned Data with Application to Spatial Transcriptomics
Zitian Wu, Susmita Datta and Arkaprava Roy
Understanding the spatiotemporal dynamics of disease progression in relation to transcriptomic profiles provides key insights into complex conditions such as Alzheimer's disease. To enable such investigations, STARmap PLUS technology offers joint profiling of high-resolution spatial transcriptomics and
protein detection within the same tissue section. Detailed visual and clustering-based analyses of STARmap PLUS data by Zeng et al. (2023) provided important insights into molecular and cellular changes associated with Alzheimer's disease pathology. Motivated by this work, we develop a kernel-weighted sparse reduced-rank regression framework that estimates associations between plaque size and neighboring
cell-level transcriptomic profiles while enabling gene selection and borrowing strength across genes,
cell types, and disease stages. The proposed approach is implemented in a fully automated manner with data-driven specification of key model components. Through simulation studies, we demonstrate the robustness of the proposed method and its superiority across a range of simulation scenarios. Applied to Alzheimer's disease data, the proposed framework uncovers biologically meaningful associations, highlighting its potential for advancing the understanding of disease mechanisms.
Read the full abstract ▾
Statistics Applications
Scattered spring: How climate change disrupts the synchrony of biological events
Jonathan Auerbach, Andrew Gelman and E. M. Wolkovich
Many biological processes, including plant leafout and flowering, occur once cumulative temperatures reach a threshold, a relationship known as the thermal-sum model. In this way, temperature is thought to coordinate the timing of biological events. An important implication is that higher temperatures cause thresholds to be reached sooner so that the timing of spring events, for example, should advance earlier in the calendar year as climates warm. But growing evidence has found that, as climates have warmed, the rate of advancement has slowed (a trend known as declining sensitivity), while the variance in the timing of spring events has increased in many cases (a trend we call declining synchrony). These trends raise questions about the resilience of temperature-based coordination to anthropogenic
climate change. To answer these questions, researchers have modified the thermal-sum model by introducing additional factors and mechanisms, such as chilling and photoperiod. We show such complexity is not necessary to explain current trends of sensitivity and synchrony. Using experimental and real-world data, we find these trends are exactly as predicted by the thermal-sum model. In particular, the model predicts that as temperatures continue to increase and springtime events shift from the equinox toward the winter solstice, those events will become less synchronized and more variable, a phenomenon we refer to as a scattered spring. By 2100, our results predict much of the American South will experience a scattered spring under a high-warming scenario, with the average time between spring flowering events increasing by several weeks in many locations.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
Giansalvo Cirrincione
A widely cited result by Dong et al. (2021) showed that
Transformers built from self-
attention alone, without skip connections or feed-forward layers, suffer from rapid rank collapse: all
token representations converge to a single direction. The proposed remedy was the MLP. We show that this picture, while correct in the regime studied by Dong, is incomplete in ways that matter for architectural understanding. Three results are established. First, layer normalisation is precisely affine-rank-neutral: it preserves the affine rank of the
token representation set exactly. The widespread claim that LN "plays no role" is imprecise; the correct statement is sharper. Second, residual connections generically obstruct rank collapse in real
Transformers such as BERT-base, in a measure-theoretic sense, without contribution from the MLP. The MLP's irreplaceable function is different: generating feature directions outside the linear span of the original
token embeddings, which no stack of
attention layers can produce. Third, a phenomenon distinct from rank collapse is identified: head-channel non-identifiability. After multi-head
attention sums per-head outputs through the output projection, individual contributions cannot be canonically attributed to a specific head; n(H-1)d_k degrees of freedom per layer remain ambiguous when recovering a single head from the mixed signal. The MLP cannot remedy this because it acts on the post-summation signal. A constructive partial remedy is proposed: a position-gated output projection (PG-OP) at parameter overhead below 1.6% of the standard output projection. The four collapse phenomena identified in the literature – rank collapse in depth, in width, head-channel non-identifiability, and
entropy collapse – are unified under a
symmetry-breaking framework, each corresponding to a distinct
symmetry of the
Transformer's forward pass.
Read the full abstract ▾