Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Thursday 13 November

Statistical Methodology

Goodness-of-fit Test for Generalized Functional Linear Models via Projection Averaging

Assessing model adequacy is a crucial step in regression analysis, ensuring the validity of statistical inferences. For Generalized Functional Linear Models (GFLMs), which are widely used for modeling relationships between scalar responses and functional predictors, there is a recognized need for formal goodness-of-fit testing procedures. Current literature on this specific topic remains limited. This paper introduces a novel goodness-of-fit test for GFLMs. The test statistic is formulated as a U-statistic derived from a Cramér-von-Mises metric integrated over all one-dimensional projections of the functional predictor. This projection averaging strategy is designed to effectively mitigate the curse of dimensionality. We establish the asymptotic normality of the test statistic under the null hypothesis and prove the consistency under the alternatives. As the asymptotic variance of the limiting null distribution can be complex for practical use, we also propose practical bootstrap resampling methods for both continuous and discrete responses to compute p-values. Simulation studies confirm that the proposed test demonstrates good power performance across various settings, showing advantages over existing methods.
Read the full abstract

All authors Feifei Chen, Kaiming Zhang, Yanni Zhang, Hua Liang

Machine Learning (Statistics) · Cross-listed from math.NA

Randomized batch-sampling Kaczmarz methods for solving linear systems

To conduct a more in-depth investigation of randomized solvers for solving linear systems, we adopt a unified randomized batch-sampling Kaczmarz framework with per-iteration costs as low as cyclic block methods, and develop a general analysis technique to establish its convergence guarantee. With concentration inequalities, we derive new expected linear convergence rate bounds. The analysis applies to any randomized non-extended block Kaczmarz methods with arbitrary static stochastic samplings. In addition, the new rate bounds are scale-invariant, which eliminate the dependence on the magnitude of the data matrix. In most experiments, the new bounds are significantly tighter than existing ones and better reflect the empirical convergence behavior of block methods. Within this new framework, the batch-sampling distribution, as a learnable parameter, provides the possibility for block methods to achieve efficient performance in specific application scenarios, which deserves further investigation.
Read the full abstract

Statistical Methodology · Cross-listed from math.OC

Bridging Constraints and Stochasticity: A Fully First-Order Method for Stochastic Bilevel Optimization with Linear Constraints

This work provides the first finite-time convergence guarantees for linearly constrained stochastic bilevel optimization using only first-order methods, requiring solely gradient information without any Hessian computations or second-order derivatives. We address the unprecedented challenge of simultaneously handling linear constraints, stochastic noise, and finite-time analysis in bilevel optimization, a combination that has remained theoretically intractable until now. While existing approaches either require second-order information, handle only unconstrained stochastic problems, or provide merely asymptotic convergence results, our method achieves finite-time guarantees using gradient-based techniques alone. We develop a novel framework that constructs hypergradient approximations via smoothed penalty functions, using approximate primal and dual solutions to overcome the fundamental challenges posed by the interaction between linear constraints and stochastic noise. Our theoretical analysis provides explicit finite-time bounds on the bias and variance of the hypergradient estimator, demonstrating how approximation errors interact with stochastic perturbations. We prove that our first-order algorithm converges to $(\delta, \epsilon)$-Goldstein stationary points using $\Theta(\delta^{-1}\epsilon^{-5})$ stochastic gradient evaluations, establishing the first finite-time complexity result for this challenging problem class and representing a significant theoretical breakthrough in constrained stochastic bilevel optimization.
Read the full abstract

Statistical Computation

Diagnostics for Semiparametric Accelerated Failure Time Models with R Package afttest

The semiparametric accelerated failure time (AFT) model offers a direct and interpretable alternative to the Cox proportional hazards model, yet practical diagnostic tools for this framework remain limited. We introduce afttest, an R package that implements martingale-residual-based goodness-of-fit procedures for semiparametric AFT models. In addition to the recently developed multiplier bootstrap diagnostics, the package introduces a new computationally efficient resampling strategy based on an influence-function linear approximation. Unlike the original approach, which requires repeatedly solving estimating equations for each bootstrap replicate, the proposed method avoids iterative optimization and substantially reduces computation time while preserving asymptotic validity. Both the standard multiplier bootstrap and the accelerated linear approximation are implemented, allowing users to balance finite-sample performance and computational scalability. The package supports rank-based and least-squares estimators, provides omnibus, link function, and functional form tests, and includes graphical tools for visualizing residual processes. An application to the Mayo Clinic primary biliary cirrhosis study illustrates the workflow.
Read the full abstract

All authors Woojung Bae, Dongrak Choi, Jun Yan, Sangwook Kang

Wednesday 12 November

Machine Learning (Statistics)

Convex Clustering Redefined: Robust Learning with the Median of Means Estimator

Clustering approaches that utilize convex loss functions have recently attracted growing interest in the formation of compact data clusters. Although classical methods like k-means and its wide family of variants are still widely used, all of them require the number of clusters k to be supplied as input, and many are notably sensitive to initialization. Convex clustering provides a more stable alternative by formulating the clustering task as a convex optimization problem, ensuring a unique global solution. However, it faces challenges in handling high-dimensional data, especially in the presence of noise and outliers. Additionally, strong fusion regularization, controlled by the tuning parameter, can hinder effective cluster formation within a convex clustering framework. To overcome these challenges, we introduce a robust approach that integrates convex clustering with the Median of Means (MoM) estimator, thus developing an outlier-resistant and efficient clustering framework that does not necessitate prior knowledge of the number of clusters. By leveraging the robustness of MoM alongside the stability of convex clustering, our method enhances both performance and efficiency, especially on large-scale datasets. Theoretical analysis demonstrates weak consistency under specific conditions, while experiments on synthetic and real-world datasets validate the method's superior performance compared to existing approaches.
Read the full abstract

All authors Sourav De, Koustav Chowdhury, Bibhabasu Mandal, Sagar Ghosh, Swagatam Das, Debolina Paul, Saptarshi Chakraborty

Statistics Applications · Cross-listed from cs.LG

A Bayesian Model for Multi-stage Censoring

Many sequential decision settings in healthcare feature funnel structures characterized by a series of stages, such as screenings or evaluations, where the number of patients who advance to each stage progressively decreases and decisions become increasingly costly. For example, an oncologist may first conduct a breast exam, followed by a mammogram for patients with concerning exams, followed by a biopsy for patients with concerning mammograms. A key challenge is that the ground truth outcome, such as the biopsy result, is only revealed at the end of this funnel. The selective censoring of the ground truth can introduce statistical biases in risk estimation, especially in underserved patient groups, whose outcomes are more frequently censored. We develop a Bayesian model for funnel decision structures, drawing from prior work on selective labels and censoring. We first show in synthetic settings that our model is able to recover the true parameters and predict outcomes for censored patients more accurately than baselines. We then apply our model to a dataset of emergency department visits, where in-hospital mortality is observed only for those who are admitted to either the hospital or ICU. We find that there are gender-based differences in hospital and ICU admissions. In particular, our model estimates that the mortality risk threshold to admit women to the ICU is higher for women (5.1%) than for men (4.5%).
Read the full abstract

All authors Shuvom Sadhuka, Sophia Lin, Bonnie Berger, Emma Pierson

Machine Learning (Statistics)

Generalized Inequality-based Approach for Probabilistic WCET Estimation

Estimating the probabilistic Worst-Case Execution Time (pWCET) is essential for ensuring the timing correctness of real-time applications, such as in robot IoT systems and autonomous driving systems. While methods based on Extreme Value Theory (EVT) can provide tight bounds, they suffer from model uncertainty due to the need to decide where the upper tail of the distribution begins. Conversely, inequality-based approaches avoid this issue but can yield pessimistic results for heavy-tailed distributions. This paper proposes a method to reduce such pessimism by incorporating saturating functions (arctangent and hyperbolic tangent) into Chebyshev's inequality, which mitigates the influence of large outliers while preserving mathematical soundness. Evaluations on synthetic and real-world data from the Autoware autonomous driving stack demonstrate that the proposed method achieves safe and tighter bounds for such distributions.
Read the full abstract

Statistical Methodology

Multiple Treatments Causal Effects Estimation with Task Embeddings and Balanced Representation Learning

The simultaneous application of multiple treatments is increasingly common in many fields, such as healthcare and marketing. In such scenarios, it is important to estimate the single treatment effects and the interaction treatment effects that arise from treatment combinations. Previous studies have proposed using independent outcome networks with subnetworks for interactions, or combining task embedding networks that capture treatment similarity with variational autoencoders. However, these methods suffer from the lack of parameter sharing among related treatments, or the estimation of unnecessary latent variables reduces the accuracy of causal effect estimation. To address these issues, we propose a novel deep learning framework that incorporates a task embedding network and a representation learning network with the balancing penalty. The task embedding network enables parameter sharing across related treatment patterns because it encodes elements common to single effects and contributions specific to interaction effects. The representation learning network with the balancing penalty learns representations nonparametrically from observed covariates while reducing distances in representation distributions across different treatment patterns. This process mitigates selection bias and avoids model misspecification. Simulation studies demonstrate that the proposed method outperforms existing baselines, and application to real-world marketing datasets confirms the practical implications and utility of our framework.
Read the full abstract

Statistics Theory

Sample Complexity of Quadratically Regularized Optimal Transport

It is well known that optimal transport suffers from the curse of dimensionality: when the prescribed marginals are approximated by i.i.d. samples, the convergence of the empirical optimal transport problem to the population counterpart slows exponentially with increasing dimension. Entropically regularized optimal transport (EOT) has become the standard bearer in many statistical applications as it avoids this curse. Indeed, EOT has parametric sample complexity, as has been shown in a series of works based on the smoothness of the EOT potentials or the strong concavity of the dual EOT problem. However, EOT produces full-support approximations to the (sparse) OT problem, leading to overspreading in applications, and is computationally unstable for small regularization parameters. The most popular alternative is quadratically regularized optimal transport (QOT), which penalizes couplings by $L^2$ norm instead of relative entropy. QOT produces sparse approximations of OT and is computationally stable. However, its potentials are not smooth (do not belong to a Donsker class) and its dual problem is not strongly concave, hence QOT is often assumed to suffer from the curse of dimensionality. In this paper, we show that QOT nevertheless has parametric sample complexity. More precisely, we establish central limit theorems for its dual potentials, optimal couplings, and optimal costs. Our analysis is based on novel arguments that focus on the regularity of the support of the optimal QOT coupling. Specifically, we establish a Lipschitz property of its sections and leverage VC theory to bound its statistical complexity. Our analysis also leads to gradient estimates of independent interest, including $C^{1,1}$ regularity of the population potentials.
Read the full abstract

Machine Learning (Statistics)

Generalized infinite dimensional Alpha-Procrustes based geometries

This work extends the recently introduced Alpha-Procrustes family of Riemannian metrics for symmetric positive definite (SPD) matrices by incorporating generalized versions of the Bures-Wasserstein (GBW), Log-Euclidean, and Wasserstein distances. While the Alpha-Procrustes framework has unified many classical metrics in both finite- and infinite- dimensional settings, it previously lacked the structural components necessary to realize these generalized forms. We introduce a formalism based on unitized Hilbert-Schmidt operators and an extended Mahalanobis norm that allows the construction of robust, infinite-dimensional generalizations of GBW and Log-Hilbert-Schmidt distances. Our approach also incorporates a learnable regularization parameter that enhances geometric stability in high-dimensional comparisons. Preliminary experiments reproducing benchmarks from the literature demonstrate the improved performance of our generalized metrics, particularly in scenarios involving comparisons between datasets of varying dimension and scale. This work lays a theoretical and computational foundation for advancing robust geometric methods in machine learning, statistical inference, and functional data analysis.
Read the full abstract

All authors Salvish Goomanee, Andi Han, Pratik Jawanpuria, Bamdev Mishra

Statistical Methodology

Modelos Empiricos de Pos-Dupla Selecao por LASSO: Discussoes para Estudos do Transporte Aereo

This paper presents and discusses forms of estimation by regularized regression and model selection using the LASSO method - Least Absolute Shrinkage and Selection Operator. LASSO is recognized as one of the main supervised learning methods applied to high-dimensional econometrics, allowing work with large volumes of data and multiple correlated controls. Conceptual issues related to the consequences of high dimensionality in modern econometrics and the principle of sparsity, which underpins regularization procedures, are addressed. The study examines the main post-double selection and post-regularization models, including variations applied to instrumental variable models. A brief description of the lassopack routine package, its syntaxes, and examples of HD, HDS (High-Dimension Sparse), and IV-HDS models, with combinations involving fixed effects estimators, is also presented. Finally, the potential application of the approach in research focused on air transport is discussed, with emphasis on an empirical study on the operational efficiency of airlines and aircraft fuel consumption.
Read the full abstract

Statistical Methodology

Distributional Treatment Effect Transportability across Heterogeneous Sites

We study distributional transportability of treatment effects in a “cross-site, one-armed target" design, where both treated and control units are observed in a source site, but only control units are observed in a target site. Our object of interest is not estimating the average treatment effect, but recovering the full treated distribution in the target site using transfer knowledge from the source site, while allowing cross-site heterogeneity in measurement systems, observed features, outcome reporting, population composition, and latent contextual factors. We model cross-site heterogeneity through a transformation between the sites that transports the joint feature–outcome distributions. This transformation is learned from comparing the observed control samples in the source and target sites, using an optimal transport criterion. The learned transformation is then applied to the source treated sample to construct a synthetic sample from the target treated distribution. We establish convergence of the resulting synthetic empirical distribution to the target treated distribution. Simulation studies across multiple data-generating scenarios and a real-world application to patient-derived xenograft (PDX) data demonstrate that our framework recovers the full distributional properties of the target treated population.
Read the full abstract

All authors Borna Bateni, Yubai Yuan, Qi Xu, Annie Qu

Machine Learning (Statistics)

Masked Mineral Modeling: Continent-Scale Mineral Prospecting via Geospatial Infilling

Minerals play a critical role in the advanced energy technologies necessary for decarbonization, but characterizing mineral deposits hidden underground remains costly and challenging. Inspired by recent progress in generative modeling, we develop a learning method which infers the locations of minerals by masking and infilling geospatial maps of resource availability. We demonstrate this technique using mineral data for the conterminous United States, and train performant models, with the best achieving Dice coefficients of $0.31 \pm 0.01$ and recalls of $0.22 \pm 0.02$ on test data at 1$\times$1 mi$^2$ spatial resolution. One major advantage of our approach is that it can easily incorporate auxiliary data sources for prediction which may be more abundant than mineral data. We highlight the capabilities of our model by adding input layers derived from geophysical sources, along with a nation-wide ground survey of soils originally intended for agronomic purposes. We find that employing such auxiliary features can improve inference performance, while also enabling model evaluation in regions with no recorded minerals.
Read the full abstract

All authors Sujay Nair, Evan Coleman, Sherrie Wang, Elsa Olivetti

Statistical Methodology

State Space Modeling of Mortgage Default Rates under Natural Hazard Shocks

Mortgage default rates, on the one hand, serve as a measure of economic health to support decision-making by insurance companies, and on the other hand, is a key risk factor in the asset-liability management (ALM) practice, as mortgage related assets constitute a significant proportion of insurers' investment portfolios. This paper studies the relationship between economic losses due to natural hazards and mortgage default rates. The topic is greatly relevant to the insurance industry, as excessive insurance losses from natural hazards can lead to a surge in mortgage defaults, creating compounded challenges for insurers. To this end, we apply a state-space modeling approach to decouple the effect of natural hazard losses on mortgage default rates after controlling for other economic determinants through the inclusion of latent variables. Moreover, we consider a sliced variant of the classical SSM to capture the subtle relationship that only emerges when natural hazard losses are sufficiently high. Our model verifies the significance of this relationship and provides insights into how natural hazard losses manifest as increased mortgage default rates.
Read the full abstract

All authors Samuel J. Eschker, Antik Chakraborty, Melanie Gall, Peter Jevtic, Jianxi Su

Statistical Methodology

Coalescent Inference for Epidemics with Latent Periods

Coalescent models are used to study the transmission dynamics of rapidly evolving pathogens from molecular sequence data obtained from infected individuals. However coalescent parameters, such as effective population size, offer limited interpretability for transmission dynamics. In this work, we derive a coalescent model for exposed-infected population dynamics that allows us to infer the number of infected individuals and the effective reproduction number over time from the sample genealogy. The model can be interpreted as a two-deme model in which coalescence is restricted to individuals from different demes (exposed and infected). We propose a new data-augmentation framework with Phase-type distribution for Bayesian inference of epidemiological parameters. We study the performance of our approach on simulations and apply it to re-analyze the 2014 Ebola outbreak in Liberia.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Boosted GFlowNets: Improving Exploration via Sequential Learning

Generative Flow Networks (GFlowNets) are powerful samplers for compositional objects that, by design, sample proportionally to a given non-negative reward. Nonetheless, in practice, they often struggle to explore the reward landscape evenly: trajectories toward easy-to-reach regions dominate training, while hard-to-reach modes receive vanishing or uninformative gradients, leading to poor coverage of high-reward areas. We address this imbalance with Boosted GFlowNets, a method that sequentially trains an ensemble of GFlowNets, each optimizing a residual reward that compensates for the mass already captured by previous models. This residual principle reactivates learning signals in underexplored regions and, under mild assumptions, ensures a monotone non-degradation property: adding boosters cannot worsen the learned distribution and typically improves it. Empirically, Boosted GFlowNets achieve substantially better exploration and sample diversity on multimodal synthetic benchmarks and peptide design tasks, while preserving the stability and simplicity of standard trajectory-balance training.
Read the full abstract

All authors Pedro Dall'Antonia, Tiago da Silva, Daniel Augusto de Souza, César Lincoln C. Mattos, Diego Mesquita

Statistics Applications · Cross-listed from physics.optics

Bayesian inference for precise and uncertainty-quantified single-shot widefield interferometric geometrical nanometrology

Advanced geometrical nanometrology is critical for process control in semiconductor manufacturing, supporting applications in, e.g., photonic integrated circuits, nanoelectronics, and emerging quantum and optoelectronic technologies. Widefield interferometric approach provide a cost-effective, non-destructive solution for characterizing semiconductor optical waveguides, which are fundamental to nanophotonic devices. This work presents a Bayesian inference framework, implemented using Dynamic Nested Sampling, for estimating geometric parameters - such as width and height - of a semiconductor optical waveguide from a single widefield interferogram. The proposed framework reduces the need of leveraging near field scanning microscopy methods for measurements. The notable advantage is that Bayesian statistics not only provide the estimated parameter values but also quantify the uncertainty of the inference results and the fitness of the used model. The proposed full-field, single-shot interferometric approach, supported by Bayesian-based data analysis, achieves high accuracy and sensitivity - down to successful measurement of 8 nm rib waveguide - while remaining resilient to noise. Thus, the demonstrated methodology provides a cost-effective, robust, and scalable tool for semiconductor fabrication monitoring and process verification, as confirmed by both numerical simulations and experimental validation on optical waveguides. This method contributes to high-precision nanometrology by integrating advanced statistical modeling and inference techniques.
Read the full abstract

All authors Damian Suski, Maria Cywinska, Julianna Winnik, Michal Jozwik, Piotr Zdankowski, Azeem Ahmad, Balpreet S. Ahluwalia, Maciej Trusiak

Machine Learning (Statistics)

Siegel Neural Networks

Riemannian symmetric spaces (RSS) such as hyperbolic spaces and symmetric positive definite (SPD) manifolds have become popular spaces for representation learning. In this paper, we propose a novel approach for building discriminative neural networks on Siegel spaces, a family of RSS that is largely unexplored in machine learning tasks. For classification applications, one focus of recent works is the construction of multiclass logistic regression (MLR) and fully-connected (FC) layers for hyperbolic and SPD neural networks. Here we show how to build such layers for Siegel neural networks. Our approach relies on the quotient structure of those spaces and the notation of vector-valued distance on RSS. We demonstrate the relevance of our approach on two applications, i.e., radar clutter classification and node classification. Our results successfully demonstrate state-of-the-art performance across all datasets.
Read the full abstract

Statistical Methodology

Local Interaction Autoregressive Model for High Dimension Time Series Data

High-dimensional matrix and tensor time series often exhibit local dependency, where each entry interacts mainly with a small neighborhood. Accounting for local interactions in a prediction model can greatly reduce the dimensionality of the parameter space, leading to more efficient inference and more accurate predictions. We propose a Local Interaction Autoregressive (LIAR) framework and study Separable LIAR, a variant with shared row and column components, for high-dimensional matrix/tensor time series forecasting problems. We derive a scalable parameter estimation algorithm via parallel least squares with a BIC-type neighborhood selector. Theoretically, we show consistency of neighborhood selection and derive error bounds for kernel and auto-covariance estimation. Numerical simulations show that the BIC selector recovers the true neighborhood with high success rates, the LIAR achieves small estimation errors, and the forecasts outperform matrix time-series baselines. In real data applications, a Total Electron Content (TEC) case study shows the model can identify localized spatio-temporal propagation and improved prediction as compared with non-local time series prediction models.
Read the full abstract

Statistics Applications · Cross-listed from q-bio.NC

A thermoinformational formulation for the description of neuropsychological systems

Complex systems produce high-dimensional signals that lack macroscopic variables analogous to entropy, temperature, or free energy. This work introduces a thermoinformational formulation that derives entropy, internal energy, temperature, and Helmholtz free energy directly from empirical microstate distributions of arbitrary datasets. The approach provides a data-driven description of how a system reorganizes, exchanges information, and moves between stable and unstable states. Applied to dual-EEG recordings from mother-infant dyads performing the A-not-B task, the formulation captures increases in informational heat during switches and errors, and reveals that correct choices arise from more stable, low-temperature states. In an independent optogenetic dam-pup experiment, the same variables separate stimulation conditions and trace coherent trajectories in thermodynamic state space. Across both human and rodent systems, this thermoinformational formulation yields compact and physically interpretable macroscopic variables that generalize across species, modalities, and experimental paradigms.
Read the full abstract

Machine Learning (Statistics)

Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula

We study the problem of denoising when only the noise level is known, not the noise distribution. Independent noise $Z$ corrupts a signal $X$, yielding the observation $Y = X + \sigma Z$ with known $\sigma \in (0,1)$. We propose universal denoisers, agnostic to both signal and noise distributions, that recover the signal distribution $P_X$ from $P_Y$. When the focus is on distributional recovery of $P_X$ rather than on individual realizations of $X$, our denoisers achieve order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, which achieves $O(\sigma^2)$ accuracy. They shrink $P_Y$ toward $P_X$ with $O(\sigma^4)$ and $O(\sigma^6)$ accuracy in matching generalized moments and densities. Drawing on optimal transport theory, our denoisers approximate the Monge–Ampère equation with higher-order accuracy and can be implemented efficiently via score matching. Let $q$ denote the density of $P_Y$. For distributional denoising, we propose replacing the Bayes-optimal denoiser, $$\mathbf{T}^*(y) = y + \sigma^2 \nabla \log q(y),$$ with denoisers exhibiting less-aggressive distributional shrinkage, $$\mathbf{T}_1(y) = y + \frac{\sigma^2}{2} \nabla \log q(y),$$ $$\mathbf{T}_2(y) = y + \frac{\sigma^2}{2} \nabla \log q(y) - \frac{\sigma^4}{8} \nabla \!\left( \frac{1}{2} \| \nabla \log q(y) \|^2 + \nabla \cdot \nabla \log q(y) \right)\!.$$
Read the full abstract

Machine Learning (Statistics)

A general framework for adaptive nonparametric dimensionality reduction

Dimensionality reduction is a fundamental task in modern data science. Several projection methods specifically tailored to take into account the non-linearity of the data via local embeddings have been proposed. Such methods are often based on local neighbourhood structures and require tuning the number of neighbours that define this local structure, and the dimensionality of the lower-dimensional space onto which the data are projected. Such choices critically influence the quality of the resulting embedding. In this paper, we exploit a recently proposed intrinsic dimension estimator which also returns the optimal locally adaptive neighbourhood sizes according to some desirable criteria. In principle, this adaptive framework can be employed to perform an optimal hyper-parameter tuning of any dimensionality reduction algorithm that relies on local neighbourhood structures. Numerical experiments on both real-world and simulated datasets show that the proposed method can be used to significantly improve well-known projection methods when employed for various learning tasks, with improvements measurable through both quantitative metrics and the quality of low-dimensional visualizations.
Read the full abstract

Machine Learning (Statistics)

Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions

Diffusion and flow matching approaches to generative modeling have shown promise in domains where the state space is continuous, such as image generation or protein folding & design, and discrete, exemplified by diffusion large language models. They offer a natural fit when the number of elements in a state is fixed in advance (e.g. images), but require ad hoc solutions when, for example, the length of a response from a large language model, or the number of amino acids in a protein chain is not known a priori. Here we propose Branching Flows, a generative modeling framework that, like diffusion and flow matching approaches, transports a simple distribution to the data distribution. But in Branching Flows, the elements in the state evolve over a forest of binary trees, branching and dying stochastically with rates that are learned by the model. This allows the model to control, during generation, the number of elements in the sequence. We also show that Branching Flows can compose with any flow matching base process on discrete sets, continuous Euclidean spaces, smooth manifolds, and `multimodal' product spaces that mix these components. We demonstrate this in three domains: small molecule generation (multimodal), antibody sequence generation (discrete), and protein backbone generation (multimodal), and show that Branching Flows is a capable distribution learner with a stable learning objective, and that it enables new capabilities.
Read the full abstract

All authors Lukas Billera, Hedwig Nora Nordlinder, Jack Collier Ryder, Anton Oresten, Aron Stålmarck, Theodor Mosetti Björk, Ben Murrell

Statistics Applications

The trade-off between model flexibility and accuracy of the Expected Threat model in football

With an average football (soccer) match recording over 3,000 on-ball events, effective use of this event data is essential for practitioners at football clubs to obtain meaningful insights. Models can extract more information from this data, and explainable methods can make them more accessible to practitioners. The Expected Threat model has been praised for its explainability and offers an accessible option. However, selecting the grid size is a challenging key design choice that has to be made when applying the Expected Threat model. Using a finer grid leads to a more flexible model that can better distinguish between different situations, but the accuracy of the estimates deteriorates with a more flexible model. Consequently, practitioners face challenges in balancing the trade-off between model flexibility and model accuracy. In this study, the Expected Threat model is analyzed from a theoretical perspective and simulations are performed based on the Markov chain of the model to examine its behavior in practice. Our theoretical results establish an upper bound on the error of the Expected Threat model for different flexibilities. Based on the simulations, a more accurate characterization of the model's error is provided, improving over the theoretical bound. Finally, these insights are converted into a practical rule of thumb to help practitioners choose the right balance between the model flexibility and the desired accuracy of the Expected Threat model.
Read the full abstract

All authors Koen W. van Arem, Jakob Söhl, Mirjam Bruinsma, Geurt Jongbloed

Statistical Methodology

Family-wise error rate control in clinical trials with overlapping populations

We consider clinical trials with multiple, overlapping patient populations, that test multiple treatment policies specifically tailored to these populations. Such designs may lead to multiplicity issues, as false statements will affect several populations. For type I error control, often the family-wise error rate (FWER) is controlled, which is the probability to reject at least one true null hypothesis. If the joint distribution of the test statistics is known, the FWER level can be exhausted by determining critical values or adjusted $\alpha$-levels. The adjustment is typically done under the common ANOVA assumptions. However, the performed tests are then only valid under the rather strong assumption of homogeneous null effects, i.e., when the null hypothesis applies to all subpopulations and their intersections. We show that under cancelling null effects, when heterogeneous effects cancel out in some or all subpopulations, this procedure does not provide FWER control. We also suggest different alternatives and compare them in terms of FWER control and their power.
Read the full abstract

Statistics Theory

A Riemannian Generalization of Wilks' Lambda for Testing Differences Among Brain Connectomes

Frćhet analysis of variance is a well-established tool for comparing samples of non-Euclidean data such as brain connectomes, symmetric positive definite (SPD) matrices that encode coactivation of different brain regions. But by relying solely on various squared distances, it can fail to detect real group differences. We introduce a Riemannian generalization of Wilks' Lambda for SPD-valued data. Our central result is a power comparison. Against a location shift of size $\theta$, the non-centrality of the proposed test behaves as $O(\theta^2)$ while that of Fréchet ANOVA behaves as $O(\theta^4)$, so the new test is more powerful for small-to-moderate effects. We confirm this on Human Connectome Project data, where the test detects sex and acquisition-site differences in functional connectivity that Frćhet ANOVA misses while the two tests agree at negligible and large effects. The method is implemented in an open-source R package.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Supporting Evidence for the Adaptive Feature Program across Diverse Models

Theoretically exploring the advantages of neural networks might be one of the most challenging problems in the AI era. An adaptive feature program has recently been proposed to analyze feature learning, the characteristic property of neural networks, in a more abstract way. Motivated by the celebrated Le Cam equivalence, we advocate the over-parameterized sequence models to further simplify the analysis of the training dynamics of adaptive feature program and present several pieces of supporting evidence for the adaptive feature program. More precisely, after having introduced the feature error measure (FEM) to characterize the quality of the learned feature, we show that the FEM is decreasing during the training process of several concrete adaptive feature models including linear regression, single/multiple index models, etc. We believe that this hints at the potential successes of the adaptive feature program.
Read the full abstract

Statistics Theory

Pseudo-Differential Operators and Generalized Random Fields over Tori

Matérn covariance functions are ubiquitous in spatial statistics, valued for their interpretable parameters and well-understood sample path properties in Euclidean settings. This paper extends the theory of Matérn fields over tori through the pseudo-differential calculus. We first establish that a Matérn process on the $d$-dimensional torus has sample paths in $C^{\nu^-}_{\mathrm{loc}}$ for every $\nu>0$ and no more, exactly as in the Euclidean case. Our main results concern symbols whose order varies with position. Given an order function $m \in C^\infty(\mathbb{T}^d;\mathbb{R})$ we construct a Gaussian field whose sample-path Hölder exponent is $H(x) = m(x)-d/2$ at each point, and we prove that the existence threshold $m(x)>d/2$ is local: the regularity of the field near a point depends on $m$ only through its germ at that point. Finally we examine the canonical field, which supplies a direct check on both the threshold and the exponent. We use it to prove a rigidity statement: weighting the Matérn spectral density by $|k|^{-2}$ produces a superposition of ordinary Matérn fields of higher smoothness.
Read the full abstract

Statistical Methodology

Density ratio model for multiple types of survival data with empirical likelihood

The density ratio model (DRM) is a semiparametric model that relates the distributions from multiple samples to a nonparametrically defined reference distribution via exponential tilting, with finite-dimensional parameters governing their differences in shape. When multiple types of partially observed (censored/truncated) failure time data are collected in an observational study, the DRM can be utilized to conduct a single unified analysis of the combined data. In this paper, we extend the methodology for censored length-biased/truncated data to the DRM framework and formulate the inference using empirical likelihood. We develop an EM algorithm to compute the DRM-based maximum empirical likelihood estimators of the model parameters and survival function, and assess its performance through extensive simulations under correct model specification, overspecification, and misspecification, across a range of failure-time distributions and censoring proportions. We also illustrate the efficacy of our method by analyzing the duration of time spent from admission to discharge in a Montreal-area hospital in Canada. The R code that implements our method is available on GitHub at \href{https://github.com/gozhang/DRM-combined-survival}{DRM-combined-survival}.
Read the full abstract

Statistical Methodology

Designing Efficient Hybrid and Single-Arm Trials: External Control Borrowing and Sample Size Calculation

External controls (ECs) from historical trials or real-world data have gained increasing attention as a way to augment hybrid and single-arm trials, especially when balanced randomization is infeasible. While most existing work has focused on post-trial inference using ECs, their role in prospective trial design remains less explored. We address this gap by focusing on the sample size determination and power analysis for an experimental design problem that encompasses standard randomized controlled trials (RCTs), hybrid trials, and single-arm trials. Building on estimators derived from the efficient influence function, we develop hybrid and single-arm design strategies that leverage comparable EC data to reduce the required sample size of the current study. We derive asymptotic variance expressions for these estimators in terms of interpretable, population-level quantities and introduce a pre-experimental variance estimation procedure to guide sample size calculation, ensuring prespecified type I error and power for the relevant hypothesis test. Simulation studies demonstrate that the proposed hybrid and single-arm designs maintain valid type I error and achieve target power across diverse scenarios while requiring substantially fewer subjects in the current study than RCT designs. A real data application further illustrates the practical utility and advantages of the proposed hybrid and single-arm designs.
Read the full abstract