Machine Learning (Statistics)
Colorectal Cancer Histopathological Grading using Multi-Scale Federated Learning
Md Ahasanul Arafath, Abhijit Kumar Ghosh, Md Rony Ahmed et al. (+5 more)
Colorectal cancer (CRC) grading is a critical prognostic factor but remains hampered by inter-observer variability and the
privacy constraints of multi-institutional data sharing. While deep learning offers a path to automation, centralized training models conflict with data governance regulations and neglect the diagnostic importance of multi-scale analysis. In this work, we propose a scalable,
privacy-preserving
federated learning (FL) framework for CRC histopathological grading that integrates multi-scale feature learning within a distributed training paradigm. Our approach employs a dual-stream ResNetRS50 backbone to concurrently capture fine-grained
nuclear detail and broader tissue-level context. This architecture is integrated into a robust FL system stabilized using FedProx to mitigate client drift across heterogeneous data distributions from multiple hospitals. Extensive evaluation on the CRC-HGD
dataset demonstrates that our framework achieves an overall accuracy of 83.5%, outperforming a comparable centralized model (81.6%). Crucially, the system excels in identifying the most aggressive Grade III tumors with a high recall of 87.5%, a key clinical priority to prevent dangerous false negatives. Performance further improves with higher magnification, reaching 88.0% accuracy at 40x. These results validate that our
federated multi-scale approach not only preserves patient
privacy but also enhances model performance and generalization. The proposed modular pipeline, with built-in preprocessing, checkpointing, and error handling, establishes a foundational step toward deployable,
privacy-aware clinical AI for digital pathology.
Read the full abstract ▾
All authors Md Ahasanul Arafath, Abhijit Kumar Ghosh, Md Rony Ahmed, Sabrin Afroz, Minhazul Hosen, Md Hasan Moon, Md Tanzim Reza, Md Ashad Alam
Machine Learning (Statistics) · Cross-listed from astro-ph.IM
Addressing prior dependence in hierarchical Bayesian modeling for PTA data analysis I: Methodology and implementation
Luigi D'amico, Eleonora Villa, Fatima Modica Bittordo et al. (+4 more)
Complex inference tasks, such as those encountered in
Pulsar Timing Array (PTA) data analysis, rely on
Bayesian frameworks. The high-dimensional parameter space and the strong interdependencies among astrophysical,
pulsar noise, and nuisance parameters introduce significant challenges for efficient learning and robust inference. These challenges are emblematic of broader issues in decision science, where model over-parameterization and prior sensitivity can compromise both computational tractability and the reliability of the results. We address these issues in the framework of hierarchical
Bayesian modeling by introducing a reparameterization strategy. Our approach employs Normalizing Flows (NFs) to decorrelate the parameters governing hierarchical priors from those of astrophysical interest. The use of NF-based mappings provides both the flexibility to realize the reparametrization and the tractability to preserve proper probability densities. We further adopt i-nessai, a flow-guided nested sampler, to accelerate exploration of complex posteriors. This unified use of NFs improves statistical robustness and computational efficiency, providing a principled methodology for addressing hierarchical
Bayesian inference in PTA analysis.
Read the full abstract ▾
All authors Luigi D'amico, Eleonora Villa, Fatima Modica Bittordo, Aldo Barca, Francesco Alì, Massimo Meneghetti, Luca Naso
Machine Learning (Statistics) · Cross-listed from cs.LG
Towards Formalizing Reinforcement Learning Theory: A Robbins-Siegmund Approach
Shangtong Zhang
In this paper, we formalize the almost sure convergence of $Q$-learning and linear temporal difference (TD) learning with Markovian samples using the Lean 4
theorem prover based on the Mathlib library. $Q$-learning and linear TD are among the earliest and most influential reinforcement learning (RL)
algorithms. The investigation of their convergence properties is not only a major research topic during the early development of the RL field but also receives significant
attention nowadays. This paper formally verifies their almost sure convergence in a unified framework based on the Robbins-Siegmund
theorem. The framework developed in this work can potentially be extended to convergence rates and other modes of convergence. This work thus makes an important step towards fully formalizing convergent RL results. The code is available at https://github.com/ShangtongZhang/rl-theory-in-lean.
Read the full abstract ▾
Machine Learning (Statistics)
Vector-valued self-normalized concentration inequalities beyond sub-Gaussianity
Diego Martinez-Taboada, Tomas Gonzalez and Aaditya Ramdas
The study of self-normalized processes plays a crucial role in a wide range of applications, from sequential decision-making to econometrics. While the behavior of self-normalized concentration has been widely investigated for scalar-valued processes, vector-valued processes remain comparatively underexplored, especially outside of the sub-Gaussian framework. In this contribution, we provide concentration bounds for self-normalized processes with light tails beyond sub-Gaussianity (such as Bennett or Bernstein bounds). We illustrate the relevance of our results in the context of online linear regression, with applications in (kernelized) linear bandits.
Read the full abstract ▾
Statistical Methodology
Bayesian Topological Analysis of Functional Brain Networks
Xukun Zhu, Michael W Lutz and Tananun Songdechakraiwut
Subtle alterations in
brain network
topology often evade detection by traditional statistical methods. To address this limitation, we introduce a
Bayesian inference framework for topological comparison of
brain networks that probabilistically models within- and between-group dissimilarities. The framework employs Markov chain Monte Carlo sampling to estimate posterior distributions of test statistics and Bayes factors, enabling graded evidence assessment beyond binary significance testing. Simulations confirmed statistical consistency to permutation testing. Applied to fMRI data from the Duke-UNC Alzheimer's Disease Research Center, the framework detected
topology-based network differences that conventional permutation tests failed to reveal, highlighting its enhanced sensitivity to early or subtle
brain network alterations in clinical neuroimaging.
Read the full abstract ▾
Statistics Applications
Accounting for Heavy Censoring in Evaluating the Risk Stratification Abilities of Existing Models for Time to Diagnosis of Huntington Disease
Kyle F. Grosser, Abigail G. Foes, Stellen Li et al. (+3 more)
Huntington disease (HD) is a neurodegenerative disease with progressively worsening symptoms. Accurately modeling time to HD diagnosis is essential for clinical trial design. Langbehn's model, the CAG-Age Product (CAP) model, the Prognostic Index Normed (PIN) model, and the Multivariate Risk Score (MRS) model have all been proposed for this task. However, these models may yield conflicting predictions and few studies have systematically compared their performance. Further, those that have could be misleading due to testing the models on the same data used to train them and failing to account for high rates of right censoring (80%+) in performance metrics. We discuss the theoretical foundations of these models, offering intuitive comparisons about their practical feasibility. We externally validate their risk stratification abilities using data from the ENROLL-HD study and two censoring-appropriate performance metrics, guiding model selection for HD clinical trial design. As these models were developed in HD studies that ended more than a decade ago, we compared their predictive performance using published parameters versus updated ones (re-estimated using ENROLL-HD). We show how these models can be used to estimate sample sizes for an HD clinical trial. Based on either metric and using published or updated parameters, the MRS model, which incorporates the most
covariates, performed best. However, the simpler PIN model offered similarly good performance while requiring fewer variables, many of which would require patients to undergo additional tests. In illustrating an HD clinical trial design, we defined an optimal threshold based on model performance metrics to determine which patients are more likely to be diagnosed. Sample size calculations using an optimal threshold based on metrics that did not account for censoring, as in previous studies, are shown to lead to underpowered trials.
Read the full abstract ▾
All authors Kyle F. Grosser, Abigail G. Foes, Stellen Li, Vraj Parikh, Tanya P. Garcia, Sarah C. Lotspeich
Statistics Applications
Post-2024 U.S. Presidential Election Analysis of Election and Poll Data: Real-life Validation of Prediction via Small Area Estimation and Uncertainty Quantification
Zheshi Zheng, Yuanyuan Li, Peter X.K. Song et al. (+1 more)
We carry out a post-election analysis of the 2024 U.S. Presidential Election (USPE) using a prediction model derived from the Small Area Estimation (SAE) methodology. With pollster data obtained one week prior to the election day, retrospectively, our SAE-based prediction model can perfectly predict the Electoral College election results in all 44 states where polling data were available. In addition to such desirable prediction accuracy, we introduce the probability of incorrect prediction (PoIP) to rigorously analyze prediction uncertainty. Since the standard bootstrap method appears inadequate for estimating PoIP, we propose a conformal inference method that yields reliable uncertainty quantification. We further investigate potential pollster biases by the means of sensitivity analyses and conclude that swing states are particularly vulnerable to polling bias in the prediction of the 2024 USPE.
Read the full abstract ▾
All authors Zheshi Zheng, Yuanyuan Li, Peter X.K. Song, Jiming Jiang
Statistics Theory
The Structure of Cross-Validation Error: Stability, Covariance, and Minimax Limits
Ido Nachum, Rüdiger Urbanke and Thomas Weinberger
Despite ongoing theoretical research on cross-validation (CV), many theoretical questions remain widely open. This motivates our investigation into how properties of
algorithm-distribution pairs can affect the choice for the number of folds in $k$-fold CV. Our results consist of a novel decomposition of the mean-squared error of cross-validation for risk estimation, which explicitly captures the correlations of error estimates across overlapping folds and includes a novel algorithmic stability notion, squared loss stability, that is considerably weaker than the typically required hypothesis stability in other comparable works. Furthermore, we prove: 1. For any learning
algorithm that minimizes empirical risk, the mean-squared error of the $k$-fold cross-validation
estimator $\widehat{L}_{\mathrm{CV}}^{(k)}$ of the population risk $L_{D}$ satisfies the following minimax lower bound: \[ \min_{k \mid n} \max_{D} \mathbb{E}\left[\big(\widehat{L}_{\mathrm{CV}}^{(k)} - L_{D}\big)^{2}\right]=\Omega\big(\sqrt{k^*}/n\big), \] where $n$ is the sample size, $k$ the number of folds, and $k^*$ denotes the number of folds attaining the minimax optimum. This shows that even under idealized conditions, for large values of $k$, CV cannot attain the optimum of order $1/n$ achievable by a validation set of size $n$, reflecting an inherent penalty caused by dependence between folds. 2. Complementing this, we exhibit learning rules for which \[ \max_{D}\mathbb{E}\!\left[\big(\widehat{L}_{\mathrm{CV}}^{(k)} - L_{D}\big)^{2}\right]=\Omega(k/n), \] matching (up to constants) the accuracy of a hold-out
estimator of a single fold of size $n/k$. Together these results delineate the fundamental trade-off in resampling-based risk estimation: CV cannot fully exploit all $n$ samples for unbiased risk evaluation, and its minimax performance is pinned between the $k/n$ and $\sqrt{k}/n$ regimes.
Read the full abstract ▾
Statistics Theory
Asymptotics of the maximum likelihood estimator of the location parameter of Pearson Type VII distribution
Kazuki Okamura
We study the maximum likelihood
estimator of the location parameter of the Pearson Type VII distribution with known scale. We rigorously establish precise asymptotic properties such as strong consistency, asymptotic normality, Bahadur efficiency and asymptotic variance of the maximum likelihood
estimator. Our focus is the heavy-tailed case, including the Cauchy distribution. The main difficulty lies in the fact that the likelihood equation may have multiple roots; nevertheless, the maximum likelihood
estimator performs well for large samples.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Why Less is More (Sometimes): A Theory of Data Curation
Elvis Dohmatob, Mohammad Pezeshki and Reyhane Askari-Hemmat
This paper introduces a theoretical framework to resolve a central paradox in modern machine learning: When is it better to use less data? This question has become critical as classical scaling laws suggesting “more is more” (Sun et al., 2025) are challenged by methods like LIMO (“less is more”) and s1 (Ye et al., 2025; Muenighoff et al., 2025), which achieve superior performance with small, aggressively curated
datasets. Here, we study data curation strategies where an imperfect oracle selects the training examples according to their difficulty and correctness. Our results provide exact scaling law curves for test error under both label-agnostic and label-aware curation rules, revealing when and why keeping only a subset of data can improve generalization. In contrast to classical scaling laws, we show that under certain conditions, small curated
datasets can outperform full
datasets, and we provide analytical conditions for this by deriving precise phase transition curves tied to data size and quality. We validate these theoretical claims with empirical results on ImageNet, confirming our predictions about when curation improves accuracy and can even mitigate model collapse. Furthermore, our framework provides a principled explanation for the contradictory curation strategies recently observed in LLM mathematical
reasoning.
Read the full abstract ▾
Statistical Methodology
The Bradley-Terry Stochastic Block Model
Lapo Santi and Nial Friel
The Bradley-Terry model is widely used for the analysis of pairwise comparison data and, in essence, produces a ranking of the items under comparison. We embed the Bradley-Terry model within a
stochastic block model, allowing items to cluster. The resulting Bradley-Terry SBM (BT-SBM) ranks clusters so that items within a cluster share the same tied rank. We develop a fully
Bayesian specification in which all quantities-the number of blocks, their strengths, and item assignments-are jointly learned via a fast Gibbs sampler derived through a Thurstonian data augmentation. Despite its efficiency, the sampler yields coherent and interpretable posterior summaries for all model components. Our motivating application analyzes men's tennis results from ATP tournaments over the seasons 2000-2022. We find that the top 100 players can be broadly partitioned into three or four tiers in most seasons. Moreover, the size of the strongest tier was small from the mid-2000s to 2018 and has increased since, providing evidence that men's tennis has become more competitive in recent years.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.OC
A Support-Set Algorithm for Optimization Problems with Nonnegative and Orthogonal Constraints
Lei Wang, Xin Liu and Xiaojun Chen
In this paper, we investigate optimization problems with nonnegative and orthogonal constraints, where any feasible
matrix of size $n \times p$ exhibits a sparsity pattern such that each row accommodates at most one nonzero entry. Our analysis demonstrates that, by fixing the support set, the global solution of the minimization subproblem for the proximal linearization of the objective function can be computed in closed form with at most $n$ nonzero entries. Exploiting this structural property offers a powerful avenue for dramatically enhancing computational efficiency. Guided by this insight, we propose a support-set
algorithm preserving strictly the feasibility of iterates. A central ingredient is a strategically devised update scheme for support sets that adjusts the placement of nonzero entries. We establish the convergence of the support-set
algorithm to a first-order stationary point, and show that its iteration complexity required to reach an $\epsilon$-approximate first-order stationary point is $O (\epsilon^{-2})$. Numerical results are strongly in favor of our
algorithm in real-world applications, including nonnegative PCA, clustering, and community detection.
Read the full abstract ▾
Statistical Methodology
Multi-layer dissolution exponential-family models for weighted signed networks
Alberto Caimo and Isabella Gollini
Understanding the structure of weighted signed networks is essential for analysing social systems in which relationships vary both in sign and strength. Despite significant advances in statistical network analysis, there is still a lack of statistical models that can jointly and rigorously account for both the sign and strength of relationships in networks. We introduce a multi-layer dissolution exponential random graph modelling framework that jointly captures the signed and weighted processes, conditional on the observed interaction structure. The framework enables rigorous assessment of structural balance effects while fully accounting for edge weights. To enhance inference, we adopt a fully-probabilistic
Bayesian hierarchical approach that partially pools information across layers, with parameters estimated via an adaptive approximate exchange
algorithm. We demonstrate the flexibility and explanatory power of the proposed methodology by applying it to bill sponsorship data from the 108th US Senate, revealing complex patterns of signed and weighted interactions and structural balance effects that traditional approaches are unable to capture.
Read the full abstract ▾
Statistical Methodology
Bayesian Causal Effect Estimation for Categorical Data using Staged Tree Models
Andrea Cremaschi, Manuele Leonelli and Gherardo Varando
We propose a fully
Bayesian approach for causal inference with multivariate categorical data based on staged tree models, a class of probabilistic graphical models capable of representing asymmetric and context-specific dependencies. To account for uncertainty in both structure and parameters, we introduce a flexible family of prior distributions over staged trees. These include product partition models to encourage parsimony, a novel distance-based prior to promote interpretable dependence patterns, and an extension that incorporates continuous
covariates into the learning process. Posterior inference is achieved via a tailored Markov Chain Monte Carlo
algorithm with split-and-merge moves, yielding posterior samples of staged trees from which average treatment effects and uncertainty measures are derived. Posterior summaries and uncertainty measures are obtained via techniques from the
Bayesian nonparametrics literature. Two case studies on electronic fetal monitoring and cesarean delivery and on anthracycline therapy and cardiac dysfunction in breast cancer illustrate the methods.
Read the full abstract ▾
Statistical Methodology
Comment on: "Model uncertainty and missing data: An Objective Bayesian Perspective"
Stefan Franssen
We give a contributed discussion on "Model uncertainty and missing data: An Objective
Bayesian Perspective", where we discuss frequentist perspectives on the proposed methodology.
Machine Learning (Statistics) · Cross-listed from cs.CL
Silenced Biases: The Dark Side LLMs Learned to Refuse
Rom Himelstein, Amit LeVi, Brit Youngmann et al. (+2 more)
Safety-aligned large
language models (LLMs) are becoming increasingly widespread, especially in sensitive applications where
fairness is essential and biased outputs can cause significant harm. However, evaluating the
fairness of models is a complex challenge, and approaches that do so typically utilize standard question-answer (QA) styled schemes. Such methods often overlook deeper issues by interpreting the model's refusal responses as positive
fairness measurements, which creates a false sense of
fairness. In this work, we introduce the concept of silenced biases, which are unfair preferences encoded within models' latent space and are effectively concealed by safety-
alignment. Previous approaches that considered similar indirect biases often relied on
prompt manipulation or handcrafted implicit queries, which present limited scalability and risk contaminating the evaluation process with additional biases. We propose the Silenced Bias
Benchmark (SBB), which aims to uncover these biases by employing activation steering to reduce model refusals during QA. SBB supports easy expansion to new demographic groups and subjects, presenting a
fairness evaluation framework that encourages the future development of fair models and tools beyond the masking effects of
alignment training. We demonstrate our approach over multiple LLMs, where our findings expose an alarming distinction between models' direct responses and their underlying
fairness issues.
Read the full abstract ▾
All authors Rom Himelstein, Amit LeVi, Brit Youngmann, Yaniv Nemcovsky, Avi Mendelson
Statistics Applications · Cross-listed from cs.LG
A Probabilistic Approach to Pose Synchronization for Multi-Reference Alignment with Applications to MIMO Wireless Communication Systems
Rob Romijnders, Gabriele Cesa, Christos Louizos et al. (+2 more)
From molecular imaging to wireless communications, the ability to align and reconstruct signals from multiple misaligned observations is crucial for system performance. We study the problem of multi-reference
alignment (MRA), which arises in many real-world problems, such as cryo-EM, computer
vision, and, in particular, wireless communication systems. Using a probabilistic approach to model MRA, we find a new
algorithm that uses relative poses as nuisance variables to marginalize out – thereby removing the global
symmetries of the problem and allowing for more direct solutions and improved convergence. The decentralization of this approach enables significant computational savings by avoiding the cubic scaling of centralized methods through cycle consistency. Both proposed
algorithms achieve lower reconstruction error across experimental settings.
Read the full abstract ▾
All authors Rob Romijnders, Gabriele Cesa, Christos Louizos, Kumar Pratik, Arash Behboodi
Statistics Theory · Cross-listed from cs.LG
Decoupled Entropy Minimization
Jing Ma, Hanlin Li and Xiang Xiang
Entropy Minimization (EM) is beneficial to reducing class overlap, bridging domain gap, and restricting uncertainty for various tasks in machine learning, yet its potential is limited. To study the internal mechanism of EM, we reformulate and decouple the classical EM into two parts with opposite effects: cluster aggregation driving factor (CADF) rewards dominant classes and
prompts a peaked output distribution, while gradient mitigation calibrator (GMC) penalizes high-confidence classes based on predicted probabilities. Furthermore, we reveal the limitations of classical EM caused by its coupled formulation: 1) reward collapse impedes the contribution of high-certainty samples in the learning process, and 2) easy-class bias induces misalignment between output distribution and label distribution. To address these issues, we propose Adaptive Decoupled
Entropy Minimization (AdaDEM), which normalizes the reward brought from CADF and employs a marginal
entropy calibrator (MEC) to replace GMC. AdaDEM outperforms DEM*, an upper-bound variant of classical EM, and achieves superior performance across various imperfectly supervised learning tasks in noisy and dynamic environments.
Read the full abstract ▾
Other Statistics · Cross-listed from q-bio.PE
Topography, climate, land cover, and biodiversity: Explaining endemic richness and management implications on a Mediterranean island
Aristides Moustakas and Ioannis N Vogiatzakis
Island endemism is shaped by complex interactions among environmental, ecological, and evolutionary factors, yet the relative contributions of topography,
climate, and land cover remain incompletely quantified. We investigated the drivers of endemic plant richness across Crete, a Mediterranean biodiversity hotspot, using spatially explicit data on species distributions, topographic complexity, climatic variability, land cover, and soil characteristics. Artificial Neural Network models, a machine learning tool, were employed to assess the relative importance of these predictors and to identify hotspots of endemism. We found that total species richness, elevation range, and climatic variability were the strongest predictors of endemic richness, reflecting the role of biodiversity, topographic heterogeneity, and climatic gradients in generating diverse habitats and micro-refugia that promote speciation and buffer extinction risk. Endemic hotspots only partially overlapped with areas of high total species richness, indicating that total species richness was the optimal from the ones examined, yet an imperfect surrogate. These environmentally heterogeneous areas also provide critical ecosystem services, including soil stabilization, pollination, and cultural value, which are increasingly threatened by tourism, renewable energy development, land-use change, and
climate impacts. Our findings underscore the importance of prioritizing mountainous and climatically variable regions in conservation planning, integrating ecosystem service considerations, and accounting for within-island spatial heterogeneity. By explicitly linking the environmental drivers of endemism to both biodiversity patterns and ecosystem function, this study provides a framework for evidence-based conservation planning in Crete and other Mediterranean islands with similar geological and biogeographic contexts.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
Unbiased Regression-Adjusted Estimation of Average Treatment Effects in Randomized Controlled Trials
Alberto Abadie, Mehrdad Ghadiri, Ali Jadbabaie et al. (+1 more)
This article introduces a leave-one-out regression adjustment (LOORA) for estimating average treatment effects in randomized controlled trials. In finite samples, LOORA removes the bias of conventional regression adjustment and yields exact variance formulas for regression-adjusted Horvitz-Thompson and difference-in-means
estimators. Ridge regularization curbs the influence of high-leverage observations, improving stability and precision in small samples. In large samples, LOORA matches the variance of the regression-adjusted
estimator in Lin (2013) while remaining exactly unbiased. Two within-subject experimental applications, each providing a realistic joint distribution of potential outcomes as ground truth, show that LOORA removes substantial bias and achieves confidence interval coverage close to the nominal level.
Read the full abstract ▾
All authors Alberto Abadie, Mehrdad Ghadiri, Ali Jadbabaie, Mahyar JafariNodeh
Machine Learning (Statistics)
RKUM: An R Package for Robust Kernel Unsupervised Methods
Md Ashad Alam
RKUM is an R package developed for implementing robust kernel-based unsupervised methods. It provides functions for estimating the robust kernel covariance operator (CO) and the robust kernel cross-covariance operator (CCO) using generalized loss functions instead of the conventional quadratic loss. These operators form the foundation of robust kernel learning and enable reliable analysis under contaminated or noisy data conditions. The package includes implementations of robust kernel canonical correlation analysis (Kernel CCA), as well as the influence function (IF) for both standard and multiple kernel CCA frameworks. The influence function quantifies sensitivity and helps detect influential or outlying observations across two-view and multi-view
datasets. Experiments using synthesized two-view and multi-view data demonstrate that the IF of the standard kernel CCA effectively identifies outliers, while the robust kernel methods implemented in RKUM exhibit reduced sensitivity to contamination. Overall, RKUM provides an efficient and extensible platform for robust kernel-based analysis in high-dimensional data applications.
Read the full abstract ▾
Machine Learning (Statistics)
Provable Separations between Memorization and Generalization in Diffusion Models
Zeqi Ye, Qijie Zhu, Molei Tao et al. (+1 more)
Diffusion models have achieved remarkable success across diverse domains, but they remain vulnerable to memorization – reproducing training data rather than generating novel outputs. This not only limits their creative potential but also raises concerns about
privacy and safety. While empirical studies have explored mitigation strategies, theoretical understanding of memorization remains limited. We address this gap through developing a dual-separation result via two complementary perspectives: statistical estimation and network approximation. From the estimation side, we show that the ground-truth score function does not minimize the empirical denoising loss, creating a separation that drives memorization. From the approximation side, we prove that implementing the empirical score function requires network size to scale with sample size, spelling a separation compared to the more compact network representation of the ground-truth score function. Guided by these insights, we develop a
pruning-based method that reduces memorization while maintaining generation quality in
diffusion transformers.
Read the full abstract ▾
All authors Zeqi Ye, Qijie Zhu, Molei Tao, Minshuo Chen
Machine Learning (Statistics) · Cross-listed from cs.LG
Cross-Modal Alignment via Variational Copula Modelling
Feng Wu, Tsai Hor Chan, Fuying Wang et al. (+2 more)
Various data modalities are common in real-world applications (e.g., electronic health records, medical images and clinical notes in healthcare). It is essential to develop
multimodal learning methods to aggregate various information from multiple modalities. The main challenge is how to appropriately align and fuse the representations of different modalities into a joint distribution. Existing methods mainly rely on concatenation or the Kronecker product, oversimplifying the interaction structure between modalities and indicating a need to model more complex interactions. Additionally, the joint distribution of latent representations with higher-order interactions is underexplored. Copula is a powerful statistical structure for modelling the interactions among variables, as it naturally bridges the joint distribution and marginal distributions of multiple variables. We propose a novel copula-driven
multimodal learning framework, which focuses on learning the joint distribution of various modalities to capture the complex interactions among them. The key idea is to interpret the copula model as a tool to align the marginal distributions of the modalities efficiently. By assuming a Gaussian mixture distribution for each modality and a copula model on the joint distribution, our model can generate accurate representations for missing modalities. Extensive experiments on public MIMIC
datasets demonstrate the superior performance of our model over other competitors. The code is available at https://github.com/HKU-MedAI/CMCM.
Read the full abstract ▾
All authors Feng Wu, Tsai Hor Chan, Fuying Wang, Guosheng Yin, Lequan Yu
Statistics Theory
Statistical Properties of Rectified Flow
Gonzalo Mena, Arun Kumar Kuchibhotla and Larry Wasserman
Rectified flow (Liu et al., 2022; Liu, 2022; Wu et al., 2023) is a method for defining a transport map between two distributions, and enjoys popularity in machine learning, although theoretical results supporting the validity of these methods are scant. The rectified flow can be regarded as an approximation to optimal transport, but in contrast to other transport methods that require optimization over a function space, computing the rectified flow only requires standard statistical tools such as regression or density estimation, which we leverage to develop empirical versions of transport maps. We study some structural properties of the rectified flow, including existence, uniqueness, and regularity, as well as the related statistical properties, such as rates of convergence and central limit
theorems, for some selected
estimators. To do so, we analyze the bounded and unbounded cases separately as each presents unique challenges. In both cases, we are able to establish convergence at faster rates than those for the usual nonparametric regression and density estimation.
Read the full abstract ▾
Statistical Methodology
On Ignorability of Preferential Sampling in Geostatistics
Changqing Lu, Ganggang Xu, Junho Yang et al. (+1 more)
Preferential sampling has attracted considerable
attention in geostatistics since the pioneering work of Diggle et al. (2010). A variety of likelihood-based approaches have been developed to correct estimation bias by explicitly modelling the sampling mechanism. While effective in many applications, these methods are often computationally expensive and can be susceptible to model misspecification. In this paper, we present a surprising finding: some existing non-likelihood-based methods that ignore preferential sampling can still produce unbiased and consistent
estimators under the widely used framework of Diggle et al. (2010) and its extensions. We investigate the conditions under which preferential sampling can be ignored and develop relevant
estimators for both regression and covariance parameters without specifying the sampling mechanism parametrically. Simulation studies demonstrate clear advantages of our approach, including reduced estimation error, improved confidence interval coverage, and substantially lower computational cost. To show the practical utility, we further apply it to a tropical forest data set.
Read the full abstract ▾
All authors Changqing Lu, Ganggang Xu, Junho Yang, Yongtao Guan
Statistics Applications
Modeling Headway in Heterogeneous and Mixed Traffic Flow: A Statistical Distribution Based on a General Exponential Function
Natchaphon Leungbootnak, Zihao Li, Zihang Wei et al. (+2 more)
The ability of existing headway distributions to accurately reflect the diverse behaviors and characteristics in heterogeneous traffic (different types of vehicles) and mixed traffic (human-driven vehicles with autonomous vehicles) is limited, leading to unsatisfactory goodness of fit. To address these issues, we modified the exponential function to obtain a novel headway distribution. Rather than employing Euler's number (e) as the base of the exponential function, we utilized a real number base to provide greater flexibility in modeling the observed headway. However, the proposed is not a probability function. We normalize it to calculate the probability and derive the closed-form equation. In this study, we utilized a comprehensive experiment with five open
datasets: highD, exiD, NGSIM, Waymo, and Lyft to evaluate the performance of the proposed distribution and compared its performance with six existing distributions under mixed and heterogeneous traffic flow. The results revealed that the proposed distribution not only captures the fundamental characteristics of headway distribution but also provides physically meaningful parameters that describe the distribution shape of observed headways. Under heterogeneous flow on highways (i.e., uninterrupted traffic flow), the proposed distribution outperforms other candidate distributions. Under urban road conditions (i.e., interrupted traffic flow), including heterogeneous and mixed traffic, the proposed distribution still achieves decent results.
Read the full abstract ▾
All authors Natchaphon Leungbootnak, Zihao Li, Zihang Wei, Dominique Lord, Yunlong Zhang
Machine Learning (Statistics)
Provable Accelerated Bayesian Optimization with Knowledge Transfer
Haitao Lin, Boxin Zhao, Mladen Kolar et al. (+1 more)
We study how to accelerate
Bayesian optimization (BO) on a target task by transferring historical knowledge from related source tasks. Existing work on BO with knowledge transfer either lacks theoretical guarantees or achieves the same regret as BO in the non-transfer setting, $\widetilde{O}(\sqrt{T \gamma_f})$, where $T$ is the number of evaluations of the target function and $\gamma_f$ denotes its information gain. In this paper, we propose the DeltaBO
algorithm, which builds a novel uncertainty-quantification approach on the difference function $\delta$ between the source and target functions, which are allowed to belong to different Reproducing Kernel
Hilbert Spaces (RKHSs). Under mild assumptions, we prove that the regret of DeltaBO is of order $\widetilde{O}(\sqrt{T (T/N + \gamma_\delta)})$, where $N$ denotes the number of evaluations from source tasks and typically $N \gg T$. In many applications, source and target tasks are similar, which implies that $\gamma_\delta$ can be much smaller than $\gamma_f$. Empirical studies on both real-world hyperparameter-tuning tasks and synthetic functions show that DeltaBO outperforms other baseline methods and also verify our theoretical claims. Our code is available on GitHub.
Read the full abstract ▾
All authors Haitao Lin, Boxin Zhao, Mladen Kolar, Chong Liu
Statistics Applications · Cross-listed from physics.med-ph
SDE-based Monte Carlo dose calculation for proton therapy validated against Geant4
Christopher B.C. Dean, Maria L. Pérez-Lara, Emma Horton et al. (+3 more)
Objective: To assess the accuracy and computational performance of a
stochastic differential equation (SDE)–based model for
proton beam dose calculation by benchmarking against Geant4 in simplified phantom geometries. Approach: Building on Crossley et al. (2025), we implemented the SDE model using standard approximations to interaction cross sections and mean excitation energies, enabling straightforward adaptation to new materials and configurations. The model was benchmarked against Geant4 in homogeneous, longitudinally heterogeneous and laterally heterogeneous phantoms to assess depth–dose behaviour, lateral transport and material heterogeneities. Main results: Across all phantoms and beam energies, the SDE model reproduced the main depth–dose characteristics predicted by Geant4, with
proton range agreement within 0.2 mm for 100 MeV beams and 0.6 mm for 150 MeV beams. Voxel–wise comparisons yielded gamma pass rates exceeding 95% under 2%/0.5 mm criteria with a 1% dose threshold. Differences were localised to steep dose gradients or material interfaces, while overall lateral beam dispersion was well reproduced. The SDE model achieved speed-up factors of about 2.5–3 relative to single-threaded Geant4. Significance: The SDE approach reproduces key dosimetric features with good accuracy at lower computational cost and is amenable to parallel and GPU implementations, supporting fast
proton therapy dose calculations.
Read the full abstract ▾
All authors Christopher B.C. Dean, Maria L. Pérez-Lara, Emma Horton, Matthew Southerby, Jere Koskela, Andreas E. Kyprianou
Statistical Methodology
Beyond Maximum Likelihood: Variational Inequality Estimation for Generalized Linear Models
Linglingzhi Zhu, Jonghyeok Lee and Yao Xie
Generalized linear models (GLMs) are fundamental tools for statistical modeling, with maximum likelihood estimation (MLE) serving as the classical approach for parameter inference. While MLE performs well for canonical GLMs, it can become computationally challenging in more general settings with non-canonical, non-smooth, or nonlinear link functions, where the resulting optimization landscape may be ill-conditioned, non-convex, or non-differentiable. In this paper, we study an alternative estimation framework based on variational inequalities (VIs), which formulates GLM estimation through an operator-based
equilibrium condition rather than likelihood minimization. We analyze the VI
estimator from a statistical perspective and establish finite-sample error bounds and asymptotic normality under mild regularity conditions, together with convergence guarantees for fixed-point and
stochastic approximation
algorithms. The framework accommodates a broad class of link functions, including non-canonical and non-monotone cases satisfying a strong Minty-type condition, and extends naturally to generalized additive models via basis expansion. Numerical experiments demonstrate that the VI approach achieves competitive finite-sample accuracy and improved numerical stability relative to MLE, particularly in GLMs and GAMs with non-canonical or non-smooth link functions.
Read the full abstract ▾