Machine Learning (Statistics) · Cross-listed from cs.LG
Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning
Adrien Weihs and Hayden Schaeffer
We study the approximation and statistical complexity of learning collections of operators in a shared multi-task setting, with a focus on the Multiple Neural Operators (MNO) architecture. For broad classes of Lipschitz multiple operator maps, we derive near-optimal upper bounds for approximation and statistical generalization. On the lower-bound side, we establish a curse of parametric complexity and prove corresponding minimax rates. Together, these results show that shared representations across tasks do not increase the overall cost: multi-task operator learning follows the same scaling laws as single operator learning. We also compare MNO with a multi-task extension of DeepONet based on concatenated task inputs and show that, from a worst-case approximation-complexity perspective, both architectures satisfy essentially the same asymptotic rates.
Read the full abstract ▾
Statistics Applications
Comparison of probabilistic nowcasts and forecasts of SARS-CoV-2 variant proportions made by hierarchical multinomial linear regression models
Isaac MacArthur, Thomas Robacker, Evan L. Ray et al. (+3 more)
Nowcasting and
forecasting of infectious diseases have become increasingly important since the SARS-CoV-2 pandemic. In particular, methods for modeling the composition of circulating variants at a given time have seen more use in part due to a large increase in the frequency of genomic sequencing conducted as a part of routine surveillance. However, methods must take into account that locations have different amounts of data and sometimes have different trends. We discuss hierarchical multinomial logistic regression (HMLR), a commonly used method for
forecasting SARS-CoV-2 variants, which allows for data sharing across locations. We show how it has been used in the literature, and define a class of HMLR models for SARS-CoV-2 variant nowcasting and
forecasting. We rigorously test a subset of this class of models using the framework of the US SARS-CoV-2 Variant Nowcast Hub, a collaborative modeling project that launched in 2024. We created two years of weekly predictions based on retrospective
datasets, with the prediction dates ranging from Wednesday, August 3, 2022, to Wednesday, August 7, 2024. We tested 12 HMLR models against a baseline model on these
datasets. We found that the HMLR models outperformed the baseline both in terms of probabilistic accuracy, as measured by the energy score, as well as point accuracy, as measured by the Brier score. Overall, we find that HMLR models perform best with respect to the baseline model in locations with more data, and more complex HMLR models also showed more improvement in those high-data locations; however, there was no one best model across all metrics, and simpler HMLR models perform better in low-data locations. We find that HMLR models perform well in practice for nowcasting and
forecasting SARS-CoV-2 variants.
Read the full abstract ▾
All authors Isaac MacArthur, Thomas Robacker, Evan L. Ray, Benjamin W. Rogers, Nicholas G. Reich, Maryclare Griffin
Statistical Methodology
Positive-definiteness in separable priors: effects on prior interpretability and inference
Jack Storror Carter and David Rossell
A popular class of priors for symmetric positive-definite
matrices assumes independent entries and adds a truncation to ensure positive-definiteness. While conceptually simple and often computationally convenient, unless done carefully this truncation can have unintended effects. If the truncated prior or its margins are significantly different from their untruncated counterpart, then its interpretability may suffer, its shrinkage properties become harder to characterise, and posterior inference may be affected in unanticipated ways. We investigate the effect of the truncation both for dense and sparse
matrices, and show how to set prior parameters such as the variance of off-diagonal entries such that said effect is mitigated as the
matrix dimension grows. We pay particular
attention to sparse inference where, unless prior parameters are set carefully, the truncated prior and hence its corresponding posterior assign systematically higher mass to sparser structures than the untruncated prior.
Read the full abstract ▾
Statistical Methodology
A new class of functional conditional autoregressive models
Sooran Kim
We introduce a new class of conditional autoregressive models for spatially dependent functional data, formulated through conditional means given neighboring functional observations and characterized by a covariance operator and a spatial dependence parameter. Our estimation strategy consists of three components: (i) estimating the covariance operator using conditionally centered data, (ii) estimating the spatial dependence parameter by maximizing the likelihood of projected observations, and (iii) applying a novel profile-based approach to obtain the final
estimators. Under an expanding
lattice framework, we establish two key theoretical results. First, we establish the consistency of the proposed covariance
estimator, which is not attainable using naive methods based on marginally centered data. Second, we prove that the spatial dependence parameter
estimator is superconsistent and asymptotically normal, where the latter property enables statistical inference for spatial dependence in functional data – a contribution that is novel in the existing literature. Numerical studies support the theoretical results and demonstrate the computational efficiency of our method. Finally, we illustrate its practical utility by analyzing weekly PM$_{2.5}$ concentration trajectories in 2019 across counties in the Midwestern United States.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CL
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
Meimingwei Li, Yuanhao Ding, Esteban Garces Arias et al. (+1 more)
Recent work has identified a counterintuitive phenomenon termed "Hyperfitting", where fine-tuning Large
Language Models (LLMs) to near-zero training loss on small
datasets surprisingly enhances open-ended generation quality and mitigates repetition in greedy decoding. While effective, the underlying mechanism remains poorly understood, with the extremely low-
entropy output distributions suggesting a potential equivalence to simple temperature scaling. In this work, we demonstrate that this phenomenon is fundamentally distinct from distribution sharpening;
entropy-matched control experiments reveal that temperature scaling fails to replicate the diversity gains of hyperfitting. Furthermore, we falsify the hypothesis of static vocabulary reweighting, showing through ablation studies that hyperfitting relies on a dynamic, context-dependent rank reordering mechanism. Layer-wise analysis localizes this effect to a "Terminal Expansion" in the final
transformer block, where a substantial
geometric expansion of the feature space (Delta Dim approx +80.8) facilitates the promotion of deep-tail
tokens. Additionally, we introduce Late-Stage LoRA, a targeted fine-tuning strategy that updates only the final 5 layers, yielding robust generation with minimal parameter updates
Read the full abstract ▾
All authors Meimingwei Li, Yuanhao Ding, Esteban Garces Arias, Christian Heumann
Machine Learning (Statistics)
A Martingale Kernel Independence Test
Felix Laumann, Zhaolu Liu and Mauricio Barahona
The
Hilbert-Schmidt Independence Criterion (HSIC) and its joint-independence extension $d\mathrm{HSIC}$ are degenerate $V$-statistics whose data-dependent weighted-$\chi^2$ null limits force a permutation calibration that multiplies the per-test cost by the number of permutations, in practice two orders of magnitude. Adapting the recent martingale MMD construction for two-sample testing to the (joint) independence problem, we introduce two studentised statistics whose null distributions are standard normal regardless of the data law, so that a single normal-
quantile lookup replaces the permutation step entirely. The first, $m\mathrm{HSIC}$, is a self-normalised lower-triangular sum of the Hadamard product of two empirically centred Gram
matrices. Under independence and bounded-fourth-moment kernels it converges to a standard normal. It is consistent against every fixed alternative, and runs at quadratic cost in the sample size without any sample split, matching the biased HSIC $V$-statistic. Our second statistic, $md\mathrm{HSIC}$, achieves finite-sample consistency with a single half-sample split: the centring is estimated on one half and the lower-triangular self-normalised martingale is run on the other, shrinking the conditional-mean residual to a quantity that is exponentially small in $d$, so the statistic is asymptotically standard normal at every fixed number of jointly tested variables, with a per-test cost that grows only linearly in $d$. On synthetic data with per-variable input dimension from $1$ to $500$ and between $2$ and $10$ jointly tested variables, both statistics match the empirical type-I error rate and test power of permutation-calibrated baselines while running $25$ to $60\times$ faster.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Generative Modeling by Value-Driven Transport
Pablo Moreno-Muñoz, Adrian Müller and Gergely Neu
We propose a new framework for generative modeling based on a discrete-time
stochastic control formulation of measure transport. Adapting classic results from control theory, we formulate our problem as a linear program whose dual variables correspond to the optimal value function of the control problem, which directly encodes the optimal control policy. Exploiting this LP formulation, we develop an efficient simulation-free primal-dual
algorithm for computing approximately optimal value functions and the associated value-driven transport (VDT) policies which approximate the true optimal policy. We show that well-trained VDT policies enjoy numerous favorable properties in comparison with other state-of-the-art methods based on flows,
diffusions, or Schrödinger bridges: they lead to straight transport paths which can be simulated quickly and robustly, and can be enhanced in all the same ways as
diffusion and flow-based models (e.g., conditional generation, classifier-free guidance, unpaired data-to-data translation are all easy to incorporate). We evaluate our methodology in a range of experiments, with results that indicate strong performance and good potential for scalability.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks
Donald Flynn, Hadas Yaron Goldhirsh, Jonathan P. Keating et al. (+1 more)
Backdoor poisoning attacks behave counter-intuitively in high dimensions: stronger training triggers can help the defender. We study regularised generalised linear models on Gaussian-mixture data in the proportional regime ($p/n \to \kappa$), varying the training trigger strength $\alpha$ against a fixed test trigger. Three phenomena emerge: (i) clean test accuracy increases with $\alpha$; (ii) attack success peaks at a finite $\alpha$ and then declines; and (iii) the most damaging trigger direction is the minimum eigenvector of the data covariance. We prove all three results in closed form for the squared loss, and extend (i) and (ii) to general convex GLM losses via a Gaussian-proxy fixed-point system. We identify a finite-sample noise floor proportional to $\kappa$ as the mechanism behind (i), invisible to classical $n \gg p$ analysis. Experiments on CIFAR-10 and Gaussian surrogates match the theory closely; ResNet-18 experiments show the same phenomena beyond the convex setting.
Read the full abstract ▾
All authors Donald Flynn, Hadas Yaron Goldhirsh, Jonathan P. Keating, Inbar Seroussi
Machine Learning (Statistics)
Do Not Trust The Auctioneer: Learning to Bid in Feedback-Manipulated Auctions
Luigi Foscari, Matilde Tullii and Vianney Perchet
Shilling is the use of artificial bids to make competition appear stronger and push prices upward. We study repeated first-price auctions in which shilling affects feedback but not allocation: the learner wins or loses against the real competing bid, but after a loss observes the maximum of the real bid and an independent shill bid. Thus the manipulation changes what the learner observes and hence how it learns to bid, without changing the outcome of the current auction. We analyze regret with respect to the best bid
benchmark, assuming that the shill-bid distribution is known. Even then, shilling can mask the real bid, while useful side information appears only through intermittent low-shill events. Our
algorithm combines a robust interval-elimination branch, which ignores the shilled report and achieves the dynamic-pricing rate $\tilde{\mathcal{O}}(T^{2/3})$, with an optimistic branch that debiases losing-side reports and exploits the resulting suffix information when it is reliable and achieves the first-price auctions rate $\tilde{\mathcal{O}}(\sqrt{T})$. A validation and racing procedure lets the
algorithm use these optimistic updates without knowing the right scale or feedback geometry in advance. We complement the upper bounds with a matching lower bound, up to logarithmic factors, in the single-active-region case. Overall, the results show that even feedback-only shilling can sharply alter the statistical difficulty of repeated bidding.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.NE
Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions
Gabriel Kronberger, Fabricio Olivetti de Franca, Deaglan J. Bartlett et al. (+2 more)
Symbolic regression with genetic programming (GPSR) may suffer from overfitting and structural bloat, especially when noise is present. In this paper we evaluate description length (DL) and fractional Bayes factor (FBF) criteria as principled, data-efficient alternatives to heuristics for selecting compact expressions that generalise well. We implement DL using a Fisher-information-based parameter encoding and compare it to AIC and BIC across multiple
datasets, including noisy synthetic
benchmarks and real-world regression problems. We study three search/selection strategies: (i) multi-objective search for accuracy and program length followed by DL/FBF selection; (ii) multi-objective search using DL directly as an objective; and (iii) single-objective optimisation with DL/FBF as the fitness. Across
datasets we find that DL/FBF post-selection improves test performance compared to AIC/BIC baseline and that BIC in combination with the same function complexity penalty from DL/FBF produces similar results. In contrast, using DL/FBF directly as a fitness function in single-objective GPSR frequently induces premature convergence to overly simple models. We conclude with practical guidance for using DL/FBF as robust model-selection tools in genetic programming workflows.
Read the full abstract ▾
All authors Gabriel Kronberger, Fabricio Olivetti de Franca, Deaglan J. Bartlett, Harry Desmond, Pedro G. Ferreira
Statistical Methodology
From Volterra Series to Kunchenko Stochastic Polynomials: Half a Century of Non-Gaussian Estimation Methodology
Serhii Zabolotnii
This paper reconstructs the half-century
evolution of the scientific school founded by Yuriy P. Kunchenko (1939–2006) as the development of a semiparametric methodology for non-Gaussian estimation. Starting with Kunchenko's 1972/1973 dissertation applying Volterra series to estimate parameters of random processes, the trajectory is followed through 2006–2026. Kunchenko
stochastic polynomials are presented as a coherent family of moment-cumulant procedures: the
polynomial maximization method (PMM) for parameter estimation,
polynomial criteria for hypothesis testing, and decomposition in spaces with a generating element. The paper details the school's structure: a verified genealogy of 15 defended dissertations, collaborations in Poland, Slovakia, and Germany, and the R package EstemPMM. A recent 2026 paper on Volterra-based signal processing is analyzed, showing how Kunchenko's nonlinear formulation reappears in applied
radio engineering. We build a formal bridge between finite Volterra models and generalized Kunchenko
polynomials, while separating the MMSE/L2 criterion from PMM: the former is a covariance projection for kernel adaptation, whereas PMM is a parameter-dependent moment procedure. PMM efficiency claims are stated conditionally: gains require that moments exist, the centered correlant
matrix is nondegenerate, and the variance reduction coefficient is below one. The concluding research program operationalizes the historical reconstruction into testable statistical and signal-processing tasks.
Read the full abstract ▾
Statistics Theory · Cross-listed from q-bio.PE
Spatiotemporal dynamics and ecological risk factors of highly pathogenic avian influenza A(H5N1) in Canadian wildlife: A One Health surveillance analysis
Hammed Olawale Fatoyinbo and Hoyeon Jeong
Highly pathogenic avian influenza A(H5N1) has expanded geographically and ecologically, affecting wild birds, mammalian wildlife, domestic animals, and humans. Wildlife surveillance provides critical early warning for One Health preparedness, yet national-scale analyses integrating host ecology, spatial patterns, seasonality, viral lineage, and risk factors remain limited. This study analysed Canadian wildlife HPAI A(H5N1) surveillance records from 2022 to 2026 to characterise spatiotemporal dynamics and identify factors associated with detection counts. A retrospective analysis of 2,657 detections across 13 provinces and territories was conducted using descriptive epidemiology, spatial clustering methods, and Negative Binomial mixed models. Detections were predominantly avian, with waterfowl and raptors as the major host groups, while mammals accounted for a smaller but epidemiologically important proportion. Detection burden was highest in 2022, with increased activity in autumn and spring. Ontario, Alberta, and British Columbia were identified as major hotspots, with evidence of local clustering in parts of the Prairie region. Reassortant Eurasian-North American lineages dominated detections and were strongly associated with higher detection counts. Modelling results identified year, season, and lineage as key predictors. These findings support risk-based One Health surveillance prioritising high-burden regions, migration-associated periods, key avian host groups, reassortant viral lineages, and continued monitoring of mammalian wildlife.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Partial Fusion of Neural Networks: Efficient Tradeoffs Between Ensembles and Weight Aggregation
Fabian Morelli and Stephan Eckstein
Ensembles of neural networks typically outperform individual networks but incur large computational costs, whereas weight aggregation produces less costly, yet also less accurate, aggregate models. We introduce partial fusion of networks, which interpolates between ensembles and weight aggregation and thus allows for a flexible tradeoff between computational cost and performance. A direct way to achieve this is to extend existing weight aggregation methods based on
neuron-level similarity between different networks, where partial fusion then only aggregates weights of
neurons which are most similar. We showcase one particular method to jointly identify which
neurons are most similar and match them via partial optimal transport. Further, we consider the more general perspective of weight aggregation and partial fusion as generalized
pruning of ensemble models, where
neurons cannot just be deleted, but also linearly combined. Finally, we show that generalized
pruning applied to a single network yields similar benefits as partial fusion by allowing for a tradeoff between isolating, deleting, and linearly combining
neurons based on similarity. Our code is available at https://github.com/Fabian-Mor/partial_fusion_nn.
Read the full abstract ▾
Machine Learning (Statistics)
The ASE-LSE Disagreement Landscape: An End-to-End Characterisation of Extremes and Structural Drivers
Minh Triet Pham and Ian Gallagher
Two of the most widely used methods for analysing graph data, Adjacency Spectral
Embedding and
Laplacian Spectral
Embedding, often produce different results when applied to the same graph. Yet the structural reasons behind this disagreement remain incompletely understood. This paper provides an end-to-end account of ASE-LSE latent subspace disagreement. We first prove that the two methods produce identical latent subspaces for every
embedding dimension whenever the
Laplacian is a scalar multiple of the adjacency
matrix, and show that this scalar relationship holds if and only if the graph is either regular or bipartite biregular. This anchor result identifies a sufficient condition for perfect agreement that pins down the floor of the disagreement spectrum and supplies the baseline for the perturbation analysis. We then prove that no maximal-disagreement graph or family of graphs exists: the disagreement is always strictly below its theoretical ceiling, and we exhibit a witness family demonstrating that no finite maximum is attainable, so the disagreement landscape has no maximiser. With both endpoints established, we derive a Regularity Departure Bound whose two terms isolate degree heterogeneity and eigengap as the primary structural factors influencing disagreement in the middle regime. Empirical validation across thousands of simulated graphs confirms the mechanisms predicted by the bound: heterogeneity pushes disagreement up, eigengap suppresses it, and their joint ratio emerges as a unified predictor of ASE-LSE disagreement, suggesting when the two
embeddings can be treated as interchangeable and when they cannot.
Read the full abstract ▾
Statistical Methodology
Chained Markov melding using divide and conquer sequential Monte Carlo
Yixuan Liu and Robert J. B. Goudie
Specifying a full
Bayesian model that integrates multiple data sources can be challenging. One natural approach is to specify each individual model separately and join them afterwards. This is the approach adopted in Markov melding. However, when adjacent submodels share common quantities, as in chained Markov melding, posterior inference can be challenging for existing MCMC-based approaches. In this paper, we propose a new multi-stage sampler for chained Markov models involving an arbitrary number of submodels. The proposed sampler adopts a divide-and-conquer sequential Monte Carlo approach for the tree-structured model that fits naturally with the structure of chained Markov melding. The resulting multi-stage sampler provides a flexible alternative for sampling from complex joint models, as its separate sampling scheme for different submodels avoids the need for directly sampling from the full model. We demonstrate applications of the sampler through two examples. The first is a toy example involving 11 submodels of various types. The second example considers an ecologically integrated population model that combines multiple
datasets to estimate immigration and reproduction rates.
Read the full abstract ▾
Statistical Computation
moveEZ: An R Package for Animated Biplots
Raeesa Ganey and Johané Nienkemper-Swanepoel
The moveEZ (pronounced move easy) R package provides tools for constructing animated PCA biplots that reveal how multivariate structure evolves across the ordered levels of a categorical variable. Built as an extension to the biplotEZ package, moveEZ offers three animation frameworks of increasing methodological complexity: a fixed variable frame, in which variable vectors remain constant and only sample positions are animated; and two dynamic frames, in which both sample positions and variable vectors are recomputed and animated at each level. The dynamic frames support Procrustes
alignment and reflection to ensure visual continuity across levels, and are compatible with high-dimensional
datasets including grouped structures. The package integrates with gganimate to produce high-quality animations suitable for publications and presentations, and supports both animated and static faceted displays via a single argument. Although originally motivated by tracking shifts in African
climate indicators, moveEZ is domain-agnostic and applicable wherever multivariate measurements are recorded repeatedly across an ordered categorical variable, including economic, ecological, and biological settings.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.DG
Empirical Hodge Laplacians: Spectral Convergence and Harmonic Forms from Point Clouds
Hông Vân Lê
Let $M^n\subset\mathbb R^d$ be a closed, connected, orientable $C^4$-smooth Riemannian submanifold of dimension $n\ge3$. We construct, for each degree $0\le k\le n$, a family of deformed Hodge
Laplacians $\Delta_t^k$, $t>0$, defined in terms of the extrinsic geometry of $M^n$, and prove that $\Delta_t^k$ converges uniformly to the classical Hodge
Laplacian $\Delta^k$ as $t\to0^+$. Given an i.i.d. uniformly distributed point cloud $S_m\subset M^n$, we define empirical Hodge operators $\widehat\Delta_{t,S_m}^k$. Under the scaling $t=m^{-1/(2n)}$, we prove uniform consistency in probability and compact Mosco convergence in probability of the associated quadratic forms. Consequently, the empirical spectral cluster near zero contains exactly the $k$-th Betti number $b_k$ of
eigenvalues, counted with multiplicity, and converges in the transported discrete $L^2$-sense to the space of harmonic $k$-forms. We also construct consistent empirical
estimators of the tangent projection, the second fundamental form, the Riemannian
curvature tensor, and the Weitzenböck
curvature endomorphisms. As applications, we obtain consistent recovery of the Betti numbers and harmonic representatives of de Rham
cohomology, as well as of the Pontryagin forms and their periods. Evaluating these periods on a homology basis recovers the corresponding real Pontryagin classes from sampled data.
Read the full abstract ▾
Statistical Methodology
Bayesian Nonparametrics: Principles and Practice
Nils Lid Hjort, Chris Holmes, Peter Mueller et al. (+1 more)
This extended preface [to the Book `
Bayesian Nonparametrics', Cambridge University Press, 2010, by NL Hjort, CC Holmes, P Mueller, SG Walker] is meant to explain why you are right to be curious about
Bayesian nonparametrics – why you may actually need it and how you can manage to understand it and use it. The preface also serves as an introductory chapter, giving an overview of the aims and contents of the book. We also explain the background for how the book came into existence, delve briefly on the history of the still relatively young field of
Bayesian nonparametrics, and offer some concluding remarks, pertaining to various challenges and likely future developments of the area.
Read the full abstract ▾
All authors Nils Lid Hjort, Chris Holmes, Peter Mueller, Stephen G. Walker
Statistics Applications · Cross-listed from cs.LG
Explainable AI for Data-Driven Design of High-Dimensional Predictive Studies
Junyu Yan, Damian Machlanski, Kurt Butler et al. (+4 more)
Predictive modelling is important for health data analysis and data-driven clinical decision-making. However, predictive studies are challenging to design optimally by hand when tens or even hundreds of features require selection, transformation, or interaction modelling. While complex machine learning models offer high performance, their "black-box" nature limits the clinical trust, transparency, and interpretability required for decision-making. We developed and evaluated an Exploratory AI Recommender that provides data-driven recommendations to improve predictive performance of existing interpretable statistical models. The developed framework uses flexible AI modelling to capture complex data patterns and explainable AI techniques to translate the patterns into three recommendation types: feature exclusion, non-linear terms, and feature interactions. We evaluated the framework by comparing predictive performance of a baseline (i.e., no interactions or non-linear terms) Cox Proportional Hazards (CPH) model against an augmented CPH incorporating recommendations suggested by our method. The primary analysis predicts the time to the first occurrence of a fall or related injury in 245,614 patients. Our method recommended excluding 23 features, including non-linear terms for two features, and including 221 suggested feature interactions. The C-index improved from 0.805 (95% CI 0.798-0.812) to 0.815 (95% CI 0.809-0.822), and so did calibration (intercept: -0.006 to 0.003; slope: 1.063 to 0.950). All recommendations were supported by existing literature. The method also proved effective on two additional public
datasets, demonstrating wider applicability. The proposed Exploratory AI Recommender demonstrates the potential of explainable AI and data-driven study design to improve the process of developing, and the performance of high-dimensional transparent predictive models.
Read the full abstract ▾
All authors Junyu Yan, Damian Machlanski, Kurt Butler, Panagiotis Dimitrakopoulos, Ewen M Harrison, Bruce Guthrie, Sotirios A Tsaftaris
Machine Learning (Statistics) · Cross-listed from cs.LG
From Sequential Nodes to GPU Batches: Parallel Branch and Bound for Optimal $k$-Sparse GLMs
Jiachang Liu and Andrea Lodi
GPUs have significantly accelerated first-order methods for large-scale optimization, especially in continuous optimization. However, this success has not transferred cleanly to problems with discrete variables, combinatorial structure, and nonlinear objectives, such as certifying optimal solutions for cardinality-constrained generalized linear models. Major challenges include the sequential processing of heterogeneous nodes in branch and bound (BnB) and frequent data movement between the CPU and GPU. We propose a simple, generic, and modular CPU–GPU framework that processes multiple BnB nodes in batches on GPUs. The framework is built around a small set of GPU-efficient routines and uses padding together with lightweight custom kernels to handle irregular node data structures. Experiments show one to two orders of magnitude speedups and zero optimality gap on challenging instances. The framework can also be extended to collect the entire Rashomon set, enabling downstream statistical analysis such as variable-importance analysis and model selection under secondary user-specific measures (e.g., AUC in classification).
Read the full abstract ▾
Other Statistics
A critical comparison of handling zeros in high-dimensional compositional count data
Wenqi Tang, Kamila Fačevicová, Klaus Nordhausen et al. (+1 more)
The growing use of high-throughput sequencing (HTS) has enabled the large-scale production of compositional count data, driving progress in microbiome research. However, such count data are often high-dimensional, over-dispersed, and heavily zero-inflated, and they conflict with the continuity assumptions underlying log-ratio-based compositional data analysis (CoDA), creating substantial methodological challenges. This review provides an overview of zero-handling strategies in compositional data, covering zero-tolerant transformations, imputation approaches for rounded zeros, and statistical models for essential zeros. We specifically highlight the problems that arise when applying the log-ratio framework to sequencing-derived compositional count data, where violations of continuity can induce numerical instabilities and biased statistical inferences. Motivated by these issues, we systematically examine how existing imputation strategies behave when adapted to discrete, zero-inflated count data, including an evaluation of how the discrete,
lattice-valued nature of the data affects imputation performance. Overall, this review consolidates scattered methodological developments, clarifies appropriate use cases, and identifies open challenges that motivate future zero-handling frameworks capable of jointly accommodating compositional constraints, zero inflation, and the
lattice nature of count data, while also providing a detailed discussion of the comparison results.
Read the full abstract ▾
All authors Wenqi Tang, Kamila Fačevicová, Klaus Nordhausen, Sara Taskinen
Machine Learning (Statistics)
From Betting to Empirical Bernstein LIL
Francesco Orabona
This is a verbatim copy of a technical report I wrote in 2017-2018 to obtain the law of the iterated logarithm using the guarantee on the wealth of an online betting strategy.
Machine Learning (Statistics) · Cross-listed from cs.LG
Aerodynamic force reconstruction using physics-informed Gaussian processes
Gledson Rodrigo Tondo, Igor Kavrakov and Guido Morgenthal
Accurate modeling of aerodynamic loads is essential for understanding and predicting the responses of complex structural systems. However, these models often rely on simplifications of the true physical forces, introducing assumptions that can limit their accuracy. Validating such models becomes particularly challenging in the presence of noisy or incomplete data. To address this, we introduce a probabilistic physics-informed machine learning approach designed to reconstruct the underlying aerodynamic loads from noisy measurements of structural dynamic responses. The model avoids overfitting, eliminates the need for regularization schemes, and allows for the use of heterogeneous and multi-fidelity data during the training process. The efficacy of the approach is demonstrated through the reconstruction of aerodynamic loads on the Great Belt East Bridge, simulated under a linear unsteady assumption. Results show a strong agreement between true and predicted loads, particularly related to root mean squared errors, magnitude, phase angle and peak values of the signals. The method for load reconstructing holds broad applicability, such as modeling validation, future load estimation, and structural damage prognosis.
Read the full abstract ▾
Statistical Methodology
Two-stage Ensemble Clustering of Functional Data Using Random Projections
Sourav Chakrabarty, Anirvan Chakraborty and Shyamal K. De
We propose a computationally simple framework for clustering functional data based on Gaussian-process-generated random projections. In this approach, each curve is first projected onto a large collection of independent Gaussian process realizations. The resulting high-dimensional representations are clustered using the Mean Absolute Difference of Distances (MADD), a dissimilarity measure well suited for high-dimensional settings. A population-level analysis of this dissimilarity provides insight into how random projections help capture distributional differences between functional populations. We introduce a second stage of clustering to additionally leverage on data-driven projection directions. Thus, in Stage I, an initial clustering is obtained using a set of prespecified projection families. In Stage II, this partition is refined by constructing Gaussian random projections based on an estimated covariance operator that uses the first stage of cluster labels. Finally, a normalized cost function is used to select the optimal clustering among candidate solutions. The proposed clustering
algorithm is broadly applicable to diverse functional data regimes including irregular and partially observed data. Through extensive simulations and real-data applications, we show that the proposed method achieves a high degree of accuracy and outperforms many of the state-of-the-art methods across a wide range of functional data settings.
Read the full abstract ▾
Statistics Theory
A Circular Chatterjee's Correlation Coefficient
Sourav Majumdar
Chatterjee's rank correlation is a directed measure of association designed to detect whether one variable can be predicted as a function of another. While the original coefficient is naturally defined for real-valued data, circular data poses additional difficulty. Applying the usual construction requires cutting each circle at an arbitrary point and treating it as a line. Different choices of cut points can lead to different finite-sample values, even though the underlying circular relationship is unchanged. This paper proposes a circular version of Chatterjee's coefficient that removes this arbitrary choice. The population construction averages over response cuts in circular rank space, and the finite-sample construction averages over sample cut gaps and reduces to a simple statistic based only on cyclic ranks. The resulting coefficient is intrinsic to the circular ordering of the data, remains directed, and retains the key interpretation of Chatterjee's original coefficient. Under non-atomic circular marginals, it is zero exactly under independence and one exactly when the circular response is a measurable function of the circular predictor. We prove consistency and derive its distribution-free null behavior under independence. Simulations show that the proposed coefficient is especially useful for detecting multi-winding circular relationships, such as cases where the response goes around the circle twice or four times as the predictor goes around once, where standard circular correlations can be nearly blind.
Read the full abstract ▾
Statistical Methodology
A Mixed Self-Exciting Process to Model Epileptic Seizures
Karen Kanaster, Giovani L. Silva, Peter Mueller et al. (+2 more)
Epilepsy is a neurological disorder characterized by recurrent seizures affecting more than 70 million people worldwide. Often, an individual with epilepsy is more likely to experience subsequent seizures following an initial seizure, a process we call seizure clustering. Motivated by seizure diary data collected over three years from 407 individuals newly diagnosed with focal epilepsy in the Human Epilepsy Project (HEP), we propose a
Bayesian mixed Hawkes process model that addresses seizure clustering and heterogeneity between individuals. In the Hawkes process, the intensity is accelerated each time an event occurs, through the composition of background and excitation intensity functions. The proposed model incorporates a Weibull baseline intensity to model a trend in background seizure rates over time, while the excitation process accounts for seizure clustering within individuals. We model heterogeneity among individuals by including
covariates and random effects in both the background and excitation intensities. In the HEP study, the average time between primary and secondary seizures within an individual is 1.57 (95% CrI: 1.43, 1.70) days, with an average of 2.20 (1.96, 2.47) seizures per cluster. We demonstrate that omitting random effects in the presence of heterogeneity leads to underestimation of the background intensity and overestimation of excitation rates.
Read the full abstract ▾
All authors Karen Kanaster, Giovani L. Silva, Peter Mueller, Jacob Pellinen, Elizabeth Juarez-Colunga
Statistical Computation
Eigen for Statistical and Machine Learning Computing: A Lightweight C++ Tutorial with Python Bindings
Seyoung Lee and Kwan-Young Bak
This note provides a lightweight tutorial on using Eigen, a C++ template library for linear
algebra, to implement statistical and machine learning
algorithms. The emphasis is practical rather than methodological: we show how common
matrix operations, decomposition-based solvers, and vectorized updates can be written in readable C++ and then connected to Python through pybind11. Two examples are used throughout the tutorial: kernel ridge regression and
matrix factorization with
stochastic gradient descent. The examples are intentionally small enough to be studied as code, but they contain many operations that appear in larger research software projects, including kernel
matrix construction, regularized linear system solving, row-wise updates, and NumPy–Eigen data conversion. The goal is to provide a reproducible starting point for researchers who want to move from mathematical formulas to efficient C++ implementations while retaining a convenient Python workflow.
Read the full abstract ▾
Statistical Methodology
Testing for Serial Independence via Auto Hilbert-Schmidt Independence Criterion
Muyi Li, Yuqing Xu and Zhou Zhou
We develop a
Hilbert–Schmidt independence criterion (HSIC)-based framework for testing serial independence in strictly stationary time series. The proposed auto
Hilbert–Schmidt independence criterion (AutoHSIC) measures dependence between an observation and its lagged counterpart, providing a kernel-based approach to detecting nonlinear serial dependence. The empirical AutoHSIC statistic is a lagged U-statistic constructed from overlapping observations, and hence inherits temporal dependence even under the i.i.d. null. Its asymptotic analysis therefore differs from standard i.i.d. HSIC theory and must account for degeneracy under the null. We establish the limiting behaviour of the resulting single-lag and portmanteau tests under the null and under fixed alternatives. Since the limiting null distribution is non-pivotal, we develop a wild bootstrap procedure for critical value approximation and prove its asymptotic validity. The framework is further extended to residual-based model diagnostics, where parameter estimation affects the null distribution. Simulations and empirical applications illustrate its ability to detect nonlinear serial dependence in multivariate, functional and
matrix time series.
Read the full abstract ▾
Machine Learning (Statistics)
Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
Margalit Glasgow and Joan Bruna
We consider one-hidden layer neural networks trained in the feature-learning regime using gradient descent, and relate the output of the finite-width network $f_{\hat{\rho}_t^m}$ to its infinite-width counterpart $f_{\rho_t^{MF}}$, which evolves in the mean-field dynamics. While constant-time horizon bounds for $\|f_{\rho_t^{MF}} - f_{\hat{\rho}_t^m}\|$ may be obtained via standard Grönwall estimates, the long-time behavior of the fluctuation is a more delicate matter. Uniform-in-time bounds often rely on (local) strong convexity in the landscape or Logarithmic Sobolev inequalities present in noisy gradient dynamics. In this work, we establish non-asymptotic weak propagation-of-chaos that holds uniformly in time, obtained by exploiting instead the convergence rate of the mean-field deterministic Wasserstein-gradient-flow dynamics. Specifically, denoting by $L_t$ the mean-field excess MSE loss at time $t$ and $m$ the number of
neurons, under standard regularity assumptions and the condition $\int_0^\infty L_t^{1/2} dt =O(\log d)$, we obtain the uniform in time bound $\|f_{\rho_t^{MF}}- f_{\hat{\rho}_t^m}\|^2 \lesssim \text{poly}(d) m^{-\min(1,c/6)}$ whenever $L_t \lesssim t^{-c}$. Our result holds in a noiseless setting and does not make any assumptions on the geometry of the landscape near the optimum, and extends seamlessly to other forms of discretization, including finite number of samples and time discretization. A key takeaway of our result is that whenever the convergence rate of the mean-field, population-loss dynamics is faster than $t^{-2}$, we can attain a loss of $\epsilon$ with only $\text{poly}(d/\epsilon)$
neurons, training samples, and GD steps.
Read the full abstract ▾
Statistical Methodology
Selecting Informative Conformal Prediction Sets with an Optimized FCR-Controlled Approach
Israela Solomon, Etienne Roquain, Saharon Rosset et al. (+1 more)
Conformal methods provide prediction sets for outcomes with confidence guarantees. We study their use in a selective inference setting, where inference is performed only when the prediction set is informative. The analyst may consider as informative, for example, cases with prediction sets that are sufficiently small, exclude null values, or satisfy other appropriate monotone constraints. Because inference is typically restricted to informative cases in practical applications, accounting for the resulting selection bias is crucial to maintaining false coverage rate (FCR) control. A general framework for constructing such informative conformal prediction sets while controlling the FCR on the selected sample was suggested in Gazin et al. (2025). In this work we focus on oracle-guided procedures. We derive the optimal decision policy under a suitable power objective in the oracle setting where the probability of belonging to each prediction set can be computed. In practice, of course, only estimated probabilities are available. We therefore introduce calibration procedures that adjust the oracle policy to maintain finite-sample FCR control. We show that this approach can achieve substantially higher power than available alternatives. We demonstrate the effectiveness of our new methods for classification outcomes on both real and simulated data.
Read the full abstract ▾
All authors Israela Solomon, Etienne Roquain, Saharon Rosset, Ruth Heller