Machine Learning (Statistics) · Cross-listed from cs.LG
The ecosystem of machine learning competitions: Platforms, participants, and their impact on AI development
Ioannis Nasios
Machine learning competitions (MLCs) play a pivotal role in advancing artificial intelligence (AI) by fostering innovation, skill development, and practical problem-solving. This study provides a comprehensive analysis of major competition platforms such as Kaggle and Zindi, examining their workflows, evaluation methodologies, and reward structures. It further assesses competition quality, participant expertise, and global reach, with particular
attention to demographic trends among top-performing competitors. By exploring the motivations of competition hosts, this paper underscores the significant role of MLCs in shaping AI development, promoting collaboration, and driving impactful technological progress. Furthermore, by combining literature synthesis with platform-level data analysis and practitioner insights a comprehensive understanding of the MLC ecosystem is provided. Moreover, the paper demonstrates that MLCs function at the intersection of academic research and industrial application, fostering the exchange of knowledge, data, and practical methodologies across domains. Their strong ties to open-source communities further promote collaboration, reproducibility, and continuous innovation within the broader ML ecosystem. By shaping research priorities, informing industry standards, and enabling large-scale crowdsourced problem-solving, these competitions play a key role in the ongoing
evolution of AI. The study provides insights relevant to researchers, practitioners, and competition organizers, and includes an examination of the future trajectory and sustained influence of MLCs on AI development.
Read the full abstract ▾
Statistics Theory
Consistency of the Bayesian Information Criterion for Model Selection in Exploratory Factor Analysis
Hien Duy Nguyen and Kei Hirose
We study model selection by the
Bayesian information criterion (BIC) in fixed-dimensional exploratory factor analysis over a fixed finite family of compact covariance classes. Our main result shows that the BIC is strongly consistent for the pseudo-true factor order under misspecification, provided that all globally optimal models share a common pseudo-true covariance set, the population Gaussian criterion has a local quadratic margin away from that set, and the BIC complexity counts are order-separating at the pseudo-true order. The candidate models may have an unknown mean vector, exact-zero restrictions in the loading
matrix, and either diagonal or spherical error covariance structures, and the selection target is the smallest candidate factor order that yields the best Gaussian approximation, in Kullback–Leibler divergence, to the data-generating covariance structure. The proof works directly in covariance space, so it does not require a regular loading parametrization and accommodates the familiar singularities caused by rotations and redundant factors. Under correct specification, the assumptions reduce to familiar properties of the true covariance
matrix. More generally, the same argument applies to other information criteria whose penalties satisfy the same gap conditions, including several BIC-type modifications.
Read the full abstract ▾
Statistics Applications
Socio-demographic inequalities in the maximum human lifespan: evidence from Belgium and the Netherlands
Jens Robben and Torsten Kleinow
The existence of an upper limit to the human lifespan has been widely debated, with studies offering both supporting and opposing evidence. Existing studies typically treat this limit, if it exists, as a population-wide constant, overlooking potential heterogeneity across individuals. Using unique individual-level death and population records for all individuals aged 90 and older in Belgium and the Netherlands between 1995 and 2022, we use extreme value theory to estimate the upper endpoint of the lifespan distribution, conditional on survival beyond a high threshold age. We find that this endpoint is finite in both countries, but that it is not well described by a single population-wide value: differences in sex, household type, civil status, and origin translate into differences of several years in the estimated upper endpoint. We conclude that socio-demographic inequality persists even at the most advanced ages.
Read the full abstract ▾
Statistical Methodology
Unsupervised Learning Under a General Semiparametric Clusterwise Elliptical Distribution: Efficient Estimation, Optimal Clustering, and Consistent Cluster Selection
Jen-Chieh Teng, Sheng-Hsin Fan, Chin-Tsang Chiang et al. (+2 more)
A general semiparametric clusterwise elliptical distribution is introduced for assessing how latent cluster structure shapes continuous outcomes. Using a subjectwise representation, cluster-specific mean vectors are estimated by minimizing a weighted sum of squares criterion augmented with a separation penalty. An initialization scheme based on iterative Mahalanobis reassignment and a computational
algorithm with guaranteed convergence accompany the estimation procedure. Under the localization conditions, the separation penalty criterion admits a local oracle property, and the corresponding localized partition recovers the realized latent cluster partition. The practical iterative Mahalanobis reassignment does not impose these truth-centered neighborhoods and is evaluated separately through numerical studies. The resulting first-phase partition seeds a second phase of pseudo-likelihood-based estimation and cluster reassignment. A semiparametric information criterion is also proposed for selecting the number of clusters. Monte Carlo simulations identify settings in which the proposed procedures improve on the evaluated parametric
benchmarks, while additional experiments reveal sensitivity to cluster imbalance and departures from ellipticity. Applications illustrate the resulting customer and patient profiles.
Read the full abstract ▾
All authors Jen-Chieh Teng, Sheng-Hsin Fan, Chin-Tsang Chiang, Ming-Yueh Huang, Alvin Lim
Machine Learning (Statistics) · Cross-listed from math.OC
Unified Precision-Guaranteed Stopping Rules for Contextual Learning
Mingrui Ding, Qiuhong Zhao, Siyang Gao et al. (+1 more)
Contextual learning seeks to learn a decision policy that maps an individual's characteristics to an action through data collection. In operations management, such data may come from various sources, and a central question is when data collection can stop while still guaranteeing that the learned policy is sufficiently accurate. We study this question under two precision criteria: a context-wise criterion and an aggregate policy-value criterion. We develop unified stopping rules for contextual learning with unknown sampling variances in both unstructured and structured linear settings. Our approach is based on generalized likelihood ratio (GLR) statistics for pairwise action comparisons. To calibrate the corresponding sequential boundaries, we derive new time-uniform deviation inequalities that directly control the self-normalized GLR evidence and thus avoid the conservativeness caused by decoupling mean and variance uncertainty. Under the Gaussian sampling model, we establish finite-sample precision guarantees for both criteria. Numerical experiments on synthetic instances and two case studies demonstrate that the proposed stopping rules achieve the target precision with substantially fewer samples than
benchmark methods. The proposed framework provides a practical way to determine when enough information has been collected in personalized decision problems. It applies across multiple data-collection environments, including historical
datasets, simulation models, and real systems, enabling practitioners to reduce unnecessary sampling while maintaining a desired level of decision quality.
Read the full abstract ▾
All authors Mingrui Ding, Qiuhong Zhao, Siyang Gao, Jing Dong
Machine Learning (Statistics)
Intensity Dot Product Graphs
Giulio Valentino Dalla Riva and Matteo Dalla Riva
Latent-position random graph models usually treat the node set as fixed once the sample size is chosen, while graphon-based and random-measure constructions allow more randomness at the cost of weaker
geometric interpretability. We introduce Intensity Dot Product Graphs (IDPGs), which extend Random Dot Product Graphs by replacing a fixed collection of latent positions with a Poisson point process on a Euclidean latent space. This yields a model with random node populations, RDPG-style dot-product affinities, and a population-level intensity that links continuous latent structure to finite observed graphs. We define the heat map and the desire operator as continuous analogues of the probability
matrix, prove a spectral consistency result connecting adjacency singular values to the operator spectrum, compare the construction with graphon and digraphon representations, and show how classical RDPGs arise in a concentrated limit. Because the model is parameterized by an evolving intensity, temporal extensions through partial differential equations arise naturally.
Read the full abstract ▾
Machine Learning (Statistics)
Order-Optimal Sequential 1-Bit Mean Estimation in General Tail Regimes
Ivan Lau and Jonathan Scarlett
In this paper, we study the problem of mean estimation under 1-bit communication constraints. We propose a novel adaptive mean
estimator based solely on randomized threshold queries, where each 1-bit outcome indicates whether a given sample exceeds a sequentially chosen threshold. Our
estimator is $(\epsilon, \delta)$-PAC for any distribution with a bounded mean $\mu \in [-\lambda, \lambda]$ and a bounded $k$-th central moment $\mathbb{E}[|X-\mu|^k] \le \sigma^k$ for any fixed $k > 1$. Moreover, our sample complexity is order-optimal in all such tail regimes, i.e., for every such $k$ value. For $k \neq 2$, our
estimator's sample complexity matches the unquantized minimax lower bounds plus an unavoidable $O(\log(\lambda/\sigma))$ localization cost. For the finite-variance case ($k=2$), our
estimator's sample complexity has an extra multiplicative $O(\log(\sigma/\epsilon))$ penalty, and we establish a novel information-theoretic lower bound showing that this penalty is a fundamental limit of 1-bit quantization. We also establish a significant adaptivity gap: for both threshold queries and more general interval queries, the sample complexity of any non-adaptive
estimator must scale linearly with the search space parameter $\lambda/\sigma$, rendering it vastly less sample efficient than our adaptive approach. Finally, we present algorithmic variants that (i) handle an unknown sampling budget, (ii) adapt to an unknown scale parameter $\sigma$ given (possibly loose) bounds, (iii) require only two stages of adaptivity to achieve order-optimal sample complexity at the expense of more general 1-bit queries, and (iv) leverage multiple local samples per 1-bit query to proportionally reduce communication costs.
Read the full abstract ▾
Statistical Methodology
Structured Semiparametric Estimation of Average Treatment Effects with Treatment-Specific Non-Gaussian Error Distributions
Mijeong Kim
This paper studies average treatment effect (ATE) estimation for continuous outcomes when the treatment-
covariate mean is structured but the error distributions are unknown and may differ across treatment arms in scale, skewness, or tail behavior. We introduce a semiparametric model with a finite-dimensional mean, or a prespecified basis expansion, and separate unrestricted additive error laws for treated and control outcomes. The central theoretical contribution is the ATE-efficient influence function under two treatment-specific error nuisance spaces: it combines arm-specific regression-efficient scores with variation in the marginal
covariate law and is distinct from both the unrestricted AIPW gradient and a pooled location-shift score. We establish the exact nested ordering of the efficiency bounds under pooled common-error, treatment-specific-error, and unrestricted causal models, including equality conditions. A local-misspecification result further shows that the variance reduction obtained by valid pooling equals the maximal squared first-order bias incurred along unit treatment-specific directions excluded by the pooled model. We develop efficient cross-fitted plug-in and ATE-targeted implementations requiring only two one-dimensional density estimates; the targeted refinement adds a scalar update. Simulations show substantial precision gains from valid pooling under common non-Gaussian errors and protection against invalid pooling when treatment changes error shape, including under weak positivity. Applications to ACTG175 and National Supported Work data illustrate the resulting precision–robustness tradeoff for irregular continuous outcomes.
Read the full abstract ▾
Statistical Methodology
Bayesian Tensor-on-Tensor Varying Coefficient Model for Forecasting Alzheimer's Disease Progression
Yajie Liu, Hengrui Luo and Suprateek Kundu
We propose a novel tensor-on-tensor modeling framework that flexibly models nonlinear voxel-level relationships using Gaussian process (GP) priors, while incorporating the spatial structure of the output tensor through low-rank tensor-based coefficients. Spatial heterogeneity is captured through patch-to-voxel mappings, enabling each output voxel to depend on its spatial neighborhood. The proposed interpretable and flexible
Bayesian tensor-on-tensor framework is able to capture nonlinearity, spatial information, and spatial heterogeneity. We develop an efficient Markov chain Monte Carlo (MCMC)
algorithm that exploits parallel structure to sample voxel-specific GP atoms and update low-rank tensor coefficients. Extensive simulations reveal advantages of the proposed approach over existing methods in terms of coefficient estimation, inference, prediction, and scalability to high-dimensional images. Applied to longitudinal image prediction with T1-weighted MRIs from the Alzheimer's Disease Neuroimaging Initiative (ADNI), the proposed method can accurately forecast future cortical thickness. The predicted images also enable reliable prediction of
brain aging, underscoring their biological relevance. Overall, the ADNI analysis highlights the model's ability to forecast future neurobiological changes that has important implications for early detection of AD.
Read the full abstract ▾
Statistical Methodology
Fixed-Effects Models for Causal Inference in Longitudinal Cluster Randomized and Quasi-Experimental Trials
Kenneth M. Lee and Fan Li
This article investigates the model-robustness of fixed-effects models for analyzing a broad class of longitudinal cluster trials (CTs) such as stepped-wedge, parallel-with-baseline and crossover designs, encompassing both randomized (CRTs) and quasi-experimental (CQTs) designs. We clarify a longstanding misconception in biostatistics, demonstrating that fixed-effects models, traditionally perceived as targeting only finite-sample conditional estimands, can effectively target super-population marginal estimands through an M-estimation framework. We comprehensively prove that linear and log-link fixed-effects models with correctly specified treatment effect structures can broadly yield consistent and asymptotically normal
estimators for nonparametrically defined treatment effect estimands in longitudinal CRTs, even under arbitrary misspecification of other model components. We identify that the constant treatment effect
estimator generally targets the period-average treatment effect for the overlap population (P-ATO); accordingly, some CRT designs don't even require correct specification of the treatment effect structure for model-robustness. We further characterize conditions where fixed-effects models can maintain consistency by adjusting for both cluster-level and individual-level time-invariant confounding in longitudinal CQTs. Altogether, supported by simulation and a case study re-analysis, we establish fixed-effects models as a robust and potentially preferable alternative to mixed-effects models for longitudinal CT analysis.
Read the full abstract ▾
Machine Learning (Statistics)
Sparse $\epsilon$ insensitive zone bounded asymmetric elastic net support vector machines for pattern classification
Haiyan Du and Hu Yang
Existing support vector machines(SVM) models are sensitive to noise and lack sparsity, which limits their performance. To address these issues, we combine the elastic net loss with a robust loss framework to construct a sparse $\varepsilon$-insensitive bounded asymmetric elastic net loss, and integrate it with SVM to build $\varepsilon$ Insensitive Zone Bounded Asymmetric Elastic Net Loss-based SVM($\varepsilon$-BAEN-SVM). $\varepsilon$-BAEN-SVM is both sparse and robust. Sparsity is proven by showing that samples inside the $\varepsilon$-insensitive band are not support vectors. Robustness is theoretically guaranteed because the influence function is bounded. To solve the non-convex optimization problem, we design a half-quadratic
algorithm based on clipping dual coordinate descent. It transforms the problem into a series of weighted subproblems, improving computational efficiency via the $\varepsilon$ parameter. Experiments on simulated and real
datasets show that $\varepsilon$-BAEN-SVM outperforms traditional and existing robust SVMs. It balances sparsity and robustness well in noisy environments. Statistical tests confirm its superiority. Under the Gaussian kernel, it achieves better accuracy and noise insensitivity, validating its effectiveness and practical value.
Read the full abstract ▾
Machine Learning (Statistics)
The Condition-Number Principle for Prototype Clustering
Romano Li and Jianfei Cao
We develop a
geometric framework that links objective accuracy to structural recovery in prototype-based clustering. The analysis is
algorithm-agnostic and applies to a broad class of admissible loss functions. We define a clustering condition number that compares within-cluster scale to the minimum loss increase required to move a point across a cluster boundary. When this quantity is small, any solution with a small suboptimality gap must also have a small misclassification error relative to a
benchmark partition. The framework also clarifies a fundamental trade-off between robustness and sensitivity to cluster imbalance, leading to sharp phase transitions for exact recovery under different objectives. The guarantees are deterministic and non-asymptotic, and they separate the role of algorithmic accuracy from the intrinsic
geometric difficulty of the instance. We further show that errors concentrate near cluster boundaries and that sufficiently deep cluster cores are recovered exactly under strengthened local margins. Together, these results provide a
geometric principle for interpreting low objective values as reliable evidence of meaningful clustering structure.
Read the full abstract ▾
Statistics Theory · Cross-listed from econ.EM
Identification in (Endogenously) Nonlinear SVARs Is Easier Than You Think
James A. Duffy and Sophocles Mavroeidis
We study identification in structural vector autoregressions (SVARs) in which the endogenous variables enter nonlinearly on the left-hand side of the model, a feature we term endogenous nonlinearity, to distinguish it from the more familiar case in which nonlinearity arises only through exogenous or predetermined variables. This class of models accommodates asymmetric impact multipliers, endogenous regime switching, and occasionally binding constraints. We show that, under weak regularity conditions, the model parameters and structural shocks are (nonparametrically) identified up to an orthogonal transformation, exactly as in a linear SVAR. Our results have the powerful implication that most existing identification schemes for linear SVARs extend directly to our nonlinear setting, with the number of restrictions required to achieve exact identification remaining unchanged. We specialise our results to piecewise affine SVARs, which provide a convenient framework for the modelling of endogenous regime switching, and their smooth transition counterparts. We illustrate our methodology with an application to the nonlinear Phillips curve, providing a test for the presence of nonlinearity that is robust to the choice of identifying assumptions, and finding significant evidence for state-dependent inflation dynamics.
Read the full abstract ▾
Statistical Computation
Vine Copulas for Analyzing Multivariate Conditional Dependencies in Electronic Health Records Data
Manar D. Samad, Yina Hou, Megan A. Witherow et al. (+1 more)
Electronic health records (EHR) store hundreds of demographic and laboratory variables from large patient populations. Traditional statistical methods have limited capacity in processing mixed-type data (continuous, ordinal) and capturing non-linear relationships in large multivariate data when oversimplified assumptions are made about the distribution (e.g., Gaussian) of disparate variables in EHR data. This paper addresses the limitations mentioned above by repurposing the vine copula method, which is primarily used to synthesize a multivariate distribution from many bivariate cumulative distribution functions (copulas). Vine copulas produce tree structures that represent bivariate conditional dependencies at varying hierarchical levels, decomposing a multivariate distribution. The tree structure is used to rank variables by conditional dependence and to identify a subset of central variables with local dependence, thus simplifying probabilistic mining of high-dimensional EHR data. The proposed application of vine copulas is used to identify conditional dependence between co-morbid conditions and is validated for characterizing different cohorts of EHR patients. The contribution of this paper is a novel approach to probabilistic mining and exploration of healthcare data that provides data-driven explanations, visualization, and variable selection to prognosticate a healthcare outcome. The source code is shared publicly.
Read the full abstract ▾
All authors Manar D. Samad, Yina Hou, Megan A. Witherow, Norou Diawara
Machine Learning (Statistics)
On the Unique Recovery of Transport Maps and Vector Fields from Finite Measure-Valued Data
Jonah Botvinick-Greenhouse and Yunan Yang
We establish guarantees for the unique recovery of vector fields and transport maps from finite measure-valued data, yielding new insights into generative models, data-driven dynamical systems, and PDE inverse problems. In particular, we provide general conditions under which a diffeomorphism can be uniquely identified from its pushforward action on finitely many densities, i.e., when the data $\{(\rho_j,f_\#\rho_j)\}_{j=1}^m$ uniquely determines $f$. As a corollary, we introduce a new metric which compares diffeomorphisms by measuring the discrepancy between finitely many pushforward densities in the space of probability measures. We also prove analogous results in an infinitesimal setting, where derivatives of the densities along a smooth vector field are observed, i.e., when $\{(\rho_j,\text{div} (\rho_j v))\}_{j=1}^m$ uniquely determines $v$. Our analysis makes use of the Whitney and Takens
embedding theorems, which provide estimates on the required number of densities $m$, depending only on the intrinsic dimension of the problem. We additionally interpret our results through the lens of Perron–Frobenius and Koopman operators and demonstrate how our techniques lead to new guarantees for the well-posedness of certain PDE inverse problems related to continuity, advection, Fokker–Planck, and advection-
diffusion-reaction equations. Finally, we present illustrative numerical experiments demonstrating the unique identification of transport maps from finitely many pushforward densities, and of vector fields from finitely many weighted divergence observations.
Read the full abstract ▾