Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Thursday 30 October

Machine Learning (Statistics)

Overspecified Mixture Discriminant Analysis: Exponential Convergence, Statistical Guarantees, and Remote Sensing Applications

This study explores the classification error of Mixture Discriminant Analysis (MDA) in scenarios where the number of mixture components exceeds those present in the actual data distribution, a condition known as overspecification. We use a two-component Gaussian mixture model within each class to fit data generated from a single Gaussian, analyzing both the algorithmic convergence of the Expectation-Maximization (EM) algorithm and the statistical classification error. We demonstrate that, with suitable initialization, the EM algorithm converges exponentially fast to the Bayes risk at the population level. Further, we extend our results to finite samples, showing that the classification error converges to Bayes risk with a rate $n^{-1/2}$ under mild conditions on the initial parameter estimates and sample size. This work provides a rigorous theoretical framework for understanding the performance of overspecified MDA, which is often used empirically in complex data settings, such as image and text classification. To validate our theory, we conduct experiments on remote sensing datasets.
Read the full abstract

All authors Arman Bolatov, Alan Legg, Igor Melnykov, Amantay Nurlanuly, Maxat Tezekbayev, Zhenisbek Assylbekov

Statistics Theory

Advanced Distribution Theory for Significance in Scale Space

Smoothing methods find signals in noisy data. A challenge for Statistical inference is the choice of smoothing parameter. SiZer addressed this challenge in one-dimension by detecting significant slopes across multiple scales, but was not a completely valid testing procedure. This was addressed by the development of an advanced distribution theory that ensures fully valid inference in the 1-D setting by applying extreme value theory. A two-dimensional extension of SiZer, known as Significance in Scale Space (SSS), was developed for image data, enabling the detection of both slopes and curvatures across multiple spatial scales. However, fully valid inference for 2-D SSS has remained unavailable, largely due to the more complex dependence structure of random fields. In this paper, we use a completely different probability methodology which gives an advanced distribution theory for SSS, establishing a valid hypothesis testing procedure for both slope and curvature detection. When applied to pure noise images (no true underlying signal), the proposed method controls the Type I error, whereas the original SSS identifies spurious features across scales. When signal is present, the proposed method maintains a high level of statistical power, successfully identifying important true slopes and curvatures in real data such as gamma camera images.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Quantitative Bounds for Length Generalization in Transformers

We study the problem of length generalization (LG) in transformers: the ability of a model trained on shorter sequences to maintain performance when evaluated on much longer, previously unseen inputs. Prior work by Huang et al. (2025) established that transformers eventually achieve length generalization once the training sequence length exceeds some finite threshold, but left open the question of how large it must be. In this work, we provide the first quantitative bounds on the required training length for length generalization to occur. Motivated by previous empirical and theoretical work, we analyze LG in several distinct problem settings: $\ell_\infty$ error control vs. average error control over an input distribution, infinite-precision softmax attention vs. finite-precision attention (which reduces to an argmax) in the transformer, and one- vs. two-layer transformers. In all scenarios, we prove that LG occurs when the internal behavior of the transformer on longer sequences can be "simulated" by its behavior on shorter sequences seen during training. Our bounds give qualitative estimates for the length of training data required for a transformer to generalize, and we verify these insights empirically. These results sharpen our theoretical understanding of the mechanisms underlying extrapolation in transformers, and formalize the intuition that richer training data is required for generalization on more complex tasks.
Read the full abstract

Statistical Methodology

Refined thresholds for inconsistency: The effect of the graph associated with incomplete pairwise comparisons

The inconsistency of pairwise comparisons remains difficult to interpret in the absence of acceptability thresholds. The popular 10% cut-off rule proposed by Saaty has recently been applied to incomplete pairwise comparison matrices, which contain some unknown comparisons. This paper refines these inconsistency thresholds: we uncover that they depend not only on the size of the matrix and the number of missing entries, but also on the undirected graph whose edges represent the known pairwise comparisons. Therefore, using our exact thresholds is especially important if the filling in patterns coincide for a large number of matrices, as has been recommended in the literature. The strong association between the new threshold values and the spectral radius of the representing graph is also demonstrated. Our results can be integrated into software to continuously monitor inconsistency during the collection of pairwise comparisons and immediately detect potential errors.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.AI

Causal Masking on Spatial Data: An Information-Theoretic Case for Learning Spatial Datasets with Unimodal Language Models

Language models are traditionally designed around causal masking. In domains with spatial or relational structure, causal masking is often viewed as inappropriate, and sequential linearizations are instead used. Yet the question of whether it is viable to accept the information loss introduced by causal masking on nonsequential data has received little direct study, in part because few domains offer both spatial and sequential representations of the same dataset. In this work, we investigate this issue in the domain of chess, which naturally supports both representations. We train language models with bidirectional and causal self-attention mechanisms on both spatial (board-based) and sequential (move-based) data. Our results show that models trained on spatial board states - even with causal masking - consistently achieve stronger playing strength than models trained on sequential data. While our experiments are conducted on chess, our results are methodological and may have broader implications: applying causal masking to spatial data is a viable procedure for training unimodal LLMs on spatial data, and in some domains is even preferable to sequentialization.
Read the full abstract

Statistical Methodology

Generalized Maximum Entropy: When and Why you need it

The classical Maximum-Entropy Principle (MEP) based on Shannon entropy is widely used to construct least-biased probability distributions from partial information. However, the Shore-Johnson axioms that single out the Shannon functional hinge on strong system independence, an assumption often violated in real-world, strongly correlated systems. We provide a self-contained guide to when and why practitioners should abandon the Shannon form in favour of the one-parameter Uffink-Jizba-Korbel (UJK) family of generalized entropies. After reviewing the Shore and Johnson axioms from an applied perspective, we recall the most commonly used entropy functionals and locate them within the UJK family. The need for generalized entropies is made clear with two applications, one rooted in economics and the other in ecology. A simple mathematical model worked out in detail shows the power of generalized maximum entropy approaches in dealing with cases where strong system independence does not hold. We conclude with practical guidelines for choosing an entropy measure and reporting results so that analyses remain transparent and reproducible.
Read the full abstract

Statistical Methodology

LLMs are Overconfident: Evaluating Confidence Interval Calibration with FermiEval

Large language models (LLMs) excel at numerical estimation but struggle to correctly quantify uncertainty. We study how well LLMs construct confidence intervals around their own answers and find that they are systematically overconfident. To evaluate this behavior, we introduce FermiEval, a benchmark of Fermi-style estimation questions with a rigorous scoring rule for confidence interval coverage and sharpness. Across several modern models, nominal 99% intervals cover the true answer only 65% of the time on average. With a conformal prediction based approach that adjusts the intervals, we obtain accurate 99% observed coverage, and the Winkler interval score decreases by 54%. We also propose direct log-probability elicitation and quantile adjustment methods, which further reduce overconfidence at high confidence levels. Finally, we develop a perception-tunnel theory explaining why LLMs exhibit overconfidence: when reasoning under uncertainty, they act as if sampling from a truncated region of their inferred distribution, neglecting its tails.
Read the full abstract

All authors Elliot L. Epstein, John Winnicki, Thanawat Sornwanee, Rajat Dwaraknath

Statistical Computation

Robust fuzzy clustering for high-dimensional multivariate time series with outlier detection

Fuzzy clustering provides a natural framework for modeling partial memberships, particularly important in multivariate time series (MTS) where state boundaries are often ambiguous. For example, in EEG monitoring of driver alertness, neural activity evolves along a continuum (from unconscious to fully alert, with many intermediate levels of drowsiness) so crisp labels are unrealistic and partial memberships are essential. However, most existing algorithms are developed for static, low-dimensional data and struggle with temporal dependence, unequal sequence lengths, high dimensionality, and contamination by noise or artifacts. To address these challenges, we introduce RFCPCA, a robust fuzzy subspace-clustering method explicitly tailored to MTS that, to the best of our knowledge, is the first of its kind to simultaneously: (i) learn membership-informed subspaces, (ii) accommodate unequal lengths and moderately high dimensions, (iii) achieve robustness through trimming, exponential reweighting, and a dedicated noise cluster, and (iv) automatically select all required hyperparameters. These components enable RFCPCA to capture latent temporal structure, provide calibrated membership uncertainty, and flag series-level outliers while remaining stable under contamination. On driver drowsiness EEG, RFCPCA improves clustering accuracy over related methods and yields a more reliable characterization of uncertainty and outlier structure in MTS.
Read the full abstract

All authors Ziling Ma, Ángel López-Oriona, Hernando Ombao, Ying Sun

Statistical Methodology

The Interplay between Bayesian Inference and Conformal Prediction

Conformal prediction has emerged as a cutting-edge methodology in statistics and machine learning, providing prediction intervals with finite-sample frequentist coverage guarantees. Yet, its interplay with Bayesian statistics, often criticised for lacking frequentist guarantees, remains underexplored. Recent work has suggested that conformal prediction can serve to "calibrate" Bayesian credible sets, thereby imparting frequentist validity and motivating deeper investigation into frequentist-Bayesian hybrids. We further argue that Bayesian procedures have the potential to enhance conformal prediction, not only in terms of more informative intervals, but also for achieving nearly optimal solutions under a decision-theoretic framework. Thus, the two paradigms can be jointly used for a principled balance between validity and efficiency. This work provides a basis for bridging this gap. After surveying existing ideas, we formalise the Bayesian conformal inference framework, covering challenging aspects such as statistical efficiency and computational complexity.
Read the full abstract

Statistical Methodology

Finite Sample MIMO System Identification with Multisine Excitation: Nonparametric, Direct, and Two-step Parametric Estimators

Multisine excitations are widely used for identifying multi-input multi-output systems due to their periodicity, data compression properties, and control over the input spectrum. Despite their popularity, the finite sample statistical properties of frequency-domain estimators under multisine excitation, for both nonparametric and parametric settings, remain insufficiently understood. This paper develops a finite-sample statistical framework for least-squares estimation of the frequency response function (FRF) and its implications for parametric modeling. First, we derive exact distributional and covariance properties of the FRF estimator, explicitly accounting for aliasing effects under slow sampling regimes, and establish conditions for unbiasedness, uncorrelatedness, and consistency across multiple experiments. Second, we show that the FRF estimate is a sufficient statistic for any parametric model under Gaussian noise, leading to an exact equivalence between optimal two stage frequency-domain methods and time-domain prediction error and maximum likelihood estimation. This equivalence is shown to yield finite-sample concentration bounds for parametric maximum likelihood estimators, enabling rigorous uncertainty quantification, and closed-form prediction error method estimators without iterative optimization. The theoretical results are demonstrated in a representative case study.
Read the full abstract

All authors Rodrigo A. González, Koen Classens, Cristian R. Rojas, Tom Oomen, Håkan Hjalmarsson

Statistical Methodology

A novel generalized additive scalar-on-function regression model for partially observed multidimensional functional data: An application to air quality classification

In this work we propose a generalized additive functional regression model for partially observed functional data. Our approach accommodates functional predictors of varying dimensions without requiring imputation of missing observations. Both the functional coefficients and covariates are represented using basis function expansions, with B-splines used in this study, though the method is not restricted to any specific basis choice. Model coefficients are estimated via penalized likelihood, leveraging the mixed model representation of penalized splines for efficient computation and smoothing parameter estimation.The performance of the proposed approach is assessed through two simulation studies: one involving two one-dimensional functional covariates, and another using a two-dimensional functional covariate. Finally, we demonstrate the practical utility of our method in an application to air-pollution classification in Dimapur, India, where images are treated as observations of a two-dimensional functional variable. This case study highlights the models ability to effectively handle incomplete functional data and to accurately discriminate between pollution levels.
Read the full abstract

Statistical Methodology

Residual Distribution Predictive Systems

Conformal predictive systems are sets of predictive distributions with theoretical out-of-sample calibration guarantees. The calibration guarantees are typically that the set of predictions contains a forecast distribution whose prediction intervals exhibit the correct marginal coverage at all levels. Conformal predictive systems are constructed using conformity measures that quantify how well possible outcomes conform with historical data. However, alternative methods have been proposed to construct predictive systems with more appealing theoretical properties. We study an approach to construct predictive systems that we term Residual Distribution Predictive Systems. In the split conformal setting, this approach nests conformal predictive systems with a popular class of conformity measures, providing an alternative perspective on the classical approach. In the full conformal setting, the two approaches differ, and the new approach has the advantage that it does not rely on a conformity measure satisfying fairly stringent requirements to ensure that the predictive system is well-defined; it can readily be implemented alongside any point-valued regression method to yield predictive systems with out-of-sample calibration guarantees. The empirical performance of this approach is assessed using simulated data, where it is found to perform competitively with conformal predictive systems. However, the new approach offers considerable scope for implementation with alternative regression methods.
Read the full abstract

Statistics Theory

On the class of exponential statistical structures of type B

The article is devoted to the study of exponential statistical structures of type B, which constitute a subclass of exponential families of probability distributions. This class is characterized by a number of analytical and probabilistic properties that make it a convenient tool for solving both theoretical and applied problems in statistics. The relevance of this research lies in the need to generalize known classes of distributions and to develop a unified framework for their analysis, which is essential for applications in stochastic modeling, machine learning, financial mathematics. The paper proposes a formal definition of type B. Necessary and sufficient conditions for a statistical structure to belong to class B are established, and it is proved that such structures can be represented through a dominating measure with an explicit Laplace transform. The obtained results make it possible to describe a wide range of well-known one-dimensional and multivariate distributions, including the Binomial, Poisson, Normal, Gamma, Polynomial, Logarithmic distributions, as well as specific cases such as the Borel-Tanner and Random Walk distributions. Particular attention is given to the proof of structural theorems that determine the stability of class B under linear transformations and the addition of independent random vectors. Recursive relations for initial and central moments as well as for semi-invariants are obtained, providing an efficient analytical and computational framework for their evaluation. Furthermore, the tails of type B distributions are investigated using the properties of the Laplace transform. New exponential inequalities for estimating the probabilities of large deviations are derived. The obtained results can be applied in theoretical studies and in practical problems of stochastic modeling.
Read the full abstract

Machine Learning (Statistics)

A Unified Theory for Causal Inference: Direct Debiased Machine Learning via Bregman-Riesz Regression

This note introduces a unified theory for causal inference that integrates Riesz regression, covariate balancing, density-ratio estimation (DRE), targeted maximum likelihood estimation (TMLE), and the matching estimator in average treatment effect (ATE) estimation. In ATE estimation, the balancing weights and the regression functions of the outcome play important roles, where the balancing weights are referred to as the Riesz representer, bias-correction term, and clever covariates, depending on the context. Riesz regression, covariate balancing, DRE, and the matching estimator are methods for estimating the balancing weights, where Riesz regression is essentially equivalent to DRE in the ATE context, the matching estimator is a special case of DRE, and DRE is in a dual relationship with covariate balancing. TMLE is a method for constructing regression function estimators such that the leading bias term becomes zero. Nearest Neighbor Matching is equivalent to Least Squares Density Ratio Estimation and Riesz Regression.
Read the full abstract

Statistical Methodology

A KL-divergence based test for elliptical distribution

We conduct a KL-divergence based procedure for testing elliptical distributions. The procedure simultaneously takes into account the two defining properties of an elliptically distributed random vector: independence between length and direction, and uniform distribution of the direction. The test statistic is constructed based on the $k$ nearest neighbors ($k$NN) method, and two cases are considered where the mean vector and covariance matrix are known and unknown. First-order asymptotic properties of the test statistic are rigorously established by creatively utilizing sample splitting, truncation and transformation between Euclidean space and unit sphere, while avoiding assuming Fréchet differentiability of any functionals. Debiasing and variance inflation are further proposed to treat the degeneration of the influence function. Numerical implementations suggest better size and power performance than the state of the art procedures.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Deep sequence models tend to memorize geometrically; it is unclear why

Deep sequence models are said to store atomic facts predominantly in the form of associative memory: a brute-force lookup of co-occurring entities. We identify a dramatically different form of storage of atomic facts that we term as geometric memory. Here, the model has synthesized embeddings encoding novel global relationships between all entities, including ones that do not co-occur in training. Such storage is powerful: for instance, we show how it transforms a hard reasoning task involving an $\ell$-fold composition into an easy-to-learn $1$-step navigation task. From this phenomenon, we extract fundamental aspects of neural embedding geometries that are hard to explain. We argue that the rise of such a geometry, as against a lookup of local associations, cannot be straightforwardly attributed to typical supervisory, architectural, or optimizational pressures. Counterintuitively, a geometry is learned even when it is more complex than the brute-force lookup. Then, by analyzing a connection to Node2Vec, we demonstrate how the geometry stems from a spectral bias that – in contrast to prevailing theories – indeed arises naturally despite the lack of various pressures. This analysis also points out to practitioners a visible headroom to make Transformer memory more strongly geometric. We hope the geometric view of parametric memory encourages revisiting the default intuitions that guide researchers in areas like knowledge acquisition, capacity, discovery, and unlearning.
Read the full abstract

All authors Shahriar Noroozizadeh, Vaishnavh Nagarajan, Elan Rosenfeld, Sanjiv Kumar

Statistical Computation

Phantom types for robust hierarchical models with typegeist

In Bayesian hierarchical models, group-level parameter arrays must be mapped to the observation axis, often using explicit indexing. In complex models with numerous incompatible data and parameter sets, this introduces the potential for bugs, as indexing with the incorrect indices typically fails silently. Here we present typegeist, a type system for Python that uses static type analysis to enable specification and enforcement of data-parameter-index correspondences. We show how this can be used with common probabilistic programming frameworks to help guarantee model correctness with minimal run-time overhead.
Read the full abstract

Machine Learning (Statistics)

Bridging the Gap between Empirical Welfare Maximization and Conditional Average Treatment Effect Estimation in Policy Learning

The goal of policy learning is to train a policy function that recommends a treatment given covariates to maximize population welfare. There are two major approaches in policy learning: the empirical welfare maximization (EWM) approach and the plug-in approach. The EWM approach is analogous to a classification problem, where one first builds an estimator of the population welfare, which is a functional of policy functions, and then trains a policy by maximizing the estimated welfare. In contrast, the plug-in approach is based on regression, where one first estimates the conditional average treatment effect (CATE) and then recommends the treatment with the highest estimated outcome. This study bridges the gap between the two approaches by showing that both are based on essentially the same optimization problem. In particular, we prove an exact equivalence between EWM and least squares over a reparameterization of the policy class. As a consequence, the two approaches are interchangeable in several respects and share the same theoretical guarantees under common conditions. Leveraging this equivalence, we propose a regularization method for policy learning. The reduction to least squares yields a smooth surrogate that is typically easier to optimize in practice. At the same time, for many natural policy classes the inherent combinatorial hardness of exact EWM generally remains, so the reduction should be viewed as an optimization aid rather than a universal bypass of NP-hardness.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Budgeted Multiple-Expert Deferral

Learning to defer uncertain predictions to costly experts offers a powerful strategy for improving the accuracy and efficiency of machine learning systems. However, standard training procedures for deferral algorithms typically require querying all experts for every training instance, an approach that becomes prohibitively expensive when expert queries incur significant computational or resource costs. This undermines the core goal of deferral: to limit unnecessary expert usage. To overcome this challenge, we introduce the budgeted deferral framework, which aims to train effective deferral algorithms while minimizing expert query costs during training. We propose new algorithms for both two-stage and single-stage multiple-expert deferral settings that selectively query only a subset of experts per training example. While inspired by active learning, our setting is fundamentally different: labels are already known, and the core challenge is to decide which experts to query in order to balance cost and predictive performance. We establish theoretical guarantees for both of our algorithms, including generalization bounds and label complexity analyses. Empirical results across several domains show that our algorithms substantially reduce training costs without sacrificing prediction accuracy, demonstrating the practical value of our budget-aware deferral algorithms.
Read the full abstract

All authors Giulia DeSalvo, Clara Mohri, Mehryar Mohri, Yutao Zhong

Machine Learning (Statistics)

Assessment of the conditional exchangeability assumption in causal machine learning models: a simulation study

Observational studies developing causal machine learning (ML) models for the prediction of individualized treatment effects (ITEs) seldom conduct empirical evaluations to assess the conditional exchangeability assumption. We aimed to evaluate the performance of these models under conditional exchangeability violations and the utility of negative control outcomes (NCOs) as a diagnostic. We conducted a simulation study to examine confounding bias in ITE estimates generated by causal forest and X-learner models under varying conditions, including the presence or absence of true heterogeneity. We simulated data to reflect real-world scenarios with differing levels of confounding, sample size, and NCO confounding structures. We then estimated and compared subgroup-level treatment effects on the primary outcome and NCOs across settings with and without unmeasured confounding. When conditional exchangeability was violated, causal forest and X-learner models failed to recover true treatment effect heterogeneity and, in some cases, falsely indicated heterogeneity when there was none. NCOs successfully identified subgroups affected by unmeasured confounding. Even when NCOs did not perfectly satisfy its ideal assumptions, it remained informative, flagging potential bias in subgroup level estimates, though not always pinpointing the subgroup with the largest confounding. Violations of conditional exchangeability substantially limit the validity of ITE estimates from causal ML models in routinely collected observational data. NCOs serve a useful empirical diagnostic tool for detecting subgroup-specific unmeasured confounding and should be incorporated into causal ML workflows to support the credibility of individualized inference.
Read the full abstract

All authors Gerard T. Portela, Jason B. Gibbons, Sebastian Schneeweiss, Rishi J. Desai

Machine Learning (Statistics)

Action-Driven Processes for Continuous-Time Control

At the heart of reinforcement learning are actions – decisions made in response to observations of the environment. Actions are equally fundamental in the modeling of stochastic processes, as they trigger discontinuous state transitions and enable the flow of information through large, complex systems. In this paper, we unify the perspectives of stochastic processes and reinforcement learning through action-driven processes, and illustrate their application to spiking neural networks. Leveraging ideas from control-as-inference, we show that minimizing the Kullback-Leibler divergence between a policy-driven true distribution and a reward-driven model distribution for a suitably defined action-driven process is equivalent to maximum entropy reinforcement learning.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

On Measuring Localization of Shortcuts in Deep Networks

Shortcuts, spurious rules that perform well during training but fail to generalize, present a major challenge to the reliability of deep networks (Geirhos et al., 2020). However, the impact of shortcuts on feature representations remains understudied, obstructing the design of principled shortcut-mitigation methods. To overcome this limitation, we investigate the layer-wise localization of shortcuts in deep models. Our novel experiment design quantifies the layer-wise contribution to accuracy degradation caused by a shortcut-inducing skew by counterfactual training on clean and skewed datasets. We employ our design to study shortcuts on CIFAR-10, Waterbirds, and CelebA datasets across VGG, ResNet, DeiT, and ConvNeXt architectures. We find that shortcut learning is not localized in specific layers but distributed throughout the network. Different network parts play different roles in this process: shallow layers predominantly encode spurious features, while deeper layers predominantly forget core features that are predictive on clean data. We also analyze the differences in localization and describe its principal axes of variation. Finally, our analysis of layer-wise shortcut-mitigation strategies suggests the hardness of designing general methods, supporting dataset- and architecture-specific approaches instead.
Read the full abstract

Statistics Theory

Linear regression with known noise distribution up to a scale: The reward of not using the OLSE

While the ordinary least squares estimator (OLSE) is still the most used estimator in linear regression models, other estimators can be more efficient when the error distribution is not Gaussian. In this paper, our goal is to evaluate this efficiency in the case of the Maximum Likelihood estimator (MLE) when the noise distribution belongs to a scale family. Under some regularity conditions, we show that (\beta_n,s_n), the MLE of the unknown regression vector \beta_0 and the scale s_0 exists and give the expression of the asymptotic efficiency of \beta_n over the OLSE. For given three scale families of densities, we quantify the true statistical gain of the MLE as a function of their deviation from the Gaussian family. To illustrate the theory, we present simulation results for different settings and also compare the MLE to the OLSE for the real market fish dataset.
Read the full abstract

Statistical Computation · Cross-listed from physics.soc-ph

Life-cycle Modeling and the Walking Behavior of the Pedestrian-Group as an Emergent Agent: With Empirical Data on the Cohesion of the Group Formation

This article investigates the pedestrian group as an emergent agent. The article explores empirical data to derive emergent agency and formation state spaces and outline recurring patterns of walking behavior. In this analysis, pedestrian trajectories extracted from surveillance videos are used along with manually annotated pedestrian group memberships. We conducted manual expert evaluation of observed groups, produced new manual annotations for relevant events pertaining to group behavior and extracted metrics relevant group formation. This information along with quantitative analysis was used to model the life-cycle and formation of the group agent. Those models give structure to expectations around walking behavior of groups; from pedestrian walking independently to the emergence of a collective intention where group members tended to maintain bounded distance between each other. Disturbances to this bounded distance often happened in association with changes in either their agency or their formation states. We summarized the patterns of behavior along with the sequences of state transitions into abstract patterns, which can aid in the development of more detailed group agents in simulation and in the design of engineering systems to interact with such groups.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

Higher-Order Regularization Learning on Hypergraphs

Higher-Order Hypergraph Learning (HOHL) was recently introduced as a principled alternative to classical hypergraph regularization, enforcing higher-order smoothness via powers of multiscale Laplacians induced by the hypergraph structure. Prior work established the well- and ill-posedness of HOHL through an asymptotic consistency analysis in geometric settings. We extend this theoretical foundation by proving the consistency of a truncated version of HOHL and deriving explicit convergence rates when HOHL is used as a regularizer in fully supervised learning. We further demonstrate its strong empirical performance in active learning and in datasets lacking an underlying geometric structure, highlighting HOHL's versatility and robustness across diverse learning settings.
Read the full abstract

Statistics Applications · Cross-listed from cs.PF

Approximating Heavy-Tailed Distributions with a Mixture of Bernstein Phase-Type and Hyperexponential Models

Heavy-tailed distributions, prevalent in a lot of real-world applications such as finance, telecommunications, queuing theory, and natural language processing, are challenging to model accurately owing to their slow tail decay. Bernstein phase-type (BPH) distributions, through their analytical tractability and good approximations in the non-tail region, can present a good solution, but they suffer from an inability to reproduce these heavy-tailed behaviors exactly, thus leading to inadequate performance in important tail areas. On the contrary, while highly adaptable to heavy-tailed distributions, hyperexponential (HE) models struggle in the body part of the distribution. Additionally, they are highly sensitive to initial parameter selection, significantly affecting their precision. To solve these issues, we propose a novel hybrid model of BPH and HE distributions, borrowing the most desirable features from each for enhanced approximation quality. Specifically, we leverage an optimization to set initial parameters for the HE component, significantly enhancing its robustness and reducing the possibility that the associated procedure results in an invalid HE model. Experimental validation demonstrates that the novel hybrid approach is more performant than individual application of BPH or HE models. More precisely, it can capture both the body and the tail of heavy-tailed distributions, with a considerable enhancement in matching parameters such as mean and coefficient of variation. Additional validation through experiments utilizing queuing theory proves the practical usefulness, accuracy, and precision of our hybrid approach.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection

Model and hyperparameter selection are critical but challenging in machine learning, typically requiring expert intuition or expensive automated search. We investigate whether large language models (LLMs) can act as in-context meta-learners for this task. By converting each dataset into interpretable metadata, we prompt an LLM to recommend both model families and hyperparameters. We study two prompting strategies: (1) a zero-shot mode relying solely on pretrained knowledge, and (2) a meta-informed mode augmented with examples of models and their performance on past tasks. Across synthetic and real-world benchmarks, we show that LLMs can exploit dataset metadata to recommend competitive models and hyperparameters without search, and that improvements from meta-informed prompting demonstrate their capacity for in-context meta-learning. These results highlight a promising new role for LLMs as lightweight, general-purpose assistants for model selection and hyperparameter optimization.
Read the full abstract

All authors Youssef Attia El Hili, Albert Thomas, Malik Tiomoko, Abdelhakim Benechehab, Corentin Léger, Corinne Ancourt, Balázs Kégl

Statistics Applications

Variational System Identification of Aircraft

Variational system identification is a new formulation of maximum likelihood for estimation of parameters of dynamical systems subject to process and measurement noise, such as aircraft flying in turbulence. This formulation is an alternative to the filter-error method that circumvents the solution of a Riccati equation and does not have problems with unstable predictors. In this paper, variational system identification is demonstrated for estimating aircraft parameters from real flight-test data. The results show that, in real applications of practical interest, it has better convergence properties than the filter-error method, reaching the optimum even when null initial guesses are used for all parameters and decision variables. This paper also presents the theory behind the method and practical recommendations for its use.
Read the full abstract

Statistical Methodology

Discovering Causal Relationships Between Time Series With Spatial Structure

Causal discovery is the subfield of causal inference concerned with estimating the structure of cause-and-effect relationships in a system of interrelated variables, as opposed to quantifying the strength or describing the form of causal effects. As interest in causal discovery builds in fields such as ecology, public health, and environmental sciences, where data are regularly collected with spatial and temporal structures, approaches must evolve to manage autocorrelation and complex confounding. As it stands, the few proposed causal discovery algorithms for spatiotemporal data require summarizing across locations, ignore spatial autocorrelation, and/or scale poorly to high dimensions. Here, we introduce our framework to extend time-series causal discovery to systems with spatial structure. Building upon work on causal discovery across contexts and methods for handling spatial confounding in causal effect estimation, we prove the soundness and completeness of our extension in the large sample limit. We close by discussing the feasibility of the framework, outlining remaining gaps in the literature, and identifying directions for future research.
Read the full abstract

Statistical Methodology

Statistical Inference for Matching Decisions via Matrix Completion under Dependent Missingness

This paper studies decision-making and statistical inference for two-sided matching markets via matrix completion. In contrast to the independent sampling assumed in classical matrix completion literature, the observed entries, which arise from past matching data, are constrained by matching capacity. This matching-induced dependence poses new challenges for both estimation and inference in the matrix completion framework. We propose a non-convex algorithm based on Grassmannian gradient descent and establish near-optimal entrywise convergence rates for three canonical mechanisms, i.e., one-to-one matching, one-to-many matching with one-sided random arrival, and two-sided random arrival. To facilitate valid uncertainty quantification and hypothesis testing on matching decisions, we further develop a general debiasing and projection framework for arbitrary linear forms of the reward matrix, deriving asymptotic normality with finite-sample guarantees under matching-induced dependent sampling. Our empirical experiments demonstrate that the proposed approach provides accurate estimation, valid confidence intervals, and efficient evaluation of matching policies.
Read the full abstract

All authors Congyuan Duan, Wanteng Ma, Dong Xia, Kan Xu