Machine Learning (Statistics) · Cross-listed from cs.LG
BoundAD: Boundary-Aware Negative Generation for Time Series Anomaly Detection
Xiancheng Wang, Lin Wang, Zhibo Zhang et al. (+2 more)
Contrastive learning methods for time series anomaly detection (TSAD) heavily depend on the quality of negative sample construction. However, existing strategies based on random perturbations or pseudo-anomaly injection often struggle to simultaneously preserve temporal semantic consistency and provide effective decision-boundary supervision. Most existing methods rely on prior anomaly injection, while overlooking the potential of generating hard negatives near the data
manifold boundary directly from normal samples themselves. To address this issue, we propose a reconstruction-driven boundary negative generation framework that automatically constructs hard negatives through the reconstruction process of normal samples. Specifically, the method first employs a reconstruction network to capture normal temporal patterns, and then introduces a reinforcement learning strategy to adaptively adjust the optimization update magnitude according to the current reconstruction state. In this way, boundary-shifted samples close to the normal data
manifold can be induced along the reconstruction trajectory and further used for subsequent contrastive representation learning. Unlike existing methods that depend on explicit anomaly injection, the proposed framework does not require predefined anomaly patterns, but instead mines more challenging boundary negatives from the model's own learning dynamics. Experimental results show that the proposed method effectively improves anomaly representation learning and achieves competitive detection performance on the current
dataset.
Read the full abstract ▾
All authors Xiancheng Wang, Lin Wang, Zhibo Zhang, Rui Wang, Minghang Zhao
Statistics Applications · Cross-listed from cs.LG
Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling
Junzhuo Ma, Chenghuang Shen, Yi Yu et al. (+15 more)
Technical-service LLM
agents are entering production workflows, where value depends on whether engineers adopt generated replies. Service tickets hide decision logic, contain noisy single-reference responses, and make reward evaluation costly, making standard post-training brittle. Existing post-training and LLM-as-a-Judge approaches improve grounding or feedback, but do not jointly model latent decision logic, response diversity, and reward cost. We address this gap by coupling latent logic augmentation, robust noise reduction, and hybrid reward modeling. The framework augments supervised fine-tuning data with Planning-Aware Trajectory Modeling and
Reasoning Augmentation, builds dual-filtered Multiple Ground Truths, and trains the policy with a hybrid reward that combines a Reranker with an LLM-as-a-Judge. On real Cloud technical-service tasks, the adapted Qwen3-4B achieves the highest Multi-ECS (0.441), lower reward cost, and the highest production adoption rate (46.63%).
Read the full abstract ▾
All authors Junzhuo Ma, Chenghuang Shen, Yi Yu, Xingyan Liu, Jing Gu, Hangyi Sun, Guangquan Hu, Jianfeng Liu, Weiting Liu, Pu Mingyue, Wang Yu, Zhengdong Xiao, Rui Xie, Longjiu Luo, Qianrong Wang, Gurong Cui, Honglin Qiao, Wenlian Lu
Statistical Methodology
On min-Storey estimators for multiple testing and conformal novelty detection
Gao Zijun and Roquain Etienne
In a multiple testing task, finding an appropriate
estimator of the proportion $\pi_0$ of non-signal in the data to boost power of false discovery rate (FDR) controlling procedures is a long-standing research theme, sometimes referred to as 'adaptive FDR control'. The interest in this theme has been reinforced in the recent years with conformal novelty detection, for which it turns out that similar tools can be used in combination with any 'blackbox' machine learning
algorithm. Nevertheless, perhaps surprisingly, finding a solution for 'adaptive FDR control' that is optimal in a broad sense is still an open problem. This paper fills this gap by introducing new $\pi_0$-
estimators, referred to as min-Storey (MS) and interval-min-Storey (IMS), which are built upon the so-called 'Storey
estimator'. Plugging these
estimators in the adaptive Benjamini-Hochberg (BH) procedure is shown to deliver FDR control both in the independent and conformal settings. In addition, these methods satisfy an optimal power property over any (regular) alternative distribution. The excellent behaviors of the new adaptive procedures are illustrated with numerical experiments both in the independent and conformal models for various distribution structures.
Read the full abstract ▾
Statistical Methodology
Multi-Armed Sequential Hypothesis Testing by Betting
Ricardo J. Sandoval, Ian Waudby-Smith and Michael I. Jordan
We consider a variant of sequential testing by betting where, at each time step, the statistician is presented with multiple data sources (arms) and obtains data by choosing one of the arms. We consider the composite global null hypothesis $\mathscr{P}$ that all arms are null in a certain sense (e.g. all dosages of a treatment are ineffective) and we are interested in rejecting $\mathscr{P}$ in favor of a composite alternative $\mathscr{Q}$ where at least one arm is non-null (e.g. there exists an effective treatment dosage). We posit an optimality desideratum that we describe informally as follows: even if several arms are non-null, we seek $e$-processes and sequential tests whose performance are as strong as the ones that have oracle knowledge about which arm generates the most evidence against $\mathscr{P}$. Formally, we generalize notions of log-optimality and expected rejection time optimality to more than one arm, obtaining matching lower and upper bounds for both. A key technical device in this optimality analysis is a modified upper-confidence-bound-like
algorithm for unobservable but sufficiently "estimable" rewards. In the design of this
algorithm, we derive nonasymptotic concentration inequalities for optimal wealth growth rates in the sense of Kelly [1956]. These may be of independent interest.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CL
Yue Zhao, Jiatao Gu, Paloma Jeretič et al. (+1 more)
Understanding the distance between human
languages is central to linguistics, anthropology, and tracing human evolutionary history. Yet, while linguistics has long provided rich qualitative accounts of cross-linguistic variation, a unified and scalable quantitative approach to measuring
language distance remains lacking. In this paper, we introduce a method that leverages pretrained
multilingual language models as systematic instruments for linguistic measurement. Specifically, we show that the spontaneously emerged
attention mechanisms of these models provide a robust, tokenization-agnostic measure of cross-linguistic distance, termed
Attention Transport Distance (ATD). By treating
attention matrices as probability distributions and measuring their
geometric divergence via optimal transport, we quantify the representational distance between
languages during translation. Applying ATD to a large and diverse set of
languages, we demonstrate that the resulting distances recover established linguistic groupings with high fidelity and reveal patterns aligned with geographic and contact-induced relationships. Furthermore, incorporating ATD as a regularizer improves transfer performance in low-resource machine translation. Our results establish a principled foundation for testing linguistic hypotheses using artificial neural networks. This framework transforms
multilingual models into powerful tools for quantitative linguistic discovery, facilitating more equitable
multilingual AI.
Read the full abstract ▾
All authors Yue Zhao, Jiatao Gu, Paloma Jeretič, Weijie Su
Machine Learning (Statistics)
A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models
Leonardo Defilippis, Florent Krzakala, Bruno Loureiro et al. (+1 more)
Understanding when learning is statistically possible yet computationally hard is a central challenge in high-dimensional statistics. In this work, we investigate this question in the context of single- and multi-index models, classes of functions widely studied as
benchmarks to probe the ability of machine learning methods to discover features in high-dimensional data. Our main contribution is to show that a Noise Sensitivity Exponent (NSE) - a simple quantity determined by the activation function - governs the existence and magnitude of statistical-to-computational gaps within a broad regime of these models. We first establish that, in single-index models with large additive noise, the onset of a computational bottleneck is fully characterized by the NSE. We then demonstrate that the same exponent controls a statistical-computational gap in the specialization transition of large separable multi-index models, where individual components become learnable. Finally, in hierarchical multi-index models, we show that the NSE governs the optimal computational rate in which different directions are sequentially learned. Taken together, our results identify the NSE as a unifying property linking noise robustness, computational hardness, and feature specialization in high-dimensional learning.
Read the full abstract ▾
All authors Leonardo Defilippis, Florent Krzakala, Bruno Loureiro, Antoine Maillard
Statistics Applications
NFL step-and-turn: A generative framework for evaluating player movement in American football
Quang Nguyen and Ronald Yurko
In sports analytics, player tracking data have driven significant advancements in the task of player evaluation. We present a novel generative framework for evaluating the observed frame-by-frame player positioning against a distribution of hypothetical alternatives. We illustrate our approach by modeling the within-play movement of an individual ball carrier in the National Football League (NFL). Specifically, we develop
Bayesian multilevel models for frame-level player movement based on two components: step length (distance between successive locations) and turn angle (change in direction between successive steps). Using the step-and-turn models, we perform posterior predictive simulation to generate hypothetical ball carrier steps at each frame during a play. This enables comparison of the observed player movement with a distribution of simulated alternatives using common valuation measures in American football. We apply our framework to tracking data from the first nine weeks of the 2022 NFL season and derive novel player performance metrics based on hypothetical evaluation.
Read the full abstract ▾
Statistical Methodology
Bivariate deconvolution for cancer detection after surgery
Nuria Senar, Stavros Makrodimitris, Michel H. Hof et al. (+3 more)
Detection of minimal residual disease (MRD) in cancer patients after surgery can provide an early marker for disease recurrence and guide subsequent treatment decisions. Accurate and sensitive estimation of tumour burden after cancer surgery may be obtained through liq- uid biopsies, measuring circulating tumour DNA (ctDNA) using, for example, mutation-based Variant Allele Frequency (VAF) values. However, to be applicable to all patients this ei- ther requires tumour-informed, patient-specific mutation panels or sensitive, tumour-agnostic genome-wide measurements. We propose a solution that accounts for patient-specific charac- teristics in genome-wide screens. For that, we introduce a bivariate deconvolution model to estimate tumour proportion from circulating
cell-free DNA (cfDNA) methylation profiles of patients before and after surgery. The observations are modelled as a convolution of two bivariate latent variables, corresponding to tumour and background signals, mixed by the tumour proportion at each measurement. This bivariate approach links pre- and post-surgery measurements improving estimation of the tumour proportion after surgery, when the tumour signal is potentially very weak, or absent. We approximate likelihood of the convolution through a discretisation of the bivariate density for each latent variable into a two-dimensional grid for each pair of observations which allows for fast maximum likelihood estimation. We evaluate the predictive performance of the estimated post-surgery tumour proportions based on cfDNA methylation against available mutation-based VAF values in one-year recurrence-free survival.
Read the full abstract ▾
All authors Nuria Senar, Stavros Makrodimitris, Michel H. Hof, Cornelis Verhoef, Saskia M. Wilting, Mark A. van de Wiel
Statistics Applications · Cross-listed from cs.CR
Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation
Iakovos-Christos Zarkadis and Christos Douligeris
Supervised detection of network attacks has always been a critical part of network intrusion detection systems (NIDS). Nowadays, in a pivotal time for artificial intelligence (AI), with even more sophisticated attacks that utilize advanced techniques, such as generative artificial intelligence (GenAI) and reinforcement learning, it has become a vital component if we wish to protect our personal data, which are scattered across the web. In this paper, we address two tasks, in the first unified multi-modal NIDS
dataset, which incorporates flow-level data, packet payload information and temporal contextual features, from the reprocessed CIC-IDS-2017, CIC-IoT-2023, UNSW-NB15 and CIC-DDoS-2019, with the same feature space. In the first task we use machine learning (ML)
algorithms, with stratified cross validation, in order to prevent network attacks, with stability and reliability. In the second task we use
adversarial learning
algorithms to generate synthetic data, compare them with the real ones and evaluate their fidelity, utility and
privacy using the SDV framework, f-divergences, distinguishability and non-parametric statistical tests. The findings provide stable ML models for intrusion detection and generative models with high fidelity and utility, by combining the Synthetic Data Vault framework, the TRTS and TSTR tests, with non-parametric statistical tests and f-divergence measures.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.NT
Murmurations, Mestre–Nagao sums, and Convolutional Neural Networks for elliptic curves
Joanna Bieri, Edgar Costa, Alyson Deines et al. (+5 more)
We apply one-dimensional convolutional neural networks to the Frobenius traces of
elliptic curves over $\mathbb{Q}$ and evaluate and interpret their predictive capacity. In keeping with similar experiments by Kazalicki–Vlah, Bujanović–Kazalicki–Novak, and Pozdnyakov, we observe high accuracy predictions for the analytic rank across a range of conductors. We interpret the prediction using saliency curves and explore the interesting interplay between saliency, murmurations and Mestre–Nagao sums.
Read the full abstract ▾
All authors Joanna Bieri, Edgar Costa, Alyson Deines, Kyu-Hwan Lee, David Lowry-Duda, Thomas Oliver, Yidi Qi, Tamara Veenstra
Statistical Methodology
More with Less – Bethel Allocation and Precision-Preserving Sample Size Reduction via Hierarchical Bayes Modelling
Siu-Ming Tam
Statistical offices face a familiar and intensifying dilemma: rising demand for detailed regional and domain-level estimates under budgets that are fixed or shrinking. National statistical offices (NSOs) either ignore the problem of optimal sample allocation for multiple target variables when designing a multi-purpose survey, or address it incorrectly - relying on ad hoc approaches such as computing Neyman allocations separately per variable and taking the element-wise maximum, a practice that simultaneously wastes budget and fails to guarantee precision across all domains. This paper presents a practical two-stage strategy that reframes the question: not how to allocate a given sample, but how small the sample can be made while still meeting pre-defined precision targets for all target variables across all geographic domains at once. The innovation lies not in inventing new methods, but in the novel combination of two well-established techniques applied to this cost-reduction problem: (i) multivariate constrained optimisation via Bethel allocation, which finds the globally minimum sample satisfying all precision constraints simultaneously; and (ii) Hierarchical Bayes (HB) small area modelling, which borrows strength across strata and permits a further reduction of the Bethel sample. The approach is validated using a Monte Carlo study (B = 1,000 replications) based on a synthetic labour-force population of one million individuals, where known population truth allows rigorous evaluation of precision, accuracy, and credible-interval coverage. Keywords: Bethel allocation; Hierarchical Bayes; small area estimation; sample size reduction; multivariate optimisation; labour force survey; coefficient of variation.
Read the full abstract ▾
Machine Learning (Statistics)
rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks
Suryasis Jana and Abhik Ghosh
Neural networks are central to modern artificial intelligence, yet their training remains highly sensitive to data contamination. Standard neural classifiers are trained by minimizing the categorical cross-
entropy loss, corresponding to maximum likelihood estimation under a multinomial model. While statistically efficient under ideal conditions, this approach is highly vulnerable to contaminated observations including label noises corrupting supervision in the output space, and
adversarial perturbations inducing worst-case deviations in the input space. In this paper, we propose a unified and statistically grounded framework for robust neural classification that addresses both forms of contamination within a single learning objective. We formulate neural network training as a minimum-divergence estimation problem and introduce rSDNet, a robust learning
algorithm based on the general class of $S$-divergences. The resulting training objective inherits robustness properties from classical statistical estimation, automatically down-weighting aberrant observations through model probabilities. We establish essential population-level properties of rSDNet, including Fisher consistency, classification calibration implying Bayes optimality, and robustness guarantees under uniform label noise and infinitesimal feature contamination. Experiments on three
benchmark image classification
datasets show that rSDNet improves robustness to label corruption and
adversarial attacks while maintaining competitive accuracy on clean data, Our results highlight minimum-divergence learning as a principled and effective framework for robust neural classification under heterogeneous data contamination.
Read the full abstract ▾
Statistical Methodology
Prediction with Missing Data: Target Probabilities and Missingness Mechanisms
Pierre Catoire, Robin Genuer and Cecile Proust-Lima
Conditions ensuring optimal parameter estimation in the presence of missing data are well established in inference, typically relying on the Missing-at-Random (MAR) assumption. In prediction, similar principles are often assumed to apply. However, methods considered biased in inference, such as pattern sub-modelling or unconditional imputation, have been shown to achieve optimal predictive performance under any missingness mechanism, including non-MAR (MNAR). To explain this apparent contradiction, we introduce a new formal framework for describing missingness in prediction. Central to this framework is a distinction between two prediction targets, defined according to whether or not the indicator of observation of the predictors is exploited to predict the outcome. This distinction leads to a classification of the missingness mechanisms describing the conditions under which these targets are equal, and when consistent prediction of each is achievable. A key result is that both targets may be consistently predicted under conditions weaker than MAR. We discuss the implications of this paradigm for handling missing data in prediction, distinguishing between missingness at development, validation and deployment of a forecaster. The findings are illustrated using simulated data and a real-world application with the prediction of significant injury after trauma upon arrival at the emergency department.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Identifying Latent Actions and Dynamics from Offline Data via Demonstrator Diversity
Felix Schur
Can latent actions and environment dynamics be recovered from offline trajectories when actions are never observed? We study this question in a setting where trajectories are action-free but tagged with demonstrator identity. We assume that each demonstrator follows a distinct policy, while the environment dynamics are shared across demonstrators and identity affects the next observation only through the chosen action. Under these assumptions, the conditional next-observation distribution $p(o_{t+1}\mid o_t,e)$ is a mixture of latent action-conditioned transition kernels with demonstrator-specific mixing weights. We show that this induces, for each state, a column-
stochastic nonnegative
matrix factorization of the observable conditional distribution. Using sufficiently scattered policy diversity and rank conditions, we prove that the latent transitions and demonstrator policies are identifiable up to permutation of the latent action labels. We extend the result to continuous observation spaces via a Gram-determinant minimum-volume criterion, and show that continuity of the transition map over a connected state space upgrades local permutation ambiguities to a single global permutation. A small amount of labeled action data then suffices to fix this final ambiguity. These results establish demonstrator diversity as a principled source of identifiability for learning latent actions and dynamics from offline RL data.
Read the full abstract ▾
Machine Learning (Statistics)
Gaussian Process Limit Reveals Structural Benefits of Graph Transformers
Nil Ayday, Lingchu Yang and Debarghya Ghoshdastidar
Graph
transformers are the state-of-the-art for learning from graph-structured data and are empirically known to avoid several pitfalls of message-passing architectures. However, there is limited theoretical analysis on why these models perform well in practice. In this work, we prove that
attention-based architectures have structural benefits over graph convolutional networks in the context of node-level prediction tasks. Specifically, we study the neural network gaussian process limits of graph
transformers (GAT, Graphormer, Specformer) with infinite width and infinite heads, and derive the node-level and edge-level kernels across the layers. Our results characterise how the node features and the graph structure propagate through the graph
attention layers. As a specific example, we prove that graph
transformers structurally preserve community information and maintain discriminative node representations even in deep layers, thereby preventing oversmoothing. We provide empirical evidence on synthetic and real-world graphs that validate our theoretical insights, such as integrating informative priors and positional encoding can improve performance of deep graph
transformers.
Read the full abstract ▾
Machine Learning (Statistics)
Consistency of the $k$-Nearest Neighbor Regressor under Complex Survey Designs
Caren Hasler
We study the consistency of the $k$-nearest neighbor regressor under complex survey designs. While consistency results for this
algorithm are well established for independent and identically distributed data, corresponding results for complex survey data are lacking. We show that the $k$-nearest neighbor regressor is consistent under regularity conditions on the sampling design and the distribution of the data. We derive lower bounds for the rate of convergence and show that these bounds exhibit the curse of dimensionality, as in the independent and identically distributed setting. Empirical studies based on simulated and real data illustrate our theoretical findings.
Read the full abstract ▾
Machine Learning (Statistics)
Mirror Descent on Riemannian Manifolds
Jiaxin Jiang, Lei Shi and Jiyuan Tan
Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian
manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a
stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and
stochastic RMD. As an application to the Stiefel
manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the
stochastic RMD framework to the Stiefel setting, we obtain a
stochastic extension of CGD, which effectively addresses large-scale
manifold optimization problems.
Read the full abstract ▾
Statistical Methodology
A lightweight framework for characterising extreme precipitation events in climate ensembles
Dáire Healy, Isadora Antoniano-Villalobos, Claudia Collarin et al. (+3 more)
This article summarises the methods used by the team “Ca' Foscari" for the EVA 2025 Data Challenge. The questions of the challenge concern the estimation of exceedance probabilities across several locations. Rather than modelling the spatial dependence structure, we reduce the problems to univariate ones by considering relevant spatial order statistics across the sites. Within a Peaks over Threshold framework, we model the marginal distributions of exceedances using generalised Pareto distributions. Generalised additive models are employed to allow the parameters to vary as functions of external predictors, which for all questions are reduced to the month. For questions 1 and 2, the required estimates and confidence intervals are obtained by generating samples from our fitted models. Question 3 involves the dependence between two consecutive observed statistics. To account for this temporal dependence, we fit a conditional extreme value model and derive empirical estimates of persistent extreme events by simulating from this model.
Read the full abstract ▾
All authors Dáire Healy, Isadora Antoniano-Villalobos, Claudia Collarin, Nathan Huet, Ilaria Prosdocimi, Emilia Siviero
Statistics Applications
A Weight-Dependent 1RM Prediction Equation Optimized on 303,494 Near-Failure Sets Across 388 Exercises
Thiago Marzagao
Classical equations for predicting one-repetition maximum (1RM) from submaximal performance were derived from small samples performing a single exercise, yet are routinely applied to hundreds of exercises. All use a fixed conversion factor relating repetitions to estimated 1RM, regardless of exercise or load. We used large-scale observational data from a consumer fitness app (303,494 near-failure sets from 14,966 users across 388 exercises spanning 16 muscle groups) to derive and evaluate a generalization in which the conversion factor varies logarithmically with the weight lifted: 1RM = w * (1 + (r - 1)^0.85 / (-2.55 + 4.58 * ln(w))). Because the
dataset contains no directly measured maxima, we optimized and evaluated the formula using an internal consistency criterion – the degree to which different weight-repetition combinations from the same person, exercise, and time window yield the same estimated 1RM. The proposed formula reduced inconsistency by 17-22% relative to four classical
benchmarks, with the improvement positive for every one of the 183 exercises with sufficient data. Five-fold user-level cross-validation confirmed near-zero overfitting. An ablation analysis attributed 91% of the improvement to the weight-dependent conversion factor and 9% to the sub-linear repetition exponent. The conversion factor increases with load: at light weights each additional repetition implies a larger fraction of maximal capacity than at heavy weights, consistent with prior evidence that the repetitions-%1RM relationship varies by exercise. Classical equations, by applying a single conversion factor across all loads, systematically underestimate this variation – and the discrepancy is largest for the lighter, more diverse exercises that dominate real-world training programs.
Read the full abstract ▾
Statistical Methodology
Fast and scalable inference in hidden Markov models with Gaussian fields
Jan-Ole Fischer
Hidden Markov models (HMMs) are powerful tools for analysing time series data that depend on discrete underlying but unobserved states. As such, they have gained prominence across numerous empirical disciplines, in particular ecology, medicine, and economics. However, the increasing complexity of empirical data is often accompanied by additional latent structure such as spatial effects, temporal trends, or measurement perturbations. Gaussian fields provide an attractive building block for incorporating such structured latent variation into HMMs. Fast inference methods for Gaussian fields have emerged through the
stochastic partial differential equation (SPDE) approach. Due to their sparse representation, these integrate well with novel frequentist estimation methods for random-effects models via the use of automatic differentiation and the Laplace approximation. Scaling to high dimensions requires tools such as (R)TMB to exploit sparsity in the Hessian w.r.t. the latent variables - a property satisfied by SPDE fields but violated by the HMM likelihood. We present a modified forward
algorithm to compute the HMM likelihood, constructing sparsity in the Hessian and consequently enabling fast and scalable inference. We demonstrate the practical feasibility and the usefulness through simulations and two case studies exploring the detection of
stellar flares as well as modelling the movement of lions.
Read the full abstract ▾
Statistical Computation · Cross-listed from math.DS
A Full-Density Approach to Simulating Random Iteration Equations with Applications
Wolfgang Hoegele
The goal of this study is to introduce a unified framework for simulating random iteration equations (RIE), understood as iteration equations containing random variables. The main idea is to propagate approximations of the full state density from one iteration to the next, rather than estimating it from many repeated pathwise Monte Carlo simulations. The presentation of the RIE modeling framework is conceptually simple based on recent work on static random equations and designed to be accessible. The modeling requirements for RIEs allow for potential nonsmooth nonlinearities and stochasticities in the transfer function. Additionally, the RIE computational strategy for full-density propagation is presented based on iterative likelihood / posterior calculations. As results, illustrative applications of nonlinear random and
stochastic differential equation simulations, a new full-density gradient descent method (FDGD) for global optimization under uncertainty and examples of chaotic mappings are presented in order to demonstrate the breadth of the utility of this framework. In total, the character of the presentation is explorative and encourages new applications and theoretical studies.
Read the full abstract ▾
Statistics Applications
Multivariate GARCH and portfolio variance prediction: A forecast reconciliation perspective
Massimiliano Caporin, Daniele Girolimetto and Emanuele Lopetuso
We assess the advantage of combining univariate and multivariate portfolio risk forecasts with the aid of forecast reconciliation techniques. In our analyzes, we assume knowledge of portfolio weights, a standard for portfolio risk management applications. With an extensive simulation experiment, we show that, if the true covariance is known, forecast reconciliation improves over a standard multivariate approach, in particular when the adopted multivariate model is misspecified. However, if noisy proxies are used, correctly specified models and the misspecified ones (for instance, neglecting spillovers) turn out to be, in several cases, indistinguishable, with forecast reconciliation still providing improvements. The noise in the covariance proxy plays a crucial role in determining the improvement of both the forecast reconciliation and the correct model specification. An empirical analysis shows how forecast reconciliation can be adopted with real data to improve traditional GARCH-based portfolio variance forecasts.
Read the full abstract ▾
Statistical Methodology
Algorithms for Models with Intractable Normalizing Functions
Murali Haran, Bokgyeong Kang and Jaewoo Park
In this paper we discuss a well known computing problem – inference for models with intractable normalizing functions. Models with intractable normalizing functions arise in a wide variety of areas, for instance network models, models for spatial data on
lattices, spatial point processes, flexible models for count data and gene expression, and models for permutations. Simulating from these models for fixed parameter values is well studied, starting with work dating back seventy years to the origin of the Metropolis
algorithm. On the other hand some of the most practical and theoretically justified
algorithms for inference, particularly
Bayesian inference, have only been developed within the past two decades. The most computationally efficient
algorithms often do not have well developed theory and few if any approaches exist for assessing the quality of approximations based on them. For many problems even the best
algorithms can be computationally infeasible. Hence, this is an exciting area of research with many open problems. We explain several key
algorithms, providing connections and touching upon practical advantages and disadvantages of each, with some discussion of theoretical properties where they impact practice. We discuss an approach for assessing the accuracy of approximations produced by these
algorithms; this diagnostic is particularly valuable for
algorithm tuning. While our focus is largely on models with intractable normalizing functions, we also discuss
algorithms that are more broadly applicable to models where the entire likelihood function is intractable; these methods are of course also applicable to intractable normalizing function problems.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from econ.EM
An Auditable AI Agent Loop for Empirical Economics: A Case Study in Forecast Combination
Minchul Shin
AI coding
agents, general purpose assistants that write and execute code, make empirical specification search fast and cheap, but they also widen hidden researcher degrees of freedom. This paper adapts an open-source
agent-loop architecture to an empirical economics workflow and adds a post-search holdout evaluation. In a forecast-combination illustration, independent
agent searches find methods that improve on
benchmarks from the original study. Logged search and holdout evaluation together make adaptive specification search more transparent and help distinguish robust improvements from sample-specific discoveries.
Read the full abstract ▾
Statistical Methodology
Empirical Likelihood Inference for Sen and Sen–Shorrocks–Thon Indices
Sreelakshmi N, Saparya Suresh and Sudheesh K. Kattumannil
The Sen index and Sen-Shorrocks-Thon (SST) index are widely used measures of poverty indices. Developing reliable inference for these measures enables us to compare these measures in different populations of interest in an effective way. It is important to construct confidence intervals for the Sen index and SST index, which provide better coverage probability and shorter interval length. Motivated by this, we discuss empirical likelihood (EL) and jackknife empirical likelihood (JEL) based inference for the Sen index. To derive a JEL-based confidence interval for the Sen and SST indices, we propose a new
estimator for the Sen index using the theory of U-statistics and examine its properties. The large sample properties of the EL and JEL ratio statistics are studied. We also discuss EL and JEL-based inference for the Sen-Shorrocks-Thon (SST) index. The finite sample performance of the EL and JEL-based confidence intervals of both Sen and SST indices is evaluated through a Monte Carlo simulation study. Finally, we illustrate our methods using individual-level data from the Panel Study of Income Dynamics (PSID) survey from the US as well as Indian household level income data for different states sourced from the Consumer Pyramids Household Survey (CPHS).
Read the full abstract ▾
Statistics Applications
Analysis of molecular dynamics simulation data via statistical distances between covariance matrices
Yusuke Ono, Takumi Sato, Kenji Yasuoka et al. (+1 more)
Molecular dynamics (MD) simulations are powerful tools for elucidating the macroscopic physical properties of materials from microscopic atomic behaviors. However, the massive, high-dimensional
datasets generated by MD simulations pose a significant challenge for analysis, necessitating efficient dimensionality reduction and feature extraction techniques. While existing methods such as principal component analysis and unsupervised learning have been utilized, issues regarding data efficiency and computational cost remain. In this study, we propose a statistical analysis framework focusing on the analysis of the particle data distributions through their covariance
matrices, corresponding to the second-order moments of MD trajectory data. Discrepancies between system states are quantified using statistical distances between these covariance
matrices. By applying dimensionality reduction to the resulting distance
matrix, we extract lower-dimensional features that characterize the systems' dynamics. We validate the proposed method using Lennard-Jones (LJ) particle systems under different temperature conditions, as well as separate bulk systems of ice and
liquid water. The results of LJ particles demonstrate an approximately linear correlation between the first principal component obtained through dimensionality reduction of the distance
matrix and the
diffusion coefficient. This suggests that global physical properties can be effectively inferred from local statistical information, such as covariance
matrices, offering a data-efficient alternative for analyzing complex molecular systems. Furthermore, in the case of separate bulk systems of ice and
liquid water, the method successfully distinguishes between the two phases, highlighting its potential for characterizing phase transitions and structural differences in molecular systems.
Read the full abstract ▾
All authors Yusuke Ono, Takumi Sato, Kenji Yasuoka, Linyu Peng
Statistical Methodology
Bayesian Scalar-on-Tensor Quantile Regression for Longitudinal Data on Alzheimer's Disease
Rongke Lyu, Marina Vannucci and Suprateek Kundu
As a general and robust alternative to traditional mean regression models,
quantile regression avoids the assumption of normally distributed errors, making it a versatile choice when modeling outcomes such as cognitive scores that typically have skewed distributions. Motivated by an application to Alzheimer's disease data where the aim is to explore how
brain-behavior associations change over time, we propose a novel
Bayesian tensor
quantile regression for high-dimensional longitudinal imaging data. The proposed approach distinguishes between effects that are consistent across visits and patterns unique to each visit, contributing to the overall longitudinal trajectory. A low-rank decomposition is employed on the tensor coefficients which reduces dimensionality and preserves spatial configurations of the imaging voxels. We incorporate multiway shrinkage priors to model the visit-invariant tensor coefficients and variable selection priors on the tensor margins of the visit-specific effects. For posterior inference, we develop a computationally efficient Markov chain Monte Carlo sampling
algorithm. Simulation studies reveal significant improvements in parameter estimation, feature selection, and prediction performance when compared with existing approaches. In the analysis of the Alzheimer's disease data, the flexibility of our modeling approach brings new insights as it provides a fuller picture of the relationship between the imaging voxels and the
quantile distributions of the cognitive scores.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.PR
On the structure of marginals in high dimensions
Daniel Bartl and Shahar Mendelson
Let $G, G_1,\dots,G_N$ be independent copies of a standard gaussian random vector in $\mathbb{R}^d$ and denote by $\Gamma = \sum_{i=1}^N \langle G_i,\cdot\rangle e_i$ the standard gaussian ensemble. We show that, for any set $A\subset S^{d-1}$, with exponentially high probability, \[ \sup_{x\in A} \frac{1}{N}\sum_{i=1}^N \big| (\Gamma x)^\sharp_i - q_i\big| \le c \frac{ \mathbb{E} \sup_{x\in A} \langle G,x\rangle + \log^2N }{\sqrt N }. \] Here each $q_i$ is the $\frac{i}{N+1}$-
quantile of the standard normal distribution and $(\Gamma x)^\sharp $ denotes the monotone increasing rearrangement of the vector $\Gamma x$. The estimate is sharp up to a possible logarithmic factor and significantly extends previously known bounds. Moreover, we show that similar estimates hold in much greater generality: after replacing the gaussian
quantiles by the appropriate ones, the same phenomenon persists for a broad class of random vectors.
Read the full abstract ▾
Statistics Applications
Improving causal inference in interrupted time series analysis: the triple difference design
Ariel Linden
Background: Interrupted time series analysis (ITSA) is widely used to evaluate health policy and intervention effects. While multiple-group ITSA (MG-ITSA) improves causal inference by incorporating a control group, residual confounding from unmeasured time-varying factors may remain. The triple-difference interrupted time series (DDD-ITSA) design extends this approach by adding a second control group to further isolate treatment effects, but it remains underutilized and lacks formal guidance. Methods: We formalize the DDD-ITSA framework, specify the regression model, define key parameters for estimating level and trend effects, and clarify interpretation of the triple-difference estimand. We illustrate the approach using a worked example evaluating California's Proposition 99 cigarette tax and its impact on per-capita cigarette sales. Results: In the example, all groups were balanced on pre-intervention level and trend. The triple-difference estimand indicated a statistically significant annual reduction of -1.76 per-capita cigarette packs in California relative to the secondary control (P = 0.020; 95 percent CI: -3.24, -0.28), consistent with results from the primary comparison. Differences between control groups were not significant. Conclusions: DDD-ITSA strengthens causal inference when two-group comparisons may be confounded by leveraging an additional control group to remove remaining biases and assess heterogeneity. Implementation is facilitated by updates to the itsa Stata package. Careful
attention to control selection, baseline balance, and autocorrelation remains essential.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Classifier Pooling for Modern Ordinal Classification
Noam H. Rotenberg, Andreia V. Faria and Brian Caffo
Ordinal data is widely prevalent in clinical and other domains, yet there is a lack of both modern, machine-learning based methods and publicly available software to address it. In this paper, we present a model-agnostic method of ordinal classification, which can apply any non-ordinal classification method in an ordinal fashion. We also provide an open-source implementation of these
algorithms, in the form of a Python package. We apply these models on multiple real-world
datasets to show their performance across domains. We show that they often outperform non-ordinal classification methods, especially when the number of datapoints is relatively small or when there are many classes of outcomes. This work, including the developed software, facilitates the use of modern, more powerful machine learning
algorithms to handle ordinal data.
Read the full abstract ▾