Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 12 January

Statistics Applications

Cauchy-Gaussian Overbound for Heavy-tailed GNSS Measurement Errors

Overbounds of heavy-tailed measurement errors are essential to meet stringent navigation requirements in integrity monitoring applications. This paper proposes to leverage the bounding sharpness of the Cauchy distribution in the core and the Overbounds of heavy-tailed measurement errors are essential for meeting stringent navigation requirements in integrity-monitoring applications. This paper proposes to leverage the bounding sharpness of the Cauchy distribution in the core and the Gaussian distribution in the tails to tightly bound heavy-tailedglobal navigation satellite system measurement errors. We develop a procedure to determine the overbounding parameters for both symmetric unimodal (SU)and non-symmetric unimodal (NSU) heavy-tailed errors and prove that the over-bounding property is preserved through convolution. Experiment results on both simulated and real-world data sets reveal that our method can sharply boundheavy-tailed errors in both the core and tail regions. In the position domain, the proposed method reduces the average vertical protection level by 15% for SU heavy-tailed errors compared with the single-cumulative-density-function Gaussian overbound and by 21%-47% for NSU heavy-tailed errors compared with the navigation discrete envelope and two-step Gaussian overbounds.
Read the full abstract

All authors Zhengdao Li, Penggao Yan, Weisong Wen, Li-Ta Hsu

Statistical Methodology

Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity

Covariate shift arises when covariate distributions differ between source and target populations while the conditional distribution of the response remains invariant, and it underlies problems in missing data and causal inference. We propose a minimum Wasserstein distance estimation framework for inference under covariate shift that avoids explicit modeling of outcome regressions or importance weights. The resulting W-estimator admits a closed-form expression and is numerically equivalent to the classical 1-nearest neighbor estimator, yielding a new optimal transport interpretation of nearest neighbor methods. We establish root-$n$ asymptotic normality and show that the estimator is not asymptotically linear, leading to super-efficiency relative to the semiparametric efficient estimator under covariate shift in certain regimes, and uniformly in missing data problems. Numerical simulations, along with an analysis of a rainfall dataset, underscore the exceptional performance of our W-estimator.
Read the full abstract

Machine Learning (Statistics)

Covariance-Driven Regression Trees: Reducing Overfitting in CART

Decision trees are powerful machine learning algorithms, widely used in fields such as economics and medicine for their simplicity and interpretability. However, decision trees such as CART are prone to overfitting, especially when grown deep or the sample size is small. Conventional methods to reduce overfitting include pre-pruning and post-pruning, which constrain the growth of uninformative branches. In this paper, we propose a complementary approach by introducing a covariance-driven splitting criterion for regression trees (CovRT). This method is more robust to overfitting than the empirical risk minimization criterion used in CART, as it produces more balanced and stable splits and more effectively identifies covariates with true signals. We establish an oracle inequality of CovRT and prove that its predictive accuracy is comparable to that of CART in high-dimensional settings. We find that CovRT achieves superior prediction accuracy compared to CART in both simulations and real-world tasks.
Read the full abstract

Statistical Methodology

Connections as treatment: causal inference with edge interventions in networks

Causal inference has traditionally focused on interventions at the unit level. In many applications, however, the central question concerns the causal effects of connections between units, such as transportation links, social relationships, or collaborative ties. We develop a causal framework for edge interventions in networks, where treatments correspond to the presence or absence of edges. Our framework defines causal estimands under stochastic interventions on the network structure and introduces an inverse probability weighting estimator under an unconfoundedness assumption on edge assignment. We estimate edge probabilities using exponential random graph models, a widely used class of network models. We establish consistency and asymptotic normality of the proposed estimator. Finally, we apply our methodology to China's transportation network to estimate the causal impact of railroad connections on regional economic development.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Simulated Annealing-based Candidate Optimization for Batch Acquisition Functions

Bayesian Optimization with multi-objective acquisition functions such as q-Expected Hypervolume Improvement (qEHVI) requires efficient candidate optimization to maximize acquisition function values. Traditional approaches rely on continuous optimization methods like Sequential Least Squares Programming (SLSQP) for candidate selection. However, these gradient-based methods can become trapped in local optima, particularly in complex or high-dimensional objective landscapes. This paper presents a simulated annealing-based approach for candidate optimization in batch acquisition functions as an alternative to conventional continuous optimization methods. We evaluate our simulated annealing approach against SLSQP across four benchmark multi-objective optimization problems: ZDT1 (30D, 2 objectives), DTLZ2 (7D, 3 objectives), Kursawe (3D, 2 objectives), and Latent-Aware (4D, 2 objectives). Our results demonstrate that simulated annealing consistently achieves superior hypervolume performance compared to SLSQP in most test functions. The improvement is particularly pronounced for DTLZ2 and Latent-Aware problems, where simulated annealing reaches significantly higher hypervolume values and maintains better convergence characteristics. The histogram analysis of objective space coverage further reveals that simulated annealing explores more diverse and optimal regions of the Pareto front. These findings suggest that metaheuristic optimization approaches like simulated annealing can provide more robust and effective candidate optimization for multi-objective Bayesian optimization, offering a promising alternative to traditional gradient-based methods for batch acquisition function optimization.
Read the full abstract

Statistical Methodology

Compounded Linear Failure Rate Distribution: Properties, Simulation and Analysis

This paper proposes a new extension of the linear failure rate (LFR) model to better capture real-world lifetime data. The model incorporates an additional shape parameter to increase flexibility. It helps model the minimum survival time from a set of LFR distributed variables. We define the model, derive certain statistical properties such as the mean residual life, the mean inactivity time, moments, quantile, order statistics and also discuss the results on stochastic orders of the proposed distribution. The proposed model has increasing, bathtub shaped and inverse bathtub shaped hazard rate function. We use the method of maximum likelihood estimation to estimate the unknown parameters. We conduct simulation studies to examine the behavior of the estimators. We also use three real datasets to evaluate the model, which turns out superior compared to classical alternatives.
Read the full abstract

Machine Learning (Statistics)

Multi-environment Invariance Learning with Missing Data

Learning models that can handle distribution shifts is a key challenge in domain generalization. Invariance learning, an approach that focuses on identifying features invariant across environments, improves model generalization by capturing stable relationships, which may represent causal effects when the data distribution is encoded within a structural equation model (SEM) and satisfies modularity conditions. This has led to a growing body of work that builds on invariance learning, leveraging the inherent heterogeneity across environments to develop methods that provide causal explanations while enhancing robust prediction. However, in many practical scenarios, obtaining complete outcome data from each environment is challenging due to the high cost or complexity of data collection. This limitation in available data hinders the development of models that fully leverage environmental heterogeneity, making it crucial to address missing outcomes to improve both causal insights and robust prediction. In this work, we derive an estimator from the invariance objective under missing outcomes. We establish non-asymptotic guarantees on variable selection property and $\ell_2$ error convergence rates, which are influenced by the proportion of missing data and the quality of imputation models across environments. We evaluate the performance of the new estimator through extensive simulations and demonstrate its application using the UCI Bike Sharing dataset to predict the count of bike rentals. The results show that despite relying on a biased imputation model, the estimator is efficient and achieves lower prediction error, provided the bias is within a reasonable range.
Read the full abstract

Statistics Applications · Cross-listed from cs.HC

Making Absence Visible: The Roles of Reference and Prompting in Recognizing Missing Information

Interactive systems that explain data, or support decision making often emphasize what is present while overlooking what is expected but missing. This presence bias limits users' ability to form complete mental models of a dataset or situation. Detecting absence depends on expectations about what should be there, yet interfaces rarely help users form such expectations. We present an experimental study examining how reference framing and prompting influence people's ability to recognize expected but missing categories in datasets. Participants compared distributions across three domains (energy, wealth, and regime) under two reference conditions: Global, presenting a unified population baseline, and Partial, showing several concrete exemplars. Results indicate that absence detection was higher with Partial reference than with Global reference, suggesting that partial, samples-based framing can support expectation formation and absence detection. When participants were prompted to look for what was missing, absence detection rose sharply. We discuss implications for interactive user interfaces and expectation-based visualization design, while considering cognitive trade-offs of reference structures and guided attention.
Read the full abstract

All authors Hagit Ben Shoshan, Joel Lanir, Pavel Goldstein, Osnat Mokryn

Statistics Theory

Wasserstein Concentration of Empirical Measures for Dependent Data via the Method of Moments

We establish a general concentration result for the 1-Wasserstein distance between the empirical measure of a sequence of random variables and its expectation. Unlike standard results that rely on independence (e.g., Sanov's theorem) or specific mixing conditions, our result requires only two conditions: (1) control over the variance of the empirical moments, and (2) a flexible tail condition we term $\Psi_{r_n}$-sub-Gaussianity. This approach allows for significant dependencies between variables, provided their algebraic moments behave predictably. The proof uses the method of moments combined with a polynomial approximation of Lipschitz functions via Jackson kernels, allowing us to translate moment concentration into topological concentration.
Read the full abstract

Statistical Methodology

Principal component-guided sparse reduced-rank regression

Reduced-rank regression estimates regression coefficients by imposing a low-rank constraint on the matrix of regression coefficients, thereby accounting for correlations among response variables. To further improve predictive accuracy and model interpretability, several regularized reduced-rank regression methods have been proposed. However, these existing methods cannot bias the regression coefficients toward the leading principal component directions while accounting for the correlation structure among explanatory variables. In addition, when the explanatory variables exhibit a group structure, the correlation structure within each group cannot be adequately incorporated. To overcome these limitations, we propose a new method that introduces pcLasso into the reduced-rank regression framework. The proposed method improves predictive accuracy by accounting for the correlation among response variables while strongly biasing the matrix of regression coefficients toward principal component directions with large variance. Furthermore, even in settings where the explanatory variables possess a group structure, the proposed method is capable of explicitly incorporating this structure into the estimation process. Finally, we illustrate the effectiveness of the proposed method through numerical simulations and real data application.
Read the full abstract

All authors Kanji Goto, Shintaro Yuki, Kensuke Tanioka, Hiroshi Yadohisa

Statistics Theory · Cross-listed from math.PR

Approximate FKG inequalities for phase-bound spin systems, with applications to central limit theorems for exponential random graphs

The Fortuin-Kasteleyn-Ginibre (FKG) inequality is an invaluable tool in monotone spin systems satisfying the FKG lattice condition, which provides positive correlations for all coordinate-wise increasing functions of spins. This inequality has numerous applications and plays an integral role in the proof of various central limit theorems (CLTs), including recent work on ferromagnetic exponential random graph models (ERGMs) wherein a Hamiltonian tilt promotes the presence of small subgraphs like triangles. However, the FKG lattice condition fails to hold when confining a spin system to a particular phase in the low-temperature regime of parameters. Thus it is not a priori clear if each phase internally has positive correlations for increasing functions, or if the positive correlations in the overall model (which is a mixture of phases) arise primarily from the global choice of phase. In this article, we show that the individual phases in ERGMs do indeed satisfy an approximate form of the FKG inequality internally. We use this to finish the proof of various CLTs within each individual phase in the phase-coexistence regime, answering a question posed by Bianchi, Collet, and Magnanini. We present the FKG inequality for ERGMs as a consequence of a more general result which holds under certain inputs related to metastable mixing; we expect this general result to be widely applicable, and we devote a section to spelling out the details of its application to a class of generalized higher-order ferromagnetic Curie-Weiss models where the necessary inputs are relatively transparent.
Read the full abstract

Statistical Methodology

The Covariate-Assisted Bayesian Intransitive Bradley-Terry Model via Combinatorial Hodge Theory

Pairwise comparison data are widely used to recover latent rankings, yet the models in dominant use assume stochastic transitivity. When preferences are in fact intransitive, a single scalar strength conflates genuine hierarchy with cycle-induced structure, biasing both the recovered ranking and any covariate effects attributed to it. To address this limitation, we propose the Covariate-Assisted Bayesian Intransitive Bradley-Terry (CA-BIBT) model, which uses a combinatorial Hodge decomposition to resolve the latent match-up into identifiable and mutually orthogonal flows, attributing the component lying in the covariate-induced subspace to observed covariates and assigning the remaining components to the residuals. A global-local shrinkage prior on the residual cycle-induced flow adapts the model from transitive to intransitive regimes without prespecifying the regime, and a Gibbs sampler yields, as posterior byproducts, calibrated uncertainty for each flow, the posterior probability of the level at which the entities are rankable, and two complementary decision summaries that remain well-defined under intransitivity. In simulations, the CA-BIBT model recovers all flow components accurately with near-nominal coverage, and in applications to two animal dominance datasets, it distinguishes covariate-induced from residual cyclic dominance while quantifying posterior uncertainty, and demonstrates the practical utility of two complementary decision summaries.
Read the full abstract

Machine Learning (Statistics)

Optimal Transport under Group Fairness Constraints

Ensuring fairness in matching algorithms is a key challenge in allocating scarce resources and positions. Focusing on Optimal Transport (OT), we introduce a novel notion of group fairness requiring that the probability of matching two individuals from any two given groups in the OT plan satisfies a predefined target. We first propose a modified Sinkhorn algorithm to compute perfectly fair transport plans efficiently. Since exact fairness can significantly degrade matching quality in practice, we then develop two relaxation strategies. The first one involves solving a penalized OT problem, for which we derive novel finite-sample complexity guarantees. Our second strategy leverages bilevel optimization to learn a ground cost that induces a fair OT solution, and we establish a bound on the deviation of fairness when matching unseen data. Finally, we present empirical results illustrating the performance of our approaches and the trade-off between fairness and transport cost.
Read the full abstract

All authors Linus Bleistein, Mathieu Dagréou, Francisco Andrade, Thomas Boudou, Aurélien Bellet

Sunday 11 January

Statistical Methodology

Robust Bayesian Optimization via Tempered Posteriors

Bayesian optimization (BO) iteratively fits a Gaussian process (GP) surrogate to accumulated evaluations and selects new queries via an acquisition function. Under local misspecification, this feedback loop can produce overconfidence precisely in the region guiding subsequent decisions. We develop a tempered GP-based BO framework that raises the likelihood to a power $\alpha\in(0,1]$. For a generalized family of improvement acquisitions indexed by $g$, including probability of improvement (PI, $g=0$) and expected improvement (EI, $g=1$), we derive finite-time cumulative regret bounds with adaptively learned kernel hyperparameters. The analysis shows that tempering reduces the noise-driven confidence and information-gain contributions to regret, while a deterministic RKHS term prevents arbitrarily aggressive tempering from being uniformly beneficial. It also clarifies the role of the acquisition function: positive-order $g$-EI rules preserve the usual information-gain regret behavior, whereas zero-jitter PI is more exploitative and admits a weaker worst-case guarantee. Motivated by our theoretic findings, we propose a prequential procedure for selecting $\alpha$ online: it decreases $\alpha$ when realized prediction errors exceed model-implied uncertainty and returns $\alpha$ toward one as calibration improves. Empirical results demonstrate that tempering provides a practical yet theoretically grounded tool for stabilizing BO surrogates under localized sampling.
Read the full abstract

Machine Learning (Statistics)

Robust Mean Estimation under Quantization

We consider the problem of mean estimation under quantization and adversarial corruption. We construct multivariate robust estimators that are optimal up to logarithmic factors in two different settings. The first is a one-bit setting, where each bit depends only on a single sample, and the second is a partial quantization setting, in which the estimator may use a small fraction of unquantized data.
Read the full abstract

Statistical Computation

FormulaCompiler.jl and Margins.jl: Efficient Marginal Effects in Julia

Marginal effects analysis is fundamental to interpreting statistical models, yet existing implementations face computational constraints that limit analysis at scale. We introduce two Julia packages that address this gap. Margins.jl provides a clean two-function API organizing analysis around a 2-by-2 framework: evaluation context (population vs profile) by analytical target (effects vs predictions). The package supports interaction analysis through second differences, elasticity measures, categorical mixtures for representative profiles, and robust standard errors. FormulaCompiler.jl provides the computational foundation, transforming statistical formulas into zero-allocation, type-specialized evaluators that enable O(p) per-row computation independent of dataset size. Together, these packages achieve 622x average speedup and 460x memory reduction compared to R's marginaleffects package, with successful computation of average marginal effects and delta-method standard errors on 500,000 observations where R fails due to memory exhaustion, providing the first comprehensive and efficient marginal effects implementation for Julia's statistical ecosystem.
Read the full abstract

Machine Learning (Statistics)

Local EGOP for Continuous Index Learning

We introduce the setting of continuous index learning, in which a function of many variables varies only along a small number of directions at each point. For efficient estimation, it is beneficial for a learning algorithm to adapt, near each point $x$, to the subspace that captures the local variability of the function $f$. We pose this task as kernel adaptation along a manifold with noise, and introduce Local EGOP learning, a recursive algorithm that utilizes the Expected Gradient Outer Product (EGOP) quadratic form as both a metric and inverse-covariance of our target distribution. We prove that Local EGOP learning adapts to the regularity of the function of interest, showing that under a supervised noisy manifold hypothesis, intrinsic dimensional learning rates are achieved for arbitrarily high-dimensional noise. Empirically, we compare our algorithm to the feature learning capabilities of deep learning. Additionally, we demonstrate improved regression quality compared to two-layer neural networks in the continuous single-index setting.
Read the full abstract

All authors Alex Kokot, Anand Hemmady, Vydhourie Thiyageswaran, Marina Meila

Statistical Methodology · Cross-listed from econ.EM

Empirical Bayes Estimation in Heterogeneous Coefficient Panel Models

We develop an empirical Bayes (EB) G-modeling framework for short-panel linear models with nonparametric prior for the random intercepts, slopes, dynamics, and non-spherical error variances. We establish identification and consistency of the nonparametric maximum likelihood estimator (NPMLE) under general conditions, and provide low-level sufficient conditions for several models of empirical interest. Conditions for regret consistency of the EB estimators are also established. The NPMLE is computed using a Wasserstein-Fisher-Rao gradient flow algorithm adapted to panel regressions. Using data from the Panel Study of Income Dynamics, we find that the slope coefficient for potential experience is substantially heterogeneous and negatively correlated with the random intercept, and that error variances and autoregressive coefficients vary significantly across individuals. The EB estimates reduce mean squared prediction errors relative to individual maximum likelihood estimates.
Read the full abstract

Statistical Methodology

Semiparametric Analysis of Interval-Censored Data Subject to Inaccurate Diagnoses with A Terminal Event

Interval-censoring frequently occurs in studies of chronic diseases where disease status is inferred from intermittently collected biomarkers. Although many methods have been developed to analyze such data, they typically assume perfect disease diagnosis, which often does not hold in practice due to the inherent imperfect clinical diagnosis of cognitive functions or measurement errors of biomarkers such as cerebrospinal fluid. In this work, we introduce a semiparametric modeling framework using the Cox proportional hazards model to address interval-censored data in the presence of inaccurate disease diagnosis. Our model incorporates sensitivity and specificity of the diagnosis to account for uncertainty in whether the interval truly contains the disease onset. Furthermore, the framework accommodates scenarios involving a terminal event and when diagnosis is accurate, such as through postmortem analysis. We propose a nonparametric maximum likelihood estimation method for inference and develop an efficient EM algorithm to ensure computational feasibility. The regression coefficient estimators are shown to be asymptotically normal, achieving semiparametric efficiency bounds. We further validate our approach through extensive simulation studies and an application assessing Alzheimer's disease (AD) risk. We find that amyloid-beta is significantly associated with AD, but Tau is predictive of both AD and mortality.
Read the full abstract

Statistics Applications · Cross-listed from math-ph

A PDE approach for the invariant measure of stochastic oscillators with hysteresis

This paper presents a PDE approach as an alternative to Monte Carlo simulations for computing the invariant measure of a white-noise-driven bilinear oscillator with hysteresis. This model is widely used in engineering to represent highly nonlinear dynamics, such as the Bauschinger effect. The study extends the stochastic elasto-plastic framework of Bensoussan et al. [SIAM J. Numer. Anal. 47 (2009), pp. 3374–3396] from the two-dimensional elasto-perfectly-plastic oscillator to the three-dimensional bilinear elasto-plastic oscillator. By constructing an appropriate Lyapunov function, the existence of an invariant measure is established. This extension thus enables the modelling of richer hysteretic behavior and broadens the scope of PDE alternatives to Monte Carlo methods. Two applications demonstrate the method's efficiency: calculating the oscillator's threshold crossing frequency (providing an alternative to Rice's formula) and probability of serviceability.
Read the full abstract

Machine Learning (Statistics)

Conditional Normalizing Flows for Forward and Backward Joint State and Parameter Estimation

Traditional filtering algorithms for state estimation – such as classical Kalman filtering, unscented Kalman filtering, and particle filters – show performance degradation when applied to nonlinear systems whose uncertainty follows arbitrary non-Gaussian, and potentially multi-modal distributions. This study reviews recent approaches to state estimation via nonlinear filtering based on conditional normalizing flows, where the conditional embedding is generated by standard MLP architectures, transformers or selective state-space models (like Mamba-SSM). In addition, we test the effectiveness of an optimal-transport-inspired kinetic loss term in mitigating overparameterization in flows consisting of a large collection of transformations. We investigate the performance of these approaches on applications relevant to autonomous driving and patient population dynamics, paying special attention to how they handle time inversion and chained predictions. Finally, we assess the performance of various conditioning strategies for an application to real-world COVID-19 joint SIR system forecasting and parameter estimation.
Read the full abstract

Statistical Methodology

Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests

Random forests (RFs) are widely used for prediction and variable importance analysis and are often believed to capture any types of interactions via recursive splitting. However, since the splits are chosen locally, interactions are only reliably captured when at least one involved covariate has a marginal effect. We introduce unity forests (UFOs), an RF variant designed to better exploit interactions involving covariates without marginal effects. In UFOs, the first few splits of each tree are optimized jointly across a random covariate subset to form a "tree root" capturing such interactions; the remainder is grown conventionally. We further propose the unity variable importance measure (VIM), which is based on out-of-bag split criterion values from the tree roots. Here, only a small fraction of tree root splits with the highest in-bag criterion values are considered per covariate, reflecting that covariates with purely interaction-based effects are discriminative only if a split in an interacting covariate occurred earlier in the tree. Finally, we introduce covariate-representative tree roots (CRTRs), which select representative tree roots per covariate and provide interpretable insight into the conditions - marginal or interactive - under which each covariate has its strongest effects. In a simulation study, the unity VIM reliably identified interacting covariates without marginal effects, unlike conventional RF-based VIMs. In a large-scale real-data comparison, UFOs achieved higher discrimination and predictive accuracy than standard RFs, with comparable calibration. The CRTRs reproduced the covariates' true effect types reliably in simulated data and provided interesting insights in a real data analysis.
Read the full abstract

Statistical Methodology

Localization Estimator for High Dimensional Tensor Covariance Matrices

This paper considers covariance matrix estimation of tensor data under high dimensionality. A multi-bandable covariance class is established to accommodate the need for complex covariance structures of multi-layer lattices and general covariance decay patterns. We propose a high dimensional covariance localization estimator for tensor data, which regulates the sample covariance matrix through a localization function. The statistical properties of the proposed estimator are studied by deriving the minimax rates of convergence under the spectral and the Frobenius norms. Numerical experiments and real data analysis on ocean eddy data are carried out to illustrate the utility of the proposed method in practice.
Read the full abstract

Machine Learning (Statistics)

Match Made with Matrix Completion: Efficient Learning under Matching Interference

Matching markets face increasing needs to learn the matching qualities between demand and supply for effective design of matching policies. In practice, the matching rewards are high-dimensional due to the growing diversity of participants. We leverage a natural low-rank matrix structure of the matching rewards in these two-sided markets, and propose to utilize matrix completion to accelerate reward learning with limited offline data. A unique property for matrix completion in this setting is that the entries of the reward matrix are observed with matching interference – i.e., the entries are not observed independently but dependently due to matching or budget constraints. Such matching dependence renders unique technical challenges, such as sub-optimality or inapplicability of the existing analytical tools in the matrix completion literature, since they typically rely on sample independence. In this paper, we first show that standard nuclear norm regularization remains theoretically effective under matching interference. We provide a near-optimal Frobenius norm guarantee in this setting, coupled with a new analytical technique. Next, to guide certain matching decisions, we develop a novel “double-enhanced” estimator, based off the nuclear norm estimator, with a near-optimal entry-wise guarantee. Our double-enhancement procedure can apply to broader sampling schemes even with dependence, which may be of independent interest. Additionally, we extend our approach to online learning settings with matching constraints such as optimal matching and stable matching, and present improved regret bounds in matrix dimensions. Finally, we demonstrate the practical value of our methods using both synthetic data and real data of labor markets.
Read the full abstract

Machine Learning (Statistics)

The Impact of Anisotropic Covariance Structure on the Training Dynamics and Generalization Error of Linear Networks

The success of deep neural networks largely depends on the statistical structure of the training data. While learning dynamics and generalization on isotropic data are well-established, the impact of pronounced anisotropy on these crucial aspects is not yet fully understood. We examine the impact of data anisotropy, represented by a spiked covariance structure, a canonical yet tractable model, on the learning dynamics and generalization error of a two-layer linear network in a linear regression setting. Our analysis reveals that the learning dynamics proceed in two distinct phases, governed initially by the input-output correlation and subsequently by other principal directions of the data structure. Furthermore, we derive an analytical expression for the generalization error, quantifying how the alignment of the spike structure of the data with the learning task improves performance. Our findings offer deep theoretical insights into how data anisotropy shapes the learning trajectory and final performance, providing a foundation for understanding complex interactions in more advanced network architectures.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Tractable Multinomial Logit Contextual Bandits with Non-Linear Utilities

We study the multinomial logit (MNL) contextual bandit problem for sequential assortment selection. Although most existing research assumes utility functions to be linear in item features, this linearity assumption restricts the modeling of intricate interactions between items and user preferences. A recent work (Zhang & Luo, 2024) has investigated general utility function classes, yet its method faces fundamental trade-offs between computational tractability and statistical efficiency. To address this limitation, we propose a computationally efficient algorithm for MNL contextual bandits leveraging the upper confidence bound principle, specifically designed for non-linear parametric utility functions, including those modeled by neural networks. Under a realizability assumption and a mild geometric condition on the utility function class, our algorithm achieves a regret bound of $\tilde{O}(\sqrt{T})$, where $T$ denotes the total number of rounds. Our result establishes that sharp $\tilde{O}(\sqrt{T})$-regret is attainable even with neural network-based utilities, without relying on strong assumptions such as neural tangent kernel approximations. To the best of our knowledge, our proposed method is the first computationally tractable algorithm for MNL contextual bandits with non-linear utilities that provably attains $\tilde{O}(\sqrt{T})$ regret. Comprehensive numerical experiments validate the effectiveness of our approach, showing robust performance not only in realizable settings but also in scenarios with model misspecification.
Read the full abstract

Statistical Methodology

Minimum information Markov model

The analysis of high-dimensional time series data has become increasingly important across a wide range of fields. Recently, a method for constructing the minimum information Markov kernel on finite state spaces was established. In this study, we propose a statistical model based on a parametrization of its dependence function, which we call the Minimum Information Markov Model. We show that its parametrization induces an orthogonal structure between the stationary distribution and the dependence function, and that the model arises as the optimal solution to a divergence rate minimization problem. In particular, for the Gaussian autoregressive case, we establish the existence of the optimal solution to this minimization problem, a nontrivial result requiring a rigorous proof. For parameter estimation, our approach exploits the conditional independence structure inherent in the model, which is supported by the orthogonality. Specifically, we develop several estimators, including conditional likelihood and pseudo likelihood estimators, for the minimum information Markov model in both univariate and multivariate settings. We demonstrate their practical performance through simulation studies and applications to real-world time series data.
Read the full abstract

Statistical Methodology

Likelihood-Based Regression for Weibull Accelerated Life Testing Model Under Censored Data

In this paper, we investigate accelerated life testing (ALT) models based on the Weibull distribution with stress-dependent shape and scale parameters. Temperature and voltage are treated as stress variables influencing the lifetime distribution. Data are assumed to be collected under Progressive Hybrid Censoring (PHC) and Adaptive Progressive Hybrid Censoring (APHC). A two-step estimation framework is developed. First, the Weibull parameters are estimated via maximum likelihood, and the consistency and asymptotic normality of the estimators are established under both censoring schemes. Second, the resulting parameter estimates are linked to the stress variables through a regression model to quantify the stress-lifetime relationship. Extensive simulations are conducted to examine finite-sample performance under a range of parameter settings, and a data illustration is also presented to showcase practical relevance. The proposed framework provides a flexible approach for modeling stress-dependent reliability behavior in ALT studies under complex censoring schemes.
Read the full abstract

All authors Rahul Konar, Ramnivas Jat, Neeraj Joshi, Raghu Nandan Sengupta

Machine Learning (Statistics) · Cross-listed from cs.LG

Variational decomposition autoencoding improves disentanglement of latent representations

Understanding the structure of complex, nonstationary, high-dimensional time-evolving signals is a central challenge in scientific data analysis. In many domains, such as speech and biomedical signal processing, the ability to learn disentangled and interpretable representations is critical for uncovering latent generative mechanisms. Traditional approaches to unsupervised representation learning, including variational autoencoders (VAEs), often struggle to capture the temporal and spectral diversity inherent in such data. Here we introduce variational decomposition autoencoding (VDA), a framework that incorporates a strong structural bias toward signal decomposition into variational representation learning. VDA is instantiated through variational decomposition autoencoders (DecVAEs), i.e., encoder-only neural networks that perform latent autoencoding at the level of distributions to learn multiple latent subspaces aligned with time-frequency characteristics. We evaluate DecVAEs on simulated data and three publicly available scientific datasets, spanning speech recognition, dysarthric speech evaluation, and emotional speech classification. DecVAEs improve on VAE-based methods in disentanglement quality and robustness, and their latent subspaces become factor-specific when the generative factors have distinct spectral signatures. These findings position decomposition-aware architectures as potential tools for extracting structured representations from speech, and potentially from other dynamic signals with factors exhibiting distinct spectral signatures, with applications in clinical diagnostics, human-computer interaction, and adaptive neurotechnologies.
Read the full abstract

All authors Ioannis N. Ziogas, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis

Machine Learning (Statistics)

Constrained Density Estimation via Optimal Transport

A novel framework for density estimation under expectation constraints is proposed. The framework minimizes the Wasserstein distance between the estimated density and a prior, subject to the constraints that the expected value of a set of functions adopts or exceeds given values. The framework is generalized to include regularization inequalities to mitigate the artifacts in the target measure. An annealing-like algorithm is developed to address non-smooth constraints, with its effectiveness demonstrated through both synthetic and proof-of-concept real world examples in finance.
Read the full abstract