Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 2 March

Statistical Methodology

Topological Causal Effects

Estimating causal effects is particularly challenging when outcomes arise in complex, non-Euclidean spaces, where conventional methods often fail to capture meaningful structural variation. We develop a framework for topological causal inference that defines treatment effects through differences in the topological structure of potential outcomes, summarized by power-weighted silhouette functions of persistence diagrams. We develop an efficient, doubly robust estimator in a fully nonparametric model, establish functional weak convergence, and construct a formal test of the null hypothesis of no topological effect. Empirical studies illustrate that the proposed method reliably quantifies topological treatment effects across diverse complex outcome types.
Read the full abstract

Statistical Methodology

A Simpson Based Estimation Approach for the Overlapping Coefficient of k>=2 Normal Distributions

The overlapping coefficient is a fundamental measure of similarity between probability distributions. While the case of two distributions has been extensively studied, extending this measure to multiple populations presents both analytical and computational challenges. In this paper, we propose a general estimation framework for the overlapping coefficient of k>=2 normal distributions. The method employs Simpsons numerical integration rule combined with plug-in maximum likelihood estimators of the normal parameters. The resulting estimator is shown to be consistent under standard regularity conditions. A Monte Carlo simulation study is conducted across various overlap scenarios and sample sizes. The results demonstrate that the proposed Simpson based estimator performs competitively for all overlap levels, with notable advantages in low overlap situations. This methodology offers a flexible and computationally efficient approach applicable to an arbitrary number of normal populations.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions

Selective conformal prediction can yield substantially tighter uncertainty sets when we can identify calibration examples that are exchangeable with the test example. In interventional settings, such as perturbation experiments in genomics, exchangeability often holds only within subsets of interventions that leave a target variable "unaffected" (e.g., non-descendants of an intervened node in a causal graph). We study the practical regime where this invariance structure is unknown and must be estimated from data. Our main result quantifies how coverage degrades when the estimated safe calibration set accidentally includes interventions that affect the target, and gives a conservative correction when an upper bound on this error is available. Rather than learning a full causal graph, we learn only the intervention-target relationships needed to choose calibration interventions. We give algorithms for this partial learning task and evaluate them on synthetic structural equation models and Replogle K562 CRISPR-interference data, where the experiments illustrate synthetic gains from selective calibration and finite-sample tradeoffs on real perturbation screens.
Read the full abstract

Statistics Theory · Cross-listed from cs.AI

Conformal Policy Control

An agent must try new behaviors to explore and improve. In high-stakes environments, an agent that violates safety constraints may cause harm and must be taken offline, curtailing any future interaction. Imitating old behavior is safe, but excessive conservatism discourages exploration. How much behavior change is too much? We show how to use any safe reference policy as a probabilistic regulator for any optimized but untested policy. Conformal calibration on data from the safe policy determines how aggressively the new policy can act, while provably enforcing the user's declared risk tolerance. Unlike conservative optimization methods, we do not assume the user has identified the correct model class nor tuned any hyperparameters. Unlike previous conformal methods, our theory provides finite-sample guarantees even for non-monotonic bounded loss functions, and it introduces a new policy control setting. Our experiments on applications ranging from natural language question answering to biomolecular engineering show that safe exploration is not only possible from the first moment of deployment, but can also improve performance.
Read the full abstract

All authors Drew Prinster, Clara Fannjiang, Ji Won Park, Kyunghyun Cho, Anqi Liu, Suchi Saria, Samuel Stanton

2603.02196 · 2 Mar · International Conference on Machine Learning (ICML), 2026. v4: Revised CPC theory to (a) show enforced smoothness for likelihood-ratio control parameter and (b) for general control parameter, assume smoothness only of constrained policy $\pi_t^{(\beta)}$ w.r.t. $\beta$ (rather than of $(l_i - \alpha)\cdot\pi_t^{(\beta)}$ or of conformal weights)

Statistics Applications

Comparative Analysis of Spatiotemporal Volatility Models: An Empirical Study on Financial Network Series

Various spatiotemporal and network GARCH models have recently been proposed to capture volatility interactions, such as the transmission of market risk across financial networks. These approaches rely heavily on the specification of the adjacency or spatiotemporal weight matrix, for which several alternatives exist in the literature. This paper evaluates the out-of-sample forecasting performance of a range of spatiotemporal volatility models and multivariate GARCH benchmarks under nine alternative network specifications. The empirical analysis uses daily data for 16 sectorally diversified S&P 500 stocks from 22 December 1998 to 20 October 2024. A one-step-ahead forecasting framework is implemented, and models are assessed using BIC, RMSFE, and MAFE, with forecasts evaluated against a single realised volatility proxy based on squared log-returns. The nine spatial weight matrices reflect diverse economic and statistical relationships, including Granger-filtered and EGARCH-based spillovers. Results show that some spatiotemporal models outperform standard GARCH benchmarks in out-of-sample forecasting accuracy. Notably, the Dynamic Spatiotemporal ARCH model achieves the lowest RMSFE and MAFE across all network specifications at minimal computational cost. Pairwise Diebold-Mariano tests confirm significant differences in predictive accuracy. These findings underscore the value of incorporating spatial structure into volatility modelling as a parsimonious and interpretable alternative for financial network analysis.
Read the full abstract

All authors Ariane N. Meli Chrisko, Jessie Li, Philipp Otto, Wolfgang Schmid

Machine Learning (Statistics) · Cross-listed from cs.LG

Symbol-Equivariant Recurrent Reasoning Models

Reasoning problems such as Sudoku and ARC-AGI remain challenging for neural networks. The structured problem solving architecture family of Recurrent Reasoning Models (RRMs), including Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), offer a compact alternative to large language models, but currently handle symbol symmetries only implicitly via costly data augmentation. We introduce Symbol-Equivariant Recurrent Reasoning Models (SE-RRMs), which enforce permutation equivariance at the architectural level through symbol-equivariant layers, guaranteeing identical solutions under symbol or color permutations. SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize from just training on 9x9 to smaller 4x4 and larger 16x16 and 25x25 instances, to which existing RRMs cannot extrapolate. On ARC-AGI-1 and ARC-AGI-2, SE-RRMs achieve competitive performance with substantially less data augmentation and only 2 million parameters, demonstrating that explicitly encoding symmetry improves the robustness and scalability of neural reasoning. Code is available at https://github.com/ml-jku/SE-RRM.
Read the full abstract

All authors Richard Freinschlag, Timo Bertram, Erich Kobler, Andreas Mayr, Günter Klambauer

Statistics Theory

Algebraic statistics of Hüsler-Reiss graphical models in multivariate extremes

The field of extreme value statistics is concerned with modeling and predicting rare events. In a Hüsler-Reiss graphical model, a graph represents extremal conditional independence (CI) relations between random variables. These models are exponential families parameterized by a graph Laplacian and are considered the analogue of multivariate Gaussian models in the extremal setting. We study these models from the perspective of algebraic geometry. Translating the CI relations into polynomial constraints in the parameters, we define extremal CI ideals and find a determinantal representation of their generators. In terms of parametric inference, we study the extremal maximum likelihood degree as the number of solutions to a conditionally negative definite matrix completion problem. We also define and analyze the extremal maximum likelihood threshold for Hüsler-Reiss graphical models, which provides a certificate for the existence of a surrogate MLE in terms of the dimensionality of the point configuration that realizes the underlying summary statistic as a Euclidean distance matrix. We highlight throughout many interesting similarities but also differences with respect to Gaussian graphical models.
Read the full abstract

All authors Carlos Améndola, Jane Ivy Coons, Alexandros Grosdos, Frank Röttger

Machine Learning (Statistics) · Cross-listed from cs.LG

Reservoir Subspace Injection for Online ICA under Top-n Whitening

Reservoir expansion can improve online independent component analysis (ICA) under nonlinear mixing, yet top-$n$ whitening may discard injected features. We formalize this bottleneck as reservoir subspace injection (RSI): injected features help only if they enter the retained eigenspace without displacing passthrough directions. RSI diagnostics (IER, SSO, $\rho_x$) identify a failure mode in our top-$n$ setting: stronger injection increases IER but crowds out passthrough energy ($\rho_x: 1.00\!\rightarrow\!0.77$), degrading SI-SDR by up to $2.2$\,dB. A guarded RSI controller preserves passthrough retention and recovers mean performance to within $0.1$\,dB of baseline $1/N$ scaling. With passthrough preserved, RE-OICA improves over vanilla online ICA by $+1.7$\,dB under nonlinear mixing and achieves positive SI-SDR$_{\mathrm{sc}}$ on the tested super-Gaussian benchmark ($+0.6$\,dB).
Read the full abstract

Statistical Methodology

Setwise Hierarchical Variable Selection and the Generalized Linear Step-Up Procedure for False Discovery Rate Control

Controlling the false discovery rate (FDR) in variable selection becomes challenging when predictors are correlated, as existing methods often exclude all members of correlated groups and consequently perform poorly for prediction. We introduce a new setwise variable-selection framework that identifies clusters of potential predictors rather than forcing selection of a single variable. By allowing any member of a selected set to serve as a surrogate predictor, our approach supports strong predictive performance while maintaining rigorous FDR control. We construct sets via hierarchical clustering of predictors based on correlation, then test whether each set contains any non-null effects. Similar clustering and setwise selection have been applied in the familywise error rate (FWER) control regime, but previous research has been unable to overcome the inherent challenges of extending this to the FDR control framework. To control the FDR, we develop substantial generalizations of linear step-up procedures, extending the Benjamini-Hochberg and Benjamini-Yekutieli methods to accommodate the logical dependencies among these composite hypotheses. We prove that these procedures control the FDR at the nominal level and highlight their broader applicability. Simulation studies and real-data analyses show that our methods achieve higher power than existing approaches while preserving FDR control, yielding more informative variable selections and improved predictive models.
Read the full abstract

Machine Learning (Statistics)

Instrumental and Proximal Causal Inference with Gaussian Processes

Instrumental variable (IV) and proximal causal learning (Proxy) methods are central frameworks for causal inference in the presence of unobserved confounding. Despite substantial methodological advances, existing approaches rarely provide reliable epistemic uncertainty (EU) quantification. We address this gap through a Deconditional Gaussian Process (DGP) framework for uncertainty-aware causal learning. Our formulation recovers popular kernel estimators as the posterior mean, ensuring predictive precision, while the posterior variance yields principled and well-calibrated EU. Moreover, the probabilistic structure enables systematic model selection via marginal log-likelihood optimization. Empirical results demonstrate strong predictive performance alongside informative EU quantification, evaluated via empirical coverage frequencies and decision-aware accuracy rejection curves. Together, our approach provides a unified, practical solution for causal inference under unobserved confounding with reliable uncertainty.
Read the full abstract

All authors Yuqi Zhang, Krikamol Muandet, Dino Sejdinovic, Edwin Fong, Siu Lun Chau

Statistics Theory · Cross-listed from cs.LG

Near-Optimal Regret for KL-Regularized Multi-Armed Bandits

Recent studies have shown that reinforcement learning with KL-regularized objectives can enjoy faster rates of convergence or logarithmic regret, in contrast to the classical $\sqrt{T}$-type regret in the unregularized setting. However, the statistical efficiency of online learning with respect to KL-regularized objectives remains far from completely characterized, even when specialized to multi-armed bandits (MABs). We address this problem for MABs via a sharp analysis of KL-UCB using a novel peeling argument, which yields a $\tilde{O}(\eta K\log^2T)$ upper bound: the first high-probability regret bound with linear dependence on $K$. Here, $T$ is the time horizon, $K$ is the number of arms, $\eta^{-1}$ is the regularization intensity, and $\tilde{O}$ hides all logarithmic factors except those involving $\log T$. The near-tightness of our analysis is certified by the first non-constant lower bound $\Omega(\eta K \log T)$, which follows from subtle hard-instance constructions and a tailored decomposition of the Bayes prior. Moreover, in the low-regularization regime (i.e., large $\eta$), we show that the KL-regularized regret for MABs is $\eta$-independent and scales as $\tilde{\Theta}(\sqrt{KT})$. Overall, our results provide a thorough understanding of KL-regularized MABs across all regimes of $\eta$ and yield nearly optimal bounds in terms of $K$, $\eta$, and $T$.
Read the full abstract

All authors Kaixuan Ji, Qingyue Zhao, Heyang Zhao, Qiwei Di, Quanquan Gu

Statistics Applications

Socio-Spatial Patterns of Suicide Mortality in the United States

Suicide causes more than 49,000 deaths annually in the United States, 55% involving firearms. Suicide mortality varies substantially across US counties, but the role of large-scale social networks remains underexplored. We combine county-level suicide mortality data for 2010-2022 with the Facebook Social Connectedness Index (SCI) and measures of exposure to Extreme Risk Protection Orders (ERPOs). In population-weighted two-way fixed effects models adjusting for sociodemographic and economic characteristics, COVID-19, drug-overdose and alcohol-related mortality, poor mental health days, state firearm policies, and geographical proximity, a one-standard-deviation increase in the SCI-weighted suicide mortality rate of socially connected counties was associated with 2.47 additional deaths per 100,000 residents in the focal county (95% CI [0.97, 4.00]). A one-standard-deviation increase in ERPO social exposure was associated with 0.266 fewer deaths per 100,000 (coefficient -0.266; 95% CI [-0.403, -0.128]) after adjustment for geographical proximity and state-by-year fixed effects. The social-proximity association persisted under alternative neighborhood definitions and spatial-error models, and subgroup analyses showed heterogeneity by age. These findings show that both suicide mortality and the lower mortality associated with exposure to firearm-restriction policies are patterned along inter-county social ties beyond what is explained by geographical proximity alone.
Read the full abstract

All authors Kushagra Tiwari, M. Amin Rahimian, Marie-Laure Charpignon, Philippe J. Giabbanelli, Praveen Kumar

Statistics Applications · Cross-listed from cs.SI

Political attitudes differ but share a common low-dimensional structure across social media and survey data

Does polarization online reflect the state of polarization in society? We study ideological positions and attitudes on several issues in France, a country with documented issue nonalignment. We compare distributions on X/Twitter with a nationally representative sample, focusing on two key properties: ideological polarization and issue alignment. Despite significant issue-wise divergences, positions of both the X population and the nationally representative sample present a similar bi-dimensional structure along two dominant bundles of aligned issues: a Left-Right divide, and a Global-Local divide. We then study how our results vary when accounting for key structural parameters of the online public sphere: activity, popularity, and visibility. We find that the dimensionality of attitude distributions shrinks as ideological polarization increases when selecting more active users. The divergence between political attitudes on social media and in survey data is greatly mediated by the combination of activity and popularity of social media users: users benefiting from the most exposure are also the most representative of the general public. Together, our results shed light on the structural similarities and differences between political attitudes from social media users and the general public.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?

We study scaling laws of signSGD under a power-law random features (PLRF) model that accounts for both feature and target decay. We analyze the population risk of a linear model trained with one-pass signSGD on Gaussian-sketched features. We express the risk as a function of model size, training steps, learning rate, and the feature and target decay parameters. Comparing against the SGD risk analyzed by Paquette et al. (2024), we identify a drift-normalization effect and a noise-reshaping effect unique to signSGD. We then obtain compute-optimal scaling laws under the optimal choice of learning rate. Our analysis shows that the noise-reshaping effect can make the compute-optimal slope of signSGD steeper than that of SGD in regimes where noise is dominant. Finally, we observe that the widely used warmup-stable-decay (WSD) schedule further reduces the noise term and sharpens the compute-optimal slope, when feature decay is fast but target decay is slow.
Read the full abstract

Machine Learning (Statistics)

TRAKNN: Efficient Trajectory Aware Spatiotemporal kNN for Rare Meteorological Trajectory Detection

Extreme weather events, such as windstorms and heatwaves, are driven by persistent atmospheric circulation patterns that evolve over several consecutive days. While traditional circulation-based studies often focus on instantaneous atmospheric states, capturing the temporal evolution, or trajectory, of these spatial fields is essential for characterizing rare and potentially impactful atmospheric behavior. However, performing an exhaustive similarity search on multi-decadal, continental-scale gridded datasets presents significant computational and memory challenges. In this paper, we propose TRAKNN (TRajectory Aware KNN), a fully unsupervised and data-agnostic framework for detecting geometrically rare short trajectories in spatio-temporal data with an exact kNN approach. TRAKNN leverages a recurrence-based algorithm that decouples computational complexity from trajectory length and efficient batch operations, maximizing computational intensity. These optimizations enable exhaustive analysis on standard workstations, either on CPU or on GPU. We evaluate our approach on 75 years of daily European sea-level pressure data. Our results illustrate that rare trajectories identified by TRAKNN correspond to physically coherent atmospheric anomalies and align with independent extreme-event databases.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Multiplicative Oracle Inequalities for Transductive Learning via Level-Set Aggregation

We revisit transductive learning where predictions are made with the set of all covariates known in advance. In the leave-one-out (LOO) setting, the prediction is made with labels of the remaining sample points and evaluated by the average error. In particular, we study multiplicative oracle inequalities for agnostic transductive LOO prediction for a variety of tasks, including classification with 0-1 loss, squared loss regression, density estimation, and logistic regression. Specifically, we introduce Median of Level-Set Aggregation (MLSA), an aggregation procedure built on near-ERM level sets (i.e., empirical-risk level sets around the ERM). We prove a general multiplicative oracle inequality for the LOO error of the form \[ LOO_S(MLSA) \;\le\; C \left( \frac{1}{n} \min_{h\in H} L_S(h) \;+\; \frac{\log |H|}{n}\right), \qquad C>1, \] where $H$ is the hypothesis/function class. This inequality holds for hypothesis classes under a local level-set growth condition together with losses satisfying a mild monotonicity assumption. For classification with VC classes under the $0$–$1$ loss, the $\log |H|$ factor can be improved to be $d\log n$, where $d$ is the VC dimension, recovering Long (1998) up to a $\log n$ factor. For logistic regression with bounded covariates and parameters, the $\log |H|$ factor can be improved to be $d\log n$ up to problem-dependent factors, where $d$ is the ambient dimension.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.AI

Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization

Moving beyond evaluations that collapse performance across heterogeneous prompts toward fine-grained evaluation at the prompt level, or within relatively homogeneous subsets, is necessary to diagnose generative models' strengths and weaknesses. Such fine-grained evaluations, however, suffer from a data bottleneck: human gold-standard labels are too costly at this scale, while automated ratings are often misaligned with human judgment. To resolve this challenge, we propose a novel statistical model based on tensor factorization that merges cheap autorater data with a limited set of human gold-standard labels. Specifically, our approach uses autorater scores to pretrain latent representations of prompts and generative models, and then aligns those pretrained representations to human preferences using a small calibration set. This sample-efficient methodology is robust to autorater quality, more accurately predicts human preferences on a per-prompt basis than standard baselines, and provides tight confidence intervals for key statistical parameters of interest. We also showcase the practical utility of our method by constructing granular leaderboards based on prompt qualities and by estimating model performance solely from autorater scores, eliminating the need for additional human annotations.
Read the full abstract

All authors Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque

Machine Learning (Statistics) · Cross-listed from cs.LG

Noise-Calibrated Inference from Differentially Private Sufficient Statistics in Exponential Families

Many differentially private (DP) data release systems either output DP synthetic data and leave analysts to perform inference as usual, which can lead to severe miscalibration, or output a DP point estimate without a principled way to do uncertainty quantification. This paper develops a clean and tractable middle ground for exponential families: release only DP sufficient statistics, then perform noise-calibrated likelihood-based inference and optional parametric synthetic data generation as post-processing. Our contributions are: (1) a general recipe for approximate-DP release of clipped sufficient statistics under the Gaussian mechanism; (2) asymptotic normality, explicit variance inflation, and valid Wald-style confidence intervals for the plug-in DP MLE; (3) a noise-aware likelihood correction that is first-order equivalent to the plug-in but supports bootstrap-based intervals; and (4) a matching minimax lower bound showing the privacy distortion rate is unavoidable. The resulting theory yields concrete design rules and a practical pipeline for releasing DP synthetic data with principled uncertainty quantification, validated on three exponential families and real census data.
Read the full abstract

Statistical Methodology

Analysis of Stepped-Wedge Randomised Cluster Trial using a generalized pairwise comparison approach : a simulation study

Stepped-wedge cluster randomised trials (SW-CRTs) increasingly evaluate complex interventions, yet methodological guidance for analysing composite endpoints using generalized pairwise comparisons (GPC)remains limited. This work investigates the performance of several GPC-based estimators in the presence of clustering, temporal trends, and varying correlation structures typical of SW-CRTs. We conducted an extensive simulation study covering a range of intraclass correlations (ICC), cluster autocorrelation coefficients (CAC), time effects, and treatment effect sizes. Eight analytical approaches were compared, including unadjusted estimators, cluster-stratified win odds, mixed-effects models applied to cluster-period win odds, and probabilistic index models (PIMs). Type I error control was strongly compromised for methods ignoring time or clustering, whereas only two approaches consistently maintained nominal error rates: a hierarchical mixed-effects model with sequence and cluster-level random slopes (b4) and a cluster-restricted PIM (c2). These two methods were further evaluated in terms of statistical power, where c2 generally showed higher efficiency, particularly under strong clustering, low CAC, or the presence of temporal trends, while both converged to similar performance for large treatment effects. Overall, our findings identify b4 and c2 as the most reliable GPC-based strategies for SW-CRT analysis and provide practical guidance for their application, including for ongoing trials such as ETHER.
Read the full abstract

All authors Yohan Bard, Emilie Presles, Marc Buyse, Silvy Laporte, Paul Zufferey, Frederikus A. Klok, Olivier Sanchez, Francis Couturaud, Edouard Ollier

Statistics Theory

Wasserstein-based identification of metastable states in time series data via change point detection and segment clustering

Change point detection for time series analysis is a difficult and important problem in applied statistics, for which a variety of approaches have been developed in the past several decades. Here, the Wasserstein metric is employed as a tool for change-point identification in multi-dimensional time series data in order to identify clusters in time series in an unsupervised way. We leverage the simplicity of the optimal transport cost in the 1-dimensional setting to quickly identify both a segmentation (family of change points for a trajectory) and a clustering for the data when the number of segments is much smaller than the number of data points, making no parametric assumptions about the particular distributions involved. Our change point detection method scales linearly in the size of the data and in the dimension of the samples. We test our approach on idealized synthetic data trajectories, as well as real world trajectories coming from the domain of molecular dynamics simulations and underwater acoustics. We find that segmenting these time series via change points obtained by estimating the Wasserstein metric derivative and then clustering the identified segments as measures with similarity measured by the Wasserstein metric, successfully identifies metastable states in the law of the processes.
Read the full abstract

Statistics Applications

Quantifying Uncertainty in Void Swelling Prediction: A Conformal Prediction Framework for Reactor Safety Margins

Irradiation-induced void swelling is a critical degradation mechanism for structural materials in nuclear reactors, dictating component operational lifespan and safety. While recent machine learning (ML) approaches have improved the accuracy of swelling rate predictions, they often fail to account for the inherent stochasticity of radiation damage, providing point estimates without rigorous uncertainty quantification. This lack of probabilistic context limits their applications in materials qualification, reactor licensing and risk assessment. In this work, we develop a framework that integrates ensemble ML models with Conformal Prediction (CP) to generate statistically calibrated prediction intervals. Unlike standard error estimation or Bayesian methods that often rely on rigid distributional assumptions, this approach specifically addresses the physical heteroscedasticity of swelling data, where variance transitions from the nucleation-dominated incubation regime to the growth-dominated steady-state regime. We demonstrate that log-transformed conformal prediction inference provides valid empirical coverage consistent with target confidence levels even in sparse data regimes. This framework offers a pathway to replace overly conservative upper-bound curves with Probabilistic Risk Assessment (PRA) tools for high-dose reactor core internals.
Read the full abstract

Machine Learning (Statistics)

Density-Matrix Spectral Embeddings for Categorical Data: Operator Structure and Stability

We introduce a supervised dimensionality reduction methodology for categorical (and discretized mixed-type) data based on a density-matrix construction induced by class-conditional frequencies. Given a labeled dataset encoded in a one-hot survey space, we assemble a frequency matrix whose columns aggregate feature occurrences within each class, and define a normalized Gram-type operator that satisfies the axioms of a density matrix. The resulting representation admits an intrinsic rank bound controlled by the number of classes, enabling low-dimensional spectral embeddings via dominant eigenmodes. Classification is performed in the reduced space through class-conditional kernel density estimation and a maximum-likelihood decision rule. We establish structural invariances, provide complexity estimates, and validate the approach on synthetic benchmarks probing high cardinality, sparsity, noise, and class imbalance.
Read the full abstract

Machine Learning (Statistics)

LOCUS: A Distribution-Free Loss-Quantile Score for Risk-Aware Predictions

Modern machine learning models can be accurate on average yet still make mistakes that dominate deployment cost. We introduce Locus, a distribution-free wrapper that produces a per-input loss-scale reliability score for a fixed prediction function. Rather than quantifying uncertainty about the label, Locus models the realized loss of the prediction function using any engine that outputs a predictive distribution for the loss given an input. A simple split-calibration step turns this function into a distribution-free interpretable score that is comparable across inputs and can be read as an upper loss level. The score is useful on its own for ranking, and it can optionally be thresholded to obtain a transparent flagging rule with distribution-free control of large-loss events. Experiments across 13 regression benchmarks show that Locus yields effective risk ranking and reduces large-loss frequency compared to standard heuristics.
Read the full abstract

All authors Matheus Barreto, Mário de Castro, Thiago R. Ramos, Denis Valle, Rafael Izbicki

Machine Learning (Statistics) · Cross-listed from cs.LG

Accelerating Single-Pass SGD for Generalized Linear Prediction

We study generalized linear prediction under a streaming setting, where each iteration uses only one fresh data point for a gradient-level update. While momentum is well-established in deterministic optimization, a fundamental open question is whether it can accelerate such single-pass non-quadratic stochastic optimization. We propose the first algorithm that successfully incorporates momentum via a novel data-dependent proximal method, achieving dual-momentum acceleration. Our derived excess risk bound decomposes into three components: an improved optimization error, a minimax optimal statistical error, and a higher-order model-misspecification error. The proof handles mis-specification via a fine-grained stationary analysis of inner updates, while localizing statistical error through a two-phase outer-loop analysis. As a result, we resolve the open problem posed by Jain et al. [2018a] and demonstrate that momentum acceleration is more effective than variance reduction for generalized linear prediction in the streaming setting.
Read the full abstract

Statistical Methodology

A Simulation Study to Compare Inferential Properties when Modelling Ordinal Outcomes: The Case for the (Plain but Robust) Proportional Odds Model

Ordinal measurements are common outcomes in studies within psychology, as well as in the social and behavioral sciences. Choosing an appropriate regression model for analysing such data poses a difficult task. This paper aims to facilitate modeling decisions for quantitative researchers by presenting the results of an extensive simulation study on the inferential properties of common ordinal regression models: the proportional odds model, the category-specific odds model, the location-shift model, the location-scale model, and the linear model, which incorrectly treats ordinal outcomes as metric. The simulations were conducted under different data generating processes based on each of the ordinal models and varying parameter configurations within each model class. We examined the bias of parameter estimates as well as type I error rates ($\alpha$-errors) and the power of statistical parameter testing procedures corresponding to the respective models. Our findings reveal several highlights. For parameter estimates, we observed that cumulative ordinal regression models exhibited large biases in cases of large parameter values and high skewness of the outcome distribution in the true data generation process. Regarding statistical hypothesis testing, the proportional odds model and the linear model showed the most reliable results. Due to its better fit and interpretability for ordinal outcomes, we recommend the use of the proportional odds model unless there are relevant contraindications.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Existence, properties, and parametric inference for possibly hyperuniform Gibbs perturbed lattices

This work lies at the intersection of Gibbs models and hyperuniform point processes. Classical Gibbs models, whether defined on lattices or in continuous space, provide flexible tools to describe interacting particle systems but are generally not hyperuniform. Conversely, known hyperuniform models such as the Ginibre process or perturbed lattices lack flexibility and typically cannot enforce physically relevant constraints such as hard-core interactions. We introduce a new class of models, termed Gibbs perturbed lattice models, which preserve a lattice structure while allowing interactions through a Hamiltonian defined on the perturbed particle locations. We establish existence results for the associated Gibbs measures, derive DLR-type equilibrium equations, and show that some models in this class exhibit hyperuniformity. Finally, we propose statistical inference methods based on the Takacs-Fiksel type approach and prove their asymptotic properties.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Constrained Particle Seeking: Solving Diffusion Inverse Problems with Just Forward Passes

Diffusion models have gained prominence as powerful generative tools for solving inverse problems due to their ability to model complex data distributions. However, existing methods typically rely on complete knowledge of the forward observation process to compute gradients for guided sampling, limiting their applicability in scenarios where such information is unavailable. In this work, we introduce \textbf{Constrained Particle Seeking (CPS)}, a novel gradient-free approach that leverages all candidate particle information to actively search for the optimal particle while incorporating constraints aligned with high-density regions of the unconditional prior. Unlike previous methods that passively select promising candidates, CPS reformulates the inverse problem as a constrained optimization task, enabling more flexible and efficient particle seeking. We demonstrate that CPS can effectively solve both image and scientific inverse problems, achieving results comparable to gradient-based methods while significantly outperforming gradient-free alternatives. Code is available at https://github.com/deng-ai-lab/CPS.
Read the full abstract

All authors Hongkun Dou, Zike Chen, Zeyu Li, Hongjue Li, Lijun Yang, Yue Deng

Machine Learning (Statistics) · Cross-listed from cs.LG

Uncertainty Quantification of Click and Conversion Estimates for the Autobidding

Modern e-commerce platforms employ various auction mechanisms to allocate paid slots for a given item. To scale this approach to the millions of auctions, the platforms suggest promotion tools based on the autobidding algorithms. These algorithms typically depend on the Click-Through-Rate (CTR) and Conversion-Rate (CVR) estimates provided by a pre-trained machine learning model. However, the predictions of such models are uncertain and can significantly affect the performance of the autobidding algorithm. To address this issue, we propose the DenoiseBid method, which corrects the generated CTRs and CVRs to make the resulting bids more efficient in auctions. The underlying idea of our method is to employ a Bayesian approach and replace noisy CTR or CVR estimates with those from recovered distributions. To demonstrate the performance of the proposed approach, we perform extensive experiments on the synthetic, iPinYou, and BAT datasets. To evaluate the robustness of our approach to the noise scale, we use synthetic noise and noise estimated from the predictions of the pre-trained machine learning model.
Read the full abstract

All authors Ivan Zhigalskii, Andrey Pudovikov, Aleksandr Katrutsa, Egor Samosvat

Machine Learning (Statistics) · Cross-listed from cs.LG

Phase-Type Variational Autoencoders for Heavy-Tailed Data

Heavy-tailed distributions are ubiquitous in real-world data, where rare but extreme events dominate risk and variability. However, standard Variational Autoencoders (VAEs) employ simple decoder distributions, such as Gaussian distributions, that fail to capture heavy-tailed behavior, while existing heavy-tail-aware extensions remain restricted to predefined parametric families whose tail behavior is fixed a priori. We propose the Phase-Type Variational Autoencoder (PH-VAE), whose decoder distribution is a latent-conditioned Phase-Type (PH) distribution, defined as the absorption time of a continuous-time Markov chain (CTMC). This formulation composes multiple exponential time scales, yielding a flexible and analytically tractable decoder that adapts its finite-range tail behavior directly from the observed data. Experiments on synthetic and real-world benchmarks demonstrate that PH-VAE accurately approximates diverse heavy-tailed distributions, significantly outperforming Gaussian, Student-t, and extreme-value-based VAE decoders in modeling observed tail behavior and extreme quantiles. In multivariate settings, PH-VAE captures realistic cross-dimensional tail dependence through its shared latent representation. To our knowledge, this is the first work to integrate Phase-Type distributions into deep generative modeling, bridging applied probability and representation learning.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Learning Shortest Paths with Generative Flow Networks

In this paper, we present a novel learning framework for finding shortest paths in graphs utilizing Generative Flow Networks (GFlowNets). First, we examine theoretical properties of GFlowNets in non-acyclic environments in relation to shortest paths. We prove that, if the total flow is minimized, forward and backward policies traverse the environment graph exclusively along shortest paths between the initial and terminal states. Building on this result, we reduce shortest-path finding in an arbitrary unweighted graph to finding a non-acyclic GFlowNet with minimal total flow, and approximate it in practice by training with flow regularization. We experimentally demonstrate the performance of our method in pathfinding in permutation environments and in solving Rubik's Cubes of various sizes. For the latter problems, our approach shows competitive results with state-of-the-art machine learning approaches designed specifically for this task in terms of the solution length, while requiring smaller search budget at test-time.
Read the full abstract

All authors Nikita Morozov, Ian Maksimov, Daniil Tiapkin, Sergey Samsonov