Statistics Applications · Cross-listed from cs.CR
Differentially Private Modeling of Disease Transmission within Human Contact Networks
Shlomi Hod, Debanuj Nayak, Jason R. Gantenberg et al. (+3 more)
Epidemiologic studies of infectious diseases often rely on models of contact networks to capture the complex interactions that govern disease spread, and ongoing projects aim to vastly increase the scale at which such data can be collected. However, contact networks may include sensitive information, such as sexual relationships or drug use behavior. Protecting individual
privacy while maintaining the scientific usefulness of the data is crucial. We propose a
privacy-preserving pipeline for disease spread simulation studies based on a sensitive network that integrates differential
privacy (DP) with statistical network models such as
stochastic block models (SBMs) and exponential random graph models (ERGMs). Our pipeline comprises three steps: (1) compute network summary statistics using node-level DP (which corresponds to protecting individuals' contributions); (2) fit a statistical model, like an ERGM, using these summaries, which allows generating synthetic networks reflecting the structure of the original network; and (3) simulate disease spread on the synthetic networks using an
agent-based model. We evaluate the effectiveness of our approach using a simple Susceptible-Infected-Susceptible (SIS) disease model under multiple configurations. We compare both numerical results, such as simulated disease incidence and prevalence, as well as qualitative conclusions such as intervention effect size, on networks generated with and without differential
privacy constraints. Our experiments are based on
egocentric sexual network data from the ARTNet study (a survey about HIV-related behaviors). Our results show that the noise added for
privacy is small relative to other sources of error (sampling and model misspecification). This suggests that, in principle, curators of such sensitive data can provide valuable epidemiologic insights while protecting
privacy.
Read the full abstract ▾
All authors Shlomi Hod, Debanuj Nayak, Jason R. Gantenberg, Iden Kalemaj, Thomas A. Trikalinos, Adam Smith
Statistical Methodology
Eliciting core spatial association from spatial time series: a random matrix approach
Madhuchhanda Bhattacharjee and Arup Bose
Spatial time series (STS) data are fundamental to
climate science, yet conventional approaches often conflate temporal co-
evolution with genuine spatial dependence, obscuring subtle but critical climatic anomalies. We introduce a Random
Matrix Theory (RMT)-based framework to isolate "core spatial association" by suitably trimming out strong but routine temporal signals while preserving spatial signals. Our pipeline introduces
Hilbert space filling curve technique and Bergsma's correlation measure of statistical dependence, to
climate modelling. Applied to the diurnal temperature range (DTR) data of India (1951-2022), the method reveals distinct spatial anomalies shaped by topography, mesoclimate, and urbanization. The approach uncovers temporal
evolution in spatial dependence and demonstrates how regional
climate variability is structured by both physical geography and anthropogenic influences. Beyond the Indian application, the framework is broadly applicable to diverse spatio-temporal
datasets, offering a robust statistical foundation for predictive modelling, resilience planning, and policy design in the context of accelerating
climate change.
Read the full abstract ▾
Statistical Methodology
Virtual Dummies: Enabling Scalable FDR-Controlled Variable Selection via Sequential Sampling of Null Features
Taulant Koka, Jasin Machkour, Daniel P. Palomar et al. (+1 more)
High-dimensional variable selection, particularly in genomics, requires error-controlling procedures that scale to millions of predictors. The Terminating-Random Experiments (T-Rex) selector achieves false discovery rate (FDR) control by aggregating results of early terminated random experiments, each combining original predictors with i.i.d. synthetic null variables (dummies). At biobank scales, however, explicit dummy augmentation requires terabytes of
memory. We demonstrate that this bottleneck is not fundamental. Formalizing the information flow of forward selection through a filtration, we show that compatible selectors interact with unselected dummies solely through projections onto an adaptively evolving low-dimensional subspace. For rotationally invariant dummy distributions, we derive an adaptive stick-breaking construction sampling these projections from their exact conditional distribution given the selection history, thereby eliminating dummy
matrix materialization. We prove a pathwise universality
theorem: under mild delocalization conditions, selection paths driven by generic standardized i.i.d. dummies converge to the same Gaussian limit. We instantiate the theory through Virtual Dummy LARS (VD-LARS), reducing
memory and runtime by several orders of magnitude while preserving the exact selection law and FDR guarantees of the T-Rex selector. Experiments on realistic genome-wide association study data confirm that VD-T-Rex controls FDR and achieves power at scales where all competing methods either fail or time out.
Read the full abstract ▾
All authors Taulant Koka, Jasin Machkour, Daniel P. Palomar, Michael Muma
Machine Learning (Statistics) · Cross-listed from cond-mat.stat-mech
Score Shocks: The Burgers Equation Structure of Diffusion Generative Models
Krisanu Sarkar
We analyze the score field of a
diffusion generative model through a Burgers-type
evolution law. For VE
diffusion, the heat-evolved data density implies that the score obeys viscous Burgers in one dimension and the corresponding irrotational vector Burgers system in $\R^d$, giving a PDE view of speciation transitions as the sharpening of inter-mode interfaces. For any binary decomposition of the noised density into two positive heat solutions, the score separates into a smooth background and a universal $\tanh$ interfacial term determined by the component log-ratio; near a regular binary mode boundary this yields a normal criterion for speciation. In symmetric binary Gaussian mixtures, the criterion recovers the critical
diffusion time detected by the midpoint derivative of the score and agrees with the spectral criterion of Biroli, Bonnaire, de~Bortoli, and Mézard (2024). After subtracting the background drift, the inter-mode layer has a local Burgers $\tanh$ profile, which becomes global in the symmetric Gaussian case with width $\sigma_\tau^2/a$. We also quantify exponential amplification of score errors across this layer, show that Burgers dynamics preserves irrotationality, and use a change of variables to reduce the VP-SDE to the VE case, yielding a closed-form VP speciation time. Gaussian-mixture formulas are verified to machine precision, and the local
theorem is checked numerically on a quartic double-well.
Read the full abstract ▾
Statistical Methodology
Poisson-response Tensor-on-Tensor Regression and Applications
Carlos Llosa-Vite and Daniel M. Dunlavy
We introduce Poisson-response tensor-on-tensor regression (PToTR), a novel regression framework designed to handle tensor responses composed element-wise of random Poisson-distributed counts. Tensors, or multi-dimensional arrays, composed of counts are common data in fields such as international relations, social networks, epidemiology, and medical imaging, where events occur across multiple dimensions like time, location, and dyads. PToTR accommodates such tensor responses alongside tensor
covariates, providing a versatile tool for multi-dimensional data analysis. We propose
algorithms for maximum likelihood estimation under a canonical polyadic (CP) structure on the regression coefficient tensor that satisfy the positivity of Poisson parameters and then provide an initial theoretical error analysis for PToTR
estimators. We also demonstrate the utility of PToTR through three concrete applications: longitudinal data analysis of the Integrated Crisis Early Warning System database, positron emission
tomography (PET) image reconstruction, and change-point detection of communication patterns in longitudinal dyadic data. These applications highlight the versatility of PToTR in addressing complex, structured count data across various domains.
Read the full abstract ▾
Statistics Applications · Cross-listed from astro-ph.CO
The Non-Gaussian Weak-Lensing Likelihood: A Multivariate Copula Construction and Impact on Cosmological Constraints
Veronika Oehl and Tilman Tröster
We present a framework to compute non-Gaussian likelihoods for two-point correlation functions. The non-Gaussianity is most pronounced on large scales that will be well-measured by stage-IV weak-
lensing surveys. We show how such a multivariate likelihood can be constructed and efficiently evaluated using a copula approach by incorporating exact one-dimensional marginals and a dependence structure derived from the exact multivariate likelihood. The copula likelihood is found to be in better agreement with simulated sampling distributions of correlation functions than Gaussian likelihoods, particularly on large scales. We furthermore investigate the effect of the non-Gaussian copula likelihood on posterior inference, including sampling the full parameter space of contemporary weak-
lensing analyses. We find potential parameter shifts in $S_8$ on the order of one standard deviation for $1 \ 000 \ \mathrm{deg}^2$ surveys but negligible shifts for areas of $10 \ 000 \ \mathrm{deg}^2$, suggesting Gaussian likelihoods are sufficient for stage-IV surveys, though results depend on the detailed mask geometry and data-vector structure.
Read the full abstract ▾
Statistical Methodology
Conformal Prediction with Time-Series Data via Sequential Conformalized Density Regions
M. Sampson and K.S. Chan
We propose a new conformal prediction method for time-series data with a guaranteed asymptotic conditional coverage rate, Sequential Conformalized Density Regions (SCDR), which is flexible enough to produce both prediction intervals and disconnected prediction sets, signifying the emergence of bifurcations. Our approach uses existing estimated conditional highest density predictive regions to form initial predictive regions. We then use a
quantile random forest conformal adjustment to provide guaranteed coverage while adaptively changing to take the non-exchangeable nature of time-series data into account. We show that the proposed method achieves the guaranteed coverage rate asymptotically under certain regularity conditions. In particular, the method is doubly robust – it works if the predictive density model is correctly specified and/or if the scores follow a nonlinear autoregressive model with the correct order specified. Simulations reveal that the proposed method outperforms existing methods in terms of empirical coverage rates and set sizes. We illustrate the method using two real
datasets, the Old Faithful geyser
dataset and the Australian electricity usage
dataset. Prediction sets formed using SCDR for the geyser eruption durations include both single intervals and unions of two intervals, whereas existing methods produce wider, less informative, single-interval prediction sets.
Read the full abstract ▾
Machine Learning (Statistics)
Gaussian Approximation for Asynchronous Q-learning
Artemy Rubtsov, Sergey Samsonov, Vladimir Ulyanov et al. (+1 more)
In this paper, we derive rates of convergence in the high-dimensional central limit
theorem for Polyak-Ruppert averaged iterates generated by the asynchronous Q-learning
algorithm with a
polynomial stepsize $k^{-\omega},\, \omega \in (1/2, 1]$. Assuming that the sequence of state-action-next-state triples $(s_k, a_k, s_{k+1})_{k \geq 0}$ forms a uniformly geometrically ergodic Markov chain, we establish a rate of order up to $n^{-1/6} \log^{4} (nS A)$ over the class of hyper-rectangles, where $n$ is the number of samples used by the
algorithm and $S$ and $A$ denote the numbers of states and actions, respectively. To obtain this result, we prove a high-dimensional central limit
theorem for sums of martingale differences, which may be of independent interest. Finally, we present bounds for high-order moments for the
algorithm's last iterate.
Read the full abstract ▾
All authors Artemy Rubtsov, Sergey Samsonov, Vladimir Ulyanov, Alexey Naumov
Statistics Applications · Cross-listed from physics.ins-det
Multispectral representation of Distributed Acoustic Sensing data: a framework for physically interpretable feature extraction and visualization
Sergio Morell-Monzó, Dídac Diego-Tortosa, Isabel Pérez-Arjona et al. (+1 more)
Distributed Acoustic Sensing (DAS) enables continuous monitoring of dynamic strain along tens of kilometers of optical fiber, generating massive
datasets whose interpretation and automated analysis remain challenging. DAS measurements often lack a standardized visual representation, and their physical interpretation depends strongly on acquisition conditions and signal processing choices. This work introduces a systematic framework for visualization and feature extraction of DAS data based on a multispectral signal representation. The approach decomposes strain-rate measurements into predefined frequency bands and computes band-limited energy images that describe the spatial and temporal distribution of acoustic energy across distinct spectral regimes. The framework is evaluated using DAS recordings containing Fin Whale (Balaenoptera physalus) and Blue Whale (Balaenoptera musculus) vocalizations. Three experiments are conducted to assess the approach: enhanced visualization of bioacoustic signals, unsupervised clustering of acoustic patterns, and supervised event detection using a convolutional neural network. Using multispectral composites as input, a ResNet-18 classifier achieves an accuracy of 97.3% in whale vocalization detection, demonstrating that the proposed representation captures biologically meaningful spectral structure and provides an effective feature space for automated analysis of DAS data.
Read the full abstract ▾
All authors Sergio Morell-Monzó, Dídac Diego-Tortosa, Isabel Pérez-Arjona, Víctor Espinosa
Machine Learning (Statistics)
The Theory and Practice of Highly Scalable Gaussian Process Regression with Nearest Neighbours
Robert Allison, Tomasz Maciazek and Anthony Stephenson
Gaussian process ($GP$) regression is a widely used non-parametric modeling tool, but its cubic complexity in the training size limits its use on massive data sets. A practical remedy is to predict using only the nearest neighbours of each test point, as in Nearest Neighbour Gaussian Process ($NNGP$) regression for geospatial problems and the related scalable $GPnn$ method for more general machine-learning applications. Despite their strong empirical performance, the large-$n$ theory of $NNGP/GPnn$ remains incomplete. We develop a theoretical framework for $NNGP$ and $GPnn$ regression. Under mild regularity assumptions, we derive almost sure pointwise limits for three key predictive criteria: mean squared error ($MSE$), calibration coefficient ($CAL$), and negative log-likelihood ($NLL$). We then study the $L_2$-risk, prove universal consistency, and show that the risk attains Stone's minimax rate $n^{-2\alpha/(2p+d)}$, where $\alpha$ and $p$ capture regularity of the regression problem. We also prove uniform convergence of $MSE$ over compact hyper-parameter sets and show that its derivatives with respect to lengthscale, kernel scale, and noise variance vanish asymptotically, with explicit rates. This explains the observed robustness of $GPnn$ to hyper-parameter tuning. These results provide a rigorous statistical foundation for $NNGP/GPnn$ as a highly scalable and principled alternative to full $GP$ models.
Read the full abstract ▾
Statistical Methodology
NLP-Informed Dynamic Cognitive Diagnosis Modelling
Yawen Ma, Sahoko Ishida, Kate Cain et al. (+1 more)
Digital learning platforms are increasingly used to support reading development while generating rich log files and item-level textual content. Using these data, this study proposes a dynamic cognitive diagnostic modelling (CDM) framework that incorporates text-derived semantic information to inform the estimation of the Q-
matrix. We construct item-level semantic representations of question text and response options, and use these representations to define an informative prior on the Q-
matrix. This approach treats text-derived signals as proxies for item complexity and cognitive demands, guiding the item-skill mapping in a data-driven manner. The proposed framework jointly estimates latent skill mastery profiles, item parameters, and transition dynamics over time within a
Bayesian framework. We apply the model to data from Boost Reading, a digital reading supplement, focusing on students' vocabulary and comprehension skill development. We compare the proposed framework with a baseline model without any text information and show that the text-derived prior can improve Q-
matrix recovery, particularly in settings where response data alone provide limited identification, as well as other model parameters for varying scenarios. This study provides a novel integration of natural
language processing and dynamic CDMs, offering a data-driven approach to modelling skill acquisition and item-skill relationships in digital learning environments.
Read the full abstract ▾
All authors Yawen Ma, Sahoko Ishida, Kate Cain, Gabriel Wallin
Machine Learning (Statistics)
Amortized Filtering and Smoothing with Conditional Normalizing Flows
Tiangang Cui, Xiaodong Feng, Chenlong Pei et al. (+2 more)
Bayesian filtering and smoothing are central to data assimilation in nonlinear dynamical systems. Recent advances in deep generative models provide flexible approximations of the associated non-Gaussian posterior distributions. However, several existing approaches require the score to be re-estimated or a transport map to be constructed at each assimilation step. We propose an amortized framework for filtering and smoothing that reuses trained conditional models across observation sequences and assimilation times. The framework jointly learns a shared recurrent summary network and two conditional normalizing flows from simulated state and observation trajectories. The recurrent network represents each observation history by a fixed-dimensional summary that conditions the filtering approximation and, together with the next state, the backward transition approximation. An information-theoretic analysis shows that, under the Markov assumption, a summary sufficient for filtering is also sufficient for the backward kernel. Combining the terminal filtering approximation with the backward kernels yields an approximate joint smoothing distribution. Numerical experiments on advection-
diffusion, Burgers, and Lorenz systems demonstrate the accuracy of the proposed filtering and smoothing approximations and characterize the
evolution of their errors beyond the training horizon.
Read the full abstract ▾
All authors Tiangang Cui, Xiaodong Feng, Chenlong Pei, Xiaoliang Wan, Tao Zhou
Machine Learning (Statistics) · Cross-listed from cs.LG
SBBTS: A Unified Schrödinger-Bass Framework for Synthetic Financial Time Series
Alexandre Alouadi, Grégoire Loeper, Célian Marsala et al. (+2 more)
We study the problem of generating synthetic time series that reproduce both marginal distributions and temporal dynamics, a central challenge in financial machine learning. Existing approaches typically fail to jointly model drift and
stochastic volatility, as
diffusion-based methods fix the
volatility while martingale transport models ignore drift. We introduce the Schrödinger-Bass Bridge for Time Series (SBBTS), a unified framework that extends the Schrödinger-Bass formulation to multi-step time series. The method constructs a
diffusion process that jointly calibrates drift and
volatility and admits a tractable decomposition into conditional transport problems, enabling efficient learning. Numerical experiments on the Heston model demonstrate that SBBTS accurately recovers
stochastic volatility and correlation parameters that prior SchrödingerBridge methods fail to capture. Applied to S&P 500 data, SBBTS-generated synthetic time series consistently improve downstream
forecasting performance when used for data augmentation, yielding higher classification accuracy and Sharpe ratio compared to real-data-only training. These results show that SBBTS provides a practical and effective framework for realistic time series generation and data augmentation in financial applications.
Read the full abstract ▾
All authors Alexandre Alouadi, Grégoire Loeper, Célian Marsala, Othmane Mazhar, Huyên Pham
Statistics Theory
Non-asymptotic two-sample kernel testing with the spectrally truncated normalized MMD
Perrine Lacroix, Bertrand Michel, Franck Picard et al. (+1 more)
Kernel methods provide a flexible and powerful framework for nonparametric statistical testing by
embedding probability distributions into a reproducing kernel
Hilbert space (RKHS). In this work, we study the kernel two-sample testing problem and focus on a normalized version of the Maximum Mean Discrepancy (MMD) as a test statistic, which scales the discrepancy by the within-group covariance operator to account for data variability. This normalization has been shown to improve test power in both theoretical and empirical settings. Because this normalization requires regularization, we study the non-asymptotic properties of the spectrally truncated normalized MMD (st-nMMD) and derive an exponential upper bound under the null hypothesis. Thanks to this result we propose a sharp and explicit upper bound for the corresponding non-asymptotic
quantile, along with a data-adaptive
estimator. We further propose an
algorithm to tune the hyperparameters involved in the
quantile estimation, including the truncation level, without requiring data splitting. We demonstrate the performance of the st-nMMD through numerical experiments under both the null and alternative hypotheses.
Read the full abstract ▾
All authors Perrine Lacroix, Bertrand Michel, Franck Picard, Vincent Rivoirard
Statistics Applications · Cross-listed from cs.LG
Lumbermark: Resistant Clustering by Chopping Up Mutual Reachability Minimum Spanning Trees
Marek Gagolewski
We introduce Lumbermark, a robust divisive clustering
algorithm capable of detecting clusters of varying sizes, densities, and shapes. Lumbermark iteratively chops off large limbs connected by protruding segments of a
dataset's mutual reachability minimum spanning tree. The use of mutual reachability distances smoothens the data distribution and decreases the influence of low-density objects, such as noise points between clusters or outliers at their peripheries. The
algorithm can be viewed as a divisive version of Genie and an alternative to HDBSCAN that produces partitions with user-specified sizes. A fast, easy-to-use implementation of this new method is available in the open-source lumbermark package for Python and R. We show that Lumbermark performs well on
benchmark data and hope it will prove useful to data scientists and practitioners across different fields.
Read the full abstract ▾
Statistical Methodology
Private Federated Learning for High-dimensional Time Series
Kejun Chen and Qianqian Zhu
In the era of big data, leveraging information from multiple clients while preserving data
privacy has emerged as a critical challenge in modern statistical modeling and
forecasting. This paper introduces a
privacy-preserving
federated learning framework for high-dimensional vector autoregressive models, where each client's dynamics are characterized by a common low-rank structure augmented with sparse client-specific deviations. We develop a two-stage estimation procedure that integrates differentially private representation learning for the shared component with local personalization for client-specific adjustments, enabling effective information pooling under selective
privacy constraints. Non-asymptotic error bounds are established for both the single-client and
federated estimators to characterize the inherent
privacy-utility trade-off, and consistency of a ridge-type rank selection criterion is proved. Simulation studies demonstrate that federation substantially improves estimation accuracy when local sample sizes are limited. Two empirical applications to analyzing electricity-economy linkages across U.S. states and conducting multi-task macroeconomic
forecasting across countries, highlight the superior predictive accuracy of the proposed method over existing single-client
benchmarks.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Are Stochastic Multi-objective Bandits Harder than Single-objective Bandits?
Changkun Guan and Mengfan Xu
Multi-objective bandits have attracted increasing
attention for their broad applicability, with \(d\)-dimensional reward vectors inducing Pareto regret. There has been a subtle debate over whether this added structure makes the problem fundamentally harder than single-objective bandits. We answer this by showing that, in terms of Pareto regret, it is surprisingly no harder: Pareto regret scales inversely with \(g^\dagger\), the largest objective-wise suboptimality gap, and thus matches the smallest objective-wise classical regret. We formalize this idea via a novel method with upper and lower confidence-bound
estimators for every arm-objective pair. It uses top-two races to compare arms within each objective and an uncertainty-greedy rule to allocate exploration toward the largest objective-wise gap \(g^\dagger\), until the corresponding Pareto-optimal arm is committed to. We prove that it achieves Pareto regret of \(O(\nicefrac{\log T}{g^\dagger})\), where \(T\) is the horizon, with \emph{no dependence on \(d\)}. A matching lower bound of \(\Omega(\nicefrac{\log T}{g^\dagger})\) implies optimality. We evaluate the method on synthetic and real-world
datasets, confirming the theory and achieving order-of-magnitude reductions in Pareto regret over baselines. Real-world results further show that our method commits to a Pareto optimal arm, possibly at the cost of empirical
fairness, suggesting a potential hardness absent in single-objective bandits.
Read the full abstract ▾
Statistical Methodology
Introduction to Relational Event Modelling
Martina Boschi and Ernst C. Wit
Interactions and time shape many aspects of life. Everyday activities – like conversations, emails, money transfers, citations, and even acts of violence – are relational events: interactions between a sender and a receiver at a specific moment. At the intersection of event-history analysis and network modelling, relational event models (REMs) offer a powerful framework for studying when and why these events occur. Recent advances have made it possible to express REMs as generalized additive models, allowing researchers to capture complex, non-linear patterns over time. A hands-on guide to REMs is still missing. This tutorial fills that gap. It provides a practical introduction to REMs, incorporating the latest developments in the field. It demonstrates how to simulate synthetic relational-event data and walks through several empirical applications, comparing different modeling and inference strategies. By bringing together theory, simulation, and application, this tutorial lowers the barrier to entry and makes REMs a more accessible and practical tool.
Read the full abstract ▾
Statistical Methodology
Time Series Gaussian Chain Graph Models
Qin Fang, Xinghao Qiao and Zihan Wang
Time series graphical models have recently received considerable
attention for characterizing (conditional) dependence structures in multivariate time series. In many applications, the multivariate series exhibit variable-partitioned blockwise dependence, with distinct patterns within and across blocks. In this paper, we introduce a new class of time series Gaussian chain graph models that represent contemporaneous and lagged causal relations via directed edges across blocks, while capturing within-block conditional dependencies through undirected edges. In the frequency domain, this formulation induces a cross-frequency shared group sparse plus group low-rank decomposition of the inverse spectral density
matrices, which we exploit to establish identifiability of the time series chain graph structure. Building on this, we then propose a three-stage learning procedure for estimating the undirected and directed edge sets, which involves optimizing a regularized Whittle likelihood with a group lasso penalty to encourage group sparsity and a novel tensor-unfolding
nuclear norm penalty to enforce group low-rank structure. We investigate the asymptotic properties of the proposed method, ensuring its consistency for exact recovery of the chain graph structure. The superior empirical performance of the proposed method is demonstrated through both extensive simulation studies and an application to U.S. macroeconomic data that highlights key monetary policy transmission mechanisms.
Read the full abstract ▾
Statistical Methodology
Recovering manifold structure in LLM responses through a joint Euclidean mirror
Maximilian Baum, Aranyak Acharyya, Tianyi Chen et al. (+5 more)
Understanding the behavior of black-box large
language models and determining effective means of comparing their performance is a key task in modern machine learning. We consider how large
language models respond to a specific query by analyzing how the distributions of responses vary over different values of tuning parameters. We frame this problem in a general mathematical setting, treating the mapping from model parameters to response distributions as a structured family of probability measures, endowed with a geometry via a dissimilarity measure. We show how dissimilarities between response distributions can be represented in low-dimensional Euclidean space through a joint Euclidean mirror surface encoding the underlying geometry, which permits both qualitative and quantitative analysis of large
language models and provides insight into predicting response distributions for different values of tuning parameters. We propose an estimation procedure for the underlying joint Euclidean mirror based on observed samples from the response distributions, and we prove its asymptotic properties. Additionally, we propose a statistically consistent procedure to infer the value of an unknown model parameter based on samples from the corresponding response distribution and the estimated joint Euclidean mirror. In an experimental setting with large
language models, we find that changes in different tuning parameter values correspond to distinct directions in the
embedding space, making it possible to estimate the tuning parameters that were used to generate a given response.
Read the full abstract ▾
All authors Maximilian Baum, Aranyak Acharyya, Tianyi Chen, Avanti Athreya, Youngser Park, Francesco Sanna Passino, Carey E. Priebe, Zachary Lubberts
Statistical Methodology
Covariance Correction for Permutation Statistics in Multiple Testing Problems
Merle Munko and Paavo Sattler
In qualitative statistics, permutation tests are very popular, mainly because of their finite-sample exactness under exchangeability. However, in non-exchangeable settings, the covariance structure of permuted statistics typically differs from that of the original statistic. A common solution is studentization, which restores asymptotic correctness for general hypotheses while preserving exactness under exchangeability. In multiple testing settings, however, standard studentization fails to provide the correct joint limiting distribution. Existing solutions such as prepivoting address this issue but are computationally expensive and therefore rarely used in practice. We propose a general, computationally more efficient methodology that overcomes this fundamental limitation. By appropriately correcting the covariance
matrix of multiple permutation statistics, our approach restores the correct joint asymptotic dependence structure, enabling asymptotically valid permutation tests in broad multiple testing frameworks. The proposed method is highly flexible: it accommodates singular covariance structures and is not tied to specific parameters, test statistics, or permutation schemes. This generality makes it applicable across a wide range of problems. Extensive simulation studies demonstrate that our approach results in reliable inference and outperforms existing methods across diverse settings.
Read the full abstract ▾
Statistics Applications
How Does LLM Help Regional CPI Forecast: An LLM-powered Deep Panel Modeling Framework
Tianchen Gao, Ao Sun, Yurou Wang et al. (+2 more)
Understanding regional Consumer Price Index (CPI) dynamics is essential for timely and effective economic policymaking. However, traditional modeling procedures typically rely only on parametric panel modeling with low-frequency and high-cost macroeconomic indicators, which often fail to capture rapid
market fluctuations and lead to inaccurate predictions. To this end, we propose a residual-joint-modeling framework that integrates large
language model (LLM) analyses and social media narratives via a new deep neural network based panel modeling. Specifically, we construct a large narrative corpus from a newly collected Sina Weibo
dataset, and develop a
prompt-based GPT model and a series of fine-tuned BERT models to generate high-frequency LLM-induced surrogates for regional CPI. A novel joint modeling strategy is then advocated to transfer the information from these surrogates to the target regional CPI data and hence empower CPI prediction. To solve the joint objectives, we further introduce a new deep panel learning procedure with region-wise homogeneity pursuit, which has its own significance in panel data analysis literature. In addition, conformal-based panel prediction intervals are provided to quantify the uncertainty of the LLM-powered prediction. The proposed approach significantly reduces short-term
forecasting errors and more effectively captures abrupt inflationary shifts compared to traditional econometric models. While demonstrated for regional CPI
forecasting, the proposed framework is broadly applicable for incorporating insights from LLMs to enhance traditional statistical modeling.
Read the full abstract ▾
All authors Tianchen Gao, Ao Sun, Yurou Wang, Jingyuan Liu, Cheng Hsiao
Machine Learning (Statistics)
A Data-Informed Variational Clustering Framework for Noisy High-Dimensional Data
Wan Ping Chen
Clustering in high-dimensional settings with severe feature noise remains challenging, especially when only a small subset of dimensions is informative and the final number of clusters is not specified in advance. In such regimes, partition recovery, feature relevance learning, and structural adaptation are tightly coupled, and standard likelihood-based methods can become unstable or overly sensitive to noisy dimensions. We propose DIVI, a data-informed variational clustering framework that combines global feature gating with split-based adaptive structure growth. DIVI uses informative prior initialization to stabilize optimization, learns feature relevance in a differentiable manner, and expands model complexity only when local diagnostics indicate underfit. Beyond clustering performance, we also examine runtime scalability and parameter sensitivity in order to clarify the computational and practical behavior of the framework. Empirically, we find that DIVI performs competitively under severe feature noise, remains computationally feasible, and yields interpretable feature-gating behavior, while also exhibiting conservative growth and identifiable failure regimes in challenging settings. Overall, DIVI is best viewed as a practical variational clustering framework for noisy high-dimensional data rather than as a fully
Bayesian generative solution.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Bi-Lipschitz Autoencoder With Injectivity Guarantee
Qipeng Zhan, Zhuoping Zhou, Zexuan Wang et al. (+2 more)
Autoencoders are widely used for dimensionality reduction, based on the assumption that high-dimensional data lies on low-dimensional
manifolds. Regularized autoencoders aim to preserve
manifold geometry during dimensionality reduction, but existing approaches often suffer from non-injective mappings and overly rigid constraints that limit their effectiveness and robustness. In this work, we identify encoder non-injectivity as a core bottleneck that leads to poor convergence and distorted latent representations. To ensure robustness across data distributions, we formalize the concept of admissible regularization and provide sufficient conditions for its satisfaction. In this work, we propose the Bi-Lipschitz Autoencoder (BLAE), which introduces two key innovations: (1) an injective regularization scheme based on a separation criterion to eliminate pathological local minima, and (2) a bi-Lipschitz relaxation that preserves geometry and exhibits robustness to data distribution drift. Empirical results on diverse
datasets show that BLAE consistently outperforms existing methods in preserving
manifold structure while remaining resilient to sampling sparsity and distribution shifts. Code is available at https://github.com/qipengz/BLAE.
Read the full abstract ▾
All authors Qipeng Zhan, Zhuoping Zhou, Zexuan Wang, Qi Long, Li Shen
Machine Learning (Statistics) · Cross-listed from cs.LG
Generative Cross-Entropy: A Strictly Proper Loss for Data-Efficient Classification
Qipeng Zhan, Zhuoping Zhou and Li Shen
Cross-
entropy (CE) is the default training loss for supervised classification, but its sample efficiency is limited when labels are scarce. Existing remedies primarily act on the data side, via augmentation, synthesis, or transfer from pretrained models; the training objective itself is rarely revisited. We revisit it here. Drawing on the classical observation that generative classifiers reach their asymptotic error with fewer samples than discriminative ones, we propose Generative Cross-
Entropy (GenCE), a drop-in replacement for CE that introduces a generative learning principle into a standard discriminative network without altering the architecture or fitting a separate density model. GenCE follows from a
Bayesian rewrite of the class-conditional likelihood and, in the mini-batch approximation, reduces to normalizing each sample's softmax score against the model's predictions on the batch, coupling the training signal across examples sharing a class. We extend the proper-scoring-rule framework to such non-local losses and prove that GenCE is strictly proper under a mild completeness condition: its population risk is uniquely minimized at the true posterior. Across three
datasets, on two architectures and in both balanced small-data and class-imbalanced regimes, GenCE outperforms CE and other widely used losses, while also producing better-calibrated probabilities and stronger out-of-distribution detection.
Read the full abstract ▾
Statistical Methodology
Transfer Learning for Robust Structured Regression with Bi-level Source Detection
Haoming Shi, Yang Feng and Xiaoqian Liu
High-dimensional data in modern applications, such as COVID-19 mortality, often span multiple domains. Leveraging auxiliary information from source domains to improve performance in a target domain motivates the use of transfer learning. However, a practical issue that has been overlooked is data contamination, which induces heterogeneity and can significantly degrade transfer learning performance. To address this challenge, we propose a novel approach that tackles transfer learning under data contamination within a structured regression setting. By employing the robust L2E criterion, we develop the TransL2E method that accounts for contamination in both target and source data while effectively transferring relevant information. Beyond robust estimation, TransL2E introduces a data-driven bi-level source detection mechanism, operating at both individual and cohort levels, which possesses multiple advantages over existing source detection approaches. Comprehensive simulation studies and a real data application demonstrate the superior performance of TransL2E in both robust estimation and structure recovery in the presence of data limitation and contamination.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
FlowAdam: Implicit Regularization via Geometry-Aware Soft Momentum Injection
Devender Singh and Tarun Sheel
Adaptive moment methods such as Adam use a diagonal, coordinate-wise preconditioner based on exponential moving averages of squared gradients. This diagonal scaling is coordinate-system dependent and can struggle with dense or rotated parameter couplings, including those in
matrix factorization, tensor decomposition, and graph neural networks, because it treats each parameter independently. We introduce FlowAdam, a hybrid optimizer that augments Adam with continuous gradient-flow integration via an ordinary differential equation (ODE). When EMA-based statistics detect landscape difficulty, FlowAdam switches to clipped ODE integration. Our central contribution is Soft Momentum Injection, which blends ODE velocity with Adam's momentum during mode transitions. This prevents the training collapse observed with naive hybrid approaches. Across coupled optimization
benchmarks, the ODE integration provides implicit regularization, reducing held-out error by 10-22% on low-rank
matrix/tensor recovery and 6% on Jester (real-world collaborative filtering), also surpassing tuned Lion and AdaBelief, while matching Adam on well-conditioned workloads (CIFAR-10). MovieLens-100K confirms benefits arise specifically from coupled parameter interactions rather than bias estimation. Ablation studies show that soft injection is essential, as hard replacement reduces accuracy from 100% to 82.5%.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.GL
The Theorems of Dr. David Blackwell and Their Contributions to Artificial Intelligence
Napoleon Paxton
Dr. David Blackwell was a mathematician and statistician of the first rank, whose contributions to statistical theory, game theory, and decision theory predated many of the algorithmic breakthroughs that define modern artificial intelligence. This survey examines three of his most consequential theoretical results the Rao Blackwell
theorem, the Blackwell Approachability
theorem, and the Blackwell Informativeness
theorem (comparison of experiments) and traces their direct influence on contemporary AI and machine learning. We show that these results, developed primarily in the 1940s and 1950s, remain technically live across modern subfields including Markov Chain Monte Carlo inference, autonomous mobile
robot navigation (SLAM), generative model training, no-regret online learning, reinforcement learning from human feedback (RLHF), large
language model
alignment, and information design. NVIDIAs 2024 decision to name their flagship GPU architecture (Blackwell) provides vivid testament to his enduring relevance. We also document an emerging frontier: explicit Rao Blackwellized variance reduction in LLM RLHF pipelines, recently proposed but not yet standard practice. Together, Blackwell
theorems form a unified framework addressing information compression, sequential decision making under uncertainty, and the comparison of information sources precisely the problems at the core of modern AI.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CE
A Rolling-Horizon Stochastic Optimization Framework for NBA Franchise Management with Distributionally Robust Risk Constraints
Siming Zhang, Zhehui Shen, Shijie Chen et al. (+1 more)
NBA franchise management is not a sequence of independent tasks, but a single dynamic control problem in which roster construction, cash-flow discipline, media strategy, external
market shocks, and player-health uncertainty interact over time. Using the New York Knicks as a case study, this paper develops a unified decision architecture for franchise management under competitive, financial, and regulatory constraints. The core layer is formulated as a rolling-horizon
stochastic mixed-integer program augmented with distributionally robust optimization and conditional value-at-risk constraints, so that long-run franchise value can be optimized while downside exposure remains explicitly controlled. On top of this core layer, we construct coordinated modules for transaction execution, league-expansion shock transmission, media-rights regime transition, and injury-triggered re-optimization. This integrated design reframes multiple managerial mechanisms inside one research problem: how should an NBA franchise allocate resources and update decisions when performance objectives and commercial objectives are jointly determined under uncertainty? The manuscript is organized around problem formulation, model architecture, empirical validation, robustness analysis, and managerial interpretation.
Read the full abstract ▾
All authors Siming Zhang, Zhehui Shen, Shijie Chen, Jian Zhou
Machine Learning (Statistics) · Cross-listed from math.OC
A Generalized Sinkhorn Algorithm for Mean-Field Schrödinger Bridge
Asmaa Eldesoukey, Yongxin Chen and Abhishek Halder
The mean-field Schrödinger bridge (MFSB) problem concerns designing a minimum-effort controller that guides a
diffusion process with nonlocal interaction to reach a given distribution from another by a fixed deadline. Unlike the standard Schrödinger bridge, the dynamical constraint for MFSB is the mean-field limit of a population of interacting
agents with controls. It serves as a natural model for large-scale multi-
agent systems. The MFSB is computationally challenging because the nonlocal interaction makes the problem nonconvex. We propose a generalization of the
Hopf-Cole transform for MFSB and, building on it, design a Sinkhorn-type recursive
algorithm to solve the associated system of integro-PDEs. Under mild assumptions on the interaction potential, we discuss convergence guarantees for the proposed
algorithm. We present numerical examples with repulsive and attractive interactions to illustrate the theoretical contributions.
Read the full abstract ▾