Peer Revenue An exchange of ideas

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Monday 2 February

Statistical Methodology

Data-Driven Uniform Inference for General Continuous Treatment Models via Minimum-Variance Weighting

Ai et al. (2021) studied the estimation of a general dose-response function (GDRF) of a continuous treatment that includes the average dose-response function, the quantile dose-response function, and other expectiles of the dose-response distribution. They specified the GDRF as a parametric function of the treatment status only and proposed a weighted regression with the weighting function estimated using the maximum entropy approach. This paper specifies the GDRF as a nonparametric function of the treatment status, proposes a weighted local linear regression for estimating GDRF, and develops a bootstrap procedure for constructing the uniform confidence bands. We propose stable weights with minimum sample variance while eliminating the sample association between the treatment and the confounding variables. The proposed weights admit a closed-form expression, allowing them to be computed efficiently in the bootstrap sampling. Under certain conditions, we derive the uniform Bahadur representation for the proposed estimator of GDRF and establish the validity of the corresponding uniform confidence bands. A fully data-driven approach to choosing the undersmooth tuning parameters and a data-driven bias-control confidence band are included. A simulation study and an application demonstrate the usefulness of the proposed approach.
Read the full abstract

Statistical Methodology

When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating

Loss-based updating, including generalized Bayes, Gibbs, and quasi-posteriors, replaces likelihoods by a user-chosen loss and produces a posterior-like distribution via exponential tilt. We give a decision-theoretic characterization that separates belief posteriors – conditional beliefs justified by the foundations of Savage and Anscombe-Aumann under a joint probability mode l– from decision posteriors – randomized decision rules justified by preferences over decision rules. We make explicit that a loss-based posterior coincides with ordinary Bayes if and only if the loss is, up to scale and a data-only term, negative log-likelihood. We then show that generalized marginal likelihood is not evidence for decision posteriors, and Bayes factors are not well-defined without additional structure. In the decision posterior regime, non-degenerate posteriors require nonlinear preferences over decision rules. Under sequential coherence and separability, these lead to an entropy-penalized variational representation yielding generalized Bayes as the optimal rule.
Read the full abstract

Statistics Applications

Bayesian brain mapping: a population-informed framework for personalized functional network topography and connectivity

The spatial topography of functional brain organization is increasingly recognized to play an important role in cognition and disease. Accounting for individual differences in functional topography is also crucial for accurately distinguishing spatial and temporal aspects of functional brain connectivity. Yet, accurate estimation of personalized functional brain networks from functional magnetic resonance imaging (fMRI) without extensive scanning remains challenging due to high noise levels. Here, we describe Bayesian Brain Mapping (BBM), a technique for personalized functional topography and connectivity informed by population information. BBM relies on population-derived priors on both spatial topography of networks and between-network functional connectivity to guide subject-level estimation and combat noise. These priors are based on existing spatial templates, such as parcellations or continuous network maps, providing correspondence to those templates. Yet BBM is highly flexible, avoiding strong spatial or temporal constraints and allowing for overlap between networks and heterogeneous patterns of engagement. BBM is designed for single-subject analysis, making it computationally efficient and translatable to clinical settings. Here, we describe the BBM model and illustrate the use of the BayesBrainMap R package to construct population-derived priors, fit the model, and perform inference to identify engagements. A demo is provided in an accompanying Github repo. We also share priors derived from the Human Connectome Project and provide code to support the construction of priors from different data sources, lowering the barrier to adoption of BBM for studies of individual brain organization.
Read the full abstract

All authors Diego Derman, Nohelia Da Silva Sanchez, Damon D. Pham, Ellyn R. Butler, Mary Beth Nebel, Amanda F. Mejia

Statistical Methodology · Cross-listed from hep-ph

HDSense: An efficient method for ranking observable sensitivity

Identifying which observables most effectively constrain model parameters can be computationally prohibitive when considering full likelihoods of many correlated observables. This is especially important for, e.g., hadronization models, where high precision is required to interpret the results of collider experiments. We introduce the High-Dimensional Sensitivity (HDSense) score, a computationally efficient metric for ranking observable sets using only one-dimensional histograms. Derived by profiling over unknown correlations in the Fisher information framework, the score balances total information content against redundancy between observables. We apply HDSense to rank a set observables in terms of their constraining power with respect to five parameters of the Lund string model of hadronization implemented in Pythia using simulated leptonic collider events at the $Z$ pole. Validation against machine-learning–based full-likelihood approximations demonstrates that HDSense successfully identifies near-optimal observable subsets. The framework naturally handles data from multiple experiments with different acceptances and incorporates detector effects. While demonstrated on hadronization models, the methodology applies broadly to generic parameter estimation problems where correlations are unknown or difficult to model.
Read the full abstract

All authors Benoît Assi, Christian Bierlich, Rikab Gambhir, Phil Ilten, Tony Menzo, Stephen Mrenna, Manuel Szewc, Michael K. Wilkinson, Jure Zupan

Machine Learning (Statistics) · Cross-listed from cs.LG

Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum

We establish an optimal sample complexity of $O(\epsilon^{-2})$ for obtaining an $\epsilon$-optimal global policy using a single-timescale actor-critic (AC) algorithm in infinite-horizon discounted Markov decision processes (MDPs) with finite state-action spaces, improving upon the prior state of the art of $O(\epsilon^{-3})$. Our approach applies STORM (STOchastic Recursive Momentum) to reduce variance in the critic updates. However, because samples are drawn from a nonstationary occupancy measure induced by the evolving policy, variance reduction via STORM alone is insufficient. To address this challenge, we maintain a buffer of small fraction of recent samples and uniformly sample from it for each critic update. Importantly, these mechanisms are compatible with existing deep learning architectures and require only minor modifications, without compromising practical applicability.
Read the full abstract

All authors Navdeep Kumar, Tehila Dahan, Lior Cohen, Ananyabrata Barua, Giorgia Ramponi, Kfir Yehuda Levy, Shie Mannor

2602.01505 · 2 Feb · Following further internal verification, we identified foundational issues in the analytical framework, including unresolved problems in the treatment of nonstationary sampling and parts of the coupled convergence analysis under the stated assumptions. Addressing these issues requires a substantial overhaul of the theoretical framework beyond a standard revision

Sunday 1 February

Machine Learning (Statistics) · Cross-listed from cs.LG

Fubini Study geometry of representation drift in high dimensional data

High dimensional representation drift is commonly quantified using Euclidean or cosine distances, which presuppose fixed coordinates when comparing representations across time, training or preprocessing stages. While effective in many settings, these measures entangle intrinsic changes in the data with variations induced by arbitrary parametrizations. We introduce a projective geometric view of representation drift grounded in the Fubini Study metric, which identifies representations that differ only by gauge transformations such as global rescalings or sign flips. Applying this framework to empirical high dimensional datasets, we explicitly construct representation trajectories and track their evolution through cumulative geometric drift. Comparing Euclidean, cosine and Fubini Study distances along these trajectories reveals that conventional metrics systematically overestimate change whenever representations carry genuine projective ambiguity. By contrast, the Fubini Study metric isolates intrinsic evolution by remaining invariant under gauge-induced fluctuations. We further show that the difference between cosine and Fubini Study drift defines a computable, monotone quantity that directly captures representation churn attributable to gauge freedom. This separation provides a diagnostic for distinguishing meaningful structural evolution from parametrization artifacts, without introducing model-specific assumptions. Overall, we establish a geometric criterion for assessing representation stability in high-dimensional systems and clarify the limits of angular distances. Embedding representation dynamics in projective space connects data analysis with established geometric programs and yields observables that are directly testable in empirical workflows.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Predicting and improving test-time scaling laws via reward tail-guided search

Test-time scaling has emerged as a critical avenue for enhancing the reasoning capabilities of Large Language Models (LLMs). Though the straight-forward ”best-of-$N$” (BoN) strategy has already demonstrated significant improvements in performance, it lacks principled guidance on the choice of $N$, budget allocation, and multi-stage decision-making, thereby leaving substantial room for optimization. While many works have explored such optimization, rigorous theoretical guarantees remain limited. In this work, we propose new methodologies to predict and improve scaling properties via tail-guided search. By estimating the tail distribution of rewards, our method predicts the scaling law of LLMs without the need for exhaustive evaluations. Leveraging this prediction tool, we introduce Scaling-Law Guided (SLG) Search, a new test-time algorithm that dynamically allocates compute to identify and exploit intermediate states with the highest predicted potential. We theoretically prove that SLG achieves vanishing regret compared to perfect-information oracles, and achieves expected rewards that would otherwise require a polynomially larger compute budget required when using BoN. Empirically, we validate our framework across different LLMs and reward models, confirming that tail-guided allocation consistently achieves higher reward yields than Best-of-$N$ under identical compute budgets. Our code is available at https://github.com/PotatoJnny/Scaling-Law-Guided-search.
Read the full abstract

Statistical Methodology · Cross-listed from cs.LG

Causal Preference Elicitation

We propose causal preference elicitation, a Bayesian framework for expert-in-the-loop causal discovery that actively queries local edge relations to concentrate a posterior over directed acyclic graphs (DAGs). From any black-box observational posterior, we model noisy expert judgments with a three-way likelihood over edge existence and direction. Posterior inference uses a flexible particle approximation, and queries are selected by an efficient expected information gain criterion on the expert's categorical response. Experiments on synthetic graphs, protein signaling data, and a human gene perturbation benchmark show faster posterior concentration and improved recovery of directed effects under tight query budgets.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability

How can we understand gradient-based training over non-convex landscapes? The edge of stability phenomenon, introduced in Cohen et al. (2021), indicates that the answer is not so simple: namely, gradient descent (GD) with large step sizes often diverges away from the gradient flow. In this regime, the "Central Flow", recently proposed in Cohen et al. (2025), provides an accurate ODE approximation to the GD dynamics over many architectures. In this work, we propose Rod Flow, an alternative ODE approximation, which carries the following advantages: (1) it rests on a principled derivation stemming from a physical picture of GD iterates as an extended one-dimensional object – a "rod"; (2) it better captures GD dynamics for simple toy examples and matches the accuracy of Central Flow for representative neural network architectures, and (3) is explicit and cheap to compute. Theoretically, we prove that Rod Flow correctly predicts the critical sharpness threshold and explains self-stabilization in quartic potentials. We validate our theory with a range of numerical experiments.
Read the full abstract

Machine Learning (Statistics)

Density-Informed Pseudo-Counts for Calibrated Evidential Deep Learning

Evidential Deep Learning (EDL) is a popular framework for uncertainty-aware classification that models predictive uncertainty via Dirichlet distributions parameterized by neural networks. Despite its popularity, its theoretical foundations and behavior under distributional shift remain poorly understood. In this work, we provide a principled statistical interpretation by proving that EDL training corresponds to amortized variational inference in a hierarchical Bayesian model with a tempered pseudo-likelihood. This perspective reveals a major drawback: standard EDL conflates epistemic and aleatoric uncertainty, leading to systematic overconfidence on out-of-distribution (OOD) inputs. To address this, we introduce Density-Informed Pseudo-count EDL (DIP-EDL), a new parametrization that decouples class prediction from the magnitude of uncertainty by separately estimating the conditional label distribution and the marginal covariate density. This separation preserves evidence in high-density regions while shrinking predictions toward a uniform prior for OOD data. Theoretically, we prove that DIP-EDL achieves asymptotic concentration. Empirically, we show that our method enhances interpretability and improves robustness and uncertainty calibration under distributional shift.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts

Self-attention has greatly contributed to the success of the widely used Transformer architecture by enabling learning from data with long-range dependencies. In an effort to improve performance, a gated attention model that leverages a gating mechanism within the multi-head self-attention has recently been proposed as a promising alternative. Gated attention has been empirically demonstrated to increase the expressiveness of low-rank mapping in standard attention and even to eliminate the attention sink phenomenon. Despite its efficacy, a clear theoretical understanding of gated attention's benefits remains lacking in the literature. To close this gap, we rigorously show that each entry in a gated attention matrix or a multi-head self-attention matrix can be written as a hierarchical mixture of experts. By recasting learning as an expert estimation problem, we demonstrate that gated attention is more sample-efficient than multi-head self-attention. In particular, while the former needs only a polynomial number of data points to estimate an expert, the latter requires exponentially many data points to achieve the same estimation error. Furthermore, our analysis also provides a theoretical justification for why gated attention yields higher performance when a gate is placed at the output of the scaled dot product attention or the value map rather than at other positions in the multi-head self-attention architecture.
Read the full abstract

All authors Viet Nguyen, Tuan Minh Pham, Thinh Cao, Tan Dinh, Huy Nguyen, Nhat Ho, Alessandro Rinaldo

Machine Learning (Statistics)

Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function

The sigmoid gate in mixture-of-experts (MoE) models has been empirically shown to outperform the softmax gate across several tasks, ranging from approximating feed-forward networks to language modeling. Additionally, recent efforts have demonstrated that the sigmoid gate is provably more sample-efficient than its softmax counterpart under regression settings. Nevertheless, there are three notable concerns that have not been addressed in the literature, namely (i) the benefits of the sigmoid gate have not been established under classification settings; (ii) existing sigmoid-gated MoE models may not converge to their ground-truth; and (iii) the effects of a temperature parameter in the sigmoid gate remain theoretically underexplored. To tackle these open problems, we perform a comprehensive analysis of multinomial logistic MoE equipped with a modified sigmoid gate to ensure model convergence. Our results indicate that the sigmoid gate exhibits a lower sample complexity than the softmax gate for both parameter and expert estimation. Furthermore, we find that incorporating a temperature into the sigmoid gate leads to a sample complexity of exponential order due to an intrinsic interaction between the temperature and gating parameters. To overcome this issue, we propose replacing the vanilla inner product score in the gating function with a Euclidean score that effectively removes that interaction, thereby substantially improving the sample complexity to a polynomial order.
Read the full abstract

All authors Tuan Minh Pham, Thinh Cao, Viet Nguyen, Huy Nguyen, Nhat Ho, Alessandro Rinaldo

Machine Learning (Statistics) · Cross-listed from math.NA

Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics

Designing sampling algorithms for multimodal targets that remain stable under refinement of the finite-dimensional approximation of an underlying function-space problem is a central challenge. Annealed Langevin dynamics (ALD) is a natural alternative to classical Langevin in this context, since it is often observed to improve exploration across modes. Yet a gap remains between its empirical success and existing theory: under which conditions can ALD be guaranteed to remain stable across dimensions? In this paper, we bridge this gap by providing a uniform-in-dimension analysis of continuous-time ALD for Gaussian-mixture targets. Along an explicit annealing path obtained by gradually removing Gaussian smoothing from the target, we identify spectral conditions linking the smoothing covariance to the component covariances under which ALD achieves a prescribed accuracy in Kullback-Leibler divergence within a dimension-uniform time horizon. We then establish stability in a perturbative regime with imperfect initialization and approximate scores. Under a misspecified-mixture score model, we show that preconditioning ALD with an operator whose spectrum decays sufficiently fast prevents error terms from accumulating across coordinates and thereby preserves dimension-uniform control.
Read the full abstract

All authors Lorenzo Baldassari, Josselin Garnier, Knut Solna, Maarten V. de Hoop

Machine Learning (Statistics) · Cross-listed from cs.LG

Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data

The problem of corrupted data, missing features, or missing modalities continues to plague the modern machine learning landscape. To address this issue, a class of regularization methods that enforce consistency between imputed and fully observed data has emerged as a promising approach for improving model generalization, particularly in partially observed settings. We refer to this class of methods as Measure Consistency Regularization (MCR). Despite its empirical success in various applications, such as image inpainting, data imputation and semi-supervised learning, a fundamental understanding of the theoretical underpinnings of MCR remains limited. This paper bridges this gap by offering theoretical insights into when MCR yields a more favorable finite-sample estimation-error upper bound, viewed through the lens of neural network distance. Under ideal interpolation and compatibility conditions, we show that the MCR estimation-error upper bound is no larger than vanilla supervised training and becomes strictly smaller when the mixed-sample alternative is favorable. We then extend the analysis to the non-ideal regime, where optimization and compatibility residuals can potentially offset this finite-sample advantage. Guided by these insights, we propose a novel practical diagnostic that leverages the duality gap and a calibrated drift estimator to infer the potential benefit of MCR training. We present detailed empirical evidence to support our theoretical claims and to show the effectiveness and accuracy of our practical diagnostic. We further provide simulations on real-world datasets to show the versatility of MCR under different model architectures designed for different data sources.
Read the full abstract

Statistics Theory · Cross-listed from cs.LG

Phase Transitions for Feature Learning in Neural Networks

According to a popular viewpoint, neural networks learn from data by first identifying low-dimensional representations, and subsequently fitting the best model in this space. Recent works provide a formalization of this phenomenon when learning multi-index models. In this setting, we are given $n$ i.i.d. pairs $({\boldsymbol x}_i,y_i)$, where the covariate vectors ${\boldsymbol x}_i\in\mathbb{R}^d$ are isotropic, and responses $y_i$ only depend on ${\boldsymbol x}_i$ through a $k$-dimensional projection ${\boldsymbol \Theta}_*^{{\sf T}}{\boldsymbol x}_i$. Feature learning amounts to learning the latent space spanned by ${\boldsymbol \Theta}_*$. In this context, we study the gradient descent dynamics of two-layer neural networks under the proportional asymptotics $n,d\to\infty$, $n/d\to\delta$, while the dimension of the latent space $k$ and the number of hidden neurons $m$ are kept fixed. Earlier work establishes that feature learning via polynomial-time algorithms is possible if $\delta> \delta_{\text{alg}}$, for $\delta_{\text{alg}}$ a threshold depending on the data distribution, and is impossible (within a certain class of algorithms) below $\delta_{\text{alg}}$. Here we derive an analogous threshold $\delta_{\text{NN}}$ for two-layer networks. Our characterization of $\delta_{\text{NN}}$ opens the way to study the dependence of learning dynamics on the network architecture and training algorithm. The threshold $\delta_{\text{NN}}$ is determined by the following scenario. Training first visits points for which the gradient of the empirical risk is large and learns the directions spanned by these gradients. Then the gradient becomes smaller and the dynamics becomes dominated by negative directions of the Hessian. The threshold $\delta_{\text{NN}}$ corresponds to a phase transition in the spectrum of the Hessian in this second phase.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

DCD: Decomposition-based Causal Discovery from Autocorrelated and Non-Stationary Temporal Data

Multivariate time series in domains such as finance, climate science, and healthcare often exhibit long-term trends, seasonal patterns, and short-term fluctuations, complicating causal inference under non-stationarity and autocorrelation. Existing causal discovery methods typically operate on raw observations, making them vulnerable to spurious edges and misattributed temporal dependencies. We introduce a decomposition-based causal discovery framework that separates each time series into trend, seasonal, and residual components and performs component-specific causal analysis. Trend components are assessed using stationarity tests, seasonal components using kernel-based dependence measures, and residual components using constraint-based causal discovery. The resulting component-level graphs are integrated into a unified multi-scale causal structure. This approach isolates long- and short-range causal effects, reduces spurious associations, and improves interpretability. Across extensive synthetic benchmarks and real-world climate data, our framework more accurately recovers ground-truth causal structure than state-of-the-art baselines, particularly under strong non-stationarity and temporal autocorrelation.
Read the full abstract

Machine Learning (Statistics)

Robust Generalization with Adaptive Optimal Transport Priors for Decision-Focused Learning

Few-shot learning requires models to generalize under limited supervision while remaining robust to distribution shifts. Existing Sinkhorn Distributionally Robust Optimization (DRO) methods provide theoretical guarantees but rely on a fixed reference distribution, which limits their adaptability. We propose a Prototype-Guided Distributionally Robust Optimization (PG-DRO) framework that learns class-adaptive priors from abundant base data via hierarchical optimal transport and embeds them into the Sinkhorn DRO formulation. This design enables few-shot information to be organically integrated into producing class-specific robust decisions that are both theoretically grounded and efficient, and further aligns the uncertainty set with transferable structural knowledge. Experiments show that PG-DRO achieves stronger robust generalization in few-shot scenarios, outperforming both standard learners and DRO baselines.
Read the full abstract

Machine Learning (Statistics)

Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

Importance-weighted variational inference approximates intractable densities by optimizing multi-sample Monte Carlo objectives whose tightness increases with the number of importance samples $N$. When reparameterization is unavailable, optimization relies on score-function (REINFORCE) gradient estimators. In this paper, we provide a comprehensive analysis of score-function gradient estimators for importance-weighted objectives by characterizing their asymptotic expectation and variance. We establish that standard variational inference for Monte Carlo objectives (VIMCO) gradient estimators suffer from a vanishing signal-to-noise ratio (SNR) as $N$ increases, which arises from a suboptimal control variate choice and can impair optimization. To address this issue, we derive a general mechanism for constructing VIMCO gradient estimators that avoid SNR collapse. We identify the High-Effective Sample Size optimal variance importance sampling estimator as belonging to this category of estimators and establish its $\sqrt{N}$ SNR scaling. This estimator does not, however, have zero variance at optimality and achieving its $\sqrt{N}$ SNR rate may require prohibitively large $N$ when the importance weights are degenerate. We overcome these limitations by introducing VIMCO gradient estimators that attain zero variance at optimality and support importance weight tempering. Experiments with discrete latent-variable and state-space models validate the theoretical predictions and illustrate the benefits of our estimators.
Read the full abstract

Machine Learning (Statistics)

Online Social Welfare Function-based Resource Allocation

In many real-world settings, a centralized decision-maker must repeatedly allocate finite resources to a population over multiple time steps. Individuals who receive a resource derive some stochastic utility; to characterize the population-level effects of an allocation, the expected individual utilities are then aggregated using a social welfare function (SWF). We formalize this setting and present a general confidence sequence framework for SWF-based online learning and inference, valid for any monotonic, concave, and Lipschitz-continuous SWF. Our key insight is that monotonicity alone suffices to lift confidence sequences from individual utilities to anytime-valid bounds on optimal welfare. Building on this foundation, we propose SWF-UCB, a SWF-agnostic online learning algorithm that achieves near-optimal $\tilde{O}(n+\sqrt{nkT})$ regret (for $k$ resources distributed among $n$ individuals at each of $T$ time steps). We instantiate our framework on three normatively distinct SWF families: Weighted Power Mean, Kolm, and Gini, providing bespoke oracle algorithms for each. Experiments confirm $\sqrt{T}$ scaling and reveal rich interactions between $k$ and SWF parameters. This framework naturally supports inference applications such as sequential hypothesis testing, optimal stopping, and policy evaluation.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

An Odd Estimator for Shapley Values

The Shapley value is a ubiquitous framework for attribution in machine learning, encompassing feature importance, data valuation, and causal inference. However, its exact computation is generally intractable, necessitating efficient approximation methods. While the most effective and popular estimators leverage the paired sampling heuristic to reduce estimation error, the theoretical mechanism driving this improvement has remained opaque. In this work, we provide an elegant and fundamental justification for paired sampling: we prove that the Shapley value depends exclusively on the odd component of the set function, and that paired sampling orthogonalizes the regression objective to filter out the irrelevant even component. Leveraging this insight, we propose OddSHAP, a novel consistent estimator that performs polynomial regression solely on the odd subspace. By utilizing the Fourier basis to isolate this subspace and employing a proxy model to identify high-impact interactions, OddSHAP overcomes the combinatorial explosion of higher-order approximations. Through an extensive benchmark, we find that OddSHAP achieves state-of-the-art estimation accuracy at larger sampling budgets.
Read the full abstract

All authors Fabian Fumagalli, Landon Butler, Justin Singh Kang, Kannan Ramchandran, R. Teal Witter

Machine Learning (Statistics) · Cross-listed from cs.CL

On the Power of (Approximate) Reward Models for Inference-Time Scaling

Inference-time scaling has recently emerged as a powerful paradigm for improving the reasoning capability of large language models. Among various approaches, Sequential Monte Carlo (SMC) has become a particularly important framework, enabling iterative generation, evaluation, rejection, and resampling of intermediate reasoning trajectories. A central component in this process is the reward model, which evaluates partial solutions and guides the allocation of computation during inference. However, in practice, true reward models are never available. All deployed systems rely on approximate reward models, raising a fundamental question: Why and when do approximate reward models suffice for effective inference-time scaling? In this work, we provide a theoretical answer. We identify the Bellman error of the approximate reward model as the key quantity governing the effectiveness of SMC-based inference-time scaling. For a reasoning process of length $T$, we show that if the Bellman error of the approximate reward model is bounded by $O(1/T)$, then combining this reward model with SMC reduces the computational complexity of reasoning from exponential in $T$ to polynomial in $T$. This yields an exponential improvement in inference efficiency despite using only approximate rewards.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.CL

Context Dependence and Reliability in Autoregressive Language Models

Large language models (LLMs) generate outputs by utilizing extensive context, which often includes redundant information from prompts, retrieved passages, and interaction history. In critical applications, it is vital to identify which context elements actually influence the output, as standard explanation methods struggle with redundancy and overlapping context. Minor changes in input can lead to unpredictable shifts in attribution scores, undermining interpretability and raising concerns about risks like prompt injection. This work addresses the challenge of distinguishing essential context elements from correlated ones. We introduce RISE (Redundancy-Insensitive Scoring of Explanation), a method that quantifies the unique influence of each input relative to others, minimizing the impact of redundancies and providing clearer, stable attributions. Experiments demonstrate that RISE offers more robust explanations than traditional methods, emphasizing the importance of conditional information for trustworthy LLM explanations and monitoring.
Read the full abstract

All authors Poushali Sengupta, Shashi Raj Pandey, Sabita Maharjan, Frank Eliassen

Statistical Methodology

A Fractional M/M/1 Queue Governed by Stretched Non-Local Time Operators

We introduce a non-Markovian generalization of the classical M/M/1 queue by incorporating extended nonlocal time dynamics into Kolmogorov forward equations. We obtain the model by replacing the standard time derivative with an extended Caputo-type operator. It preserves the birth-death transition structure of the standard queue while introducing memory effects into the temporal evolution. We derive explicit representations for transient state probabilities in terms of the Kilbas-Saigo function, which naturally emerges as the relaxation kernel associated with the stretched operator, using Laplace transform techniques. We construct a time-varying interpretation and show that the fractional queue can be viewed as a distribution of a classical M/M/1 process evaluated at a non-decreasing random time. It is observed that the fractional queue can be viewed as a distribution of a classical M/M/1 process evaluated at a non-decreasing random time. We prove that under the standard stability condition $\rho<1$, the steady-state distribution remains geometric and coincides with the distribution of the classical queue, whilst we prove that the stretched fractional parameters significantly affect the convergence rate in the transient regime. Numerical examples based on Monte Carlo simulations highlight the effect of the parameters $(\alpha,\gamma)$ on the distribution of empty states, tail length distributions, and the average tail evolution, and validate the flexibility of the proposed framework in capturing long-memory tail dynamics.
Read the full abstract

Statistics Applications · Cross-listed from cs.CR

Adaptive Quantum-Safe Cryptography for 6G Vehicular Networks via Context-Aware Optimization

Powerful quantum computers in the future may be able to break the security used for communication between vehicles and other devices (Vehicle-to-Everything, or V2X). New security methods called post-quantum cryptography can help protect these systems, but they often require more computing power and can slow down communication, posing a challenge for fast 6G vehicle networks. In this paper, we propose an adaptive post-quantum cryptography (PQC) framework that predicts short-term mobility and channel variations and dynamically selects suitable lattice-, code-, or hash-based PQC configurations using a predictive multi-objective evolutionary algorithm (APMOEA) to meet vehicular latency and security constraints.However, frequent cryptographic reconfiguration in dynamic vehicular environments introduces new attack surfaces during algorithm transitions. A secure monotonic-upgrade protocol prevents downgrade, replay, and desynchronization attacks during transitions. Theoretical results show decision stability under bounded prediction error, latency boundedness under mobility drift, and correctness under small forecast noise. These results demonstrate a practical path toward quantum-safe cryptography in future 6G vehicular networks. Through extensive experiments based on realistic mobility (LuST), weather (ERA5), and NR-V2X channel traces, we show that the proposed framework reduces end-to-end latency by up to 27%, lowers communication overhead by up to 65%, and effectively stabilizes cryptographic switching behavior using reinforcement learning. Moreover, under the evaluated adversarial scenarios, the monotonic-upgrade protocol successfully prevents downgrade, replay, and desynchronization attacks.
Read the full abstract

All authors Poushali Sengupta, Mayank Raikwar, Sabita Maharjan, Frank Eliassen, Yan Zhang

Statistics Theory · Cross-listed from cs.LG

High-accuracy sampling for diffusion models and log-concave distributions

We present algorithms for diffusion model sampling which obtain $\delta$-error in $\mathrm{polylog}(1/\delta)$ steps, given access to $\widetilde O(\delta)$-accurate score estimates in $L^2$. This is an exponential improvement over all previous results. Specifically, under minimal data assumptions, the complexity is $\widetilde O(d_\star \mathrm{polylog}(1/\delta))$ where $d_\star$ is the intrinsic dimension of the data. Further, under a non-uniform $L$-Lipschitz condition, the complexity reduces to $\widetilde O(L \mathrm{polylog}(1/\delta))$. Our approach also yields the first $\mathrm{polylog}(1/\delta)$ complexity sampler for general log-concave distributions using only gradient evaluations.
Read the full abstract

All authors Fan Chen, Sinho Chewi, Constantinos Daskalakis, Alexander Rakhlin

Machine Learning (Statistics) · Cross-listed from cs.LG

Multi-LLM Adaptive Conformal Inference for Reliable LLM Responses

Ensuring factuality is essential for the safe use of Large Language Models (LLMs) in high-stakes domains such as medicine and law. Conformal inference provides distribution-free guarantees, but existing approaches are either overly conservative, discarding many true-claims, or rely on adaptive error rates and simple linear models that fail to capture complex group structures. To address these challenges, we reformulate conformal inference in a multiplicative filtering setting, modeling factuality as a product of claim-level scores. Our method, Multi-LLM Adaptive Conformal Inference (MACI), leverages ensembles to produce more accurate factuality-scores, which in our experiments led to higher retention, while validity is preserved through group-conditional calibration. Experiments show that MACI consistently achieves user-specified coverage with substantially higher retention and lower time cost than baselines. Our repository is available at https://github.com/MLAI-Yonsei/MACI
Read the full abstract

All authors Kangjun Noh, Seongchan Lee, Ilmun Kim, Kyungwoo Song

Statistical Methodology

Explicit Expressions for Multidimensional Value-at-Risk under Archimedean Copulas

This paper studies multivariate Value-at-Risk (VaR) for financial portfolios with a focus on modeling dependence structures through Archimedean copulas. Using the generator representation of Archimedean copulas, we derive explicit analytical expressions for the marginal lower-tail multivariate VaR in arbitrary dimensions. Closed-form formulas are obtained for several commonly used copula families, including Clayton, Frank, Gumbel-Hougaard, Joe and Ali–Mikhail–Haq copulas, allowing a direct assessment of the impact of dependence on multivariate risk. These results complement existing approaches, which largely rely on numerical or simulation-based methods, by providing tractable alternatives for theoretical and applied risk analysis. Monte Carlo simulations are conducted to evaluate the finite-sample performance of the proposed VaR estimator and to illustrate the role of different dependence structures. The proposed analytical setting offers transparent tools for multivariate risk measurement and systemic risk assessment.
Read the full abstract

Statistical Methodology

Estimation of Tsallis entropy and its applications to goodness-of-fit tests

In this paper, we consider the problem of estimating Tsallis entropy from a given data set. We propose four different estimators for Tsallis entropy measure based on higher-order sample spacings, and then discuss estimation of Tsallis divergence measure. We compare the performance of the proposed estimators by means of bias and mean squared error and also examine their robustness to outliers. Next, we propose a spacings-based estimator for Tsallis entropy under progressive type-II censoring and study its performance using Monte Carlo simulations. Another estimator for Tsallis entropy is proposed using quantile function and its consistency and asymptotic normality are studied, and its performance is evaluated through Monte Carlo simulations. Goodness-of-fit tests for normal and exponential distributions as applications are developed using Tsallis divergence measure. The performance of the proposed tests are then compared with some known tests using simulations and it is shown that the proposed tests perform very well. Also, an exponentiality test under progressive type-II censoring is proposed, its performance is compared with an existing entropy-based test using simulation. It is observed that the proposed test performs well. Finally, some real data sets are analysed for illustrative purposes.
Read the full abstract

Statistics Theory

Estimating Conditional Distributions via Sklar's Theorem and Empirical Checkerboard Approximations, with Consequences to Nonparametric Regression

We tackle the natural question of whether it is possible to estimate conditional distributions via Sklar's theorem by separately estimating the conditional distributions of the underlying copula and the marginals. Working with so-called empirical checkerboard/Bernstein approximations with suitably chosen resolution/degree, we first show that uniform weak convergence to the true underlying copula can be established under very mild regularity assumptions. Building upon these results and plugging in the univariate empirical marginal distribution functions we then provide an affirmative answer to the afore-mentioned question and prove strong consistency of the resulting estimators for the conditional distributions. Moreover, we show that aggregating our estimators allows to construct consistent nonparametric estimators for the mean, the quantile, and the expectile regression function, and beyond. Some simulations illustrating the performance of the estimators and a real data example complement the established theoretical results.
Read the full abstract

Statistics Applications

Simultaneous Estimation of Seabed and Its Roughness With Longitudinal Waves

This paper introduces an infinite-dimensional Bayesian framework for acoustic seabed tomography, leveraging wave scattering to simultaneously estimate the seabed and its roughness. Tomography is considered an ill-posed problem where multiple seabed configurations can result in similar measurement patterns. We propose a novel approach focusing on the statistical isotropy of the seabed. Utilizing fractional differentiability to identify seabed roughness, the paper presents a robust numerical algorithm to estimate the seabed and quantify uncertainties. Extensive numerical experiments validate the effectiveness of this method, offering a promising avenue for large-scale seabed exploration.
Read the full abstract