Statistical Methodology
Forecasting multiple density-valued time series
Han Lin Shang
Density-valued time series are common in practice. However, since these functions are non-negative and must integrate to one, they do not form a linear vector space, which complicates the direct application of standard time-series methods. To address these constraints, we introduce two transformations, based on the centered log-ratio transformation and the cumulative distribution function, for modeling and forecasting balanced panel data represented by multiple densities. Both transformations map densities bijectively into an unconstrained function space in which cross-sectional and serial dependence can be modeled directly. Within the resulting unconstrained space, we consider two approaches: multi-population functional time-series modeling to account for correlations among populations, and gap modeling between gender and region. Using Japanese subnational age-specific life-table death counts from 1973 to 2024, we evaluate and compare the one- to 20-step-ahead point and interval forecast accuracy of the two approaches for each transformation, and offer some general recommendations.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.CL
Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation
Thomas Reiter, Christoph Kern, Fedor Miasnikov et al. (+4 more)
Large
language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task designs, three independent runs, and 3,000 tweets labeled for offensive
language and hate
speech. Repeating the same model and task design produced high agreement (median Fleiss' $\kappa = 0.91$). Agreement fell when we changed the task design for the same tweets (median Cohen's $\kappa = 0.76$). Task design and model choice increased the variance of estimated prevalence by factors of 76.7 for offensive
language and 110.6 for hate
speech compared with sampling variance alone. Variation across LLM task designs reached 560-572 basis points, compared with 270-331 basis points across five human instrument versions. Confidence scores did not solve this problem. They tracked repeated model outputs more closely than agreement with human labels, and grouping six tweets in one
prompt lowered mean offensive-
language confidence by 660 basis points. We call the variation caused by task design and model choice instrument uncertainty. Researchers can measure it only by comparing reasonable task designs. Repeating one setup or relying on confidence scores cannot replace that test.
Read the full abstract ▾
All authors Thomas Reiter, Christoph Kern, Fedor Miasnikov, Sofiia Nikolenko, Rob Chew, Stephanie Eckman, Frauke Kreuter
Machine Learning (Statistics)
Stochastic Inertial Krasnosel'skii-Mann Iteration Achieves Near-Optimal Sample Complexity
Tong Yang, Tao Jiang, Yuejie Chi et al. (+2 more)
We analyze a simple
stochastic inertial Krasnosel'skii–Mann (iKM) method for finding a fixed point of a nonexpansive operator in a real
Hilbert space. Our method is obtained simply by adding two inertial extrapolations to
stochastic KM [Bravo and Cominetti, 2024], and it retains one call to a possibly biased
stochastic oracle per update and achieves sharp rates in both the
stochastic and deterministic regimes. Specifically, with our proposed parameter schedule, we prove the following last-iterate fixed-point residual bound: \[ {O}\!\left(\frac{1}{K} +\frac{\sigma\log K}{\sqrt K} +\frac{B_K\log K}{K}\right), \] where $K$ is the horizon, $\sigma$ is the noise level and $B_K$ is the accumulated root-mean-square bias. When $B_K=O(\sqrt K)$, this yields $\widetilde O(\epsilon^{-2})$ sample complexity that matches, up to a logarithmic factor, the
stochastic-oracle lower bound given under the unbiased subclass of our model [Foster et al., 2019,
Theorem 2]. It also improves the best-known $O(\epsilon^{-4})$ random-iterate guarantee for
stochastic KM [Bravo and Cominetti, 2024, Corollary 5.4]. To our knowledge, this is the first single-loop method for general nonexpansive fixed-point problems to attain this near-optimal sample complexity without variance reduction or batching. When the oracle is exact, the same method attains the worst-case-optimal $O(K^{-1})$ last-iterate residual rate [Park and Ryu, 2022,
Theorem 4.6], improving the $O(K^{-1/2})$ rate of classical KM [Cominetti et al., 2014; Bravo and Cominetti, 2018].
Read the full abstract ▾
All authors Tong Yang, Tao Jiang, Yuejie Chi, Ashok Cutkosky, Lin Xiao
Machine Learning (Statistics)
Sequential Confidence Sets for Coverage-Constrained Conformal Model Selection
Jing Li and Haibin Zhu
Modern conformal forecasting systems often maintain several adaptive pipelines that differ in base forecasters, conformity scores, calibration windows, and update rules. Comparing them is difficult because coverage is a hard constraint, whereas efficiency should be optimized only among feasible pipelines. We formulate this problem as sequential inference for a
stochastic constrained argmin. At each time, the target is the set of minimum-cost pipelines satisfying multiple prefix-average conditional miscoverage constraints. We introduce Coverage-Constrained Sequential Model Confidence Sets (CC-SMCS), which separate certifiably feasible, possibly feasible, and possibly constrained-optimal pipelines. Using simultaneous martingale confidence sequences, CC-SMCS projects a rectangular confidence region onto the constrained argmin and admits an exact closed-form rule. With probability at least $1-\delta$, it contains every constrained-optimal pipeline simultaneously over all times. This finite-sample guarantee requires no stationarity or mixing assumptions and remains valid under data-dependent stopping. We also establish an impossibility result for safe certification at the coverage boundary and extend the construction to delayed multi-horizon feedback and outcome-dependent efficiency objectives.
Read the full abstract ▾
Machine Learning (Statistics)
Certified Task-Conditioned Active Observability
Linzhe Zhang and Changming Xu
Before acting upon an unobservable physical system, an autonomous
agent must determine which latent distinctions govern downstream tasks, how many active interventions are necessary to certify them, and when to abstain to prevent catastrophic errors. Classical observability treats state reconstruction as an unconditioned binary predicate, failing when passive observations cannot break latent degeneracies without perturbation, full microscopic inversion is prohibitively costly, and distinguishing task-irrelevant degrees of freedom wastes interaction budgets. We formalize task-conditioned active observability complexity: the minimum worst-case expected interaction cost required to identify task-relevant states under certified error and safe abstention guarantees. We prove that task-predictive equivalence induces the unique minimal sufficient quotient $\mathcal{H}/\!\sim_\tau$, leaving active observability complexity strictly invariant while eliminating superfluous distinctions. In deterministic regimes, this complexity is characterized by an optimal adaptive distinguishing tree and Bellman recursion; in noisy regimes, it obeys a stopped-transcript relative-
entropy lower bound and adaptive martingale certificates that compose without independence assumptions. We instantiate a prospective certified observer with staged recovery: a nominal verifier defers candidate compilation, triggering active probing only upon evidence, while a history-measurable score shell prunes hypotheses without sacrificing risk bounds. Stress audits across high-dimensional physical systems and thousands of operational trials demonstrate certified state recovery with zero false acceptances and substantial reductions in sensor reads and model steps.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CR
The Like Trap: Multi-Stage Poisoning against Agents in Similarity-based Recommendation Systems
Yue Xing, Pengfei He and Zitao Li
With recent advancements in large
language models (LLMs) and LLM-based
agents, these
agents are becoming increasingly autonomous and gaining broader access to act on users' behalf on the internet. However, the vulnerability of automated
agents deployed on social media platforms (e.g., for managing a user's personal account) remains underexplored. Existing studies on
agent poisoning typically assume that the adversary can expose poisoned content to the
agent. Although such an attack is direct and effective, it is more easily detected and mitigated. In the context of social media platforms, this leaves open whether the recommendation system itself would surface such content to the
agent in a more subtle manner. Through theoretical analysis, we show that the like-score mechanism used in OASIS can be exploited, and we characterize the conditions under which a multi-stage chain of poisoned posts can steer the
agent's feed. Based on these insights, we further develop an
algorithm that crafts realistic poisoned posts. Experiments support our theoretical findings and demonstrate the effectiveness of the proposed
algorithm. Notably, by exploiting the like-score feedback loop, the attack causes the recommendation system to select poisoned posts even when their user-post similarity falls below the
retrieval threshold.
Read the full abstract ▾
Statistical Methodology
Bayesian Nonparametric Approaches to Ordinal Drought Modeling in the United States
Mostafa Shams, Robert Erhardt and Staci Hepler
Data observed over space and time can exhibit dependence across both dimensions, and
datasets can grow very large in both the number of locations and the number of time periods. This dependence and data size mean that many statistical models cannot be fit at a reasonable computational cost as a result of dense
matrix inversions, large parameter spaces, or
memory and storage challenges. When the data are ordinal, this only adds to the computational complexity of model fitting. Many ordinal models rely on a latent continuous variable designed to capture dependence in a computationally efficient manner, and then partitioned according to cutoff parameters to yield the observed ordinal response data. A very common choice is a latent Gaussian distribution, which can accommodate different dependence structures and permits Gibbs sampling in a
Bayesian framework. Unfortunately, this model can be overly restrictive and fails to provide the flexibility needed to capture a range of ordinal outcomes. In this paper, we demonstrate the use of
Bayesian nonparametric (BNP) methods to enhance the model flexibility of a Gaussian latent model for ordinal drought data observed over space and time, with Dirichlet process priors inducing clustering among time periods within each spatial location. In this work, we model ordinal drought data separately at each spatial location while accounting for temporal dependence, but we do not model spatial dependence across locations. We show that these BNP models often outperform
Bayesian parametric approaches at a reasonable computational cost.
Read the full abstract ▾
Statistics Theory · Cross-listed from quant-ph
Dual Boundary Condition Inference: A Two-Boundary Product Rule and Its Implications
Luis Razo and Eliahu Cohen
Many inference problems are constrained from two sides: forward information and a second constraint on outcomes. Dual Boundary Condition Inference (DBCI) multiplies them and renormalises. The rule is not new: it is the product-of-experts form and the unit-exponent member of the logarithmic-pooling family. The paper contributes a classification and reducibility characterisation. The
algebra does not fix the exponent at which the inputs combine: given the statistic log p_fwd + log p_bwd, which already builds in the product form, maximum
entropy supplies the family (p_fwd p_bwd)^lambda and selects no member. If the inputs are read as two equally weighted opinions, three requirements select lambda = 1/2: unanimity preservation, consistency under a common
Bayesian update, and minimal symmetric Kullback-Leibler divergence. If they are read as separately applied factors, two requirements select lambda = 1: a neutral second input must leave the first unchanged, and an update to one input must pass through unchanged. For a rank-one intermediate projective measurement, the Aharonov-Bergmann-Lebowitz (ABL) rule realises this form with no extra parameter. Every positive exponent orders outcomes identically, so the choice is invisible to picking the likeliest outcome, but not to scoring a class by total mass. Under the relevant identifications, the same product form gives Bayes' rule, while a final effect proportional to the identity returns the ordinary forward Born probabilities. The second result is a Two-Boundary Reducibility Criterion: DBCI factors through the forward boundary exactly when the effective backward boundary, up to positive rescaling, does. So a fixed prior or model-fixed structural constraint adds no distinctions between cases sharing the same forward boundary, though it may encode substantial information and still materially affect the result.
Read the full abstract ▾
Statistical Methodology
A Bayesian framework for multilevel data under model mis-specification
Widemberg S. Nobre, David A. Stephens, Alexandra M. Schmidt et al. (+1 more)
We propose a
Bayesian framework for uncertainty quantification from the perspective that the working model is mis-specified in settings of a multilevel data-generating process. We focus on settings in which the mis-specification fails to match the functional form of the mean structure, and discuss
Bayesian estimation of target parameters under dependence induced by a mismatch between working and data-generating models. The proposal represents a
Bayesian semi-parametric procedure aimed at estimating population-level parameters while accounting for cluster- and unit-level variation in the estimating function. The proposal extends the regular
Bayesian bootstrap to account for cluster- and unit-level variation using multilevel weights from an enriched Dirichlet model. Simulation studies indicate that the proposed approach has good frequentist properties when the data-generating process and the proposed model induce a partially exchangeable sequence associated with the unknown quantity of interest. Applications to radon (Gelman and Hill, 2007), Programme for International Student Assessment 2022 (OECD, 2023), and tuberculosis (Nobre et al., 2023)
datasets are presented for illustrative purposes. The results demonstrate that the proposed method is competitive with variations of multilevel models, with major differences observed in the range of credible intervals, which are justified by the nonparametric assumptions underlying the proposed method.
Read the full abstract ▾
All authors Widemberg S. Nobre, David A. Stephens, Alexandra M. Schmidt, Erica E. M. Moodie
Statistical Methodology
Formulating Cross-World Mediation Estimands Through Single-World Mixtures
Razieh Nabi and David Benkeser
In causal mediation analysis, natural direct and indirect effects are defined through nested counterfactuals that combine the outcome under one exposure level with the mediator value under another and are therefore inherently cross-world. Their canonical identification additionally relies on cross-world independence assumptions. Consequently, both the estimands and their identifying assumptions remain controversial. In this paper, we ask what additional single-world structure would be required to re-express cross-world estimands as single-world quantities, identifiable under single-world assumptions. We show that these quantities can be written as mixtures of controlled single-world effects under assumptions involving a susceptibility marker, a possibly latent baseline variable that encodes the "would-be" mediator value under a reference treatment arm. If observed, the marker would make several implications of the marker restrictions empirically testable. These results provide a transparent single-world formulation of cross-world mediation effects while making explicit the assumptions required. Although these conditions clarify how cross-world estimands can be interpreted within a single-world framework, their practical relevance depends on whether such susceptibility markers can be justified in specific applications. We discuss implications for principal stratification and show in the Supplementary Material how the formulation extends to ordered mediators and settings with exposure-induced mediator-outcome confounding.
Read the full abstract ▾
Statistical Methodology
CVaR anchor regression protects against rare shifts
Malte Londschien
We study prediction in new environments when training data contain rare, large shifts. Anchor regression penalizes the average of the squared mean residual across environments. It protects against shifts in an ellipsoid determined by the second moment of the training shifts. Covering rare shifts may therefore require a large penalty, expanding the ellipsoid in every direction and reducing accuracy on common environments. We propose CVaR anchor regression, which replaces the average of the squared mean residuals with a tail average. Unlike CVaR or GroupDRO applied directly to prediction risks, it does not give environments more weight solely because their noise levels are high. We prove an exact worst-case risk guarantee under a linear structural model that allows for heteroscedastic noise. For discrete environments, decreasing the CVaR tail fraction expands the robustness set from an ellipsoid to a scaled convex hull of the training shifts and their negatives. A separate parameter controls its scale. Examples show how the method can improve protection against rare shifts while retaining accuracy on common environments. We illustrate the method on New York City taxi data.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps
Abbas Mammadov, Jerry Y. Huang, Justin Lin et al. (+5 more)
Reward fine-tuning aims to update a pre-trained flow-based generative model to improve the downstream reward of its generated samples. Existing methods typically formulate this problem as sampling from a reward-tilted distribution, the solution to a KL-regularized reward-maximization problem. Here, we introduce an optimal transport regularizer built directly from the pre-trained drift. Unlike KL reward tilting, the resulting objective transports individual samples toward higher reward rather than reweighting the base distribution. We show that the resulting problem is equivalent to a deterministic optimal control problem on the flow. Given a pre-trained flow map, this equivalence yields a simulation-free reinforcement learning
algorithm for fine-tuning generative flows. We call the resulting framework Wasserstein-Tilted Flow Maps (WTF), the first end-to-end fine-tuning recipe native to flow maps. The output is a fine-tuned flow map that retains strong reward-aligned performance at few-step inference budgets without post-hoc
distillation. Experiments on ImageNet-256 and text-to-image show that WTF achieves higher reward with comparable or higher diversity than baselines, while requiring up to $280\times$ less training compute. More broadly, we argue that accelerated samplers such as flow maps are essential infrastructure for efficient post-training, and that the dominant KL-regularized formulation is only one of many choices worth revisiting.
Read the full abstract ▾
All authors Abbas Mammadov, Jerry Y. Huang, Justin Lin, Partha Kaushik, Sheel Shah, Kartik Nair, Yee Whye Teh, Nicholas M. Boffi
Machine Learning (Statistics)
Tight Regret Bound for Online Inverse Linear Optimization via Multiscale Matrix Weights
Shinsaku Sakaue
We study online inverse linear optimization with a fixed unknown linear utility: in each round, an environment presents a compact action set, the learner recommends an action from it, and the environment returns an action that maximizes the utility over the same set. When the utility vector and the actions lie in the $d$-dimensional Euclidean unit ball, we give a randomized
algorithm whose regret—the cumulative utility shortfall relative to optimal actions—is $O(\sqrt d)$ in expectation for every time horizon, without knowledge of the horizon. The dependence on $d$ is optimal up to a constant factor by the known $\Omega(\sqrt d)$ lower bound for horizons $T\ge d$. Our
algorithm maintains
matrix multiplicative weights on
polynomial feature spaces at geometrically spaced scales. It selects a recommendation distribution by solving a linear program and updates its score
matrices by comparing the available actions with the feedback action. With rational oracle outputs and feedback actions, an implementation computable relative to a linear-optimization oracle preserves the $O(\sqrt d)$ regret bound. Whether the same rate is attainable with running time
polynomial in the dimension, horizon, and input length remains open.
Read the full abstract ▾
Statistics Theory
Rolling Conformal Prediction in Sequential Model Training
Chen Cheng, Ruiting Liang and Rina Foygel Barber
We introduce Rolling Conformal Prediction (rolling-CP), a distribution-free predictive inference method for the setting of sequential model training. Specifically, given a data stream $(X_1,Y_1),(X_2,Y_2),\dots$, at each time $n$ the trained model may depend on the observed history $\{(X_i,Y_i)\}_{i<n}$. This setting arises naturally in modern sequential training, including one-pass training over massive
datasets and continual fine-tuning or test-time adaptation of
language models during deployment. Rolling-CP first calibrates each incoming observation against the current predictor and then rolls it into future training. In this way, we avoid the need for data splitting. Remarkably, although the models at times $n=1,2,\dots$ may have entirely different properties and accuracy levels, for exchangeable data it is nonetheless possible to establish a guarantee of marginal coverage, with a familiar universal factor-two guarantee (a worst case guarantee of $1-2\alpha$ coverage, as compared to the target level $1-\alpha$), without any assumptions of stability or any restrictions on the model training process. For i.i.d. data streams, we further prove high-probability training-conditional validity uniformly over time; under stability conditions, coverage guarantees sharpen towards $1-\alpha$. Numerical experiments on sequential regression, multiclass SGD, and one-pass neural-network training further demonstrate the practical effectiveness of rolling-CP.
Read the full abstract ▾
Statistical Methodology
Additive Nonparametric Regression with Spatial and Network Objects
Rajarshi Guhaniyogi, Pritam Dey, Krishnendu Chandra et al. (+2 more)
This article is motivated by an imaging application from the Adolescent
Brain Cognitive Development (ABCD) study, aiming to predict task-based
brain activation maps from t-fMRI using cortical metrics from structural MRI (s-MRI) and
brain connectivity data from resting-state fMRI (rs-fMRI). Hierarchical
Bayesian modeling is well-suited for integrating diverse imaging data and quantifying prediction uncertainty. However, progress in this field is limited due to challenges in designing joint priors that capture the structures and interconnections between different imaging modalities, along with computational complexity and lack of theoretical assurances. To address these challenges, the article introduces a novel regression framework that treats t-fMRI and s-MRI images as functional data, incorporating additive non-linear effects of both network and functional predictors on the functional response. Specifically, we employ Gaussian process (GP) priors on coefficients related to the functional predictors to capture their intricate functional dependencies with the response. Furthermore, a GP prior is assigned to encapsulate the non-linear nodal effects of the network predictor on the response function. The method is supported by theoretical results on predictive accuracy for the functional response, and is empirically validated through simulation studies and analysis of multi-modal neuroimaging data from the ABCD study.
Read the full abstract ▾
All authors Rajarshi Guhaniyogi, Pritam Dey, Krishnendu Chandra, Aaron Scheffler, Bani K. Mallick
Machine Learning (Statistics) · Cross-listed from math.NA
Gaussian-process surrogate indicators for residual-based adaptive GMsFEM
Siqing Liu, Eric Chung and Yiran Wang
Residual-based adaptive GMsFEM for high-contrast elliptic problems repeatedly evaluates local weighted $H^{-1}$ indicators on every coarse neighborhood, making indicator evaluation a recurring cost in repeated-query settings. We introduce a non-intrusive Gaussian-process (GP) surrogate for the indicator scores used in Dörfler marking. The operational predictor is the GP posterior mean, algebraically equivalent to a kernel ridge regression (KRR)
estimator under the stated convention; it uses compressed local solution and spectral features without changing the multiscale solve, local spectral construction, or basis enrichment. A nonuniform perturbed-marking result quantifies how pointwise score errors affect the exact indicator mass captured by surrogate-selected neighborhoods, while a conditional bounded-discrepancy KRR pathway identifies sufficient assumptions for such score bounds. In controlled held-out in-distribution tests, the surrogate-guided method gives error-versus-DoF trends comparable with classical $H^{-1}$-residual offline adaptivity and evaluates the online indicator component 2.0-2.1 times faster, excluding offline data generation and GP training.
Read the full abstract ▾