Machine Learning (Statistics) · Cross-listed from cs.CL
Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation
Muhammad Aimal Rehman and Chi-Kuang Yeh
Retrieval-augmented generation (RAG) can ground large
language models in external evidence, but retrieved context does not guarantee that generated claims are factually supported. This problem is especially relevant in multi-hop RAG, where
retrieval and
reasoning proceed through multiple dependent stages. We study whether claim-level conformal factuality control, previously developed for RAG, remains effective in this setting. We apply split-conformal claim filtering to multi-hop RAG and evaluate it on HotpotQA, Natural Questions, and TriviaQA using Llama 3.1 8B and GPT-4o-mini, together with a single-hop reference experiment. Across all six multi-hop model-
dataset configurations, increasingly stringent conformal targets consistently increase the fraction of responses whose retained claims are fully supported. At the 95% target, this rate ranges from 95.80% to 97.20%, compared with 55.60%-76.03% without filtering. However, the improvement is strongly selective: only 4.41%-31.09% of generated claims are retained and 9.70%-51.40% of responses remain non-empty at the 95% target. These results show that conformal factuality extends to multi-hop RAG, while demonstrating that nominal reliability must be interpreted jointly with claim retention and abstention.
Read the full abstract ▾
Statistical Methodology
Calibrated Order-Randomized Rosenblatt Tests
Mehrdad Pournaderi
We test whether a multivariate vector X conforms to a specified distribution F, a problem in copula modelling and density forecasting. The Rosenblatt transform reduces it to a test of uniformity, but depends on an arbitrary coordinate ordering that strongly affects power under dependence. We study order randomization: applying the transform under many random orderings and merging the evidence with dependence-robust rules. Reordering conserves the total Mahalanobis signal energy and merely redistributes it, so one ordering is a lucky or unlucky draw. In simulations we observe significant gains in calibrated power over both the expected single random ordering and order-invariant references. Two ingredients are essential: a two-sided base statistic, and a re-estimating parametric bootstrap that restores level under an estimated null and unlocks the gain. The calibrated pooled tests are robust to the departure's shape; no order-invariant reference we compare is: the symmetric-root test collapses on diffuse departures, while the shape-flat chi-squared test trails on concentrated ones. We apply it to a Gaussian foreign-exchange risk model over a decade of daily data on nine currencies, where it detects episodes such as Brexit and COVID. Though we focus on Gaussian nulls, the procedure extends to any null whose conditional distributions can be computed and simulated from.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces
Han Chen and Yingrui Li
Equivalent probability requests can lead to different decisions even when both reports are valid. We introduce probability contracts, a
benchmark that connects exact finite-world posteriors, validated event
alignment, interface coherence, and failure-aware decision evaluation. Across four model-interface configurations on 1,000 worlds, Kev has lower aggregate canonical posterior error than Jev but larger complement and coarsening residuals; accuracy ordering varies by stratum. Jev's Event and Choice interfaces change the binary action on 32.8% of valid pairs at defer cost 0.10. Post-hoc analyses show that disagreement certifies only 11-52% of mean binary pair error and does not consistently outperform confidence for selection. An action-region characterization and a standard scoring-rule identity explain averaging's expected Brier guarantee relative to random interface selection, but not a decision-loss guarantee at each cost. The loss contrast takes both signs on a 99-cost grid for every configuration; small penalties where both policies beat deferral have pointwise intervals containing zero. Secondary checks specified before collection include a separate 400-root cohort, where Event/Choice effects remain configuration-dependent. A joint surface-order and answer-ID intervention shifts posttrained probabilities. Both bounded
reasoning arms yield no valid probability reports, leaving their probability accuracy undefined. Probability contracts make these distinctions measurable by evaluating event semantics, posterior error, coverage, and decision cost together.
Read the full abstract ▾
Machine Learning (Statistics)
Wasserstein Causal Forests for Distribution-Valued Outcomes
Hugo Gobato Souto
This paper proposes Wasserstein Causal Forests (WCF) for settings in which each unit's outcome is itself a probability distribution. This study also defines finite-grid transformed average and conditional average treatment effects, including a reference-distance contrast that asks whether treatment moves unit-level distributions toward a prespecified
benchmark. Simulations cover null effects, location and shape changes, limited overlap, equal-mean but different laws, heterogeneous effects, multimodality, and structural zeros. WCF is most accurate on the conditional-law metric in most reported designs and sharply improves reference-effect estimation in the principal location-and-shape settings, but it is less accurate than the forest baselines for
multimodal settings. WCF is applied to the famous Project
STAR \citep{word1990state}, revealing that small classes alter more than the mean: they raise within-grade mathematics achievement by $0.161$ standard deviations on average (SE $0.028$); but the gain is not a uniform location shift, it is larger in the upper part of the classroom score distribution ($+0.179$ at the ninetieth percentile versus $+0.091$ at the tenth) and, in descriptive stratum estimates, largest in the schools serving the most economically disadvantaged students (highest free-lunch quartile, $+0.262$, versus $+0.092$ to $+0.164$ elsewhere), while overall dispersion is essentially unchanged.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Posterior Regimes and Latent Deception: Variational Bayesian Inference in Hidden Markov Models for Sequential Fraud Detection in Financial Transactions
Joseph Uririoghene Obukofe, Anthony O'Hare and Chioma Sandra Dike
We present a three-tier progression of Hidden Markov Models: maximum-likelihood (Baum-Welch), variational
Bayesian (VBEM), and a neural variational extension (Neural VBEM), that model each customer's transaction history as a trajectory through a small number of latent behavioural regimes, one of which is empirically identified as fraud-associated. The Neural VBEM HMM replaces the fixed Gaussian-multinomial emission family with a learned encoder, compressing a 741-dimensional transaction representation into a 64-dimensional latent space in which the VBEM HMM's posterior operates; a UMAP projection of this space reveals that the discovered regimes are not discrete clusters but ordered segments of a single continuous behavioural
manifold, with confirmed fraud concentrated at its extreme. We show that the model's natural output, that is, the posterior probability of regime membership, is routinely mistaken for a fraud probability, and quantify the resulting miscalibration (the regime-membership interpretation error, MRIE); a corrected posterior-predictive score, closes most of this gap. We further distinguish batch (smoothed) inference, which uses look-ahead unavailable at deployment time, from filtered (forward-only) inference, and report both. On IEEE-CIS transaction data, the neural tier achieves a 14.4$\times$ fraud enrichment in its identified regime; while its AUPRC trails a discriminative XGBoost baseline, we show this gap is structural and not incidental, and argue the model is best positioned as a calibrated triage and interpretability layer rather than a drop-in ranking replacement.
Read the full abstract ▾
Machine Learning (Statistics)
HARMONIA: Interpretable Graph Learning through Mixtures of Neural Bases
Quan D. Bui, Nguyen Do, An Nguyen Dang et al. (+3 more)
Existing interpretable graph additive models still face limitations in either computational scalability or modeling flexibility. In terms of structural modeling, previous approaches either face quadratic scaling costs or sacrifice explicit source-to-target contribution decomposition. In terms of feature components, they rely either on per-feature neural networks or on single shared bases with limited feature specialization. We address both problems by introducing HARMONIA: Interpretable Graph Learning through Mixtures of Neural Bases, an interpretable-by-design framework. For feature modeling, HARMONIA introduces a Mixture of Neural Bases (MoNB), which routes features to specialized basis experts, enabling parameter sharing without sacrificing feature-specific specialization. For structural modeling, HARMONIA uses Relative Random Walk Probabilities (RRWP) to capture multi-hop and multi-path relationships, and proposes Sparse RRWP Aggregation (SRA) to compute these interactions through sparse graph propagation without quadratic pairwise complexity. HARMONIA retains a simple additive form in which predictions decompose into feature responses modulated by structural influence. Empirically, HARMONIA achieves stronger explanation recovery than existing interpretable graph baselines while maintaining competitive predictive performance and scaling to graphs with millions of nodes. These results show that interpretable graph learning can remain both faithful and scalable without sacrificing predictive effectiveness.
Read the full abstract ▾
All authors Quan D. Bui, Nguyen Do, An Nguyen Dang, Huyen Nguyen, Nhu Duc Minh Nguyen, My T. Thai
Machine Learning (Statistics)
Two-Sample Testing for Inhomogeneous Random Graphs in Non-Integral $L_r$ Norms
Soham Dan
Testing whether two populations of networks share the same edge probabilities is a basic problem in network inference. How hard it is depends on the norm used to measure the difference. For the inhomogeneous Erdős–Rényi (IER) model, the optimal sample complexity is known for every integer $L_r$ norm and for $1\le r<2$. For non-integral $r>2$, however, the known upper and lower bounds do not match, and the lower bound was conjectured to be tight. We study this gap for two-sample testing on aligned vertices. We propose a test that runs two published statistics, of orders $2$ and $\lceil r\rceil$, on the same data and rejects if either one rejects. Its thresholds come from Hölder interpolation, so that both statistics have the same sample cost. We prove that this test attains the conjectured rate. Combined with earlier results, this shows that for every fixed $r\ge1$ the minimax sample complexity is of order $n^{\max\{4/r-1,\,2/r\}}/\epsilon^2$, even when the separation changes with $n$. In simulations with $n$ between 32 and 256, the number of graphs needed for 80% power at level $0.05$ grows with $n$ at a rate consistent with the theory. For $r=2.5$, for example, the fitted exponent is $0.78$, against the theoretical value $0.8$. Interestingly, the two statistics split the work as the interpolation argument suggests: the higher-order statistic is more powerful when only a few edges change, and the $L_2$ statistic when many edges change.
Read the full abstract ▾
Statistical Computation · Cross-listed from math.OC
Mixed-Noise Plug-and-Play with Infimal Convolution Fidelities and Multiple Priors
Ziqi Qin, Hong Ye Tan, Ander Biguri et al. (+2 more)
Plug-and-Play (PnP)
algorithms are a class of iterative methods for inverse imaging. Within an optimization
algorithm, they combine a flexible fidelity term, encoding the forward operator, and a pretrained image denoiser, in order to deal with more severe corruptions such as blurring or downsampling when reconstructing an image. This work studies provably convergent PnP methods for mixed-noise forward processes by using the infimal convolution as a fidelity term, providing a statistical interpretation as a joint maximum a-posteriori
estimator over both noises, and preserving the
Bayesian MAP interpretation of PnP methods. Independently, we extend the PnP formulation to multiple prior terms using the Davis–Yin three-operator splitting. This extension can be combined with either standard fidelities or the proposed mixed-noise infimal-convolution fidelities. We verify that these generalized PnP methods are convergent under standard Kurdyka–Lojasiewicz conditions. Numerical experiments on Laplace-Gaussian and Poisson–Gaussian noise demonstrate stable convergence of single-prior and multiple-prior PnP methods, and divergence under fidelity mismatch. Furthermore, PnP with infimal convolution fidelities are able to scale to noise up to 38% standard deviation, with multiple priors reaching different stationary points that qualitatively preserve more textural properties.
Read the full abstract ▾
All authors Ziqi Qin, Hong Ye Tan, Ander Biguri, Jingwei Liang, Carola-Bibiane Schönlieb
Statistics Applications
Understanding Sectoral Responses to Tariff Uncertainty via a High-Dimensional Mediation Analysis
Qilan Hong and Runze Li
This paper conducts an empirical mediation analysis to examine how U.S. stock-
market sectors responded to the 2025 tariff-policy episode. Using S&P 500 stock data along with hundreds of firm-level financial variables, we study whether sector membership was associated with differences in stock returns and whether these differences were statistically related to firm financial characteristics. For each of five economically distinct tariff-policy windows, we use high-dimensional linear causal mediation models to estimate the direct sector effects. We use a linear regression model to estimate the total sector effects, and then estimate the indrect sector effects by using the difference between the total sector effects and the direct sector effects. To accommodate a large number of candidate mediators, we use a partially penalized least-squares procedure that regularizes financial-variable coefficients while unpenalizes coefficients of all sector indicators. Wald and F-type tests are used to examine whether indirect and direct effects are significant or not, respectively. The results show that tariff news did not produce one uniform
market response. Sector effects vary across five economically distinct tariff-policy windows. The clearest findings are in the direct sector effects. Technology was the sector most negatively affected during the initial decline and escalation collapse, but it also rebounded most strongly during the April 9 relief rally. The results indicate that in addition to Technology, Energy and other trade-exposed sectors were particularly affected during the initial decline and escalation phases. The selected mediators provide evidence that growth, investment, liquidity, profitability, and cash-flow characteristics helped characterize firms associated with sector-level return differences.
Read the full abstract ▾
Statistics Theory
Conformal Coverage of Time Series: Validity and Inference
Percy S. Zhai, Maggie Cheng and Wei Biao Wu
Conformal prediction provides marginal coverage guarantees, yet practitioners may wonder if the observed coverage is truly abnormal or consistent with sampling variation. Inference for realized coverage has received comparatively little
attention, especially for time series. We study split conformal prediction with adjacent calibration and test sets of temporally dependent data. Using the functional dependence measure, we derive non-asymptotic bounds on marginal coverage error without mixing assumptions, which can be difficult to verify and may fail even for simple short-
memory models. We establish a Bahadur representation to derive, to our knowledge, the first central limit
theorem for realized coverage of split conformal prediction under temporal dependence. A consistent block-based
estimator of the standard error yields an asymptotically justified test. We further study long-
memory time series, which remain understudied in conformal prediction. For Gaussian linear processes, we show how very strong temporal dependence can lead to a non-Gaussian limiting law of realized coverage and establish block-sampling inference with an estimated normalization. The resulting theory explains how temporal dependence changes coverage uncertainty.
Read the full abstract ▾
Statistics Theory
Valid and Efficient Split Conformal Regression for Time Series
Percy S. Zhai, Maggie Cheng and Wei Biao Wu
We study conformalized
quantile regression and conformalized median regression that fit a model on one block of a time series and calibrate the conformal interval on the adjacent block. The existing theory of conformal prediction for time series rests largely on mixing conditions, which are hard to verify from a time-series model and fail for many standard processes, including simple ones with short
memory. We replace this theoretical toolbox with the functional dependence measure, which in principle accommodates long-
memory observations. The accuracy of the conformal interval length for time series has been understudied. To the best of our knowledge, this paper is the first work that establishes non-asymptotic coverage guarantees and accuracy of interval length simultaneously for split conformal regression on time series. Furthermore, for Gaussian linear processes with long
memory, where both the estimation of the center and its calibration converge slowly, we establish a sharper rate for the length error. We show that the calibrated length converges faster than the estimated center itself, and provide a matching lower bound for the usual centers when the calibration block is sufficiently large relative to the training block. To our knowledge, this is the first theoretical analysis of conformal interval length dedicated to long
memory.
Read the full abstract ▾
Statistical Methodology
Quantifying Inconsistent Mediation: The Mediated Magnitude Share and the Degree of Suppression
Md. Niamul Islam Sium, Shafayet Khan Shafee, Mohammad Hridoy Patwary et al. (+1 more)
The proportion mediated (PM), defined as the indirect effect divided by the total effect, is intuitive when direct and indirect effects operate in the same direction. Under inconsistent mediation, however, opposing pathways partially cancel each other, causing PM to fall outside the unit interval or become unstable as the total effect approaches zero. We introduce two complementary effect-scale measures: the Mediated Magnitude Share (MMS), which quantifies the absolute magnitude of an indirect pathway relative to the combined absolute magnitudes of the direct and indirect pathways, and the degree of suppression, which quantifies the fraction of this combined magnitude that is cancelled by opposing pathways. We establish the bounds and limiting behavior of both measures in the two-pathway case, show that MMS reduces to PM under consistent mediation, and extend both measures to settings with multiple indirect pathways. For binary outcomes, we extend the framework to odds-ratio and risk-ratio scales by using logarithms to convert multiplicative effects into additive ones, enabling direct application of MMS and the degree of suppression. Together, these measures provide bounded and interpretable characterizations of mediation systems in which pathway directions are not uniform.
Read the full abstract ▾
All authors Md. Niamul Islam Sium, Shafayet Khan Shafee, Mohammad Hridoy Patwary, Bishal Sarker
Statistical Methodology
ViBR-WM: Visual Bayesian Regression for World Modeling
Jifan Li and Ning Ning
Modeling temporal dependence and uncertainty is central to forecasting with world models. The Visual
Bayesian Regression World Model combines visual features, physical histories and known
covariates through interpretable regression, within a modular architecture supporting trend, seasonal and cycle dynamics. Visual compression reduces representation dimension, while
Bayesian variable selection reduces active regression dimension. Posterior prediction combines forecasts across predictor subsets using their posterior probabilities as weights and accounts for parameter uncertainty and future disturbances. The model forecasts joint visual–physical states recursively and physical targets directly. Across four forecasting tasks spanning object motion, vegetation greenness and solar power, ViBR-WM achieves lower mean overall physical-target error than Temporal Straightening, ConvLSTM, PredRNN and SimVP on every task. Repeated fitting and resampling support these overall gains.
Read the full abstract ▾
Machine Learning (Statistics)
An Active-Bottleneck Mechanism for Weak-to-Strong Generalization
Mohammad Zeinalpour and Amir Najafi
Weak-to-strong generalization (W2SG) occurs when a student trained on a teacher's predictions outperforms that teacher. We study when this happens under fully converged, ridgeless two-stage learning, with no early stopping, no explicit regularization, and no assumption that the student is more expressive than the teacher. In two-stage linear regression, a teacher is fit from $n$ labeled examples and a student is trained solely on the teacher's predictions on $m$ fresh, unlabeled inputs. Although both stages share the same hypothesis class and the same training rule, we show that the student outperforms the teacher exactly when $m$ lies in an explicit intermediate range: too few pseudo-labels leave the student without enough signal, too many let it inherit the teacher's noise. Under power-law covariance, we derive this range in closed form as a function of the spectral decay and noise level, including regimes where the improving region splits into two disjoint intervals of $m$. We then study a random-feature model in which the student has strictly more features than the teacher, and identify two regimes, again given by explicit thresholds: one where improvement occurs only for $m$ in a bounded interval, and one where it occurs only once the student width $N_S$ exceeds an explicit threshold. Both regimes are governed by a single "active-bottleneck" principle: whichever of $m$ or $N_S$ is scarcer controls how much teacher error is filtered out, while increasing the other resource only reduces estimation noise. Together, these results show that finite data and finite width can themselves regularize a two-stage learner, with no explicit mechanism doing so.
Read the full abstract ▾