Machine Learning (Statistics) · Cross-listed from cs.LG
Active Learning with Selective Time-Step Acquisition for PDEs
Yegon Kim, Hyunsu Kim, Gyeonghoon Ko et al. (+1 more)
Accurately solving partial differential equations (PDEs) is critical to understanding complex scientific and engineering phenomena, yet traditional numerical solvers are computationally expensive. Surrogate models offer a more efficient alternative, but their development is hindered by the cost of generating sufficient training data from numerical solvers. In this paper, we present a novel framework for active learning in PDE surrogate modeling that reduces this cost. Unlike the existing AL methods for PDEs that always acquire entire PDE trajectories, our approach, STAP (**S**elective **T**ime-Step **A**cquisition for **P**DEs), strategically generates only the most important time steps with the numerical solver, while employing the surrogate model to approximate the remaining steps. This reduces the cost incurred by each trajectory and thus allows the active learning
algorithm to try out a more diverse set of trajectories given the same budget. To accommodate this novel framework, we develop an acquisition function that estimates the utility of a set of time steps by approximating its resulting variance reduction. We demonstrate the effectiveness of our method on several
benchmark PDEs.
Read the full abstract ▾
All authors Yegon Kim, Hyunsu Kim, Gyeonghoon Ko, Juho Lee
Statistical Methodology
Sparse-Smooth Spatially Varying Coefficient Quantile Regression
Hou Jian, Meng Tan and Tian Maozai
We develop a convex framework for spatially varying coefficient
quantile regression that, for each predictor, separates a location-invariant global effect from a spatial deviation. An adaptive group penalty selects whether a predictor varies over space, while a graph\textendash
Laplacian quadratic promotes spatial continuity of the deviations on irregular networks. The formulation is identifiable via degree-weighted centering and scales with sparse linear
algebra. We provide two practical solvers\textemdash an ADMM
algorithm with closed-form proximal maps for the check loss and a smoothed proximal-gradient scheme based on the Moreau envelope\textemdash together with implementation guidance (projection for identifiability, stopping diagnostics, and preconditioning). Under mild conditions on the sampling design,
covariates, error density, and graph geometry, we establish selection consistency for the deviation groups, mean-squared error bounds that balance
Laplacian bias and
stochastic variability, and root-\(n\) asymptotic normality for the global coefficients with an oracle property. Simulations mimicking air-pollution applications demonstrate accurate recovery of global vs.\ local effects and competitive predictive performance under heteroskedastic, heavy-tailed noise. We discuss graph construction, spatially blocked cross-validation (to prevent leakage), and options for robust standard errors under spatial dependence.
Read the full abstract ▾
Statistical Methodology
A sensitivity analysis for non-inferiority studies with non-randomised data
Daijiro Kabata and Takumi Imai
Background: Non-inferiority studies based on non-randomised data are increasingly used in clinical research but remain prone to unmeasured confounding. The classical E-value offers a simple way to quantify such bias but has been applied almost exclusively with respect to the statistical null. We reformulated the E-value framework to make explicit its applicability to predefined clinical margins, thereby extending its utility to non-inferiority analyses. Development: Using the bias-factor formulation by Ding and VanderWeele, we defined the non-inferiority E-value as the minimum strength of association that an unmeasured confounder would need with both treatment and outcome, on the risk-ratio scale, to move the 95% confidence-limit estimate to the prespecified non-inferiority margin. Application: This approach was applied to three observational studies and one single-arm trial with external controls to illustrate interpretation and range. The resulting non-inferiority E-values for the confidence limits varied from about one to three, depending on design and findings. In the single-arm trial, a large gap between the confidence-limit and point-estimate NIEs reflected small sample size and wide confidence intervals, highlighting that both should be reported for a balanced assessment of robustness. Conclusion: This study reformulates the E-value to focus on clinically meaningful margins rather than the statistical null, enabling its application to non-inferiority analyses. Although the non-inferiority E-value inherits the limitations of the original method and cannot address all bias sources, it offers a transparent framework for interpreting non-randomised evidence and for generating insights that inform the design of future, more definitive randomised controlled trials.
Read the full abstract ▾
Statistical Methodology
Sequential Bootstrap for Out-of-Bag Error Estimation: A 100-Seed Replication Study and Variance-Structure Analysis
Cheng Peng
Out-of-Bag (OOB) estimation is the standard internal diagnostic for bootstrap-aggregated tree ensembles. Under the classical multinomial bootstrap, the number of distinct training observations in each replicate, $U_b$, is itself random, but its contribution to OOB-based variability has rarely been isolated empirically. We use Sequential Bootstrap (SB) – a resampling scheme that holds $U_b$ at a fixed target $k_n = \lfloor 0.632 n\rfloor$ – as a controlled perturbation of the bootstrap mechanism, and ask whether stabilizing $U_b$ produces any measurable change in OOB-based diagnostics. We reproduce Breiman's five OOB experimental families on twelve synthetic and real
datasets, but unlike the three-seed presentation common in this literature, we run 100 independent random seeds with 50 internal replications per seed, enabling formal paired statistical comparison (Wilcoxon signed-rank, paired-$t$, Pitman–Morgan variance test). We report three findings. First, OOB means are essentially insensitive to stabilization of $U_b$: of 57 (experiment,
dataset, metric)
cells under 100 seeds, only 6 reach $p<0.05$ on the paired mean comparison, and 4 of those 6 point in the opposite direction from what a 3-seed reading would suggest. Second, a narrow but reproducible effect survives at the variance level: SB reduces the cross-seed standard deviation of node-level classification diagnostics on real
datasets while slightly increasing it on synthetic ones (permutation $p=0.026$); the Vehicle
dataset exhibits a 21% cross-seed sd reduction (Pitman–Morgan $p=0.017$). Third, several directional claims that appear stable across three seeds flip sign under 100-seed replication, illustrating the cost of underpowered replication protocols. We therefore treat SB as a diagnostic tool for probing the distinct-sample-count term in the variance of OOB
estimators, not as an alternative to the classical bootstrap.
Read the full abstract ▾
Machine Learning (Statistics)
An operator splitting analysis of Wasserstein–Fisher–Rao gradient flows
Francesca Romana Crucinio and Sahani Pathiraja
Wasserstein-Fisher-Rao (WFR) gradient flows have been recently proposed as a powerful sampling tool that combines the advantages of pure Wasserstein (W) and pure Fisher-Rao (FR) gradient flows. Existing algorithmic developments implicitly make use of operator splitting techniques to numerically approximate the WFR partial differential equation, whereby the W flow is evaluated over a given step size and then the FR flow (or vice versa). This works investigates the impact of the order in which the W and FR operator are evaluated and aims to provide a quantitative analysis. Somewhat surprisingly, we show that with a judicious choice of step size and operator ordering, the split scheme can converge to the target distribution faster than the exact WFR flow (in terms of model time). We obtain variational formulae describing the
evolution over one time step of both splitting schemes and investigate in which settings the W-FR split should be preferred to the FR-W split. As a step towards this goal we show that the WFR gradient flow preserves log-concavity and obtain the first sharp decay bound for WFR flow.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Hierarchical Linkage Clustering Beyond Binary Trees and Ultrametrics
Maximilien Dreveton, Matthias Grossglauser, Daichi Kuroda et al. (+1 more)
Hierarchical clustering seeks to uncover nested structures in data by constructing a tree of clusters, where deeper levels reveal finer-grained relationships. Traditional methods, including linkage approaches, face three major limitations: (i) they always return a hierarchy, even if none exists, (ii) they are restricted to binary trees, even if the true hierarchy is non-binary, and (iii) they are highly sensitive to the choice of linkage function. In this paper, we address these issues by introducing the notion of a valid hierarchy and defining a partial order over the set of valid hierarchies. We prove the existence of a finest valid hierarchy, that is, the hierarchy that encodes the maximum information consistent with the similarity structure of the data set. In particular, the finest valid hierarchy is not constrained to binary structures and, when no hierarchical relationships exist, collapses to a
star tree. We propose a simple two-step
algorithm that first constructs a binary tree via a linkage method and then prunes it to enforce validity. We establish necessary and sufficient conditions on the linkage function under which this procedure exactly recovers the finest valid hierarchy, and we show that all linkage functions satisfying these conditions yield the same hierarchy after
pruning. Notably, classical linkage rules such as single, complete, and average satisfy these conditions, whereas Ward's linkage fails to do so.
Read the full abstract ▾
All authors Maximilien Dreveton, Matthias Grossglauser, Daichi Kuroda, Patrick Thiran
Statistical Methodology
On a Reinforcement Learning Methodology for Epidemic Control, with application to COVID-19
Giacomo Iannucci, Petros Barmpounakis, Alexandros Beskos et al. (+1 more)
This paper presents a real time, data driven decision support framework for epidemic control. We combine a compartmental epidemic model with sequential
Bayesian inference and reinforcement learning (RL) controllers that adaptively choose intervention levels to balance disease burden, such as intensive care unit (ICU) load, against socio economic costs. We construct a context specific cost function using empirical experiments and expert feedback. We study two RL policies: an ICU threshold rule computed via Monte Carlo grid search, and a policy based on a posterior averaged Q learning
agent. We validate the framework by fitting the epidemic model to publicly available ICU occupancy data from the COVID 19 pandemic in England and then generating counterfactual roll out scenarios under each RL controller, which allows us to compare the RL policies to the historical government strategy. Over a 300 day period and for a range of cost parameters, both controllers substantially reduce ICU burden relative to the observed interventions, illustrating how
Bayesian sequential learning combined with RL can support the design of epidemic control policies.
Read the full abstract ▾
All authors Giacomo Iannucci, Petros Barmpounakis, Alexandros Beskos, Nikolaos Demiris
Statistical Methodology
Hierarchical biomarker thresholding: a model-agnostic framework for stability
O. Debeaupuis
Many biomarker pipelines require patient-level decisions aggregated from instance-level (
cell/patch) scores. Thresholds tuned on pooled instances often fail across sites due to hierarchical dependence, prevalence shift, and score-scale mismatch. We present a selection-honest framework for hierarchical thresholding that makes patient-level decisions reproducible and more defensible. At its core is a risk decomposition
theorem for selection-honest thresholds. The
theorem separates contributions from (i) internal fit and patient-level generalization, (ii) operating-point shift reflecting prevalence and shape changes, and (iii) a stability term that penalizes sensitivity to threshold perturbations. The stability component is computable via patient-block bootstraps mapped through a monotone modulus of risk. This framework is model-agnostic, reconciles heterogeneous decision rules on a
quantile scale, and yields monotone-invariant ensembles and reportable diagnostics (e.g. flip-rate, operating-point shift).
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Learning Rate Scheduling with Matrix Factorization for Private Training
Nikita P. Kalinin and Joel Daniel Andersson
We study differentially private model training with
stochastic gradient descent under learning rate scheduling and correlated noise. Although correlated noise, in particular via
matrix factorizations, has been shown to improve accuracy, prior theoretical work focused primarily on the prefix-sum workload. That workload assumes a constant learning rate, whereas in practice learning rate schedules are widely used to accelerate training and improve convergence. We close this gap by deriving general upper and lower bounds for a broad class of learning rate schedules in both single- and multi-epoch settings. Building on these results, we propose a learning-rate-aware factorization that achieves improvements over prefix-sum factorizations under both MaxSE and MeanSE error metrics. Our theoretical analysis yields
memory-efficient constructions suitable for practical deployment, and experiments on CIFAR-10 and IMDB
datasets confirm that schedule-aware factorizations improve accuracy in private training.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
An Adaptive Resonance Theory-based Topological Clustering Algorithm with a Self-Adjusting Vigilance Parameter
Naoki Masuyama, Yuichiro Toda, Yusuke Nojima et al. (+1 more)
Clustering in stationary and nonstationary settings, where data distributions remain static or evolve over time, requires models that can adapt to distributional shifts while preserving previously learned cluster structures. This paper proposes an Adaptive Resonance Theory (ART)-based topological clustering
algorithm that autonomously adjusts its recalculation interval and vigilance threshold through a diversity-driven adaptation mechanism. This mechanism enables hyperparameter-free learning that maintains cluster stability and continuity in dynamic environments. Experiments on 24 real-world
datasets demonstrate that the proposed
algorithm outperforms state-of-the-art methods in both clustering performance and continual learning capability. These results highlight the effectiveness of the proposed parameter adaptation in mitigating catastrophic forgetting and maintaining consistent clustering in evolving data streams. Source code is available at https://github.com/Masuyama-lab/IDAT
Read the full abstract ▾
All authors Naoki Masuyama, Yuichiro Toda, Yusuke Nojima, Hisao Ishibuchi
Statistics Theory
Divergence-Minimization for Latent-Structure Models: Monotone Operators, Contraction Guarantees, and Robust Inference
Lei Li and Anand N. Vidyashankar
We develop a divergence-minimization (DM) framework for robust and efficient inference in latent-mixture models. By optimizing a residual-adjusted divergence, the DM approach recovers EM as a special case and yields robust alternatives through different divergence choices. We establish that the sample objective decreases monotonically along the iterates, leading the DM sequence to stationary points under standard conditions, and that at the population level the operator exhibits local contractivity near the minimizer. Additionally, we verify consistency and $\sqrt{n}$-asymptotic normality of minimum-divergence
estimators and of finitely many DM iterations, showing that under correct specification their limiting covariance matches the Fisher information. Robustness is analyzed via the residual-adjustment function, yielding bounded influence functions and a strictly positive breakdown bound for bounded-RAF divergences, and we contrast this with the non-robust behaviour of KL/EM. Next, we address the challenge of determining the number of mixture components by proposing a penalized divergence criterion combined with repeated sample splitting, which delivers consistent order selection and valid post-selection inference. Empirically, DM instantiations based on Hellinger and negative exponential divergences deliver accurate inference and remain stable under contamination in mixture and image-
segmentation tasks. The results clarify connections to MM and proximal-point methods and offer practical defaults, making DM a drop-in alternative to EM for robust latent-structure inference.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
On Transportability for Structural Causal Bandits
Min Woo Park and Sanghack Lee
Intelligent
agents equipped with causal knowledge can optimize their action spaces to avoid unnecessary exploration. The structural causal bandit framework provides a graphical characterization for identifying actions that are unable to maximize rewards by leveraging prior knowledge of the underlying causal structure. While such knowledge enables an
agent to estimate the expected rewards of certain actions based on others in online interactions, there has been little guidance on how to transfer information inferred from arbitrary combinations of
datasets collected under different conditions – observational or experimental – and from heterogeneous environments. In this paper, we investigate the structural causal bandit with transportability, where priors from the source environments are fused to enhance learning in the deployment setting. We demonstrate that it is possible to exploit invariances across environments to consistently improve learning. The resulting bandit
algorithm achieves a sub-linear regret bound with an explicit dependence on informativeness of prior data, and it may outperform standard bandit approaches that rely solely on online learning.
Read the full abstract ▾
Statistics Theory
Minimum $L_1$-norm estimation for fractional Ornstein-Uhlenbeck type process driven by a Hermite process
B.L.S. Prakasa Rao
We investigate the asymptotic properties of the minimum $L_1$-norm
estimator of the drift parameter for fractional Ornstein-Uhlenbeck type process driven by a Hermite process.
Statistical Methodology
The Asymptotic Distribution for a Single Joinpoint Changepoint Model
Xueheng Shi and Robert Lund
A single joinpoint changepoint model partitions a time series into two segments, joined at the changepoint time by constraining the estimated piecewise linear regression responses to be continuous. This manuscript derives the exact asymptotic distribution of the changepoint existence test statistic gauging whether or not a second segment is necessary. The identified asymptotic distribution, a supremum of a Gaussian process over the unit interval, is rather unwieldy. The work presented here provides the result and its derivation;
quantiles of the asymptotic distribution are presented for the user. This addresses a subtle gap in the changepoint literature.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Mitigating Catastrophic Forgetting in Streaming Generative and Predictive Learning via Stateful Replay
Wenzhang Du
Many deployed learning systems must update models on streaming data under
memory constraints. The default strategy, sequential fine-tuning on each new phase, is architecture-agnostic but often suffers catastrophic forgetting when later phases correspond to different sub-populations or tasks. Replay with a finite buffer is a simple alternative, yet its behaviour across generative and predictive objectives is not well understood. We present a unified study of stateful replay for streaming autoencoding, time series
forecasting, and classification. We view both sequential fine-tuning and replay as
stochastic gradient methods for an ideal joint objective, and use a gradient
alignment analysis to show when mixing current and historical samples should reduce forgetting. We then evaluate a single replay mechanism on six streaming scenarios built from Rotated MNIST, ElectricityLoadDiagrams 2011-2014, and Airlines delay data, using matched training budgets and three seeds. On heterogeneous multi task streams, replay reduces average forgetting by a factor of two to three, while on benign time based streams both methods perform similarly. These results position stateful replay as a strong and simple baseline for continual learning in streaming environments.
Read the full abstract ▾
Statistics Theory · Cross-listed from econ.EM
Limit Theorems for Network Data without Metric Structure
Wen Jiang, Yachen Wang, Zeqi Wu et al. (+1 more)
This paper develops limit
theorems for random variables with network dependence, without requiring the individuals in the network to be located in a Euclidean or metric space. This distinguishes our approach from most existing limit
theorems in network statistics and econometrics, which are based on weak dependence concepts such as strong mixing, near-epoch dependence, or $\psi$-dependence. All these weak dependence concepts presuppose an underlying metric. By relaxing the assumption of an underlying metric space, our
theorems can be applied to a broader range of network data, including financial and social networks. To derive the limit
theorems, we generalize the concept of functional dependence (also known as physical dependence) from time series to random variables with network dependence. Using this framework, we establish several inequalities, a law of large numbers, and central limit
theorems. Furthermore, we demonstrate the verifiability of our high-level conditions by deriving primitive sufficient conditions for spatial autoregressive models, which are widely used in network data analysis.
Read the full abstract ▾
All authors Wen Jiang, Yachen Wang, Zeqi Wu, Xingbai Xu
Statistical Methodology
Why Is the Double-Robust Estimator for Causal Inference Not Doubly Robust for Variance Estimation?
Hao Wu, Lucy Shao, Toni Gui et al. (+6 more)
Doubly robust
estimators (DRE) are widely used in causal inference because they yield consistent
estimators of average causal effect when at least one of the nuisance models, the propensity for treatment (exposure) or the outcome regression, is correct. However, double robustness does not extend to variance estimation; the influence-function (IF)-based variance
estimator is consistent only when both nuisance parameters are correct. This raises concerns about applying DRE in practice, where model misspecification is inevitable. The recent paper by Shook-Sa et al. (2025, Biometrics, 81(2), ujaf054) demonstrated through Monte Carlo simulations that the IF-based variance
estimator is biased. However, the paper's findings are empirical. The key question remains: why does the variance
estimator fail in double robustness, and under what conditions do alternatives succeed, such as the ones demonstrated in Shook-Sa et al. 2025. In this paper, we develop a formal theory to clarify the efficiency properties of DRE that underlie these empirical findings. We also introduce alternative strategies, including a mixture-based framework underlying the sample-splitting and crossfitting approaches, to achieve valid inference with misspecified nuisance parameters. Our considerations are illustrated with simulation and real study data.
Read the full abstract ▾
All authors Hao Wu, Lucy Shao, Toni Gui, Tsungchin Wu, Zhuochao Huang, Shengjia Tu, Xin Tu, Jinyuan Liu, Tuo Lin
Machine Learning (Statistics) · Cross-listed from cs.LG
DUPLE: An Intelligent Cross-Deployment Recognition Framework for Fiber-Optic Perimeter Security under Scarce Target Labels
Yifan He, Haodong Zhang, Qiuheng Song et al. (+4 more)
Distributed Fiber Optic Sensing (DFOS) has emerged as a promising technology for long-range and real-time perimeter security in critical infrastructure monitoring. However, DFOS signals collected from different field deployments often exhibit substantial distribution shifts caused by variations in fiber installation, structural coupling, and environmental noise. These deployment-dependent changes make reliable event recognition difficult in practical perimeter security systems, especially when labeled samples from new target sites are scarce or unavailable. To address these challenges, this paper proposes DUPLE, an intelligent cross-deployment recognition framework for fiber-optic perimeter security under label-scarce target deployments. DUPLE employs statistically guided meta-learning to enhance recognition robustness across unseen deployments. Specifically, a dual-domain multi-prototype learner jointly models temporal and frequency-domain evidence to capture intra-class variability under deployment shifts. A statistical guidance network estimates sample-specific domain reliability from raw signal statistics, while a query-aware aggregation mechanism adaptively selects relevant prototypes for each test sample. Extensive experiments on two real-world cross-deployment DFOS
benchmarks demonstrate that DUPLE consistently outperforms representative traditional machine learning, deep learning, domain generalization, and meta-learning baselines. Ablation, few-shot, per-deployment, and efficiency analyses further verify the effectiveness and practicality of DUPLE for reliable DFOS-based perimeter security monitoring.
Read the full abstract ▾
All authors Yifan He, Haodong Zhang, Qiuheng Song, Lin Lei, Zhenxuan Zeng, Haoyang He, Hongyan Wu
Machine Learning (Statistics) · Cross-listed from q-fin.MF
Arbitrage-Free Bond and Yield Curve Forecasting with Neural Filters under HJM Constraints
Xiang Gao and Cody Hyndman
We develop an arbitrage-free deep learning framework for yield curve and bond price
forecasting based on the Heath-Jarrow-Morton (HJM) term-structure model and a dynamic Nelson-Siegel parameterization of forward rates. Our approach embeds a no-arbitrage drift restriction into a neural state-space architecture by combining Kalman, extended Kalman, and particle filters with recurrent neural networks (LSTM/CLSTM), and introduces an explicit arbitrage error regularization (AER) term during training. The model is applied to U.S. Treasury and corporate bond data, and its performance is evaluated for both yield-space and price-space predictions at 1-day and 5-day horizons. Empirically, arbitrage regularization leads to its strongest improvements at short maturities, particularly in 5-day-ahead forecasts, increasing
market-consistency as measured by bid-ask hit rates and reducing dollar-denominated prediction errors.
Read the full abstract ▾
Statistical Methodology
Single Changepoint Procedures
Robert Lund and Xueheng Shi
Single changepoint tests have become a staple check for homogeneity of a
climate time series, suggesting how
climate has changed should non-homogeneity be declared. This paper summarizes the most prominent single changepoint tests used in today's
climate literature, relating them to one and other and unifying their presentations. Asymptotic
quantiles for the individual tests are presented. Derivations of the
quantiles are given, enabling the reader to tackle cases not considered within. Our work here studies both mean and trend shifts, covering the most common settings arising in climatology. SOI and global temperature series are analyzed within to illustrate the techniques.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds
Xuesong Jia, Yuanjie Shi, Ziquan Liu et al. (+2 more)
Conformal prediction (CP) is a general framework to quantify the predictive uncertainty of machine learning models that uses a set prediction to include the true label with a valid probability. To align the uncertainty measured by CP, conformal training methods minimize the size of the prediction sets. A typical way is to use a surrogate indicator function, usually Sigmoid or Gaussian error function. However, these surrogate functions do not have a uniform error bound to the indicator function, leading to uncontrollable learning bounds. In this paper, we propose a simple cost-sensitive conformal training
algorithm that does not rely on the indicator approximation mechanism. Specifically, we theoretically show that minimizing the expected size of prediction sets is upper bounded by the expected rank of true labels. To this end, we develop a rank weighting strategy that assigns the weight using the rank of true label on each data sample. Our analysis provably demonstrates the tightness between the proposed weighted objective and the expected size of conformal prediction sets. Extensive experiments verify the validity of our theoretical insights, and superior empirical performance over other conformal training in terms of predictive efficiency with 21.38% reduction for average prediction set size.
Read the full abstract ▾
All authors Xuesong Jia, Yuanjie Shi, Ziquan Liu, Yi Xu, Yan Yan
Machine Learning (Statistics) · Cross-listed from cs.LG
Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently
Bochen Lyu, Yiyang Jia, Xiaohao Cai et al. (+1 more)
Transformers can acquire Chain-of-Thought (CoT) capabilities to solve
reasoning tasks via fine-tuning. Reinforcement learning (RL) and supervised fine-tuning (SFT) are two primary approaches to this end. In this work, we examine RL with verifiable process rewards and SFT for learning $k$-sparse Boolean functions with a one-layer
transformer through intermediate
reasoning steps akin to CoT. In particular, we consider Boolean functions that can be recursively decomposed into fixed 2-sparse Boolean functions. We first analyze the learning dynamics of RL fine-tuning with verifiable process rewards and SFT in a unified way, allowing us to identify sufficient conditions under which the
transformer provably learns these functions. We then verify that the conditions hold for three examples, including $k$-PARITY, $k$-AND, and $k$-OR, thus demonstrating their learnability via both RL and SFT. Notably, we reveal that RL and SFT exhibit distinct learning behaviors depending on supervision: RL learns the whole CoT chain simultaneously, whereas SFT without teacher forcing learns the CoT step-by-step. Overall, our findings provide insights on the mechanisms underlying RL and SFT and how they differ in triggering the CoT capabilities of
transformers, and suggest that the comparison between RL and SFT should consider the intermediate supervision.
Read the full abstract ▾
All authors Bochen Lyu, Yiyang Jia, Xiaohao Cai, Zhanxing Zhu