Machine Learning (Statistics) · Cross-listed from cs.LG
Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability
Haodong Wen, Kaiyue Wen and Jiaye Teng
We study logistic regression on linearly separable data under gradient descent with a large constant stepsize $\eta$. Such dynamics may exhibit a characteristic Edge of Stability phenomenon, in which the loss initially oscillates before transitioning to a stable phase of monotone decrease. Existing work provides a tight $\Theta(1)$ bound in dimension $d=2$ as $\eta \to \infty$ and
conjectures a bound independent of $\eta$ in arbitrary dimensions $d\geq 2$. In this paper, we disprove this
conjecture by showing that, for every fixed sample size $n\geq 2$ and sufficiently small margin $\gamma$, the worst-case transition time is $$\Theta\!\left((\log\eta)^{\min\{n-2,d-2\}}\right)$$ uniformly over $d\geq2$. The key challenge in establishing a tight bound is that the sample contributing most strongly to the gradient can change repeatedly across iterations. To address this issue, we control such changes by induction on dimension and sample size, and construct matching hard instances.
Read the full abstract ▾
Statistical Computation
On skew-symmetric distributions and their use in Monte Carlo sampling algorithms: coordinate-free, Gibbs-style and manifold versions of the Barker proposal
Minh Vu, Samuel Livingstone and Pantelis Samartsidis
Skew-symmetric probability distributions provide a principled mechanism for incorporating gradient information into Markov chain Monte Carlo
algorithms. Here we review the (preconditioned) Barker proposal, a Metropolis–Hastings
algorithm built on skew-symmetric distributions, and motivate its design. We then introduce three natural extensions. First, we propose coordinate-free variants of the Barker
algorithm. Second, we introduce a Gibbs-style Barker
algorithm that re-evaluates the gradient at each partially updated coordinate. Third, we derive a simplified
manifold Barker
algorithm, producing a
manifold sampler with enhanced robustness compared to natural comparators. Numerical experiments demonstrate that the Gibbs-style variant improves raw sampling efficiency on correlated targets, that the coordinate-free variants offer limited practical advantage over the standard Barker proposal once computational costs are accounted for, and that the simplified
manifold Barker
algorithm can achieve significant advantages over simplified
manifold MALA when the local
geometric structure of the target is irregular or unreliable.
Read the full abstract ▾
Machine Learning (Statistics)
Optimal Transport Meets Reinforcement Learning: A Survey
Yujie Zhu, Charles A. Hepburn, Matthew Thorpe et al. (+1 more)
Reinforcement learning (RL)
algorithms frequently compare probability distributions, such as state visitation distributions induced by policies and experts, action distributions from learned policies and offline
datasets, or transition distributions from learned models and environments. However, commonly used divergences may become ineffective when these distributions overlap weakly, which is frequently encountered in imitation learning, offline RL, and deployment under distribution shift. Optimal transport (OT) offers an alternative by measuring the cost of moving probability mass from one distribution to another under a ground cost that encodes task geometry. This survey covers how OT is used inside RL objectives and
algorithms. For each method, we identify: the role OT plays, the distributions compared, the OT formulation used, and the treatment of temporal structure. Beyond categorising existing methods, we discuss the motivations behind different OT choices, practical considerations such as cost design and computational challenges, and highlight open problems including scalable trajectory-level transport, principled handling of mass mismatch, and theoretical analysis for OT-regularised RL.
Read the full abstract ▾
All authors Yujie Zhu, Charles A. Hepburn, Matthew Thorpe, Giovanni Montana
Statistics Applications
On relations between the Cochran–Mantel–Haenszel test and Augmented Inverse Probability Weighting
Ekkehard Glimm
This note revisits the Cochran-Mantel-Haenszel (CMH) test in light of the increasing use of causal inference methods in clinical trials. It shows that the CMH test statistic can be interpreted as a standardized version of the augmented inverse probability weighted (AIPW) estimate of the causal average treatment effect. Consequently, it is not restricted to an interpretation as a test of the common-odds ratio in a stratified logistic regression model. The note then discusses conditional and unconditional variances for this
estimator as well as variance estimation and its implications for testing. The results clarify the relationship between the classical conditional CMH test and Wald-type AIPW tests, and show that, under equal treatment allocation, the CMH test is asymptotically conservative for the weak null hypothesis of zero average treatment effect. %This note revisits the Cochran–Mantel–Haenszel (CMH) test in light of the increasing use of causal inference methods in clinical trials. For stratified randomized trials with binary endpoints and a common treatment allocation fraction across strata, we show that the numerator of the CMH statistic is proportional to an augmented inverse probability weighted
estimator of the population-average causal risk difference. This provides an interpretation of the CMH test that does not depend on the common-odds-ratio model often used to motivate it. We then compare conditional and unconditional variance concepts for this
estimator and discuss their implications for testing. The results clarify the relationship between the classical conditional CMH test and Wald-type AIPW tests, and show that, under equal treatment allocation, the CMH test is asymptotically conservative for the weak null hypothesis of zero average treatment effect.
Read the full abstract ▾
Statistics Theory
User-friendly approximation theory for statistical regression
Felix Benning
The error of regression using noisy function values naturally decomposes into an approximation error from the choice of model space and
stochastic error from the noise on the observations. Statistical theory is mostly concerned with the
stochastic error while approximation theory focuses on the approximation error, often subject to the assumption of noiseless observations. We combine the tools from approximation theory and statistics into a framework for bounding both errors in generalized least squares regression. The results cover weighted sup-norms and $L^2(\mu)$-norms, real- and complex-valued functions, and correlated noise under a range of tail and moment assumptions. Weighted Lebesgue functions and constants control the amplification of the best approximation error, while concentration inequalities provide bounds on the
stochastic error. We organize these results into reusable bounds with explicit constants and establish sharpness where possible. For the common strategy of randomized regression we prove concentration of the Lebesgue function and constant around those of the corresponding $L^2(\mu)$-projection. For
Fourier and Chebyshev regression on standard sampling grids, we obtain explicit, asymptotically tight $\mathcal O(\log m)$ bounds on the Lebesgue constants where $m$ is the model dimension. This improves upon the previous $\mathcal O(\sqrt{m})$ bound obtained for the regression setting and matches the bounds known for interpolation. A catalogue of approximation bounds and worked examples illustrates how to turn these results into explicit regression guarantees.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Clifford Sheaf Neural Networks
Kotaro Kamiya and Joel Nicholls
We introduce the Clifford Sheaf Neural Network (CSNN), an equivariant sheaf neural network for
geometric graphs that places a Clifford
algebra on each stalk of a cellular sheaf and transports multivector features along edges. The canonical choice of restriction map for sheaves with
algebra-valued stalks is
algebra homomorphism. Adding the constraint of equivariance, the naive choice becomes versor conjugation. However, versor conjugation is expressively weak, so we drop
algebra homomorphism and arrive at the K-term sandwich. The resulting sheaf
Laplacian is positive semidefinite by construction, needs no versor constraint, and still mixes grades. Our main contribution characterizes the resulting family of restriction maps along three axes: which grades a map couples, how much of the endomorphism space it reaches, and how well it is conditioned. The K-term sandwich spans half of the endomorphism space, and in Cl(3, 0, 0) it corresponds to the maps that commute with the central pseudoscalar. The number of terms controls expressivity. CSNN is the reversion member, a first-order model by construction and the grade-mixing corner of this family, developed as a sheaf construction for graph-level equivariant regression.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
Feature Selective Model Collapse in Diffusion Models: Total Replacement versus Fixed-Budget Training
Hanna Malet and Gabriel Turinici
Model collapse arises when generative models are trained on synthetic data produced by earlier models. The phenomenon has attracted considerable
attention because of its societal and technical implications. However, previous studies have reached seemingly contradictory conclusions: replacing real data with synthetic data causes collapse (Shumailov et al.), yet accumulating real data alongside synthetic data can prevent it. For
diffusion models, we study an intermediate regime typical of finite-budget pipelines: all past
datasets and the real data are kept, but each new model is trained on a fixed-size sample from this growing pool, so the real fraction vanishes without any data being removed. Experiments on a 2D spiral
dataset as well as the image
benchmarks (MNIST, Fashion-MNIST, and CIFAR-10) show that replacement protocol degrades
dataset rapidly as in the literature, whereas the fixed budget degrades only partially, sparing some features. A linear-response model of the multi-generational parameter dynamics, analyzed by
stochastic recursion, confirms that the two protocols differ: some features will be fragile and lost within a few generations for both protocols, while some will be robust and preserved over practically unbounded horizons under the fixed budget protocol.
Read the full abstract ▾
Statistical Methodology
Testing and segmentation of joint and individual components in integrative multi-source factor models
Kyoowon Kim and Sungkyu Jung
Disentangling shared (joint) structures from source-specific (individual) variations is a fundamental task in multi-source data integration. Existing joint-individual models often rely on computationally intensive optimization or loose spectral bounds, leading to suboptimal separation accuracy and poor scalability. In this paper, we propose the Multi-Source Sequential
Alignment Test (MSSAT). MSSAT leverages the
geometric observation that true joint components manifest as closely aligned score subspaces across different data sources. By deriving the asymptotic null distribution of our
alignment statistic, we develop a rigorous, resampling-free sequential testing procedure to accurately estimate the joint rank. Extensive simulations and real data applications, including a TCGA multi-omics
dataset, demonstrate that MSSAT achieves superior separation accuracy and substantially faster computation compared to competing methods.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
IQS-BO: In-Context Query Selection for Bayesian Optimisation
Luca Geminiani and Nadja Klein
Bayesian Optimisation (BO) is a powerful framework for the optimisation of expensive black-box functions, but typically requires refitting a surrogate and maximising an acquisition function at every evaluation step. In-context approaches based on Prior-data Fitted Networks (PFNs) amortise part of this cost by pre-training
transformers on functions drawn from synthetic priors. PFNs4BO amortises the surrogate but still relies on a numerically maximised acquisition function, while FIBO performs BO fully in-context by sampling optimiser locations from a learned density, which fixes the decision rule and admits no surrogate. Learned acquisition functions score a finite candidate set with a trained network, but, lacking a label for the query, learn the score by reinforcement learning on previously solved tasks. We propose IQS-BO, a PFN that learns the query decision by supervised learning on synthetic priors. In a single forward pass, IQS-BO predicts the probability that each candidate maximises the objective over the set, and we show that the minimiser of its objective is the posterior probability of this event. The model can be pre-trained without a surrogate for fully in-context BO, or take the predictions of a fixed probabilistic surrogate as additional input, amortising only the decision step. Our method proposes queries at a fraction of the cost of acquisition-based methods, while either matching or outperforming standard BO with Gaussian processes (GPs) and available in-context methods on synthetic and real-world
benchmarks. Finally, we propose a mixture prior for pre-training PFNs which combines samples from GPs with functions exhibiting warped inputs, isolated narrow optima, or plateaus that are poorly modeled by stationary kernels common in GP surrogates. We show that pre-training on this prior can lead to improved optimisation performance.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
Counterfactual Generation via Flow Matching: Coupling-Sensitive End-to-End Rates
Yunrui Guan, Krishnakumar Balasubramanian and Shiva Prasad Kasiviswanathan
Counterfactual generation seeks to sample outcomes under a hypothetical intervention or decision using observational data collected under the factual assignment mechanism. We develop a flow-matching approach that combines a sample-split, doubly robust training objective with a learned coupling between observed source outcomes and target outcomes drawn from a fitted conditional outcome model. To enable finite-step generation, we leverage a score-corrected
stochastic sampler based on a Gaussian-smoothed interpolation. Our main theoretical contribution is a coupling-sensitive KL bound for constant-step Euler discretization: the error is controlled by moments of the source–target displacement under the chosen coupling, rather than by global uniform regularity of the velocity field, and has near-linear dependence on the ambient dimension. We also establish finite-sample non-parametric guarantees for the learned velocity and score fields when both the conditional outcome model and the source-target coupling are estimated from data. These bounds separate approximation, coupling-replacement, nuisance-estimation, generalization, and Monte Carlo errors and, combined with the sampler analysis, yield an end-to-end guarantee for counterfactual generation. Experiments on synthetic and semi-synthetic image
benchmarks support the coupling-dependent theory and show that, at finite discretization budgets, the
stochastic sampler can outperform the corresponding deterministic ODE sampler.
Read the full abstract ▾
Statistical Methodology
Spatio-temporal coarse-grained Hawkes processes
Tomoya Uda and Shinsuke Koyama
Spatio-temporal Hawkes processes are widely used to model self-exciting event data, but conventional inference methods require access to the occurrence time and location of every event. In many applications, however, observations are available only as counts aggregated over temporal intervals and spatial regions. We propose a spatio-temporal coarse-grained Hawkes process, a discrete-time count model that approximates aggregated observations generated by an underlying spatio-temporal Hawkes process. The proposed model constructs effective excitation kernels by averaging the temporal and spatial triggering effects over spatio-temporal bins while explicitly incorporating excitation occurring within the same temporal bin. This formulation enables direct modeling of aggregated count data without introducing latent event times or locations. We characterize the first- and second-order properties of the proposed process and derive asymptotic approximation errors for the stationary mean and autocovariance relative to those of the corresponding aggregated Hawkes process in the joint limit of fine temporal and spatial discretization. The analysis shows that the proposed model provides a higher-order approximation than the conventional binned Poisson approximation by accounting for intra-bin excitation. Numerical experiments confirm the theoretical results and demonstrate accurate approximation over a broad range of aggregation scales. We further develop a moment-based estimation procedure and apply the proposed framework to earthquake occurrence data through an aggregated Epidemic-Type Aftershock Sequence (ETAS) model. The results indicate that the proposed approach provides a computationally efficient alternative for inference and prediction from spatio-temporally aggregated event data.
Read the full abstract ▾
Statistics Theory
Cost Comparisons for Random and Deterministic Scans in Component-Wise Markov Chains
Youngwoo Kwon
Gibbs samplers, and more generally component-wise Markov chain Monte Carlo
algorithms such as Metropolis-within-Gibbs, can be implemented using either random-scan or deterministic-scan updates. How much convergence can depend on this choice of scanning rule has been a longstanding question. We study this problem through $L^2$ spectral gaps, measuring computational cost in units of component updates. For a $d$-block Gibbs sampler, the cost of random scan is at most twice that of any fixed deterministic scan, while the reverse cost ratio is at most of order $d^2$. We extend these comparisons to general reversible component-wise updates under the global block-wise contraction condition. If $K_j$ denotes the update of block $j$ and $P_j$ its Gibbs counterpart, and $\|K_j-P_j\|\leq \lambda_0<1$ for $j=1,\ldots,d$, then the cost of random scan is at most $2/(1-\lambda_0)$ times that of deterministic scan, while the reverse cost ratio is of order at most $d^2/(1-\lambda_0)$. Examples show that the cost bounds for random scan relative to deterministic scan are asymptotically sharp and that the joint dependence on $d$ and $(1-\lambda_0)^{-1}$ in the reverse comparison cannot be improved uniformly. This work was assisted by generative AI, including for formal verification of mathematical results in Lean. The human author reviewed and verified the mathematical content and takes full responsibility for the results.
Read the full abstract ▾
Statistical Methodology
Certified Alpha Capacity: Statistical Arbitrage When Learning Takes Time
Nicolò Bonacorsi
Statistical validation takes time, and alpha can decay before the evidence justifies deployment. We quantify the surviving opportunity through Certified Alpha Capacity: the maximum expected value remaining under prescribed false-deployment and power constraints. In a canonical Gaussian experiment, we derive an exact certification frontier and the sharp law $\mathcal K_{\alpha,\beta}/\log(1/\varepsilon)$ for residual information capacity just above it. Near the frontier, a single interim decision can preserve orders of magnitude more capacity than fixed-time testing. Mapping information into cash reveals why equally certifiable signals can retain different economic values. In the specified competitive
equilibrium, lifetime information approaches the certification frontier exponentially as research costs vanish.
Read the full abstract ▾
Statistical Methodology
Polylogarithmic Sparsity of Randomly Reweighted NPMLEs for Gaussian Mixtures
Hansheng Jiang
The nonparametric maximum likelihood
estimator (NPMLE) of a Gaussian location mixture maximizes the likelihood over the infinite-dimensional space of mixing distributions. The maximizing mixing distribution can be nonunique, and the classical bound on its number of atoms grows linearly with the sample size $n$. We show that a vanishingly small random perturbation of the likelihood yields exact polylogarithmic sparsity. The resulting randomly reweighted NPMLE maximizes a weighted likelihood whose independent weights, taken to be Gamma in our analysis, concentrate around one as $n$ grows. With high probability, it is unique, has $O\{(\log n/\log\log n)^d+\log n\}$ atoms in dimension $d$, nearly maximizes the ordinary likelihood, and estimates the mixture density at a Hellinger rate that is parametric up to logarithmic factors. This sparsity holds for the
estimator itself, not for an approximation of it, and requires no support penalty. The proof rests on an effective-dimension principle for positive kernel mixtures: low-dimensional variation of the fitted values controls the support of every extreme point of the set of maximizers. Numerical illustrations verify that the reweighted NPMLE has Hellinger risk and support size comparable to those of the ordinary NPMLE.
Read the full abstract ▾
Statistics Theory
Gradient-Guided Density Peak Clustering
Yikun Zhang and Yen-Chi Chen
Density peak clustering (DPC) connects each observation to its nearest neighbor of higher density and identifies cluster centers as high-density observations with unusually large nearest neighbor uphill shifts. The resulting uphill paths from observations to cluster centers, however, can be irregular and unstable in low-density regions, making the clustering assignments sensitive to local perturbations and obscuring the population geometry of the DPC graph. In this paper, we introduce gradient-guided density peak clustering (GGDPC), which performs a gradient ascent step before each nearest neighbor uphill search. We develop a stability theory that relates the GGDPC graph to the gradient ascent flow of the population density. In particular, we establish consistency of GGDPC under five complementary criteria: recovery of local modes, adjusted Rand index, dendrogram (cluster tree), path length, and waterfall measure. Together, these results provide new statistical,
geometric, and topological interpretations of DPC-type clustering
algorithms.
Read the full abstract ▾
Statistical Methodology
Augmented Patient Preference Incorporated Reinforcement Learning (APP-RL) to Estimate the Optimal Dynamic Treatment Regime
Yingchao Zhong and Lu Wang
Dynamic treatment regimes (DTRs) are sequential decision rules that individualize treatments to each patient at each treatment stage adapting to their past clinical course. Existing literature typically accommodates each individual's medical history, but overlooks a patient's preferences. We propose a method that incorporates a patient's latent preferences through data augmentation into a tree-based reinforcement learning method to estimate optimal dynamic treatment regimes for multi-stage, multi-treatment settings. For each patient at each stage, we derive the posterior distribution of preferences given responses to a questionnaire, and then subsequently weight multiple outcomes with the estimated preferences to identify the optimal stage-wise personalized decision. For multiple stage situations, we grow a decision tree at each stage and implement the
algorithm recursively using backward induction. Our proposed method, named Augmented Patient Preference incorporated Reinforcement Learning (APP-RL) is robust, efficient, and leads to interpretable DTR estimation. The finite-sample performances of the proposed method has been thoroughly evaluated through simulation studies.
Read the full abstract ▾
Machine Learning (Statistics)
Posterior sampling by source-space MCMC via prior-based few-step transport maps
Hoang Phuc Hau Luu, Marcelo Hartmann and Zhongjian Wang
Bayesian inference increasingly uses informative but implicit priors represented only by samples, such as historical ensembles, simulator outputs, and pretrained generative models. The same computational problem appears in the test-time guidance task (generalized Bayes), where an explicit positive weight, e.g., an exponentiated reward, tilts an implicit prior. We develop a framework for source-space generalized
Bayesian inference that combines inexpensive few-step prior transports with posterior stability guarantees. Specifically, we represent the prior using a one- or few-step improved MeanFlow (iMF) map and perform posterior sampling in its Gaussian source space. We establish Wasserstein error bounds between the exact and learned posteriors in terms of the joint population iMF and auxiliary-velocity loss, decomposed into training suboptimality and model-class approximation error. In the iMF source space, we adopt parallel tempering with preconditioned Crank-Nicolson updates and introduce a hybrid variant that incorporates split
Hamiltonian Monte Carlo to improve sampling efficiency. Synthetic experiments show that the proposed framework can approximate posterior distributions accurately and efficiently, while CLIP-guided ImageNet experiments demonstrate its ability to steer a pretrained iMF image prior toward text-specified preferences.
Read the full abstract ▾
Machine Learning (Statistics)
Tolerance-Based Fairness Auditing: Violation Certification and Sensitivity Screening
Jie Tang, Chuanlong Xie and Lixing Zhu
As artificial intelligence is increasingly deployed, algorithmic unfairness has raised growing concerns and intensified demands for transparent
fairness auditing. In practice, the tolerable degree of algorithmic unfairness depends on the specific legal, ethical, or application context. Given a prespecified tolerance threshold, an important statistical question is how to determine whether a group disparity exceeds the allowable tolerance across different auditing objectives. To address this problem, we develop a unified tolerance-based
fairness auditing framework for two complementary auditing objectives: violation certification, which prioritizes control of false violation declarations, and sensitivity screening, which prioritizes reducing missed violations. For the first objective, we develop a constrained empirical likelihood test for formal settings that uses least-favorable-point calibration and can be combined with false flagging rate control for simultaneous subgroup auditing. For the second objective, we develop split empirical likelihood and adjusted split empirical likelihood tests using an adaptive boundary-proxy principle for early-warning settings. Numerical experiments show the distinct error-control–sensitivity trade-offs of these procedures. A COMPAS analysis illustrates the framework in predictive
fairness auditing.
Read the full abstract ▾
Machine Learning (Statistics)
The Price of Correlated Tests: How Strict Should a Model Release Gate Be?
Marco Pollanen
Before a machine learning model ships, it often has to pass a suite of automated tests. Requiring every test to pass looks safe, yet it can reject many models that would have served users well, and it does not say how trustworthy a passing model actually is. We treat the release gate as a design problem: choose how many tests a model must pass so that cleared models meet a stated reliability target, while keeping as many good models as possible. A two-class latent-factor model makes both costs explicit and reduces each calculation to a one-dimensional integral. We prove that when both classes share the same latent correlation, a stricter gate always raises reliability, so the gate that keeps the most good models is the most lenient one that still meets the target. Under pass-all gating, any reliability target short of perfection is attainable within the model, but the share of good models kept tends to zero as the suite grows. Correlation between tests sets the price. In one configuration, a 99 percent target needs 8 independent tests, but 74 tests at a latent correlation of 0.3 and 5,182 at 0.5, where the gate keeps fewer than one good model in ten. We also give a validation procedure, built on exact binomial bounds, that certifies a gate from labelled data even when the gate is chosen from a fixed shortlist.
Read the full abstract ▾
Statistical Methodology
Testing Procedures for Strict Pleiotropy in Genetic Association Studies
Eva Biswas, Nilanjan Chatterjee and Zheyu Wang
Pleiotropy, the phenomenon in which a single genetic variant influences multiple traits, is a fundamental feature of genome biology and an important consideration in statistical genetics. Many statistical methods exist that leverage pleiotropy to increase the power of association tests, but they are often designed to identify SNPs which are associated with at least one of the underlying traits. Only a limited number of methods exist for identification of “strict pleiotropy”, i.e. those associated with more than one trait, but these methods are either restricted to a small number of traits or become computationally challenging as the number of traits increases. In this paper, we develop two testing procedures to identify SNPs under strict pleiotropy and determine the corresponding minimum number of associated traits. Both methods rely only on z-scores available from genome-wide association studies (GWAS) and are applicable to independent as well as correlated traits. The first procedure is frequentist in nature and is designed to control the family-wise-error rate. The second procedure is based on the local false discovery rate and provides improved power while maintaining FDR control. We conduct extensive simulation studies to compare the performance of the proposed methods with alternative methods for testing strict pleiotropy. Finally, we demonstrate an application of the proposed methods to Alzheimer's disease-related biomarkers for the goal of distinguishing SNPs that may represent biomarker-specific variations from those that are related to disease-relevant biology.
Read the full abstract ▾
Statistical Methodology
Quantile Social Autoregressive Model
Liyuan Wang, Danyang Huang, Wei Lan et al. (+1 more)
Research on modelling peer effects has predominantly relied on linear-in-means models. However, averaging peer responses prevents these models from capturing the effects of extreme peer behavior. To address this limitation, we propose a
quantile social autoregressive model based on a novel
quantile social norm, which uses empirical
quantiles of peer responses. By treating the
quantile level as an unknown parameter estimated directly from the data, our approach identifies which segment of the peer response distribution most strongly influences individual behavior. To estimate the model, we introduce new moment conditions using pseudo-response instruments. Because the
quantile social norm is nonsmooth, we apply kernel smoothing to the instruments and residuals, ensuring valid statistical inference. Additionally, we establish
equilibrium existence and uniqueness and derive the identification conditions. Furthermore, we prove the consistency and asymptotic normality of our proposed
estimator. Finally, Monte Carlo experiments examine the finite-sample performance of the
estimator, and an empirical application illustrates the interpretation of the estimated
quantile levels and their corresponding peer effects.
Read the full abstract ▾
All authors Liyuan Wang, Danyang Huang, Wei Lan, Chih-Ling Tsai
Machine Learning (Statistics) · Cross-listed from cs.CV
Joint Branch-Space Transform Coding for Diffusion Activation Quantization with Classifier-Free Guidance
Mingrun Jiang, Yuejia Liu, Zishan Shao et al. (+11 more)
Post-training quantization for
diffusion models increasingly exploits timestep, feature, and layer structure. While recent work has begun incorporating CFG structure into
diffusion quantization, activation quantization still operates independently across conditional and unconditional coordinates, leaving cross-activation structure unexploited. We show that matched CFG activations form a strongly correlated two-dimensional source and that, under a fixed bit budget, the choice of branch coding basis materially affects quantization fidelity. Motivated by this observation, we introduce branch-space transform coding, which rotates matched CFG branches via an offline derived 2x2 orthogonal
matrix, requiring minimal modifications to model parameters or the quantization pipeline. We further derive the Guidance-Correlation Branch Transform (GCBT), which jointly incorporates the CFG guidance direction and cross-branch second moments. Under an equal-rate quantization-noise surrogate, GCBT admits a closed-form per-layer solution without gradient optimization or angle search. Applied on top of existing
diffusion PTQ methods, GCBT yields statistically significant fidelity gains in most evaluated comparisons with no statistically significant degradation, while leaving the underlying host quantization pipeline unchanged.
Read the full abstract ▾
All authors Mingrun Jiang, Yuejia Liu, Zishan Shao, Ting Jiang, Qinsi Wang, Hancheng Ye, Yixiao Wang, Rui-Feng Wang, Kangning Cui, Yixuan Chen, Fan Yang, Xiang Cheng, Hai Li, Yiran Chen
Machine Learning (Statistics) · Cross-listed from cs.LG
Platonic Task Arithmetic
Junghwan Park and Woojin Cho
Distinct pre-trained models specialized for the same task converge to closely similar behavior, yet the parameter updates that produce it share no common coordinate system. Weight-space task arithmetic is therefore confined to a single model, and transporting an update between models requires a structural correspondence. Drawing on Plato's allegory of the cave, we hypothesize that these model-specific updates are shadows cast by one shared, model-agnostic object, the platonic task vector. To make it operational across models of different architectures, we introduce Universal Task Descriptors,
matrices whose shape is independent of architecture and
embedding dimension, which record a task's functional effect and admit addition and negation as ordinary
matrix operations, and we transfer a descriptor into a target in two ways. A single least-squares solve returns a linear operator folded into the target's last layer, and a bank of such operators, one per source and task, realizes any composition as a signed sum of its entries. Alternatively, a low-rank adapter of the target's encoder is trained on the same objective at the price of one optimization per edit. Despite a model-specific residual comparable in norm to the shared component, transfer from another model retains 74 to 80 percent of the gain the target's own descriptors attain. Experiments across six model families, eight tasks and audio-text models confirm both realizations.
Read the full abstract ▾
Machine Learning (Statistics)
Block Optimism for Nonstationary Bandits with Latent Linear Dynamics
Taehyun Hwang, Hyunjun Choi, Heesang Ann et al. (+1 more)
We study an endogenous nonstationary
stochastic bandit problem with latent linear dynamics, where actions affect both immediate rewards and the future
evolution of an unobserved latent state. Rewards are bilinear in the current action and latent state, inducing history-dependent rewards and a nontrivial long-horizon planning problem. The existing explore-then-commit approach achieves $\tilde{O}(T^{2/3})$ regret by uniformly exploring to estimate the latent dynamics and then committing to an optimized open-loop action sequence. We show that this rate can be improved via adaptive block-level optimism. Our key step is a cyclic approximation: under stable dynamics, the infinite-
memory reward process can be truncated, and the open-loop
benchmark can be approximated by optimizing a finite-
memory block-level proxy. Building on this reduction, we propose a UCB-based block
algorithm that maintains confidence sets for the truncated dynamics parameters and selects blocks optimistically. We prove a regret bound of order $\tilde{O}(\sqrt T)$, significantly improving over the previous $\tilde{O}(T^{2/3})$ guarantee for the same model. To the best of our knowledge, this is the first $\tilde{O}(\sqrt T)$ regret guarantee for latent linear-dynamics bandits with bilinear reward observations and an open-loop action-sequence
benchmark.
Read the full abstract ▾
All authors Taehyun Hwang, Hyunjun Choi, Heesang Ann, Min-hwan Oh
Statistical Methodology
Modeling Bipartite Dynamic Networks: An Additive and Multiplicative Effects Model
Jing Luo
Researchers frequently study interactions between two distinct types of actors, represented as bipartite networks. These networks exhibit dependence patterns that differ from those in one-mode networks and therefore require models tailored to their structure. This paper develops an additive and multiplicative effects (AME) framework for longitudinal bipartite data. First, I distinguish the dependence structure and specify the corresponding modeling assumptions. Second, I introduce the bipartite dynamic AME model and develop an estimation procedure based on block coordinate descent. Third, I incorporate a squared iterative method to improve computational efficiency. Using simulations and an application to global production networks, I show that the model improves coefficient estimation, more accurately recovers the data-generating process, and better captures the multiplicative latent structure. The model reveals evolving patterns in countries' global production engagement that are not captured by observed
covariates or static specifications. I provide an R package, RAMEN, to facilitate implementation.
Read the full abstract ▾
Statistics Applications
Efficient Convex Optimization Methods for State-Space Heartbeat Dynamics Models with Gamma Generalized Linear Models
Sabrina Liu, Andrew S. Perley and Todd P. Coleman
Objective: This work proposes a dynamic state-space statistical model of heartbeat dynamics whose optimal state estimate can be solved efficiently with convex optimization and used to assess autonomic and cardiovascular health. Methods: This state-space model combines a Gauss-Markov process prior with gamma emissions using a log-link function. Parameters are estimated using a static version of the model, and alternating direction method of multipliers is used to efficiently find the optimal maximum a priori latent state estimate. Model selection is performed by finding the tradeoff between model size and model fit using the partial autocorrelation function. Results: We demonstrate in synthetically generated heartbeats that this model is able to closely recover the underlying latent weights. Then, we show using a tilt table
dataset that this statistical model is able to better capture sharp transients in heart rate patterns than traditional local averaging methods. In addition, combining heart beat timings with information from other organs such as the respiratory phase, we show that our statistical modeling framework enables the extraction of information-theoretic measures to quantify inter-organ coupling such as respiratory sinus arrythmia. Conclusion: This statistical framework for modeling heartbeat dynamics provides an efficient framework that can provide physiologically meaningful insights. Significance: This work broadens potential tools that can be used to assess heartbeat dynamics.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.NA
Uncertainty Quantification for Landweber Iteration with Randomized Normal-Operator Approximation
Anuj Abhishek and Sean Holman
Iterative methods are extremely popular for solving linear ill-posed inverse problems. While deterministic convergence, or more precisely, semi-convergence of such methods is widely studied, the problem of quantifying uncertainty in the resulting reconstructions due to random noise in the observed data has received far less
attention. In this work, we study uncertainty quantification for the Landweber iteration in linear inverse problems using the Radon transform as a motivating example. We interpret the Landweber reconstruction statistically by analyzing how uncertainty in the measured data is propagated through the reconstruction map. This perspective is closely related to generalized fiducial inference, where uncertainty about the parameter is induced by inverting the relation between the observed data and the unknown (fixed) quantity. We combine the resulting
stochastic uncertainty with a theoretical bound on the regularization bias to construct confidence intervals for the reconstructed solution. The evaluation of the resulting uncertainty estimates require repeated operations with the forward operator that can become expensive in large-scale problems. To reduce this cost, we use randomized singular value decomposition to obtain a low-rank approximation of the normal operator associated with the Radon transform and incorporate this approximation in quantifying uncertainty. {{In particular, this requires that the additional randomization error introduced due to the use of such randomized techniques be included in our approach for uncertainty quantification.}} Numerical results show that the proposed method provides accurate reconstructions and reliable uncertainty estimates at substantially reduced computational cost.
Read the full abstract ▾
Statistical Methodology
Scalar-on-function regression with measurement error in the functional regressors
Xiaochen Cai, Fenglin Xie and Todd Ogden
We consider the problem of scalar-on-function regression. Most existing methods implicitly assume that the functional
covariates are observed exactly, but in practice, they are often contaminated by measurement error. Our goal, therefore, is to deal with the problem of scalar-on-function regression when the regressor functions are observed with error. In this paper, we propose a simulation-extrapolation method to correct for the attenuation of estimated coefficient functions caused by the error. The method first estimates the error variance, establishes the relationship between a sequence of added error variance and the corresponding estimates of coefficient functions, and then extrapolates to the zero-error. We describe three methods to extrapolate the sequence of estimated coefficient functions. In a simulation study, we compare the performance of the simulation-extrapolation method with two pre-smoothing methods based on smoothing splines and functional principal component analysis. Next, we discuss the extension of the method in several directions, allowing for more complex noise covariance structures, multiple replications of functional predictors, generalized responses, and 2D and 3D functional predictors. Finally, we illustrate the methods by an application to
diffusion tensor imaging data.
Read the full abstract ▾
Statistics Applications · Cross-listed from q-bio.NC
One Inference, Four Failure Modes: Formal Models of Why Pain Location Fails
Adam Y. Shavit
Patient-reported pain location is diagnostically decisive for some presentations and nearly uninformative for others. A companion paper argues this is not one gradient of diagnostic utility but three distinct failures of localization. This paper gives those failures their mathematics and shows they are one object: a single
Bayesian generative model failing at different nodes - the likelihood, the model class, and group- or context-dependence in that same likelihood. The count is not in dispute: the first three are failures of the inference that produces a felt location, and the fourth, added here, is a failure of reporting it. Anatomical multiplexing is a non-identifiable inverse problem: a rank-deficient referral
matrix sends distinct causes to one report. Delocalized amplification is a change of generative model whose dynamics are a neural-field bifurcation, with spatial extent as the order parameter. Referred and atypical displacement is group- or context-dependence in the presentation likelihood - a probability, not a loss - whose consequence sits one layer downstream, in a decision threshold varying with group prevalence and cost. Repeated observation cannot reduce recoverable information about a fixed inferential target, an exact chain-rule identity, while practical value can fall. The fourth node is the report itself. A spatial
Bayesian model factorizes mislocalization into referral blur, an anatomical offset and a precision-weighted cognitive override, and reproduces phantom-limb, mirror-box and central-post-stroke reports as regimes of one equation. It is estimable under conditions the paper states as necessary or sufficient rather than assuming them; Sec. 8 gives the design that meets them. One control is reported against the paper's own interest: the transport statistic first used to measure migration scores a stationary, spreading profile as though it had travelled.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.AI
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira Jr. et al. (+1 more)
Existing approaches to offline goal-conditioned reinforcement learning (GCRL) struggle with long-horizon tasks. Discounting shrinks value differences between distant states until they fall below the function approximation error, leaving the
agent with no signal for ranking states. Temporal abstraction, which treats k environment steps as a single transition, restores this signal at long range, but no single fixed k suits all state-goal distances: large k preserves value differences across long temporal distances while collapsing distinctions between nearby states, and small k does the reverse. We make this trade-off explicit and introduce Generalized Implicit Temporal Abstraction (GITA), which conditions a single value function on k. GITA trains one policy by aggregating advantage-weighted supervision across multiple k values, so scales assigning larger positive advantages to a state-goal pair contribute more strongly to its update. GITA does not need to choose between local resolution and long-range signal; it retains both without committing to a single k. On OGBench, GITA outperforms a broad range of offline GCRL baselines, raising average success rate across all tasks by 25 percentage points (73% relative improvement) over HIQL. It also improves over the strongest fixed-k method, OTA, by 7 percentage points (14% relative).
Read the full abstract ▾
All authors Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira Jr., Marlos C. Machado