Statistical Methodology
Mutual Information as a Tool for Optimal Classification: Application to Identifying Rapid-Responding Behaviour
Santeri Holopainen, Jari Metsämuuronen, Mikko-Jussi Laakso et al. (+1 more)
Existing methods for identifying rapid-responding behaviour in large-scale assessments require parametric assumptions about the population. In this study, we propose a novel, non-parametric, mutual information-based framework of methods as an alternative. The methods within this framework compute the mutual information of the observed responses and discretised response times and maximise the information gain to determine a threshold that differentiates rapid responses from engaged responses. The only difference between the methods is the number of categories in the relevant variables. We present three methods explicitly. The first method uses response correctness and binarised response times. The second method uses correctness and categorises time into three groups. The third method uses raw responses and binarised times. We applied these methods to mathematics achievement data collected through the Programme for International Student Assessment in 2022. Furthermore, we examined the behaviour and usability of the first method in certain realistic conditions at the population and realised levels. The results indicated that the proposed framework is a viable alternative for identifying rapid responses. The framework's novelty lies in its non-parametric nature and its ability to utilise raw responses instead of correctness. Finally, we discuss some possible future research directions on this topic.
Read the full abstract ▾
All authors Santeri Holopainen, Jari Metsämuuronen, Mikko-Jussi Laakso, Janne V. Kujala
Statistical Methodology
Matrix Graphical Model Via Joint Estimation of Partial Correlations
Hyewon Kim and Seongoh Park
Matrix graphical models aim to characterize conditional dependence structures in
matrix-variate data under a separable covariance assumption. In this framework, the precision
matrix is decomposed as a Kronecker product, enabling separate modeling of undirected graphs across row and column domains. Existing methods have been developed for this problem, including likelihood-based approaches and regression-based procedures for graph estimation. Likelihood-based methods estimate precision
matrices directly and recover graph structures indirectly, whereas regression-based approaches directly target estimating edges among variables, thus outperforming the former. However, existing regression-based methods are based on multiple penalized regression problems, which naturally yields asymmetry in estimated graphs and computational difficulty in selecting tuning parameters. To address the limitations, we propose a joint estimation of partial correlations in
matrix graphical models. The proposed method estimates all partial correlations simultaneously within a unified optimization framework, thereby preserving
symmetry and easing the pain of selecting the best models. Numerical studies demonstrate that the proposed method improves graph recovery performance compared to existing approaches. We also analyze
protein expression data collected from patients with pulmonary tuberculosis, measured repeatedly at multiple time points, where the proposed method compares
protein networks between two groups of patients and recovers the temporal dependence structure.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Learn Your Own Thoughts: Abstract Token Curriculum
Khashayar Gatmiry, Avrajit Ghosh, Parsa Mirtaheri et al. (+4 more)
Large
Language Models (LLMs) have achieved remarkable
reasoning capabilities by utilizing chain-of-thought (CoT) as a scratchpad for intermediate stages of thinking. However, CoT techniques require explicit supervision on thinking
tokens, which requires rich, task-specific data. In this work, we propose Abstract
Token Curriculum (ATC), a novel curriculum learning framework that elicits effective continuous intermediate representations without direct supervision or manual scratchpad design. ATC gradually increases problem complexity through a sequence of distributions, training the model to develop internal abstract “thoughts” in the continuous representation space. This paper provides both theoretical and experimental evidence for the benefits of ATC and its advantages over previous methods for training continuous thoughts. Theoretically, we show that for learning parity functions with single-layer softmax
attention using ATC,
attention naturally focuses on the CoT
tokens in the context that provide the “easiest path” to predicting the next
token. Experimentally, we show ATC's effectiveness on graph reachability and arithmetic learning tasks.
Read the full abstract ▾
All authors Khashayar Gatmiry, Avrajit Ghosh, Parsa Mirtaheri, Jason D. Lee, Nika Haghtalab, Emmanuel Abbe, Peter Bartlett
Statistical Methodology
Federated multiple testing with quantized p-values under family-wise error control
Prasanjit Dubey and Xiaoming Huo
Several institutions test one family of hypotheses under family-wise error control but cannot pool their data, so each site releases only a quantized report of its p-value for each hypothesis, such as whether the p-value is significant. The report's resolution is the number of values it can take. We quantify the power lost to such reports relative to the most powerful centralized procedure given all p-values (the oracle), with reports and combining rule chosen optimally. A report from which the p-value can be recovered loses no power, so the loss is due to compression, not decentralization. Under the stated regularity and power-objective conditions, every finite resolution loses power, and the optimal loss decays as the inverse square of the resolution: equal-width intervals attain this order, and no partition into as many
cells improves it. In two-site Beta examples, optimized one-bit reports are nearly lossless for a single hypothesis but lose several times as much power for two. At level 0.05 with two Beta(1,2) hypotheses, optimally combined three-bit equal-width reports retain 99.2% of oracle power. We give the power-optimal combining rule for given reports, and a rule using only their exactly known null distribution, valid under arbitrary dependence across hypotheses.
Read the full abstract ▾
Statistical Methodology
Heterogeneity-calibrated Byzantine-robust distributed composite quantile regression
Xiaofei Wu and Jian Qing Shi
We study sparse composite
quantile regression (CQR) for distributed data with heterogeneous honest sites and Byzantine workers. Honest sites share a common slope but may differ in their
covariate distributions, error laws, and
quantile intercepts. The proposed heterogeneity-calibrated robust CQR (HC-RCQR) profiles local intercepts and calibrates scores using an approximate inverse profile Hessian. Honest workers transmit the resulting vectors, whereas Byzantine workers may send arbitrary vectors. The server updates the estimate by coordinatewise trimming and soft thresholding. A scalar example shows how unequal honest-site
curvatures allow intermediate Byzantine reports to survive trimming and how ideal calibration reduces their possible effect. We also establish nonidentification of the mean of unrestricted honest-site slopes when fault identities are unknown. Under suitable conditions, we establish conditional contraction and support-recovery guarantees. The bound separates score offset, sampling fluctuation, contamination, calibration error, and the Newton remainder. Simulations and a bike-demand study examine performance under heterogeneous data and
adversarial messages.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from quant-ph
Improving Sample Efficiency in Peptide-HLA Binding Prediction with Hybrid Quantum-Classical Neural Networks
Chenyan Jia, Cong Guo, Siyue Chen et al. (+2 more)
Peptide-HLA binding prediction is a critical step in neoantigen identification for personalized cancer immunotherapy and holds significant clinical value. However, the training data available for many HLA alleles are extremely limited, which severely constrains the performance of conventional methods on this task. Parameterized
quantum circuits are hypothesized to induce inductive biases beneficial for learning from small
datasets, yet their application to biological sequence prediction remains underexplored. To address this, we propose a hybrid
quantum-classical neural network (HQNN) specifically designed for peptide-HLA binding prediction. HQNN integrates multi-source biological feature encoding with parallel
quantum feature extractors and a
quantum-enhanced classifier. On two HLA alleles (A*02:01 and B*07:02), HQNN outperforms a parameter-matched classical CNN baseline across all training sizes, with the performance gap widening as training data decreases. Ablation studies confirm the respective contributions of the
quantum feature extraction module and the
quantum classifier. In noise-aware simulations, performance degrades only mildly, and such degradation is reasonable and acceptable under realistic
quantum hardware noise levels. These results suggest that hybrid
quantum-classical architectures can provide practical sample-efficiency gains for immunoinformatics tasks in low-data regimes.
Read the full abstract ▾
All authors Chenyan Jia, Cong Guo, Siyue Chen, Pengpeng Ye, Xiaochun Chen
Statistics Theory
Human-Anchored Inference for Ranking New Models with Large Language Model Judges
Xin Zhou, Sinian Zhang, Zhanyan Yang et al. (+3 more)
Human pairwise comparisons provide a reference for evaluating large
language models (LLMs), but collecting sufficient judgments for each new release is costly and time-consuming. LLM judges offer a scalable alternative, although their comparisons may differ systematically from human preferences and across judges. We study the ranking of a new model that has received LLM-judge comparisons but no human comparisons. We propose ANCHOR (ANchored Comparisons for Human-reference inference with Orthogonal Riesz correction), which uses historical human and LLM comparisons to learn judge-specific sensitivities to human score differences and feature-dependent judge biases. These estimates are then used to infer the new model's human-reference score from its judge comparisons. The framework allows the feature distribution to change between historical and new-model comparisons. For inference, we construct a Neyman-orthogonal
estimator through a joint Riesz correction that removes the first-order effects of estimating the historical human scores, judge sensitivities, and bias functions. We establish identification, convergence rates, and asymptotic normality with consistently estimable variance, and show that ANCHOR attains the semiparametric efficiency bound. Simulations demonstrate gains in score estimation and ranking accuracy. On Chatbot Arena, ANCHOR achieves the lowest score RMSE and insertion MAE among competing methods, with narrower score intervals on average.
Read the full abstract ▾
All authors Xin Zhou, Sinian Zhang, Zhanyan Yang, Mingyuan Xu, Molei Liu, Doudou Zhou
Statistical Methodology
Selective Inference in Growth Curve Models
Haochen Lei, Qian Zhang and Hongyuan Cao
Growth curve models are widely used in psychological research, and variable selection can help identify baseline characteristics associated with longitudinal heterogeneity. However, conventional inference after data-driven variable selection can be invalid because the same outcome data are used for both selection and inference. We develop a data-fission framework for post-selection inference in growth-curve models that separates the information used for selection and inference while retaining all participants in both stages through the addition and subtraction of Gaussian noise. The framework accommodates flexible variable-selection procedures and targets covariance-weighted linear projection parameters in the selected working model. It provides exact inference when the covariance structure is known, and we establish asymptotic validity under suitable regularity conditions when the covariance structure is estimated. Simulations show that the proposed method provides valid inference and improves efficiency over subject-level data splitting, whereas naïve post-selection inference can be biased. An application to the Longitudinal Study of American Youth illustrates the proposed framework.
Read the full abstract ▾
Statistical Methodology
Oracle high-dimensional $M$-estimation using smooth reparameterization for sparsity
Yoichi Nishiyama
This paper establishes a unified non-linear regularization framework for high-dimensional $M$-estimation, encompassing both linear models and Cox's proportional hazards models. Rather than relying on traditional additive non-convex penalties, the proposed paradigm embeds sparsity directly into the transformation for the physical parameter $\theta=\phi(\beta)$ using a smooth ($C^2$) component-wise "ReParametrization map for Sparsity (RePS)" $\phi$, and the penalty term is $\lambda \Vert \beta \Vert_1$ rather than $\lambda \Vert \theta \Vert_1$. This structural formulation dynamically adapts to local parameter scales, suppressing high-dimensional noise while simultaneously recovering unbiased oracle asymptotic normality under the large-sample limit. Through the Primal-Dual Witness method, we establish a unified oracle equivalence for both model classes under a liberated micro-penalty scaling regime $\lambda \ll n^{-1/2}$, accommodating model-specific structures such as the shift-invariance in survival analysis. Extensive Monte Carlo simulations demonstrate that the proposed framework consistently achieves superior false-positive control and high 95% confidence interval coverage in high-dimensional linear models, while also delivering performance comparable or superior in all aspects to state-of-the-art methods like SCAD and MCP in proportional hazards models.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cond-mat.mtrl-sci
Portfolio-Based Constrained Multi-Objective Bayesian Optimization for Materials Design
Sushant Sinha, Christofer Hardcastle, Robert Robinson et al. (+4 more)
Materials discovery and design campaigns can be formulated as constrained multi-objective
Bayesian optimization (CMOBO) problems, within which each experimental decision negotiates between two coupled but competing goals: discovering feasible candidates and refining the underlying Pareto front. Here we recast acquisition-function choice as an adaptive policy-selection problem over a portfolio of conventional and feasibility-focused acquisition functions. This was done using two controllers: UCB-Bandit, a modified UCB multi-armed bandit, and Agentic-Switch, a multi-
agent decision system driven by a large
language model (LLM). Both were evaluated against fixed-policy baselines in silico across five synthetic
benchmark functions and two materials design case studies. The adaptive policies performed competitively in terms of both cumulative feasibility count and feasible hypervolume improvement, while each individual acquisition function performed well for only one metric, suggesting that adaptive policies are better suited for constrained materials science problems.
Read the full abstract ▾
All authors Sushant Sinha, Christofer Hardcastle, Robert Robinson, Shakti Prasad Padhy, Brent Vela, Douglas Allaire, Raymundo Arroyave
Statistical Methodology
Vector Vine Copula Models for Multivariate Longitudinal Data
Michael Stanley Smith and Lin Deng
Multivariate longitudinal data may exhibit non-Gaussian margins, nonlinear dynamics, and response vectors with composition that varies across waves. To account for these features, we introduce a vector drawable vine (VD-vine) copula that extends conventional drawable vine copulas from scalar to vector-valued nodes. Here, the response vector at each wave forms a multivariate marginal, and serial dependence is captured through a sequence of linking vector copulas. We establish that the VD-vine is itself a vector copula and reduces to a conventional drawable vine for scalar nodes. Recursive forward and backward conditional transports are derived that enable efficient likelihood evaluation and predictive simulation, with parsimonious reductions under finite-order Markov and stationary restrictions. Unconstrained parameterizations for Gaussian and FGM linking vector copulas, flexible multivariate marginals, and
Bayesian variational inference provide a practical implementation. Simulations show improved predictive accuracy when the marginals are asymmetric and serial dependence is multivariate, with little loss under a correctly specified Gaussian panel vector autoregression. In an eight-wave Australian panel of 1,093 individuals with varying response vectors, the full VD-vine delivers the best cross-validated distributional forecasts among the models considered, establishing the benefit of capturing asymmetry and nonlinear dependence.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Compressed Active Subspaces for Scalable Bayesian Inference
Thomas Flynn, Sanket Jantre, Byung-Jun Yoon et al. (+1 more)
Active subspace methods provide a framework for quantifying predictive uncertainty in high-dimensional models by identifying and performing inference along parameter directions that have the greatest influence on the model output. However, the construction of active subspaces requires storing many full-dimensional model gradients, which becomes prohibitive as model size increases. We address this limitation by proposing Compressed Active Subspaces (CAS), a scalable approach that first maps the model parameters to a compressed space using a structured isometric
embedding and then constructs the active subspace within this reduced parameterization. Our approach substantially reduces the
memory required for active subspace construction and enables
Bayesian inference for large models where standard active subspace methods become impractical. We demonstrate the scalability of CAS on neural networks of increasing size while maintaining predictive performance and robust uncertainty estimates.
Read the full abstract ▾
All authors Thomas Flynn, Sanket Jantre, Byung-Jun Yoon, Kibaek Kim
Machine Learning (Statistics)
Next-token functional estimation
Milind Nakul, Vidya Muthukumar and Ashwin Pananjady
Suppose we observe the first $n$ points of a sequence of random variables having length $n+1$, and wish to estimate a functional of the unobserved final point and the empirical measure of the $n$ observed training points. Such next-
token functionals include the probability that the next
token is novel (also known as the surprise probability), the tail probability of the minimum distance between the next
token and training points, and the test error of a classifier trained on the observed points. All of these quantities are classically estimated by the leave-one-out method, which is inconsistent under temporal dependence. We propose a leave-a-window-out
estimator, which deletes a window of length $\tau$ after each index before forming the empirical measure and reduces to leave-one-out at $\tau = 1$. Under natural assumptions, we show that the error of our
estimator decays at a parametric rate for any stationary $\beta$-mixing process that also admits a Marton coupling. Our results thus cover several natural functionals on a large class of
stochastic processes. We complement these upper bounds with a sharp minimax lower bound for estimating the surprise probability on mixing Markov chains. Simulations on Markov chains, moving-average processes, and autoregressive processes show that our
estimator succeeds in many scenarios where leave-one-out and add-constant baselines fail.
Read the full abstract ▾
Statistics Applications · Cross-listed from eess.IV
Compression Hurts, Pooling Helps: Information Loss in Rayleigh-Scale Estimation from B-Mode Ultrasound
D. Hudson Smith and Ahmer Raza
Clinical B-mode images are widely available as potential data sources for quantitative ultrasound (QUS) analysis for tissue characterization. However, standard clinical ultrasound devices apply unknown log-compression to RF envelope data before display and storage. Previous work has demonstrated estimation of the underlying RF envelope statistics in the presence of an unknown compression law. Using Fisher information analysis, we show that finite-offset log compression causes severe information loss when estimating the Rayleigh scale $\sigma$, which controls diffuse speckle. For a single image window, unknown compression raises the minimum achievable variance for unbiased estimation of $\sigma$ by a compression-independent factor of approximately 34.98. When $M$ equal-sized windows share the same unknown compression settings, the excess variance decays as $1/M$; even in the most favorable regime, reducing the variance inflation factor below 1.1 requires 340 windows. Our analysis treats the contrast parameter $a$ as unknown and the boundary offset $b$ as known; estimating $b$ experimentally shows even larger variance. We validate this theory using synthetic estimation experiments and demonstrate RF-scale recovery on real RF-envelope windows from the OASBUD
dataset. Together, these results clarify the limitations of using routine B-mode images for QUS.
Read the full abstract ▾
Statistical Methodology
Spatial Aggregation of ROC and Precision-Recall Curves
Romain Pic, Zhongwei Zhang, Sebastian Engelke et al. (+1 more)
Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are widely used to assess the discrimination ability of forecasts for binary events, such as threshold exceedances or warnings of extreme events. In weather forecasting, forecasts are provided as spatial fields, yielding location-wise ROC and PR curves that are often aggregated to facilitate comparison. However, the effect of the aggregation strategy on performance assessment remains poorly understood. We investigate how different aggregation strategies for ROC and PR curves affect the assessment of discrimination ability. In particular, we identify conditions under which aggregation strategies satisfy two desirable properties for fair comparison: preservation of dominance between forecasts and preservation of concavity or achievability of the curves. We obtain sufficient conditions and propose two strategies satisfying them. They are compared with existing strategies from the literature, and we analyze their properties and highlight potential pitfalls that may lead to misleading interpretations. Based on these findings, we provide practical guidelines for the interpretation of aggregated ROC and PR curves. The proposed framework is illustrated with AI-based global weather forecasts, showing how different aggregation strategies can yield different rankings of competing forecasts.
Read the full abstract ▾
All authors Romain Pic, Zhongwei Zhang, Sebastian Engelke, Johanna Ziegel