Machine Learning
Counterfactual Explanations for Hypergraph Neural Networks
Fabiano Veglianti, Lorenzo Antonelli and Gabriele Tolomei
Hypergraph neural networks (HGNNs) effectively model higher-order interactions in many real-world systems but remain difficult to interpret, limiting their deployment in high-stakes settings. We introduce CF-HyperGNNExplainer, a counterfactual explanation method for HGNNs that identifies the minimal structural changes required to alter a model's prediction. The method generates counterfactual hypergraphs using actionable edits limited to removing node-hyperedge incidences or deleting hyperedges, producing concise and structurally meaningful explanations. Extensive experiments on hypergraph
benchmark datasets show that CF-HyperGNNExplainer generates valid and concise counterfactuals, highlighting the higher-order relations most critical to HGNN decisions.
Read the full abstract ▾
Machine Learning
Mosaic Learning: A Framework for Decentralized Learning with Model Fragmentation
Sayan Biswas, Davide Frey, Romaric Gaudel et al. (+7 more)
Decentralized learning (DL) enables collaborative machine learning (ML) without a central server, making it suitable for settings where training data cannot be centrally hosted. We introduce Mosaic Learning, a DL framework that decomposes models into fragments and disseminates them independently across the network. Fragmentation reduces redundant communication across correlated parameters and enables more diverse information propagation without increasing communication cost. We theoretically show that Mosaic Learning (i) shows state-of-the-art worst-case convergence rate, and (ii) leverages parameter correlation in an ML model, improving contraction by reducing the highest
eigenvalue of a simplified system. We empirically evaluate Mosaic Learning on four learning tasks and observe up to 12 percentage points higher node-level test accuracy compared to epidemic learning (EL), a state-of-the-art baseline. In summary, Mosaic Learning improves DL performance without sacrificing its utility or efficiency, and positions itself as a new DL standard.
Read the full abstract ▾
All authors Sayan Biswas, Davide Frey, Romaric Gaudel, Nirupam Gupta, Anne-Marie Kermarrec, Dimitri Lerévérend, Rafael Pires, Rishi Sharma, François Taïani, Martijn de Vos
Machine Learning · Cross-listed from stat.ML
A Bandit-Based Approach to Educational Recommender Systems: Contextual Thompson Sampling for Learner Skill Gain Optimization
Lukas De Kerpel, Arthur Thuy and Dries F. Benoit
In recent years, instructional practices in Operations Research (OR), Management Science (MS), and Analytics have increasingly shifted toward digital environments, where large and diverse groups of learners make it difficult to provide practice that adapts to individual needs. This paper introduces a method that generates personalized sequences of exercises by selecting, at each step, the exercise most likely to advance a learner's understanding of a targeted skill. The method uses information about the learner and their past performance to guide these choices, and learning progress is measured as the change in estimated skill level before and after each exercise. Using data from an online mathematics tutoring platform, we find that the approach recommends exercises associated with greater skill improvement and adapts effectively to differences across learners. From an instructional perspective, the framework enables personalized practice at scale, highlights exercises with consistently strong learning value, and helps instructors identify learners who may benefit from additional support.
Read the full abstract ▾
Machine Learning
MirrorLA: Reflecting Feature Map for Vision Linear Attention
Weikang Meng, Liangyu Huo, Yadan Luo et al. (+3 more)
Linear
attention significantly reduces the computational complexity of
Transformers from quadratic to linear, yet it consistently lags behind softmax-based
attention in performance. We identify the root cause of this degradation as the non-negativity constraint imposed on kernel feature maps: standard projections like ReLU act as "passive truncation" operators, indiscriminately discarding semantic information residing in the negative domain. We propose MirrorLA, a
geometric framework that substitutes passive truncation with active reorientation. By leveraging learnable Householder reflections, MirrorLA rotates the feature geometry into the non-negative orthant to maximize information retention. Our approach restores representational density through a cohesive, multi-scale design: it first optimizes local discriminability via block-wise isometries, stabilizes long-context dynamics using variance-aware modulation to diversify activations, and finally, integrates dispersed subspaces via cross-head reflections to induce global covariance mixing. MirrorLA achieves state-of-the-art performance across standard
benchmarks, demonstrating that strictly linear efficiency can be achieved without compromising representational fidelity.
Read the full abstract ▾
All authors Weikang Meng, Liangyu Huo, Yadan Luo, Yaowei Wang, Yingjian Li, Zheng Zhang
Machine Learning
UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching
Kou Misaki and Takuya Akiba
Test-time scaling strategies have effectively leveraged inference-time compute to enhance the
reasoning abilities of Autoregressive Large
Language Models. In this work, we demonstrate that Masked
Diffusion Language Models (MDLMs) are inherently amenable to advanced search strategies, owing to their iterative and non-autoregressive generation process. To leverage this, we propose UnMaskFork (UMF), a framework that formulates the unmasking trajectory as a search tree and employs Monte Carlo Tree Search to optimize the generation path. In contrast to standard scaling methods relying on
stochastic sampling, UMF explores the search space through deterministic partial unmasking actions performed by multiple MDLMs. Our empirical evaluation demonstrates that UMF consistently outperforms existing test-time scaling baselines on complex coding
benchmarks, while also exhibiting strong scalability on mathematical
reasoning tasks.
Read the full abstract ▾
Machine Learning
RISE: Interactive Visual Diagnosis of Fairness in Machine Learning Models
Ray Chen and Christan Grant
Evaluating
fairness under domain shift is challenging because scalar metrics often obscure exactly where and how disparities arise. We introduce RISE (Residual Inspection through Sorted Evaluation), an interactive visualization tool that converts sorted residuals into interpretable patterns. By connecting residual curve structures to formal
fairness notions, RISE enables localized disparity diagnosis, subgroup comparison across environments, and the detection of hidden
fairness issues. Through post-hoc analysis, RISE exposes accuracy-
fairness trade-offs that aggregate statistics miss, supporting more informed model selection.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Geometry-Aware Optimal Transport: Fast Intrinsic Dimension and Wasserstein Distance Estimation
Ferdinand Genans and Olivier Wintenberger
Solving large scale Optimal Transport (OT) in machine learning typically relies on sampling measures to obtain a tractable discrete problem. While the discrete solver's accuracy is controllable, the rate of convergence of the discretization error is governed by the intrinsic dimension of our data. Therefore, the true bottleneck is the knowledge and control of the sampling error. In this work, we tackle this issue by introducing novel
estimators for both sampling error and intrinsic dimension. The key finding is a simple, tuning-free
estimator of $\text{OT}_c(\rho, \hat\rho)$ that utilizes the semi-dual OT functional and, remarkably, requires no OT solver. Furthermore, we derive a fast intrinsic dimension
estimator from the multi-scale decay of our sampling error
estimator. This framework unlocks significant computational and statistical advantages in practice, enabling us to (i) quantify the convergence rate of the discretization error, (ii) calibrate the entropic regularization of Sinkhorn divergences to the data's intrinsic geometry, and (iii) introduce a novel, intrinsic-dimension-based Richardson extrapolation
estimator that strongly debiases Wasserstein distance estimation. Numerical experiments demonstrate that our geometry-aware pipeline effectively mitigates the discretization error bottleneck while maintaining computational efficiency.
Read the full abstract ▾
Machine Learning
Efficient Equivariant High-Order Crystal Tensor Prediction via Cartesian Local-Environment Many-Body Coupling
Dian Jin, Yancheng Yuan and Xiaoming Tao
End-to-end prediction of high-order
crystal tensor properties from atomic structures remains challenging: while spherical-harmonic equivariant models are expressive, their Clebsch-Gordan tensor products incur substantial compute and
memory costs for higher-order targets. We propose the Cartesian Environment Interaction Tensor Network (CEITNet), an approach that constructs a multi-channel Cartesian local environment tensor for each atom and performs flexible many-body mixing via a learnable channel-space interaction. By performing learning in channel space and using Cartesian tensor bases to assemble equivariant outputs, CEITNet enables efficient construction of high-order tensor. Across
benchmark datasets for order-2 dielectric, order-3 piezoelectric, and order-4 elastic tensor prediction, CEITNet surpasses prior high-order prediction methods on key accuracy criteria while offering high computational efficiency.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement
Chien-Chun Wang, Hung-Shin Lee, Hsin-Min Wang et al. (+1 more)
Pre-trained models for automatic
speech recognition (ASR) and
speech enhancement (SE) have exhibited remarkable capabilities under matched noise and channel conditions. However, these models often suffer from severe performance degradation when confronted with domain shifts, particularly in the presence of unseen noise and channel distortions. In view of this, we in this paper present URSA-GAN, a unified and domain-aware generative framework specifically designed to mitigate mismatches in both noise and channel conditions. URSA-GAN leverages a dual-
embedding architecture that consists of a noise encoder and a channel encoder, each pre-trained with limited in-domain data to capture domain-relevant representations. These
embeddings condition a GAN-based
speech generator, facilitating the synthesis of
speech that is acoustically aligned with the target domain while preserving phonetic content. To enhance generalization further, we propose dynamic
stochastic perturbation, a novel regularization technique that introduces controlled variability into the
embeddings during generation, promoting robustness to unseen domains. Empirical results demonstrate that URSA-GAN effectively reduces character error rates in ASR and improves perceptual metrics in SE across diverse noisy and mismatched channel scenarios. Notably, evaluations on compound test conditions with both channel and noise degradations confirm the generalization ability of URSA-GAN, yielding relative improvements of 16.16% in ASR performance and 15.58% in SE metrics.
Read the full abstract ▾
All authors Chien-Chun Wang, Hung-Shin Lee, Hsin-Min Wang, Berlin Chen
Machine Learning · Cross-listed from cs.CL
Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecification
Branislav Pecher, Michal Spiegel, Robert Belanec et al. (+1 more)
Large
language models (LLMs) are widely used as zero-shot and few-shot classifiers, where task behaviour is largely controlled through prompting. A growing number of works have observed that LLMs are sensitive to
prompt variations, with small changes leading to large changes in performance. However, in many cases, the investigation of sensitivity is performed using underspecified
prompts that provide minimal task instructions and weakly constrain the model's output space. In this work, we argue that a significant portion of the observed
prompt sensitivity can be attributed to
prompt underspecification. We systematically study and compare the sensitivity of underspecified
prompts and
prompts that provide specific instructions. Utilising performance analysis, logit analysis, and linear probing, we find that underspecified
prompts exhibit higher performance variance and lower logit values for relevant
tokens, while instruction-
prompts suffer less from such problems. However, linear probing analysis suggests that the effects of
prompt underspecification have only a marginal impact on the internal LLM representations, instead emerging in the final layers. Overall, our findings highlight the need for more rigour when investigating and mitigating
prompt sensitivity.
Read the full abstract ▾
All authors Branislav Pecher, Michal Spiegel, Robert Belanec, Jan Cegin
Machine Learning
Disentangling Intrinsic Importance from Emergent Structure in Multi-Expert Orchestration
Sudipto Ghosh, Sujoy Nath, Sunny Manchanda et al. (+1 more)
Multi-expert systems, where multiple Large
Language Models (LLMs) collaborate to solve complex tasks, are increasingly adopted for high-performance
reasoning and generation. However, the orchestration policies governing expert interaction and sequencing remain largely opaque. We introduce INFORM, an interpretability analysis that treats orchestration as an explicit, analyzable computation, enabling the decoupling of expert interaction structure, execution order, and functional attribution. We use INFORM to evaluate an orchestrator on GSM8K, HumanEval, and MMLU using a homogeneous consortium of ten instruction-tuned experts drawn from LLaMA-3.1 8B, Qwen3 8B, and DeepSeek-R1 8B, with controlled decoding-temperature variation, and a secondary heterogeneous consortium spanning 1B-7B parameter models. Across tasks, routing dominance is a poor proxy for functional necessity. We reveal a divergence between relational importance, captured by routing mass and interaction
topology, and intrinsic importance, measured via gradient sensitivity: frequently selected experts often act as interaction hubs with limited influence, while sparsely routed experts can be structurally critical. Orchestration behaviors emerge asynchronously, with expert centralization preceding stable routing confidence and expert ordering remaining non-deterministic. Targeted ablations show that masking intrinsically important experts induces disproportionate collapse in interaction structure compared to masking frequent peers, confirming that INFORM exposes functional and structural dependencies beyond accuracy metrics alone. Our code is available at https://github.com/parmanu-lcs2/inform.
Read the full abstract ▾
All authors Sudipto Ghosh, Sujoy Nath, Sunny Manchanda, Tanmoy Chakraborty
Machine Learning · Cross-listed from cs.CL
Proxy Compression for Language Modeling
Lin Zheng, Xinyu Li, Qian Liu et al. (+2 more)
Modern
language models are trained almost exclusively on
token sequences produced by a fixed tokenizer, an external lossless compressor often over UTF-8 byte sequences, thereby coupling the model to that compressor. This work introduces proxy compression, an alternative training scheme that preserves the efficiency benefits of compressed inputs while providing an end-to-end, raw-byte interface at inference time. During training, a single
language model is jointly trained on raw byte sequences and compressed views generated by external compressors; through the process, the model learns to internally align compressed sequences and raw bytes. This
alignment enables strong transfer between the two formats, even when training predominantly on compressed inputs that are discarded at inference. Extensive experiments on code
language modeling demonstrate that proxy compression substantially improves training efficiency and significantly outperforms pure byte-level baselines given fixed compute budgets. As model scale increases, these gains become more pronounced, and proxy-trained models eventually match or surpass tokenizer approaches, all while operating solely on raw bytes and retaining the inherent robustness of byte-level modeling. Our code is available at https://github.com/LZhengisme/proxy-compression.
Read the full abstract ▾
All authors Lin Zheng, Xinyu Li, Qian Liu, Xiachong Feng, Lingpeng Kong
Machine Learning · Cross-listed from cs.CL
Contextual Drag: How Errors in the Context Affect LLM Reasoning
Yun Cheng, Xingyu Zhu, Haoyu Zhao et al. (+1 more)
Central to many self-improvement pipelines for large
language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed contextual drag: the presence of failed attempts in the context biases subsequent generations toward structurally similar errors. Across evaluations of 11 proprietary and open-weight models on 8
reasoning tasks, contextual drag induces 10-20% performance drops, and iterative self-refinement in models with severe contextual drag can collapse into self-deterioration. Structural analysis using tree edit distance reveals that subsequent
reasoning trajectories inherit structurally similar error patterns from the context. We demonstrate that neither external feedback nor successful self-verification suffices to eliminate this effect. While mitigation strategies such as fallback-behavior fine-tuning and context denoising yield partial improvements, they fail to fully restore baseline performance, positioning contextual drag as a persistent failure mode in current
reasoning architectures.
Read the full abstract ▾
All authors Yun Cheng, Xingyu Zhu, Haoyu Zhao, Sanjeev Arora
Machine Learning
Convolution Operator Network for Forward and Inverse Problems (FI-Conv): Application to Plasma Turbulence Simulations
Xingzhuo Chen, Anthony Poole, Ionut-Gabriel Farcas et al. (+2 more)
We propose the Convolutional Operator Network for Forward and Inverse Problems (FI-Conv), a framework capable of predicting system
evolution and estimating parameters in complex spatio-temporal dynamics, such as
turbulence. FI-Conv is built on a U-Net architecture, in which most convolutional layers are replaced by ConvNeXt V2 blocks. This design preserves U-Net performance on inputs with high-frequency variations while maintaining low computational complexity. FI-Conv uses an initial state, PDE parameters, and
evolution time as input to predict the system future state. As a representative example of a system exhibiting complex dynamics, we evaluate the performance of FI-Conv on the task of predicting turbulent
plasma fields governed by the Hasegawa-Wakatani (HW) equations. The HW system models two-dimensional electrostatic drift-wave
turbulence and exhibits strongly nonlinear behavior, making accurate approximation and long-term prediction particularly challenging. Using an autoregressive
forecasting procedure, FI-Conv achieves accurate forward prediction of the
plasma state
evolution over short times (t ~ 3) and captures the statistic properties of derived physical quantities of interest over longer times (t ~ 100). Moreover, we develop a gradient-descent-based inverse estimation method that accurately infers PDE parameters from
plasma state
evolution data, without modifying the trained model weights. Collectively, our results demonstrate that FI-Conv can be an effective alternative to existing physics-informed machine learning methods for systems with complex spatio-temporal dynamics.
Read the full abstract ▾
All authors Xingzhuo Chen, Anthony Poole, Ionut-Gabriel Farcas, David R. Hatch, Ulisses Braga-Neto
Machine Learning · Cross-listed from cs.AI
Agent-Omit: Adaptive Context Omission for Efficient LLM Agents
Yansong Ning, Jun Fang, Naiqiang Tan et al. (+1 more)
Managing
agent context (e.g., thought and observation) during multi-turn
agent-environment interactions is an emerging strategy to improve
agent efficiency. However, existing studies treat the entire interaction trajectories equally, overlooking the thought necessity and observation utility varies across turns. To this end, we first conduct quantitative investigations into how thought and observation affect
agent effectiveness and efficiency. Based on our findings, we propose
Agent-Omit, a unified training framework that empowers LLM
agents to adaptively omit redundant thoughts and observations. Specifically, we first synthesize a small amount of cold-start data, including both single-turn and multi-turn omission scenarios, to fine-tune the
agent for omission behaviors. Furthermore, we introduce an omit-aware agentic reinforcement learning approach, incorporating a dual sampling mechanism and a tailored omission reward to incentivize the
agent's adaptive omission capability. Theoretically, we prove that the deviation of our omission policy is upper-bounded by KL-divergence. Experimental results on five
agent benchmarks show that our constructed
Agent-Omit-8B could obtain performance comparable to seven frontier LLM
agent, and achieve the best effectiveness-efficiency trade-off than seven efficient LLM
agents methods. Our code and data are available at https://github.com/usail-hkust/
Agent-Omit.
Read the full abstract ▾
All authors Yansong Ning, Jun Fang, Naiqiang Tan, Hao Liu
Machine Learning
Multi Objective Design Optimization of Non Pneumatic Passenger Car Tires Using Finite Element Modeling, Machine Learning, and Particle swarm Optimization and Bayesian Optimization Algorithms
Priyankkumar Dhrangdhariya, Soumyadipta Maiti and Venkataramana Runkana
Non Pneumatic tires offer a promising alternative to pneumatic tires. However, their discontinuous spoke structures present challenges in stiffness tuning, durability, and high speed vibration. This study introduces an integrated generative design and machine learning driven framework to optimize UPTIS type spoke geometries for passenger vehicles. Upper and lower spoke profiles were parameterized using high order
polynomial representations, enabling the creation of approximately 250 generative designs through PCHIP based
geometric variation. Machine learning models like KRR for stiffness and XGBoost for durability and vibration achieved strong predictive accuracy, reducing the reliance on computationally intensive FEM simulations. Optimization using Particle Swarm Optimization and
Bayesian Optimization further enabled extensive performance refinement. The resulting designs demonstrate 53% stiffness tunability, up to 50% durability improvement, and 43% reduction in vibration compared to the baseline. PSO provided fast, targeted convergence, while
Bayesian Optimization effectively explored multi objective tradeoffs. Overall, the proposed framework enables systematic development of high performance, next generation UPTIS spoke structures.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.CO
Bures-Wasserstein Importance-Weighted Evidence Lower Bound: Exposition and Applications
Peiwen Jiang, Takuo Matsubara and Minh-Ngoc Tran
The Importance-Weighted Evidence Lower Bound (IW-ELBO) has emerged as an effective objective for variational inference (VI), tightening the standard ELBO and mitigating the mode-seeking behaviour. However, optimizing the IW-ELBO in Euclidean space is often inefficient, as its gradient
estimators suffer from a vanishing signal-to-noise ratio (SNR). This paper formulates the optimisation of the IW-ELBO in Bures-Wasserstein space, a
manifold of Gaussian distributions equipped with the 2-Wasserstein metric. We derive the Wasserstein gradient of the IW-ELBO and project it onto the Bures-Wasserstein space to yield a tractable
algorithm for Gaussian VI. A pivotal contribution of our analysis concerns the stability of the gradient
estimator. While the SNR of the standard Euclidean gradient
estimator is known to vanish as the number of importance samples $K$ increases, we prove that the SNR of the Wasserstein gradient scales favourably as $\Omega(\sqrt{K})$, ensuring optimisation efficiency even for large $K$. We further extend this
geometric analysis to the Variational Rényi Importance-Weighted Autoencoder bound, establishing analogous stability guarantees. Experiments demonstrate that the proposed framework achieves superior approximation performance compared to other baselines.
Read the full abstract ▾
Machine Learning
Multi-Integration of Labels across Categories for Component Identification (MILCCI)
Noga Mudrik, Yuxi Chen, Gal Mishne et al. (+1 more)
Many fields collect large-scale temporal data through repeated measurements (trials), where each trial is labeled with a set of metadata variables spanning several categories. For example, a trial in a neuroscience study may be linked to a value from category (a): task difficulty, and category (b): animal choice. A critical challenge in time-series analysis is to understand how these labels are encoded within the multi-trial observations, and disentangle the distinct effect of each label entry across categories. Here, we present MILCCI, a novel data-driven method that i) identifies the interpretable components underlying the data, ii) captures cross-trial variability, and iii) integrates label information to understand each category's representation within the data. MILCCI extends a sparse per-trial decomposition that leverages label similarities within each category to enable subtle, label-driven cross-trial adjustments in component compositions and to distinguish the contribution of each category. MILCCI also learns each component's corresponding temporal trace, which evolves over time within each trial and varies flexibly across trials. We demonstrate MILCCI's performance through both synthetic and real-world examples, including voting patterns, online page view trends, and neuronal recordings.
Read the full abstract ▾
All authors Noga Mudrik, Yuxi Chen, Gal Mishne, Adam S. Charles
Machine Learning · Cross-listed from eess.SP
Aortic Valve Disease Screening from PPG via Physiology-Guided Self-Supervised Learning
Jiaze Wang, Qinghao Zhao, Zizheng Chen et al. (+4 more)
Aortic valve disease (AVD) represents a major public health burden, while its diagnosis relies on echocardiography, which is limited by cost and specialist expertise, restricting scalable screening and risk stratification. Existing portable sensing modalities are constrained by indirect representations or acquisition dependencies. In this context, photoplethysmography (PPG), a widely available optical signal capturing peripheral hemodynamic dynamics, provides a scalable physiological measurement. However, the scarcity of clinically labeled PPG data severely constrains the development of effective data-driven models. To address this limitation, we propose Physiology-Guided Self-Supervised Learning (PG-SSL), leveraging approximately 170,000 unlabeled UK Biobank PPG recordings. PG-SSL constructs physiologically derived pseudo-labels based on clinically motivated waveform phenotypes associated with aortic stenosis (AS) and aortic regurgitation (AR), enabling large-scale pretraining without AVD-specific labels. Following fine-tuning on a small labeled cohort, the model achieved AUROCs of 0.8025 for AS and 0.7669 for AR. Further analyses demonstrated robustness under clinical confounding and
covariate-balanced evaluation, as well as significant longitudinal associations with incident AVD events. This study demonstrates the feasibility of PG-SSL for leveraging large-scale unlabeled physiological signals under clinically labeled data-scarce conditions. The proposed approach provides a useful strategy for improving low-cost PPG-based screening and risk enrichment for clinically recognized AVD.
Read the full abstract ▾
All authors Jiaze Wang, Qinghao Zhao, Zizheng Chen, Zhejun Sun, Deyun Zhang, Yuxi Zhou, Shenda Hong
Machine Learning
Boosting LLM Reasoning via Human-Inspired Reward Shaping
Wenze Lin, Zhen Yang, Xitai Jiang et al. (+2 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for enhancing
reasoning in Large
Language Models (LLMs). However, existing reward formulations typically treat exploration and consolidation as a monolithic process, resulting in entangled stage-wise learning dynamics. This contradicts the natural learning behavior of human learners. In human learning, individuals adopt distinct behavioral patterns toward mastered versus unfamiliar problems. When confronting unmastered challenges, humans prioritize broad exploration to seek viable solutions. By contrast, for well-mastered problems, they focus instead on
reasoning condensation and knowledge abstraction to distill concise underlying principles. Motivated by this gap, we introduce T2T(Thickening-to-Thinning), a dynamic reward framework inspired by human learning processes. Specifically, it implements a dual-phase mechanism: (1) On incorrect attempts, T2T incentivizes "thickening" to broaden the search space and explore novel solution paths; (2) Upon achieving correctness, it shifts to "thinning", imposing length penalties to discourage redundancy, thereby fostering model confidence and crystallizing
reasoning capabilities. Extensive experiments on mathematical
benchmarks (MATH-500, AIME, AMC) across 5 mainstream LLMs demonstrate that T2T significantly outperforms standard GRPO and recent baselines, achieving superior performance.
Read the full abstract ▾
All authors Wenze Lin, Zhen Yang, Xitai Jiang, Xiaoteng Ma, Gao Huang
Machine Learning
Exponential Approximation Rates and Parameter Efficiency of Learnable Bernstein Activations
Ibrahim Albool, Malak Gamal El-Din, Salma Elmalaki et al. (+1 more)
The choice of activation function fundamentally shapes the representational capacity and parameter efficiency of deep neural networks, yet most widely used activations lack rigorous theoretical guarantees on these properties. We provide a theoretical analysis of DeepBern-Nets (DBNs) – networks employing learnable Bernstein
polynomial activations – showing that their approximation error decays with the network depth $L$ and the
polynomial order $n$ with a rate of $\mathcal{O}(n^{-L})$, exponentially faster than the
polynomial rate of ReLU architectures while remaining fully differentiable. We validate these predictions through $1{,}344$ experiments on large scientific
datasets (
HIGGS and SUSY), comparing DBNs against ReLU, Leaky ReLU, SELU, and GeLU. DBNs achieve over $70\%$ parameter reduction across the majority of architectures – reaching $99.9\%$ at scale – converge to ReLU's final loss in as few as $26\%$ of the training epochs, and attain up to $45\%$ lower final loss. These advantages hold over all tested activations, confirming that DBN's gains stem from the learnable
polynomial structure rather than mere smoothness.
Read the full abstract ▾
All authors Ibrahim Albool, Malak Gamal El-Din, Salma Elmalaki, Yasser Shoukry
Machine Learning
From Ambiguity to Action: A POMDP Perspective on Partial Multi-Label Ambiguity and Its Horizon-One Resolution
Hanlin Pan, Yuhao Tang and Wanfu Gao
In partial multi-label learning (PML), the true labels are unobserved, which makes label disambiguation important but difficult. A key challenge is that ambiguous candidate labels can propagate errors into downstream tasks such as feature engineering. To solve this issue, we jointly model the disambiguation and feature selection tasks as Partially Observable Markov Decision Processes (POMDP) to turn PML risk minimization into expected-return maximization. Stage 1 trains a
transformer policy via reinforcement learning to produce high-quality hard pseudo-labels; Stage 2 describes feature selection as a sequential reinforcement learning problem, selecting features step by step and outputting an interpretable global ranking. We further provide the theoretical analysis of PML-POMDP correspondence and the excess-risk bound that decompose the error into pseudo label quality term and sample size. Experiments in multiple metrics and data sets verify the advantages of the framework.
Read the full abstract ▾
Machine Learning
GraphVec: Cross-Domain Graph Vectorization for Graph-Level Representation Learning
Qi Feng and Jicong Fan
Learning universal graph representations across heterogeneous domains is difficult because graph
datasets differ in
topology, node-attribute semantics, feature dimensions, and even attribute availability. We propose GraphVec, a
language-model-free graph vectorization model that maps diverse graphs into transferable fixed-dimensional
embeddings for graph-level tasks. Instead of directly using incomparable raw node attributes, GraphVec constructs multi-scale global graphs over all nodes in each
dataset and extracts spectral
embeddings to obtain domain-agnostic relational features. To make these spectral features comparable across
datasets, we introduce a density-maximization mean
alignment algorithm over orthogonal transformations and prove its monotonic convergence. GraphVec further combines a GIN–Graph
Transformer backbone with a multi-layer reference distribution module, which preserves node-level distributional information beyond standard pooling. We also provide a generalization error bound for the proposed model. Experiments on 13
datasets with more than 15 comparison methods demonstrate that GraphVec consistently outperforms strong graph pretraining baselines in cross-domain few-shot graph classification and graph clustering. Beyond graph-level tasks, GraphVec also yields strong node-level representations, achieving competitive performance on few-shot node classification against representative graph
prompt learning methods.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
SPOT-Occ: Sparse Prototype-guided Transformer for Camera-based 3D Occupancy Prediction
Suzeyu Chen, Leheng Li and Ying-Cong Chen
Achieving highly accurate and real-time 3D occupancy prediction from cameras is a critical requirement for the safe and practical deployment of autonomous vehicles. While this shift to sparse 3D representations solves the encoding bottleneck, it creates a new challenge for the decoder: how to efficiently aggregate information from a sparse, non-uniformly distributed set of voxel features without resorting to computationally prohibitive dense
attention. In this paper, we propose a novel Prototype-based Sparse
Transformer Decoder that replaces this costly interaction with an efficient, two-stage process of guided feature selection and focused aggregation. Our core idea is to make the decoder's
attention prototype-guided. We achieve this through a sparse prototype selection mechanism, where each query adaptively identifies a compact set of the most salient voxel features, termed prototypes, for focused feature aggregation. To ensure this dynamic selection is stable and effective, we introduce a complementary denoising paradigm. This approach leverages ground-truth masks to provide explicit guidance, guaranteeing a consistent query-prototype association across decoder layers. Our model, dubbed SPOT-Occ, outperforms previous methods with a significant margin in speed while also improving accuracy. Source code is released at https://github.com/chensuzeyu/SpotOcc.
Read the full abstract ▾
Machine Learning
Cascading Robustness Verification: Toward Efficient Model-Agnostic Certification
Mohammadreza Maleki, Rushendra Sidibomma, Arman Adibi et al. (+1 more)
Certifying neural network robustness against
adversarial examples is challenging, as formal guarantees often require solving non-convex problems. Hence, incomplete verifiers are widely used because they scale efficiently and substantially reduce the cost of robustness verification compared to complete methods. However, relying on a single verifier can underestimate robustness because of loose approximations or misalignment with training methods. In this work, we propose Cascading Robustness Verification (CRV), which goes beyond an engineering improvement by exposing fundamental limitations of existing robustness metric and introducing a framework that enhances both reliability and efficiency. CRV is a model-agnostic verifier, meaning that its robustness guarantees are independent of the model's training process. The key insight behind the CRV framework is that, when using multiple verification methods, an input is certifiably robust if at least one method certifies it as robust. Rather than relying solely on a single verifier with a fixed constraint set, CRV progressively applies multiple verifiers to balance the tightness of the bound and computational cost. Starting with the least expensive method, CRV halts as soon as an input is certified as robust; otherwise, it proceeds to more expensive methods. For computationally expensive methods, we introduce a Stepwise Relaxation
Algorithm (SR) that incrementally adds constraints and checks for certification at each step, thereby avoiding unnecessary computation. Our theoretical analysis demonstrates that CRV achieves equal or higher verified accuracy compared to powerful but computationally expensive incomplete verifiers in the cascade, while significantly reducing verification overhead. Empirical results confirm that CRV certifies at least as many inputs as
benchmark approaches, while improving runtime efficiency by up to ~90%.
Read the full abstract ▾
All authors Mohammadreza Maleki, Rushendra Sidibomma, Arman Adibi, Reza Samavi
Machine Learning · Cross-listed from stat.ML
Provable Target Sample Complexity Improvements as Pre-Trained Models Scale
Kazuto Fukuchi, Ryuichiro Hataya and Kota Matsui
Pre-trained models have become indispensable for efficiently building models across a broad spectrum of downstream tasks. The advantages of pre-trained models have been highlighted by empirical studies on scaling laws, which demonstrate that larger pre-trained models can significantly reduce the sample complexity of downstream learning. However, existing theoretical investigations of pre-trained models lack the capability to explain this phenomenon. In this paper, we provide a theoretical investigation by introducing a novel framework, caulking, inspired by parameter-efficient fine-tuning (PEFT) methods such as adapter-based fine-tuning, low-rank adaptation, and partial fine-tuning. Our analysis establishes that improved pre-trained models provably decrease the sample complexity of downstream tasks, thereby offering theoretical justification for the empirically observed scaling laws relating pre-trained model size to downstream performance, a relationship not covered by existing results.
Read the full abstract ▾
Machine Learning
RAPO: Risk-Aware Preference Optimization for Generalizable Safe Reasoning
Zeming Wei, Qiaosheng Zhang, Xia Hu et al. (+1 more)
Large
Reasoning Models (LRMs) have achieved tremendous success with their chain-of-thought (CoT)
reasoning, yet also face safety issues similar to those of basic
language models. In particular, while
algorithms are designed to guide them to deliberately refuse harmful
prompts with safe
reasoning, this process often fails to generalize against diverse and complex jailbreak attacks. In this work, we attribute these failures to the generalization of the safe
reasoning process, particularly their insufficiency against complex attack
prompts. We provide both theoretical and empirical evidence to show the necessity of a more sufficient safe
reasoning process to defend against advanced attack
prompts. Building on this insight, we propose a Risk-Aware Preference Optimization (RAPO) framework that enables LRM to adaptively identify and address the safety risks with appropriate granularity in its thinking content. Extensive experiments demonstrate that RAPO successfully generalizes multiple LRMs' safe
reasoning adaptively across diverse attack
prompts whilst preserving general utility, contributing a robust
alignment technique for LRM safety. Our code is available at https://github.com/weizeming/RAPO.
Read the full abstract ▾
All authors Zeming Wei, Qiaosheng Zhang, Xia Hu, Xingcheng Xu
Machine Learning · Cross-listed from cs.RO
OAT: Ordered Action Tokenization
Chaoqi Liu, Xiaoshen Han, Jiawei Gao et al. (+3 more)
Autoregressive policies offer a compelling foundation for scalable
robot learning by enabling discrete abstraction,
token-level
reasoning, and flexible inference. However, applying autoregressive modeling to continuous
robot actions requires an effective action tokenization scheme. Existing approaches either rely on analytical discretization methods that produce prohibitively long
token sequences, or learned latent tokenizers that lack structure, limiting their compatibility with next-
token prediction. In this work, we identify three desiderata for action tokenization - high compression, total decodability, and a left-to-right causally ordered
token space - and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of
tokens using
transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms. The resulting
token space aligns naturally with autoregressive generation and enables prefix-based detokenization, yielding an anytime trade-off between inference cost and action fidelity. Across more than 20 tasks spanning four simulation
benchmarks and real-world settings, autoregressive policies equipped with OAT consistently outperform prior tokenization schemes and
diffusion-based baselines, while offering significantly greater flexibility at inference time.
Read the full abstract ▾
All authors Chaoqi Liu, Xiaoshen Han, Jiawei Gao, Yue Zhao, Haonan Chen, Yilun Du
Machine Learning · Cross-listed from cs.AI
Steering LLMs via Scalable Interactive Oversight
Enyu Zhou, Zhiheng Xi, Long Ma et al. (+9 more)
As Large
Language Models increasingly automate complex, long-horizon tasks such as vibe coding, a supervision gap has emerged. While models excel at execution, users often struggle to guide them effectively due to insufficient domain expertise, the difficulty of articulating precise intent, and the inability to reliably validate complex outputs. It presents a critical challenge in scalable oversight: enabling humans to responsibly steer AI systems on tasks that surpass their own ability to specify or verify. To tackle this, we propose Scalable Interactive Oversight, a framework that decomposes complex intent into a recursive tree of manageable decisions to amplify human supervision. Rather than relying on open-ended prompting, our system elicits low-burden feedback at each node and recursively aggregates these signals into precise global guidance. Validated in web development task, our framework enables non-experts to produce expert-level Product Requirement Documents, achieving a 54% improvement in
alignment. Crucially, we demonstrate that this framework can be optimized via Reinforcement Learning using only online user feedback, offering a practical pathway for maintaining human control as AI scales.
Read the full abstract ▾
All authors Enyu Zhou, Zhiheng Xi, Long Ma, Zhihao Zhang, Shihan Dou, Zhikai Lei, Guoteng Wang, Rui Zheng, Hang Yan, Tao Gui, Qi Zhang, Xuanjing Huang
Machine Learning · Cross-listed from cs.RO
SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models
Hyeonbeom Choi, Daechul Ahn, Youhan Lee et al. (+3 more)
Vision-
Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic control, with test-time scaling (TTS) gaining
attention to enhance robustness beyond training. However, existing TTS methods for VLAs require additional training, verifiers, and multiple forward passes, making them impractical for deployment. Moreover, they intervene only at action decoding while keeping visual representations fixed-insufficient under perceptual ambiguity, where reconsidering how to perceive is as important as deciding what to do. To address these limitations, we propose SCALE, a simple inference strategy that jointly modulates visual perception and action based on 'self-uncertainty', inspired by uncertainty-driven exploration in Active Inference theory-requiring no additional training, no verifier, and only a single forward pass. SCALE broadens exploration in both perception and action under high uncertainty, while focusing on exploitation when confident-enabling adaptive execution across varying conditions. Experiments on simulated and real-world
benchmarks demonstrate that SCALE improves state-of-the-art VLAs and outperforms existing TTS methods while maintaining single-pass efficiency.
Read the full abstract ▾
All authors Hyeonbeom Choi, Daechul Ahn, Youhan Lee, Taewook Kang, Seongwon Cho, Jonghyun Choi