Machine Learning · Cross-listed from cs.RO
Learning Human-Like Badminton Skills for Humanoid Robots
Yeke Chen, Shihao Dong, Xiaoyu Ji et al. (+12 more)
Realizing versatile and human-like performance in high-demand sports like badminton remains a formidable challenge for
humanoid robotics. Unlike standard locomotion or static manipulation, this task demands a seamless integration of explosive whole-body coordination and precise, timing-critical interception. While recent advances have achieved lifelike motion mimicry, bridging the gap between kinematic imitation and functional, physics-aware striking without compromising stylistic naturalness is non-trivial. To address this, we propose Imitation-to-Interaction, a progressive reinforcement learning framework designed to evolve a
robot from a "mimic" to a capable "striker." Our approach establishes a robust motor prior from human data, distills it into a compact, model-based state representation, and stabilizes dynamics via
adversarial priors. Crucially, to overcome the sparsity of expert demonstrations, we introduce a
manifold expansion strategy that generalizes discrete strike points into a dense interaction volume. We validate our framework through the mastery of diverse skills, including lifts and drop shots, in simulation. Furthermore, we demonstrate the first zero-shot sim-to-real transfer of anthropomorphic badminton skills to a
humanoid robot, successfully replicating the kinetic elegance and functional precision of human athletes in the physical world.
Read the full abstract ▾
All authors Yeke Chen, Shihao Dong, Xiaoyu Ji, Jingkai Sun, Zeren Luo, Liu Zhao, Jiahui Zhang, Wanyue Li, Ji Ma, Bowen Xu, Yimin Han, Xuanyi Li, Yudong Zhao, Liyun Li, Peng Lu
Machine Learning · Cross-listed from cs.AI
MemAdapter: Fast Alignment across Agent Memory Paradigms via Generative Subgraph Retrieval
Xin Zhang, Kailai Yang, Chenyue Li et al. (+4 more)
Memory mechanism is a core component of LLM-based
agents, enabling
reasoning and knowledge discovery over long-horizon contexts. Existing
agent memory systems are typically designed within isolated paradigms (e.g., explicit, parametric, or latent
memory) with tightly coupled
retrieval methods that hinder cross-paradigm generalization and fusion. In this work, we take a first step toward unifying heterogeneous
memory paradigms within a single
memory system. We propose MemAdapter, a
memory retrieval framework that enables fast
alignment across
agent memory paradigms. MemAdapter adopts a two-stage training strategy: (1) training a generative subgraph retriever from the unified
memory space, and (2) adapting the retriever to unseen
memory paradigms by training a lightweight
alignment module through contrastive learning. This design improves the flexibility for
memory retrieval and substantially reduces
alignment cost across paradigms. Comprehensive experiments on three public evaluation
benchmarks demonstrate that the generative subgraph retriever consistently outperforms five strong
agent memory systems across three
memory paradigms and
agent model scales. Notably, MemAdapter completes cross-paradigm
alignment within 13 minutes on a single GPU, achieving superior performance over original
memory retrievers with less than 5% of training compute. Furthermore, MemAdapter enables effective zero-shot fusion across
memory paradigms, highlighting its potential as a plug-and-play solution for
agent memory systems.
Read the full abstract ▾
All authors Xin Zhang, Kailai Yang, Chenyue Li, Hao Li, Qiyu Wei, Jun'ichi Tsujii, Sophia Ananiadou
Machine Learning · Cross-listed from cs.AI
Circuit Representations of Random Forests with Applications to XAI
Chunxi Ji and Adnan Darwiche
We make three contributions in this paper. First, we present an approach for compiling a random forest classifier into a set of
circuits, where each
circuit directly encodes the instances in some class of the classifier. We show empirically that our proposed approach is significantly more efficient than existing similar approaches. Next, we utilize this approach to further obtain
circuits that are tractable for computing the complete and general reasons of a decision, which are instance abstractions that play a fundamental role in computing explanations. Finally, we propose
algorithms for computing the robustness of a decision and all shortest ways to flip it. We illustrate the utility of our contributions by using them to enumerate all sufficient reasons, necessary reasons and contrastive explanations of decisions; to compute the robustness of decisions; and to identify all shortest ways to flip the decisions made by random forest classifiers learned from a wide range of
datasets.
Read the full abstract ▾
Machine Learning
Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations
Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong et al. (+6 more)
Fine-tuning an LLM to maximize performance on a downstream task requires finding the optimal data and model configuration. This is a black-box optimization problem that
Bayesian optimization (BO) addresses by sequentially evaluating training configurations and using observed performance feedback to adaptively guide the search towards better configurations. However, directly applying BO to the joint data-model space suffers from two fundamental challenges: the prohibitive cost of full LLM training at each iteration, and the large number of BO iterations required for effective exploration due to the high problem dimensionality. This paper introduces JoBS, a BO-based approach that co-optimizes data and model configurations without requiring a full training run at every iteration. JoBS allocates an initial portion of the optimization budget to learn a scaling-law-inspired performance predictor that estimates fully trained LLM performance from only a small number of training steps. The remaining budget is then used to run BO with this predictor, eliminating the need for full training runs and enabling JoBS to explore significantly more data and model configurations within the same budget. We analyze JoBS's average regret and derive the optimal budget allocation between predictor learning and BO iterations. Empirically, JoBS outperforms independent data and model optimization methods and existing multi-fidelity BO baselines across a diverse set of downstream tasks. Our code is available at https://github.com/chenzhiliang94/BO-for-LLMs.
Read the full abstract ▾
All authors Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong, Apivich Hemachandra, Gregory Kang Ruey Lau, Chuan-Sheng Foo, Zhengyuan Liu, Nancy F. Chen, Bryan Kian Hsiang Low
Machine Learning
All ERMs Can Fail in Stochastic Convex Optimization Lower Bounds in Linear Dimension
Tal Burla and Roi Livni
We study the sample complexity of the best-case Empirical Risk Minimizer in the setting of
stochastic convex optimization. We show that there exists an instance in which the sample size is linear in the dimension, learning is possible, but the Empirical Risk Minimizer is likely to be unique and to overfit. This resolves an open question by Feldman. We also extend this to approximate ERMs. Building on our construction we also show that (constrained) Gradient Descent potentially overfits when horizon and learning rate grow w.r.t sample size. Specifically we provide a novel generalization lower bound of $\Omega\left(\sqrt{\eta T/m^{1.5}}\right)$ for Gradient Descent, where $\eta$ is the learning rate, $T$ is the horizon and $m$ is the sample size. This narrows down, exponentially, the gap between the best known upper bound of $O(\eta T/m)$ and existing lower bounds from previous constructions.
Read the full abstract ▾
Machine Learning
ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection
Debajyoti Datta, Trishala Neeraj, Bibek Paudel et al. (+2 more)
Long-context inference is constrained by KV-cache
memory, which grows linearly with sequence length; KV-cache compression therefore hinges on reliably selecting which past
tokens to retain. Most geometry-based eviction methods score keys by cosine similarity to a global centroid, but cosine is scale-invariant and can discard magnitude cues that distinguish semantically salient
tokens. We propose ManifoldKV, a training-free scorer that ranks
tokens by Euclidean distance to the key centroid, capturing both angular and radial deviations. On the RULER
benchmark, ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression; matching the best
geometric baseline while improving robustness in two regimes where cosine scoring fails. First, on multi-key
retrieval, ManifoldKV reduces directional collisions, achieving 92.4% vs KeyDiff's 77.0% (+15.4 points) on 3-key NIAH at 50% compression. Second, to address dilution and performance collapse of global centroids at 64K context, we introduce WindowedManifoldKV, which restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff. The method requires only 3 lines of code and works across 4 architectures without tuning.
Read the full abstract ▾
All authors Debajyoti Datta, Trishala Neeraj, Bibek Paudel, Vyom Sharma, Subhabrata Mukherjee
Machine Learning
Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training
Cristian Pérez-Corral, Alberto Fernández-Hernández, Jose I. Mestre et al. (+3 more)
Despite the empirical success of DNN, their internal training dynamics remain difficult to characterize. In ReLU-based models, the activation pattern induced by a given input determines the piecewise-linear region in which the network behaves affinely. Motivated by this geometry, we investigate whether training exhibits a two-timescale behavior: an early stage with substantial changes in activation patterns and a later stage where weight updates predominantly refine the model within largely stable activation regimes. We first prove a local stability property: outside measure-zero sets of parameters and inputs, sufficiently small parameter perturbations preserve the activation pattern of a fixed input, implying locally affine behavior within activation regions. We then empirically track per-iteration changes in weights and activation patterns across fully-connected and convolutional architectures, as well as
Transformer-based models, where activation patterns are recorded in the ReLU feed-forward (MLP/FFN) submodules, using fixed validation subsets. Across the evaluated settings, activation-pattern changes decay 3 times earlier than weight-update magnitudes, showing that late-stage training often proceeds within relatively stable activation regimes. These findings provide a concrete, architecture-agnostic instrument for monitoring training dynamics and motivate further study of decoupled optimization strategies for piecewise-linear networks. For reproducibility, code and experiment configurations will be released upon acceptance.
Read the full abstract ▾
All authors Cristian Pérez-Corral, Alberto Fernández-Hernández, Jose I. Mestre, Manuel F. Dolz, Jose Duato, Enrique S. Quintana-Ortí
Machine Learning · Cross-listed from cs.NI
PACC: Protocol-Aware Cross-Layer Compression for Compact Network Traffic Representation
Zhaochen Guo, Honghao Wang, Tianyufei Zhou et al. (+2 more)
Network traffic classification underpins network security and management but faces growing challenges from pervasive encryption and evolving protocols. Existing representations involve clear trade-offs: hand-crafted flow statistics are efficient but lossy, raw-bit encodings can be accurate but costly, and pre-trained
embeddings support transfer but often flatten the protocol stack and entangle cross-layer signals. We observe substantial redundancy both across and within network layers, which existing paradigms do not explicitly address, wasting capacity and encouraging shortcut learning that degrades generalization. To address this, we propose PACC, a redundancy-aware cross-layer representation framework. PACC models protocol layers as multiview inputs and learns faithful, compact projections factorized into shared (cross-layer) and private (layer-specific) components. It incorporates an information-preserving sparse segment representation to reduce structural repetition and supervised intra-layer filtering to suppress task-irrelevant semantic redundancy. A behavior-guided shared/private decomposition aligns cross-layer consensus while retaining layer-specific information, supported by information-theoretic analysis. Across encrypted application classification, IoT device identification, benign/malicious traffic classification, and Tor traffic detection, PACC achieves the best overall performance. On encrypted traffic, it achieves an F1 score of 0.9685, substantially outperforming existing baselines while using approximately 15x fewer computes and less GPU
memory than the latest pre-trained traffic model.
Read the full abstract ▾
All authors Zhaochen Guo, Honghao Wang, Tianyufei Zhou, Ronghua Li, Shinan Liu
Machine Learning
Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference
Yifei Gao, Lei Wang, Rong-Cheng Tu et al. (+3 more)
A core bottleneck in large
language model (LLM) inference is the cost of attending over the ever-growing key-value (KV) cache. Although near-oracle top-k KV selection can preserve the quality of dense
attention while sharply reducing computation and bandwidth, existing sparse methods generally rely on posterior heuristics, i.e., selectors conditioned on observed
attention or proxy scores. Such conditioning introduces posterior bias: it tends to distort true
token importance and miss salient
tokens, thereby impairing long-range
reasoning. To tackle this problem, we propose Pre-hoc Sparsity (PrHS), which selects KV entries before
attention scoring and provides explicit accuracy control. Let the
attention mass of discarded entries be delta (the dropped mass). Through a marginal-to-mutual-information analysis, we derive an upper bound on the mutual-information loss that depends only on the dropped mass. This relation explains failure modes of posterior heuristics and enables verifiable guarantees by controlling the dropped mass in advance. Within PrHS, we instantiate three orthogonal pre-hoc selectors along the axes of time, depth, and layer. Extensive experiments on LLaMA and Mistral families validate PrHS. Across GSM8K and CoQA, PrHS reduces
retrieval overhead by over 90%, achieving 3x higher
retrieval sparsity than HShare at matched or better accuracy. It incurs under 1% average degradation on LongBench, lowers
attention FLOPs by about 15% versus prior sparse baselines, and yields a 9.9x speedup in
attention-operator latency and 2.8x higher throughput on NVIDIA A100-80GB GPUs than the dense baseline.
Read the full abstract ▾
All authors Yifei Gao, Lei Wang, Rong-Cheng Tu, Qixin Zhang, Jun Cheng, Dacheng Tao
Machine Learning
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
Yuntian Tang, Bohan Jia, Wenxuan Huang et al. (+7 more)
Chain-of-Thought (CoT)
reasoning successfully enhances the
reasoning capabilities of Large
Language Models (LLMs), yet it incurs substantial computational overhead for inference. Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios, resulting in significant performance degradation. To achieve high-fidelity, fast
reasoning, we propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the
token budget while preserving answer accuracy. To generate reliable, high-fidelity supervision, we first train a dedicated semantically-preserved compressor on mathematical CoT data with fine-grained annotations. An LLM is then fine-tuned on these compressed pairs via a mixed-ratio supervised fine-tuning (SFT), teaching it to follow a spectrum of compression budgets and providing a stable initialization for reinforcement learning (RL). We further propose Constrained and Hierarchical Ratio Policy Optimization (CHRPO) to explicitly incentivize question-solving ability under lower budgets by a hierarchical reward. Experiments on three mathematical
reasoning benchmarks show the superiority of Extra-CoT. For example, on MATH-500 using Qwen3-1.7B, Extra-CoT achieves over 73%
token reduction with an accuracy improvement of 0.6%, significantly outperforming state-of-the-art (SOTA) methods. Our source codes have been released at https://github.com/Mwie1024/Extra-CoT.
Read the full abstract ▾
All authors Yuntian Tang, Bohan Jia, Wenxuan Huang, Lianyue Zhang, Jiao Xie, Wenxi Li, Wei Li, Jie Hu, Xinghao Chen Rongrong Ji, Shaohui Lin
Machine Learning · Cross-listed from stat.ML
Is Flow Matching Just Trajectory Replay for Sequential Data?
Soon Hoe Lim, Shizheng Lin, Michael W. Mahoney et al. (+1 more)
Flow matching (FM) is increasingly used in scientific domains for time series generation and
forecasting, where data often arise from underlying dynamical systems. However, it is not well-understood whether it learns transferable dynamical structure or simply performs an effective ”trajectory replay”. We study this question by deriving the optimal velocity field targeted by the empirical FM objective on sequential data. For the Gaussian conditional paths commonly used in practice, we show that the resulting sampler is an ODE whose dynamics describe a nonparametric,
memory-augmented continuous-time dynamical system. The driving velocity field admits a closed-form expression as a weighted mixture of instantaneous velocities determined by observed transitions. This characterization provides an interpretable view of neural FM models as parametric estimation of an underlying nonparametric target and naturally leads to FreeFM: a closed-form, training-free realization of this target. FreeFM provides a transparent reference for studying sequential FM while yielding effective probabilistic forecasts directly from historical transitions.
Read the full abstract ▾
All authors Soon Hoe Lim, Shizheng Lin, Michael W. Mahoney, N. Benjamin Erichson
Machine Learning
Reusable Conditional Resampling via Flow Matching for Constraint-Based Causal Discovery
Shunyu Zhao, Yanfeng Yang, Shuai Li et al. (+1 more)
Constraint-based causal discovery requires many conditional independence (CI) tests. Generative CI tests are attractive in nonlinear and high-dimensional settings because they can model complex conditional distributions, but often suffer from high per-test costs and limited reusability across query-dependent conditional resampling tasks in graph search. We propose a generative nonlinear CI testing and graph search framework for PC-style causal discovery. We introduce Langevin-RePaint for conditional imputation. Building on Langevin-RePaint and joint flow matching modeling, we propose the Flow Matching based Conditional Independence Test (FMCIT), which reformulates generative CI testing from query-specific conditional modeling into a reusable conditional resampling framework. We establish theoretical guarantees for FMCIT's type-I error control. FMCIT reuses a single trained generative model to generate randomized copies throughout graph search. We then introduce GPC-FMCIT, which embeds FMCIT into a guided PC pipeline with screening, guidance, refinement, and orientation. By using low-cost screening to shrink the search space, constructing edge-specific candidate conditioning sets from local graph structure, and invoking FMCIT only in a budget-controlled refinement stage, GPC-FMCIT reduces both the deployment cost of individual nonlinear CI queries and the overall query complexity. On synthetic nonlinear structural equation models, GPC-FMCIT improves CPDAG recovery over PC-stable baselines and requires far fewer FMCIT queries than direct FMCIT-based PC search, and on hESC gene-expression data it yields smaller graphs than PC/FisherZ in a descriptive comparison. Overall, our results suggest that, through the joint design of reusable conditional resampling and controlled graph search, generative nonlinear CI testing can become a practical component of PC-style causal discovery.
Read the full abstract ▾
All authors Shunyu Zhao, Yanfeng Yang, Shuai Li, Kenji Fukumizu
Machine Learning
Interaction-Grounded Learning for Contextual Markov Decision Processes with Personalized Feedback
Mengxiao Zhang, Yuheng Zhang, Haipeng Luo et al. (+1 more)
In this paper, we study Interaction-Grounded Learning (IGL) [Xie et al., 2021], a paradigm designed for realistic scenarios where the learner receives indirect feedback generated by an unknown mechanism, rather than explicit numerical rewards. While prior work on IGL provides efficient
algorithms with provable guarantees, those results are confined to single-step settings, restricting their applicability to modern sequential decision-making systems such as multi-turn Large
Language Model (LLM) deployments. To bridge this gap, we propose a computationally efficient
algorithm that achieves a sublinear regret guarantee for contextual episodic Markov Decision Processes (MDPs) with personalized feedback. Technically, we extend the reward-
estimator construction of Zhang et al. [2024a] from the single-step to the multi-step setting, addressing the unique challenges of decoding latent rewards under MDPs. Building on this
estimator, we design an Inverse-Gap-Weighting (IGW)
algorithm for policy optimization. Finally, we demonstrate the effectiveness of our method in learning personalized objectives from multi-turn interactions through experiments on both a synthetic episodic MDP and a real-world user booking
dataset.
Read the full abstract ▾
All authors Mengxiao Zhang, Yuheng Zhang, Haipeng Luo, Paul Mineiro
Machine Learning
TextResNet: Decoupling and Routing Optimization Signals in Compound AI Systems via Deep Residual Tuning
Suizhi Huang, Mei Li, Han Yu et al. (+1 more)
Textual Gradient-style optimizers (TextGrad) enable gradient-like feedback propagation through compound AI systems. However, they do not work well for deep chains. The root cause of this limitation stems from the Semantic
Entanglement problem in these extended workflows. In standard textual backpropagation, feedback signals mix local critiques with upstream contexts, leading to Attribution Ambiguity. To address this challenge, we propose TextResNet, a framework that reformulates the optimization process to achieve precise signal routing via four key innovations. Firstly, in the forward pass, it enforces Additive Semantic Deltas to preserve an Identity Highway for gradient flow. Secondly, in the backward pass, it introduces Semantic Gradient Decomposition via a Semantic Projector to disentangle feedback into causally independent subspaces. Thirdly, it implements Causal Routing, which routes projected signals to their specific components. Finally, it performs Density-Aware Optimization Scheduling to leverage the disentangled signals to dynamically allocate resources to key system bottlenecks. Our results show that TextResNet not only achieves superior performance compared to TextGrad, but also exhibits remarkable stability for agentic tasks in compound AI systems where baselines collapse. Code is available at https://github.com/JeanDiable/TextResNet.
Read the full abstract ▾
All authors Suizhi Huang, Mei Li, Han Yu, Xiaoxiao Li
Machine Learning
On the Dynamics of Delayed Generalization in Linear Models
Atreya Vedantam, Nataraj Das and Chandrashekar Lakshminarayanan
Delayed generalization, where training accuracy becomes nearly perfect substantially before test accuracy, has often been studied in the context of the depth and compositional structure of deep networks. In this paper, we ask whether delayed generalization can occur in simpler settings. We show that it can arise in a linear model trained with gradient descent and an exponential-tail loss on linearly separable data. We characterize the finite-time hyperplane dynamics into three phases: a population phase, a support vector dominated unlearning phase and a generalization phase. The convergence of the residual beyond the maximum-margin direction is sufficient for the unlearning phase to begin. In the support vector dominated phase, we characterize the hyperplane dynamics. The time scale in these dynamics causes delayed generalization. We validate our theory through controlled experiments varying population and support-vector distributions, and further verify the predicted bias behavior in a deep model with a single output
neuron on MNIST. Our results provide a unified characterization of the asymptotic and finite-time dynamics underlying delayed generalization in linear models.
Read the full abstract ▾
Machine Learning
Trust-Based Incentive Mechanisms in Semi-Decentralized Federated Learning Systems
Ajay Kumar Shrestha
In
federated learning (FL), decentralized model training allows multi-ple participants to collaboratively improve a shared machine learning model without exchanging raw data. However, ensuring the integrity and reliability of the system is challenging due to the presence of potentially malicious or faulty nodes that can degrade the model's performance. This paper proposes a novel trust-based incentive mechanism designed to evaluate and reward the quality of contributions in FL systems. By dynamically assessing trust scores based on fac-tors such as data quality, model accuracy, consistency, and contribution fre-quency, the system encourages honest participation and penalizes unreliable or malicious behavior. These trust scores form the basis of an incentive mechanism that rewards high-trust nodes with greater participation opportunities and penal-ties for low-trust participants. We further explore the integration of blockchain technology and smart contracts to automate the trust evaluation and incentive distribution processes, ensuring transparency and decentralization. Our proposed theoretical framework aims to create a more robust, fair, and transparent FL eco-system, reducing the risks posed by untrustworthy participants.
Read the full abstract ▾
Machine Learning
Noise Stability of Transformer Models
Themistoklis Haris, Zihan Zhang and Yuichi Yoshida
Understanding simplicity biases in deep learning offers a promising path toward developing reliable AI. A common metric for this, inspired by Boolean function analysis, is average sensitivity, which captures a model's robustness to single-
token perturbations. We argue that average sensitivity has two key limitations: it lacks a natural generalization to real-valued domains and fails to explain the "junta-like" input dependence we empirically observe in modern LLMs. To address these limitations, we propose noise stability as a more comprehensive simplicity metric. Noise stability expresses a model's robustness to correlated noise applied to all input coordinates simultaneously. We provide a theoretical analysis of noise stability for single-layer
attention and ReLU MLP layers and tackle the multi-layer propagation problem with a covariance interval propagation approach. Building on this theory, we develop a practical noise stability regularization method. Experiments on algorithmic and next-
token-prediction tasks show that our regularizer consistently catalyzes grokking and accelerates training by approximately $35\%$ and $75\%$ respectively. Our results sculpt a new connection between signal propagation in neural networks and interpretability, with noise stability emerging as a powerful tool for understanding and improving modern
Transformers.
Read the full abstract ▾
Machine Learning
When Do Multi-Agent Systems Outperform? Analysing the Learning Efficiency of Agentic Systems
Junwei Su and Chuan Wu
Reinforcement Learning (RL) has emerged as a crucial method for training or fine-tuning large
language models (LLMs), enabling adaptive, task-specific optimizations through interactive feedback. Multi-
Agent Reinforcement Learning (MARL), in particular, offers a promising avenue by decomposing complex tasks into specialized subtasks learned by distinct interacting
agents, potentially enhancing the ability and efficiency of LLM systems. However, theoretical insights regarding when and why MARL outperforms Single-
Agent RL (SARL) remain limited, creating uncertainty in selecting the appropriate RL framework. In this paper, we address this critical gap by rigorously analyzing the comparative sample efficiency of MARL and SARL within the context of LLM. Leveraging the Probably Approximately Correct (PAC) framework, we formally define SARL and MARL setups for LLMs, derive explicit sample complexity bounds, and systematically characterize how task decomposition and
alignment influence learning efficiency. Our results demonstrate that MARL improves sample complexity when tasks naturally decompose into independent subtasks, whereas dependent subtasks diminish MARL's comparative advantage. Additionally, we introduce and analyze the concept of task
alignment, quantifying the trade-offs when enforcing independent task decomposition despite potential misalignments. These theoretical insights clarify empirical inconsistencies and provide practical criteria for deploying MARL strategies effectively in complex LLM scenarios.
Read the full abstract ▾
Machine Learning
Inverting Data Transformations via Diffusion Sampling
Jinwoo Kim, Sékou-Oumar Kaba, Jiyun Park et al. (+2 more)
We study the problem of transformation inversion on general Lie groups: a datum is transformed by an unknown group element, and the goal is to recover an inverse transformation that maps it back to the original data distribution. Such unknown transformations arise widely in machine learning and scientific modeling, where they can significantly distort observations. We take a probabilistic view and model the posterior over transformations as a Boltzmann distribution defined by an energy function on the data space. To sample from this posterior, we introduce a
diffusion process on Lie groups that keeps all updates on-
manifold and only requires computations in the associated Lie
algebra. Our method, Transformation-Inverting Energy
Diffusion (TIED), relies on a new trivialized target-score identity that enables efficient score-based sampling of the transformation posterior. As a key application, we focus on test-time equivariance, where the objective is to improve the robustness of pretrained neural networks to input transformations. Experiments on image homographies and PDE
symmetries demonstrate that TIED can restore transformed inputs to the training distribution at test time, showing improved performance over strong canonicalization and sampling baselines. Code is available at https://github.com/jw9730/tied.
Read the full abstract ▾
All authors Jinwoo Kim, Sékou-Oumar Kaba, Jiyun Park, Seunghoon Hong, Siamak Ravanbakhsh
Machine Learning
PRO-Bid: Pareto-Prioritized Regret Optimization for Constraint-Aware Generative Auto-Bidding
Binglin Wu, Yingyi Zhang, Xianneng Li et al. (+4 more)
Auto-bidding systems strive to maximize marketing value while maintaining high compliance with efficiency constraints, such as Target Cost-Per-Action (CPA). While Decision
Transformers offer powerful sequence modeling capabilities, their application to this setting faces two challenges: 1) standard Return-to-Go conditioning causes state aliasing by ignoring the cost dimension, preventing precise resource pacing; and 2) standard regression constrains the policy to mimic historical averages, limiting its capacity to optimize performance near the high-efficiency boundary. To tackle these challenges, we propose PRO-Bid, a constraint-aware generative auto-bidding framework featuring systematic redesigns across data, architecture, and training via two synergistic mechanisms: 1) Constraint-Decoupled Pareto Representation (CDPR) separates global constraints into recursive cost and value contexts to restore resource perception, while reweighting trajectories based on the empirical Pareto frontier to prioritize high-efficiency data; and 2) Counterfactual Regret Optimization (CRO) employs a global predictor to evaluate alternative actions and identify promising local adjustments. By utilizing these high-utility outcomes as weighted regression targets, the model overcomes mean regression and approaches the empirical high-efficiency boundary. Extensive experiments on two public
benchmarks and online A/B tests show that PRO-Bid achieves better constraint satisfaction and value acquisition than state-of-the-art baselines.
Read the full abstract ▾
All authors Binglin Wu, Yingyi Zhang, Xianneng Li, Ruyue Deng, Chuan Yue, Weiru Zhang, Xiaoyi Zeng
Machine Learning · Cross-listed from stat.ML
A Statistical Framework for Alignment with Biased AI Feedback
Xintao Xia, Zhiqiu Xia, Linjun Zhang et al. (+1 more)
Modern
alignment pipelines are increasingly replacing expensive human preference labels with evaluations from large
language models (LLM-as-Judge). However, AI labels can be systematically biased compared to high-quality human feedback
datasets. In this paper, we develop two debiased
alignment methods within a general framework that accommodates heterogeneous
prompt-response distributions and external human feedback sources. Debiased Direct Preference Optimization (DDPO) augments standard DPO with a residual-based correction and density-ratio reweighting to mitigate systematic bias, while retaining DPO's computational efficiency. Debiased Identity Preference Optimization (DIPO) directly estimates human preference probabilities without imposing a parametric reward model. We provide theoretical guarantees for both methods: DDPO offers a practical and computationally efficient solution for large-scale
alignment, whereas DIPO serves as a robust, statistically optimal alternative that attains the semiparametric efficiency bound. Empirical studies on sentiment generation, summarization, and single-turn dialogue demonstrate that the proposed methods substantially improve
alignment efficiency and recover performance close to that of an oracle trained on fully human-labeled data.
Read the full abstract ▾
All authors Xintao Xia, Zhiqiu Xia, Linjun Zhang, Zhanrui Cai
Machine Learning
Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers
Juncheng Dong, Bowen He, Moyang Guo et al. (+3 more)
In-context reinforcement learning (ICRL) leverages the in-context learning capabilities of
transformer models (TMs) to efficiently generalize to unseen sequential decision-making tasks without parameter updates. However, existing ICRL methods rely on explicit reward signals during pretraining, which limits their applicability when rewards are ambiguous, hard to specify, or costly to obtain. To overcome this limitation, we propose a new learning paradigm, In-Context Preference-based Reinforcement Learning (ICPRL), in which both pretraining and deployment rely solely on preference feedback, eliminating the need for reward supervision. We study two variants that differ in the granularity of feedback: Immediate Preference-based RL (I-PRL) with per-step preferences, and Trajectory Preference-based RL (T-PRL) with trajectory-level comparisons. We first show that supervised pretraining, a standard approach in ICRL, remains effective under preference-only context
datasets, demonstrating the feasibility of in-context reinforcement learning using only preference signals. To further improve data efficiency, we introduce alternative preference-native frameworks for I-PRL and T-PRL that directly optimize TM policies from preference data without requiring reward signals nor optimal action labels.Experiments on dueling bandits,
navigation, and continuous control tasks demonstrate that ICPRL enables strong in-context generalization to unseen tasks, achieving performance comparable to ICRL methods trained with full reward supervision.
Read the full abstract ▾
All authors Juncheng Dong, Bowen He, Moyang Guo, Ethan X. Fang, Zhuoran Yang, Vahid Tarokh
Machine Learning · Cross-listed from stat.ML
Discrete Adjoint Schrödinger Bridge Sampler
Wei Guo, Yuchen Zhu, Xiaochen Du et al. (+6 more)
Learning discrete neural samplers is challenging due to the lack of gradients and combinatorial complexity. While
stochastic optimal control (SOC) and Schrödinger bridge (SB) provide principled solutions, efficient SOC solvers like adjoint matching (AM), which excel in continuous domains, remain unexplored for discrete spaces. We bridge this gap by revealing that the core mechanism of AM is $\mathit{state}\text{-}\mathit{space~agnostic}$, and introduce $\mathbf{discrete~ASBS}$, a unified framework that extends AM and adjoint Schrödinger bridge sampler (ASBS) to discrete spaces. Theoretically, we analyze the optimality conditions of the discrete SB problem and its connection to SOC, identifying a necessary cyclic group structure on the state space to enable this extension. Empirically, discrete ASBS achieves competitive sample quality with significant advantages in training efficiency and scalability.
Read the full abstract ▾
All authors Wei Guo, Yuchen Zhu, Xiaochen Du, Juno Nam, Yongxin Chen, Rafael Gómez-Bombarelli, Guan-Horng Liu, Molei Tao, Jaemoo Choi
Machine Learning
Linearization Explains Fine-Tuning in Large Language Models
Zahra Rahimi Afzal, Tara Esmaeilbeig, Mojtaba Soltanalian et al. (+1 more)
Parameter-Efficient Fine-Tuning (PEFT) is a popular class of techniques that strive to adapt large models in a scalable and resource-efficient manner. Yet, the mechanisms underlying their training performance and generalization remain underexplored. In this paper, we provide several insights into such fine-tuning through the lens of linearization. Fine-tuned models are often implicitly encouraged to remain close to the pretrained model. By making this explicit, using an Euclidean distance inductive bias in parameter space, we show that fine-tuning dynamics become equivalent to learning with the positive-definite neural tangent kernel (NTK). We specifically analyze how close the fully linear and the linearized fine-tuning optimizations are, based on the strength of the regularization. This allows us to be pragmatic about how good a model linearization is when fine-tuning large
language models (LLMs). When linearization is a good model, our findings reveal a strong correlation between the
eigenvalue spectrum of the NTK and the performance of model adaptation. Motivated by this, we give spectral perturbation bounds on the NTK induced by the choice of layers selected for fine-tuning. We empirically validate our theory on Low Rank Adaptation (LoRA) on LLMs. These insights not only characterize fine-tuning but also have the potential to enhance PEFT techniques, paving the way to better informed and more nimble adaptation in LLMs.
Read the full abstract ▾
All authors Zahra Rahimi Afzal, Tara Esmaeilbeig, Mojtaba Soltanalian, Mesrob I. Ohannessian
Machine Learning
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Peng Xia, Jianwen Chen, Hanyang Wang et al. (+10 more)
Large
Language Model (LLM)
agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing
memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents
agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive
evolution. Our approach introduces an experience-based
distillation mechanism to build a hierarchical skill library SkillBank, an adaptive
retrieval strategy for general and task-specific heuristics, and a recursive
evolution mechanism that allows the skill library to co-evolve with the
agent's policy during reinforcement learning. These innovations significantly reduce the
token footprint while enhancing
reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL.
Read the full abstract ▾
All authors Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, Huaxiu Yao
Machine Learning · Cross-listed from math.OC
Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization
Ruichen Jiang, Zakaria Mhammedi, Mehryar Mohri et al. (+1 more)
We study online linear optimization with
matrix variables constrained by the operator norm, a setting where the geometry renders designing data-dependent and efficient adaptive
algorithms challenging. The best-known adaptive regret bounds are achieved by Shampoo-like methods, but they require solving a costly quadratic projection subproblem. To address this, we extend the gradient-based prediction scheme to adaptive
matrix online learning and cast
algorithm design as constructing a family of smoothed potentials for the
nuclear norm. We define a notion of admissibility for such smoothings and prove any admissible smoothing yields a regret bound matching the best-known guarantees of one-sided Shampoo. We instantiate this framework with two efficient methods that avoid quadratic projections. The first is an adaptive Follow-the-Perturbed-Leader (FTPL) method using Gaussian
stochastic smoothing. The second is Follow-the-Augmented-
Matrix-Leader (FAML), which uses a deterministic
hyperbolic smoothing in an augmented
matrix space. By analyzing the admissibility of these smoothings, we show both methods admit closed-form updates and match one-sided Shampoo's regret up to a constant factor, while significantly reducing computational cost. Lastly, using the online-to-nonconvex conversion, we derive two
matrix-based optimizers, Pion (from FTPL) and Leon (from FAML). We prove convergence guarantees for these methods in nonsmooth nonconvex settings, a guarantee that the popular
Muon optimizer lacks.
Read the full abstract ▾
All authors Ruichen Jiang, Zakaria Mhammedi, Mehryar Mohri, Aryan Mokhtari
Machine Learning · Cross-listed from cs.AI
InfiCoEvalChain: A Blockchain-Based Decentralized Framework for Collaborative LLM Evaluation
Yifan Yang, Jinjia Li, Kunxi Li et al. (+7 more)
The rapid advancement of large
language models (LLMs) demands increasingly reliable evaluation, yet current centralized evaluation suffers from opacity, overfitting, and hardware-induced variance. Our empirical analysis reveals an alarming inconsistency in existing evaluations: the standard deviation across ten repeated runs of a single model on HumanEval (1.67) actually exceeds the performance gap among the top-10 models on the official leaderboard (0.91), rendering current rankings statistically precarious. To mitigate these instabilities, we propose a decentralized evaluation framework that enables hardware and parameter diversity through large-scale benchmarking across heterogeneous compute nodes. By leveraging the blockchain-based protocol, the framework incentivizes global contributors to act as independent validators, using a robust reward system to ensure evaluation integrity and discourage dishonest participation. This collective verification transforms evaluation from a "centralized black box" into a "decentralized endorsement" where multi-party consensus and diverse inference environments yield a more stable, representative metric. Experimental results demonstrate that the decentralized evaluation framework reduces the standard deviation across ten runs on the same model to 0.28. This significant improvement over conventional frameworks ensures higher statistical confidence in model rankings. We have completely implemented this platform and will soon release it to the community.
Read the full abstract ▾
All authors Yifan Yang, Jinjia Li, Kunxi Li, Puhao Zheng, Yuanyi Wang, Zheyan Qu, Yang Yu, Jianmin Wu, Ming Li, Hongxia Yang
Machine Learning
Sparsity-Aware Evolution for Model Merging
Huan Zhang, Yanjian Zhang, Guillaume Wisniewski et al. (+2 more)
We propose a sparsity-aware evolutionary (SAE) framework for model merging that involves iterative
pruning-merging cycles to act as a novel mutation operator. We incorporate the sparsity constraints into the score function, which steers the evolutionary process to favor more sparse models, in addition to other conventional performance scores. Interestingly, the by-product of competition for sparsity introduces an extra local attraction and interplay into the evolutionary process: if one competitor has more zero elements, the other competitor's non-zero elements will occupy those positions, even though the less sparse competitor loses to the more sparse competitor in other positions. The proposed pipeline is evaluated on a variety of large-scale LLM
benchmarks. Experiments demonstrate that our approach can improve model merging reliability across multiple
benchmarks, and is easy to incorporate due to its simplicity and being orthogonal to most existing approaches.
Read the full abstract ▾
All authors Huan Zhang, Yanjian Zhang, Guillaume Wisniewski, Nadi Tomeh, Bang Liu
Machine Learning
Entropy-Generated Attention Beyond Softmax and Entmax: Kaniadakis and Reciprocal-Symmetric Abe Operators
Gunn Kim
We derive two
attention operators from generalized statistical
entropies. Kaniadakis
entropy yields an exact full-support normalization whose weights and low-score sensitivities decay algebraically, rather than exponentially as in Softmax or by exact truncation as in entmax. Classical Abe
entropy yields an implicit reciprocal-symmetric operator. With $q=e^\epsilon$, the involution $q\leftrightarrow q^{-1}$ removes every odd correction about Softmax; we obtain the normalized second- and fourth-order terms, including the deformation of the normalization multiplier. These stationary laws follow from a Fisher-metric Lagrangian on the probability simplex, whose Shannon sector recovers scaled dot-product Softmax. We also give a tangent-gradient test for deciding whether changing the
entropy changes the
attention profile or only its scale. Rényi and two-parameter Sharma–Mittal
entropies retain the Tsallis–entmax inverse-gradient shape, but their global moments make the effective temperature input dependent when the external temperature is fixed. Distinguishing profile-shape equivalence from fixed-parameter operator equivalence separates new normalization shapes from adaptive rescalings and organizes the operators by support, tail behavior, and realization complexity.
Read the full abstract ▾
Machine Learning
Distribution-Free Robust Predict-Then-Optimize in Function Spaces
Yash Patel and Ambuj Tewari
The need to rapidly solve PDEs in engineering design workflows has spurred the rise of neural surrogate models. In particular, neural operator models provide a discretization-invariant surrogate by retaining the infinite-dimensional, functional form of their arguments. Despite improved throughput, such methods lack guarantees on accuracy, unlike classical numerical PDE solvers. Optimizing engineering designs under these potentially miscalibrated surrogates thus runs the risk of producing designs that perform poorly upon deployment. In a similar vein, there is growing interest in automated decision-making under black-box predictors in the finite-dimensional setting, where a similar risk of suboptimality exists under poorly calibrated models. For this reason, methods have emerged that produce adversarially robust decisions under uncertainty estimates of the upstream model. One such framework leverages conformal prediction, a distribution-free post-hoc uncertainty quantification method, to provide these estimates due to its natural pairing with black-box predictors. We herein extend this line of conformally robust decision-making to infinite-dimensional function spaces. We first extend the typical conformal prediction guarantees over finite-dimensional spaces to infinite-dimensional Sobolev spaces. We then demonstrate how such uncertainty can be leveraged to robustly formulate engineering design tasks and characterize the suboptimality of the resulting robust optimal designs. We then empirically demonstrate the generality of our functional conformal coverage method across a diverse collection of PDEs, including the Poisson and heat equations, and showcase the significant improvement of such robust design in a
quantum state discrimination task.
Read the full abstract ▾