Artificial Intelligence · Cross-listed from cs.LG
Normalize-Then-Precondition: A Hierarchical Approach to Marginal Scale and Interaction Geometry for LLM Training
Zixuan Gong, Zeyu Gan, Jiaye Teng et al. (+1 more)
Matrix optimizers have emerged as a promising direction, with
Muon standing out as a prominent design. Revisiting
Muon through its full-Gram representation, we observe that it jointly processes marginal-scale and interaction information. This opens an alternative way to organize
geometric information hierarchically, motivating the Normalize-Then-Precondition framework. Specifically, it first uses diagonal-Gram information to construct a marginally normalized update, then applies spectral preconditioning to its directional interaction geometry. Building on this framework, we develop NormPre with NormPre-G and NormPre-L adopting global and localized spectral preconditioning, grounded in spectral-norm steepest descent and a regularized formulation followed by leading mode selection, respectively. To enable large-scale training, NormPre-G uses Newton-Schulz iterations and NormPre-L employs randomized sketching to approximate the leading interaction eigenspace. Theoretically, we establish $\mathcal{O}(T^{-1/2})$ convergence guarantees for simplified versions of NormPre. Across extensive pretraining experiments on GPT-2 Small, LLaMA and Qwen3, both variants consistently outperform AdamW,
Muon and MANO under matched training budgets. Further efficiency and spectral analyses reveal the complementary strengths of two variants and characterize their performance-efficiency trade-off. We open-source our code through a GitHub repository at https://github.com/zx-gong/NormPre.
Read the full abstract ▾
All authors Zixuan Gong, Zeyu Gan, Jiaye Teng, Yong Liu
Artificial Intelligence · Cross-listed from eess.SP
MyoCodec: A Streaming Neural Codec for Electromyography
Jihwan Lee, Kleanthis Avramidis, Junhyeok Lee et al. (+3 more)
Neural codecs encode continuous signals into compact sequences of discrete
tokens, providing an interface for efficient transmission, storage, and
token-based sequence modeling. This paradigm has been widely adopted in modern
speech and audio frameworks; however, the biosignal domain still lacks a neural codec designed specifically for low-bitrate streaming and generalization across diverse downstream tasks. We present MyoCodec, a streaming neural codec designed for electromyography (EMG). Inspired by recent neural audio codecs, MyoCodec combines causal
Transformers with residual vector quantization to encode continuous EMG signals into different levels of EMG representations spanning from continuous latent features to discrete
tokens operating at 50 Hz. Trained on twelve public EMG
datasets, MyoCodec achieves favorable performance in both intrinsic codec quality and representative downstream tasks, including typing (emg2qwerty), hand-pose (emg2pose),
speech decoding (emg2
speech), and
speech-to-EMG synthesis (
speech2emg). Across these tasks, MyoCodec exhibits strong performance against prior models while providing a compact and causal EMG representation. During streaming inference, it requires compute time of only 0.482 ms for each 20 ms frame, enabling real-time streaming. Also, the discrete
token representation provided by MyoCodec has the potential to support integration into
language-model based approaches, creating a path toward LLM-based interactive systems, where tokenized EMG representations are directly processed into such
language or
speech models. Code and model weights are released.
Read the full abstract ▾
All authors Jihwan Lee, Kleanthis Avramidis, Junhyeok Lee, Tiantian Feng, Najim Dehak, Shrikanth Narayanan
Artificial Intelligence
MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development
Xin Yu, Lizhu Zhang, Jiamu Bai et al. (+7 more)
Machine learning engineering (MLE)
agents have made substantial progress, but learning through ML experimentation remains costly in time and computation. Synthetic environments reduce these costs while introducing variations in data and experimental settings that require task-specific diagnosis. Access to diagnostic tools alone does not ensure that
agents learn when to use them or how to act on their findings. We introduce ToolMLBench, a suite of executable tools for data inspection, code verification, and experiment diagnosis, together with an SFT and RL pipeline for learning their use. Diagnostic calls acquire evidence whose value depends on subsequent decisions, so final outcomes provide limited guidance on which calls to reinforce. We address this challenge with SPICE, which measures how privileged context changes the likelihood of a sampled tool action and uses this difference as a turn-level reward alongside the final outcome. We train on 80 synthetic tasks and evaluate on 25 in-domain and 10 out-of-domain tasks. Providing tool interfaces and descriptions alone yields inconsistent gains across unadapted models. With the same diagnostic interface, our training pipeline raises in-domain success from 24.8% to 52.4% for Qwen3-8B and from 35.6% to 69.2% for Qwen3.5-35B-A3B. The latter also improves from 31% to 48% out-of-domain, supporting learned diagnostic tool use on held-out sources and targets.
Read the full abstract ▾
All authors Xin Yu, Lizhu Zhang, Jiamu Bai, Yanhong Wu, Zellux Wang, Serena Li, Weiwei Li, Lingzhou Xue, Xiangjun Fan, Bo Peng
Artificial Intelligence
FairDiff: Mitigating the Self-Reinforcing Matthew Effect in Diffusion Recommender Models
Song-Li Wu, Xianquan Wang, Zhaocheng Du et al. (+2 more)
While the "Matthew Effect" and filter bubbles are widely recognized outcome-level biases in recommender systems, we reveal that
Diffusion Recommender Models (DRMs) uniquely compound this issue through their generative dynamics. Rather than merely inheriting data imbalances, DRMs trigger a self-reinforcing amplification of popularity bias. We identify that this phenomenon is driven by two compounding mechanisms. First, while optimization loss is universally dominated by high-frequency items across recommenders, DRMs suffer from a unique structural prior mismatch during generation. Because the forward terminal distribution of long-tailed data deviates significantly from the standard Gaussian prior, reverse sampling trajectories inherently collapse toward high-density popular items, fundamentally suppressing niche item generation. To dismantle this self-reinforcing loop, we propose FairDiff, a plug-and-play
fairness-aware
diffusion framework. To overcome the popularity-dominated loss, we introduce Popularity Condition Guidance (PCG). Rather than altering the training objective, PCG acts as an inference-time distributional reweighting mechanism, mathematically reshaping the score-based gradient field to penalize high-popularity regions and guide trajectories toward niche semantics. Furthermore, we design a Semantic Calibration (SC) Module to bridge the prior mismatch, aligning the forward and reverse distributions via one-step optimal transport. Comprehensive evaluations demonstrate that FairDiff achieves state-of-the-art performance while effectively mitigating the self-reinforcing Matthew Effect, highlighting its value as a general framework for DRMs.
Read the full abstract ▾
All authors Song-Li Wu, Xianquan Wang, Zhaocheng Du, Weinan Gan, Jingyi Wang
Artificial Intelligence
FineSID: Scalable and Efficient Semantic Identifier Learning for Generative Recommendation
Song-Li Wu, Weinan Gan, Zhaocheng Du et al. (+2 more)
A critical prerequisite of generative recommendation is designing semantic identifiers (SIDs) that are both scalable to large item sets and efficiently learnable. Existing SID learning methods fundamentally rely on Top-1 hard assignment during vector quantization. While heuristic strategies – such as clustering-based initialization or forced post-hoc collision resolution – can artificially inflate codebook coverage, they often disrupt end-to-end semantic
alignment and fail to address the underlying optimization bottleneck: sparse gradient propagation. In standard Top-1 assignment, gradients concentrate on a narrow subset of frequently selected codewords, leaving the majority inherently under-trained and causing severe SID collisions. To overcome this limitation natively without relying on complex initialization priors, we propose FineSID, a unified quantization framework that moves beyond Top-1 assignment by enabling fine-grained gradient propagation across the entire codebook. Instead of updating only a single selected codeword, FineSID distributes learning signals to all codewords in a soft, differentiable manner. This design promotes globally balanced codebook optimization while strictly preserving semantic consistency, effectively alleviating SID collisions and stabilizing training in large, high-dimensional codebooks. Extensive experiments on multiple public
benchmarks demonstrate that FineSID is robust to initialization configurations and consistently improves both codebook utilization and recommendation accuracy. Our work provides a principled, initialization-agnostic solution for semantic identifier learning, advancing the practicality of generative recommendation.
Read the full abstract ▾
All authors Song-Li Wu, Weinan Gan, Zhaocheng Du, Xianquan Wang, Jingyi Wang
Artificial Intelligence · Cross-listed from cs.LG
AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization
Pengyu He, Yan Zhang, Ruien Li et al. (+1 more)
The pre-training of Large
Language Models (LLMs) is increasingly conducted across multiple data centers. As training scales to a larger number of accelerators, the fraction of time spent on computation decreases, while the fraction spent on communication increases. Therefore, frequent synchronization becomes a growing bottleneck. Local update methods reduce this cost by allowing workers to perform several optimizer steps between synchronizations. Most local update methods set the number of local optimizer steps between synchronizations before training and keep this interval fixed throughout the run. However, the best interval can change during the entire train process. If the interval and optimizer are adapted to the current training state, the communication frequency is reduced while maintaining the training performance. In this work, we introduce AutoLoCo, an adaptive training framework to reduce communication in LLM training. It adapts the local interval using scalar training statistics and corrects each outer update. Our method is motivated by two observations: 1) the appropriate local interval varies across training stages, and 2) changing the number of inner steps per interval creates a mismatch with an unchanged outer optimizer, requiring a correction to the outer update. We optimize this mismatch by correction of the outer optimizer for the momentum and the learning rate using the accumulated inner learning rate. Our experiments under communication constraints demonstrate that AutoLoCo reduces communication frequency by 27% relative to DiLoCo while maintaining training performance.
Read the full abstract ▾
All authors Pengyu He, Yan Zhang, Ruien Li, Guangwen Yang
Artificial Intelligence · Cross-listed from cs.LG
Constitutional adapters: Inference-time interventions for misalignment and misuse
Adam S. Lowet and Mark Kurzeja
Training models to act in accordance with an explicitly defined set of principles, or "constitution," has shown promise as a robust and transparent mechanism for AI
alignment. However, the generality and flexibility of such methods remain unclear. Here, we show that constitution-consistent behavior can be distilled from synthetic corpora into lightweight objects (low-rank adapters and steering vectors). Despite never seeing a harmful request or jailbreak during training, such objects increase jailbreak defense success and measured
alignment – particularly at long context lengths and against multi-turn attacks, where they outperform both prompted and steered baselines. Subtracting control-trained from constitution-trained objects further accentuates these effects, yielding defenses we call "constitutional adapters" (CAs). CAs can be trained on a base model, transferred zero-shot to its post-trained checkpoint, and scaled at inference time to predictably trade off defense for benign compliance. Taken together, these results recommend CAs as a lightweight, portable, and tunable lever for mitigating misalignment and misuse in API deployments.
Read the full abstract ▾
Artificial Intelligence
RankBuffer: Efficient Ranking-Based Rewards for Open-Ended Generation
Zixuan Yang, Yiqun Chen, Qi Liu et al. (+6 more)
Open-ended generation lacks canonical answers, making pointwise rewards difficult to calibrate for group-based reinforcement learning. Directly ranking same-query rollouts provides a more suitable relative reward signal, but existing ranking-based reward methods can incur substantial judging cost. We introduce RankBuffer, which maintains an ordered, query-specific buffer of previously judged responses as a reusable quality scale. Each rollout is first inserted into an anchor interval through an independent coarse judgment, after which only rollouts assigned to the same interval undergo local fine ranking. The resulting complete order is converted into bounded rank rewards, while boundary expansion, local refinement, and inactive-anchor
pruning adapt the buffer as the policy evolves. Across four open-ended
benchmarks, RankBuffer consistently outperforms all pointwise baselines. It also achieves nearly on-par performance with the strongest ranking-based reward baseline while substantially reducing judging cost. Ablations demonstrate the importance of both local fine ranking and anchor response content, while buffer analyses show that rollout-derived anchors progressively extend and refine the covered quality scale. These results establish response reuse as an effective approach to efficient relative reward construction.
Read the full abstract ▾
All authors Zixuan Yang, Yiqun Chen, Qi Liu, Wei Yang, Erhan Zhang, Liyi Chen, Qimeng Wang, Yan Gao, Jiaxin Mao
Artificial Intelligence · Cross-listed from cs.CV
FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution
FangZhi Zhong, Xuerui Qiu, Yuqi Pan et al. (+4 more)
Long-context
reasoning in large
language models incurs substantial computation and
memory costs. Visual text compression (VTC) reduces input length by rendering text as images, but fixed-resolution rendering creates a compression-performance trade-off: low DPI saves
tokens at the expense of legibility, whereas high DPI spends
tokens on irrelevant content. We introduce FocusVTC, which breaks this trade-off through adaptive resolution while preserving general
multimodal capabilities. It combines compressed low-DPI global views with selective region enhancement, integrating enhanced views into ongoing
reasoning. We construct 29.4K high-quality
Reasoning-Evidence Localization (REL) chain-of-thought examples (REL-CoT) that link
reasoning traces to page indices and bounding boxes. Multi-resolution REL supervised fine-tuning (REL-SFT) teaches the model to localize relevant regions, and Group Relative Policy Optimization learns when to enhance resolution and how to use the resulting observations, without a separate continual-pretraining stage. At 72 DPI on RULER v1, FocusVTC scores 87.4 at $2.9\times$ input compression, including tool observations, versus 57.5 for Glyph at $3.0\times$ input compression. It surpasses its text-input backbone on LongBench (56.40 versus 55.86), improves the MRCR macro-average by 13.91 points, and achieves a 51.19 macro-average on VTCBench. The MRCR latency evaluation also shows a $2.79\times$ online end-to-end speedup over Text. General
multimodal capabilities are preserved, with MMMU increasing from 65.12 to 66.73 and MME from 2424.02 to 2457.62.
Read the full abstract ▾
All authors FangZhi Zhong, Xuerui Qiu, Yuqi Pan, Ya Liu, Shaowei Gu, Bo Xu, Guoqi Li
Artificial Intelligence · Cross-listed from cs.RO
Where Predictive Supervision Goes Shapes What VLA Policies Learn
Hanseul Kim, Jewon Yeom, Youngjoon Jeong et al. (+2 more)
Future prediction is increasingly used to improve
vision-
language-action (VLA) policies, based on the premise that anticipating scene
evolution encourages representations useful for control. However, forecast quality alone does not establish that a policy has learned a better representation for action. This distinction matters under distribution shift, where successful control depends on preserving spatial state and likely scene change beyond familiar configurations. We study what determines whether predictive supervision improves the visual representation used by a VLA policy. Through controlled comparisons with matched target constructions, prediction horizons, and training conditions, we find that different prediction interfaces produce markedly different forecasts and visual representations, including in the spatial, dynamics, and action information that transfers beyond familiar scenes. We trace these differences to how predictive errors shape the policy's visual stream. Consistent with this controlled finding, VLA policies trained with more direct, scene-matched future supervision show stronger robustness under simulated and physical distribution shifts. Together, our results frame future prediction as a representation-learning design problem whose value for control depends on whether its supervision reaches the representations through which the policy acts.
Read the full abstract ▾
All authors Hanseul Kim, Jewon Yeom, Youngjoon Jeong, Minsoo Jo, Taesup Kim
Artificial Intelligence · Cross-listed from cs.LG
Inducing Process Supervision from Outcome-Only Reinforcement Learning
Shengda Fan, Xin Cong, Zhong Zhang et al. (+2 more)
Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time
reasoning. However, training strong PRMs remains costly: human step annotation is difficult to scale, while Monte Carlo estimation is computationally expensive and can drift from the intrinsic correctness of steps. To get effective PRMs at low cost, we introduce TIPS (Thinking-Induced Process Supervision), an outcome-only reinforcement learning (RL) framework for training generative PRMs. In TIPS, the model generates a chain-of-thought (CoT) followed by step-level labels and an outcome label. The reward depends solely on whether the predicted outcome matches the ground truth, and the resulting group-relative advantage is used to optimize the entire generated response. Intuitively, when checking intermediate steps helps determine the outcome, more accurate checks can lead to better outcome judgments and higher rewards. Outcome-only RL can therefore reinforce step-level verification without explicit process supervision. We validate the effectiveness of TIPS across math and
agent benchmarks and four backbone families. Notably, TIPS-Qwen3-4B-Thinking-2507 reaches 85.2 F1 on ProcessBench with only 3.2K outcome-labeled trajectories, surpassing all evaluated trained PRMs and strong
prompt-only judges such as GPT-5.4-Instruct and Claude-4.7-Opus, while still trailing o1-mini. Code and data are available at https://github.com/RUCBM/TIPS.
Read the full abstract ▾
All authors Shengda Fan, Xin Cong, Zhong Zhang, Haotian Chen, Yankai Lin
Artificial Intelligence · Cross-listed from cs.SE
WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses
Haomin Qi, Xiangzhe Xu, Yiming Huang et al. (+2 more)
Bug validation asks a coding
agent to produce an executable witness for a reported bug. The witness combines a concrete input with a testing harness and exposes faulty behavior during execution. Such evidence makes audit findings actionable, yet
benchmark evaluation is difficult when cases reuse public historical bugs and witnesses or require manual construction. We present WitnessGym, an automated framework for constructing bug-validation
benchmarks through bug injection. It injects bugs into test-reached paths of real projects, rebuilds each project, and retains cases exposed by a construction-time witness. Bug specifications and execution adapters allow extension to additional bug types and
languages. Bug-preserving transformations vary the surrounding structure while preserving the witness behavior. Based on real-world Java projects with test suites, WitnessGym automatically constructs 1,300
benchmark cases. The injected patches resemble historical bug patches and are difficult for the two evaluated models to distinguish in blinded comparisons. We evaluate four coding
agent frameworks in six framework/model pairings across bug types, execution contexts, and transformation depths. Witness construction remains difficult even when the bug pattern is known. Our framework,
benchmark cases, and evaluation scripts are available.
Read the full abstract ▾
All authors Haomin Qi, Xiangzhe Xu, Yiming Huang, Jingbo Shang, Chengpeng Wang
Artificial Intelligence
Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses
Ziluowen Luo, Senzhang Wang, Chaozhuo Li et al. (+9 more)
Modern
agents increasingly rely on
memories, tools, and execution logic, so their competence extends beyond model parameters. This shift exposes a limitation of conventional knowledge
distillation, which asks how a student model imitates a teacher model. We define
Agent Distillation as the persistent transfer of task-solving knowledge from a teacher
agent to a student
agent. Our study organizes the field by where transferred knowledge is retained: within the model, as artifacts, through the execution harness, or across substrates. This perspective separates transfer evidence from its outcome and clarifies how knowledge moves between
agent components. We develop an evaluation framework that relates retention to causal contribution and deployed utility. Together, these contributions establish a foundation for the reliable, maintainable, and safe development of increasingly complex agentic systems.
Read the full abstract ▾
All authors Ziluowen Luo, Senzhang Wang, Chaozhuo Li, Jun Yin, Hao Yan, Ming Cheng, Chenxu Wang, Songyang Liu, Litian Zhang, Qiwei Ye, Zheng Liu, Philip S. Yu
Artificial Intelligence
Ziyu Liu, Jun Chen and Lixu Wang
Language-model
agents increasingly rely on persistent natural-
language skills to adapt beyond their frozen model parameters. When a shared skill is repeatedly revised from a non-stationary, heterogeneous task stream, however, improvements for new tasks can overwrite procedures needed for earlier ones. In continual learning, Orthogonal Gradient Descent (OGD) addresses analogous interference by projecting a new-task gradient onto a subspace that locally preserves prior predictions. Natural-
language skill revisions, however, have neither gradients nor a canonical vector space in which such a projection can be performed. We introduce Semantic-Scope Projected
Evolution (SSPE), which transfers the functional principle of gradient projection from parameter space to behavior space. SSPE treats an unconstrained skill revision as a proposed update, identifies acquired capabilities with which it may interfere, and uses the observed gains and regressions to construct a compatible revision rather than merely rejecting the update. This enables one shared skill to evolve across latent and recurring task contexts without exposing semantic domain identities to the
evolution model. Across controlled synthetic streams and heterogeneous real-
agent benchmarks, SSPE improves final cross-domain competence and mitigates forgetting relative to strong skill-
evolution baselines. The evolved skill also retains the strongest average performance after transfer to a different executor model. These results establish semantic projection as a promising principle for stable and adaptive self
evolution of
language agents.
Read the full abstract ▾
Artificial Intelligence
DualTrack: Synchronized speech-gesture generation via symmetric coupling of pretrained priors
Yuanzhuo Hu, Zehan Liu, Xiaoyi Qin et al. (+1 more)
Joint
speech-gesture synthesis must coordinate two modalities despite limited paired data. Existing approaches often lack bidirectional interaction, have limited
language coverage, or simplify body and finger representations. We present DualTrack, which couples pretrained
speech and motion priors on a shared 12.5 Hz timeline. Causal adapters exchange previous-packet information, while current-state fusion coordinates the streams before they separately complete sixteen-codebook packets. We evaluate 43 BEAT2 recordings in four
languages, with speakers held out from joint training and validation. On the shared English/Spanish inputs, without
speech or motion prefixes, DualTrack achieves lower word error rate and full-motion Fréchet Gesture Distance, higher beat consistency and
speech naturalness than the evaluated GELINA baseline.
Read the full abstract ▾
All authors Yuanzhuo Hu, Zehan Liu, Xiaoyi Qin, Ming Li
Artificial Intelligence
Neural Structural Reasoner: A Brain-inspired Architecture for Reasoning over Structured Knowledge
Zixing Jia, Yuhang Pan and Ni Ji
Structural
reasoning, the ability to recognize and make inferences over the relational structure between objects and concepts, is a hallmark of human cognition, yet prevailing methods often collapse relational
topology into flat
embeddings, cannot discover hidden structure and lack interpretability. We introduce Neural Structural Reasoner (NSR), a
brain-inspired network that preserves relational structure directly in the connectivity and dynamics of coupled neuronal populations. NSR draws inspiration from three biological mechanisms: multi-layered architecture for encoding hierarchical knowledge, stable representations of entity and concepts, and path integration for input-driven state inference. At query time, NSR parallelizes computation over candidate relational structures and leverages confidence-weighted scores to perform link prediction. Across standard knowledge-graph
benchmarks, NSR achieves competitive accuracy without leading on every
dataset, and has lower reported training times than several neural baselines. Because
reasoning is implemented through sequences of human-readable
neuron activations, NSR affords native interpretability by tracking intermediate inference steps. The model further extracts latent relational hierarchies and compositional rules, demonstrating the
brain-inspired architecture as an effective, efficient, and highly interpretable substrate for structural
reasoning.
Read the full abstract ▾
Artificial Intelligence
Multi-Channel Mitigation of Source-Trust Shortcuts in Fact-Checking RL Agents
Jianchang Su, Yiwei Yang and Wei Zhang
Retrieval-augmented fact-checkers often receive a reliability label, such as HIGH or LOW trust, for each evidence source. These labels should adjust the model's confidence and its decision to search for more evidence, while the verdict should follow the evidence content. We introduce TrustSwap, a counterfactual test that swaps, lowers, or removes source labels while keeping every evidence text fixed, and measures its three output channels (the verdict, the confidence, and the search decision) separately. Across untrained and RL-trained models at two scales, three
datasets, and two
prompts, confidence and search respond to the labels as intended in 49 of 50 comparisons, yet a label change alone alters 4-23% of confident verdicts for Qwen3 models and up to 50% for an existing RL-trained fact-checker. Standard GRPO fine-tuning amplifies this shortcut at 8B in all six settings. To reduce it, we propose trust-swap augmentation (TSA), which trains GRPO on each claim with both its original and its label-swapped evidence under the same gold verdict. At 4B, TSA lowers the verdict flip rate by 7-35% (relative) in four of six settings, keeps accuracy and the intended confidence and search responses, outperforms reward-based alternatives in the main setting, and carries over to an unseen label-removal perturbation. An added consistency reward helps on the trained-on swap but not on unseen perturbations. At 8B, TSA's effect is not detectable, which makes scale the main open question.
Read the full abstract ▾
Artificial Intelligence
SAKI: Maximal-Coupling-Routed Teacher Supervision for On-Policy Distillation
Miteto Wei, Xiaohan Wang, Zehao Chen et al. (+7 more)
On-policy
distillation (OPD) reduces train-test state mismatch by training a student on its own generated trajectories, but weak students may visit teacher-misaligned prefixes where supervision is less representative. We introduce SAKI (Supervision Allocation with KL-constrained Interpolation), which combines a KL-constrained teacher-guided rollout with maximal coupling and reuses realized accept/correction events to route
token-level supervision. Accepted positions retain sampled-
token reverse-KL supervision, while correction positions receive direct supervision on the teacher's highest-probability
token. Under maximal coupling, the correction probability is exactly TV(p_t, q_t), so the same trust-region radius controls rollout deviation and upper-bounds intervention and specialized-supervision frequency. We further implement an engine-resident speculative verifier that preserves the exact-q trajectory distribution and coupling semantics while improving matched-workload rollout throughput by 4.22x. Across seven mathematical
reasoning benchmarks, SAKI improves the matched teacher-guided baseline in Mean@8 and Pass@8 for both 1.7B and 0.6B students. Placement controls and fixed-prefix analysis further support correction-triggered routing as a conflict-adaptive supervision signal.
Read the full abstract ▾
All authors Miteto Wei, Xiaohan Wang, Zehao Chen, Jiajun Chai, Sichao Liu, Li Wang, Haoyuan Xu, Zhaoyu Hu, Wei Lin, Guojun Yin
Artificial Intelligence · Cross-listed from math.OC
Second-Moment Stochastic Approximation Methods
Tao Jiang and Lin Xiao
Classical
stochastic approximation methods rely on
estimators of the first moment (mean) of a random regression function. We study methods that employ
estimators of both the first and the second moments, which include modern deep-learning optimizers such as Adam and
Muon as special cases. We derive second-moment
stochastic approximation methods through the lens of optimal preconditioning for solving
matrix equations, and develop a two-stage framework for their convergence analysis. The first stage focuses on the analysis of conceptual (impractical) methods that rely on the exact first and second moments. In the second stage, we replace the exact moments with their respective
estimators, and invoke Dvoretzky's
theorem to show that the resulting practical methods converge almost surely to a neighborhood of the target solution. The size of the neighborhood depends on the biases and variances of the first- and second-moment
estimators. We derive concrete bounds for
Muon and a spectral variant of Adam that determine the radius of their neighborhood of convergence.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Scaling Video Generation for Reasoning: At What Cost?
Weihang Guo, Xiaoyu Wu, Yifei Wang et al. (+2 more)
We study whether scaling
video generation enables models to reason about hidden information from the past frames, and at what computational cost. Our controlled
benchmark requires predicting nine prescribed moves of an initially solved 2x2x2 Rubik's Cube from a fixed view of three faces. Correct predictions require inferring how actions change hidden states, and the simulator provides exact ground truth for evaluation. Models learn plausible cube geometry early, while correct sticker configurations require substantially more training. Although validation MSE follows approximate power-law scaling, lower MSE loss does not reliably indicate downstream
reasoning capabilities. Smaller autoregressive models achieve higher state accuracy with limited compute, while larger models reach higher accuracy after more training. At roughly 0.1 PF-days, the 70M-parameter model correctly predicts the visible sticker configuration in 44.6% of post-action frames, compared with 0.3% for the 1B model, which reaches 83.7% at 3.14 PF-days. Symbolic state supervision raises the 20M model's frame accuracy from 31.1% to 67.3% at the same training-data budget, suggesting that learning representations of state changes can complement scaling.
Read the full abstract ▾
All authors Weihang Guo, Xiaoyu Wu, Yifei Wang, Niloofar Mireshghallah, Lydia E. Kavraki
Artificial Intelligence · Cross-listed from cs.RO
Simple Agentic Memory for Generalist Robot Policies
Yuyou Zhang, Yunbei Zhang, Miao Li et al. (+4 more)
Visual-
memory systems commonly retain or compress past observations.
Robot control additionally requires interaction-derived state that no individual frame may explicitly represent, such as persistent identity relations, accumulated progress, or ordered procedures. We introduce Simple Agentic
Robot Memory (SimpleARM), a training-free
memory layer for frozen generalist
robot policies. From the task instruction, SimpleARM specifies what to monitor; frozen perceptual tools maintain compact typed state online; structured access retrieves that state only when a proposed subgoal depends on history; and current-view grounding resolves recalled entities before execution. We evaluate SimpleARM on RoboMME, a
benchmark of
memory-dependent
robot manipulation tasks that require history information no longer available in the current observation. Across all 16 tasks and three policy seeds, SimpleARM achieves 67.17% mean success, compared with 44.51% for the strongest non-oracle baseline. Matched ablations show mechanism specificity: removing relation, reference, progress, or route state produces large losses where the affected state is retrieved for control, while largely sparing other tasks. These results support a state-based view of
robot memory: effective
memory for control is not simply retained visual history, but compact task-relevant state derived from the interaction history.
Read the full abstract ▾
All authors Yuyou Zhang, Yunbei Zhang, Miao Li, Janet Wang, Zijian Jin, Shilong Liu, Ding Zhao
Artificial Intelligence · Cross-listed from cs.GR
Text2Sim: Agentic Physics-Based Simulation Generation with Distilled Expertise
Xiaoyu Xiong, Tsun-Hsuan Wang, Yi-Ling Qiao et al. (+2 more)
Creating diverse physical simulations remains labor-intensive because assets, layout, physical parameters, motion, control, and rendering must be designed and debugged jointly. We present Text2Sim, a simulation-specialized agentic pipeline that converts a text-only request into an executable, editable dynamic case. Built on Genesis, Text2Sim uses a hierarchical agentic structure that combines a Planner with specialized Writers, asset-generation tools, and an independent Critic. Compact skills (Debug Cards) distilled from graphics demonstrations provide role-specific physical guidance for execution-based repair. We evaluate physical quality, visual quality, and human preference on 42 held-out
prompts spanning rigid, articulated, deformable, and cloth phenomena, with a paper-level split between experience construction and evaluation. We design automatic physical and visual scorers to evaluate the quality of the results, and Text2Sim achieves higher scores than all four state-of-the-art baselines on both metrics. In blinded user studies with these baselines, significantly more participants prefer Text2Sim than prefer the baselines, which is consistent with the results from our automatic scorers. The pipeline also supports a broad range of downstream applications; we select
dataset construction and extension to
multimodal input as two representative examples. We will release the code, the Debug Card library, and a
dataset of generated cases, each pairing the text
prompt and rendered
video with the executable program, assets, physical parameters, controls, and recorded states.
Read the full abstract ▾
All authors Xiaoyu Xiong, Tsun-Hsuan Wang, Yi-Ling Qiao, Tao Du, Minchen Li
Artificial Intelligence · Cross-listed from cs.RO
Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning
Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu et al. (+12 more)
We study reinforcement learning (RL) methods for cooperative multi-
agent Vision-
Language-Action (VLA) models. This problem is challenging because VLAs are pretrained on large-scale single-
agent data and therefore lack the fine-grained coordination skills required for inter-
robot collaboration. Supervised fine-tuning (SFT) on multi-
robot demonstrations partially bridges this gap, but its performance is bounded by the demonstration data and cannot improve from its own experience. We present a three-stage reinforced fine-tuning (RFT) pipeline for multi-
agent VLAs. First, initialization-aware data collection sweeps over initial configurations and invokes human demonstrations only when the pretrained VLA repeatedly fails, yielding robustness to initialization shift with reduced human cost. Second, offline credit-filtered tuning assigns credit to individual
agents and fine-tunes on per-
agent trajectories with positive advantage rather than on entire joint rollouts. Third, we find existing online RL for VLAs are less effective for hard multi-
agent tasks, which we attribute to noisy co-exploration and unstable updates. We instead use online latent-space fine tuning, which freeze the VLA and perform RL in its latent noise space. We evaluate our multi-
agent VLA with both $\pi_0$ and $\pi_{0.5}$ backbones across 11 tasks in RoboTwin, RoboFactory and real-world manipulation with two Franka
robots. Our multi-
agent VLA improves the average success rate by $+23.1\%$, $+16.4\%$, and $+44\%$ on RoboTwin, RoboFactory, and real-world tasks, respectively. Code available at https://anonymous.4open.science/r/mavla_rft-2BC0/.
Read the full abstract ▾
All authors Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu, Jianing Guo, Hanxiao Li, Kejian Shi, Shuning Zhang, Pu Feng, Yongjia Ma, Yuqing Ma, Kai Chen, Qi Dou, Yaodong Yang, Xianglong Liu, Simin Li
Artificial Intelligence
Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It
Zehao Jin, Ruixuan Deng and Junran Wang
Pretrained
transformers use little of their depth to follow references in context. Thirteen base models reliably follow only 1.4-3.6 lines, and extra pretrained loops add little. A task-trained rank-8 LoRA at one early layer extends this computation with all model weights frozen. Qwen3-8B improves from 15.5% to 99% exact accuracy on 24-line chains; a longer-trained LoRA reaches 50 lines. Ouro-1.4B reaches 60 lines after four loops and at least 160 after eight. The LoRA starts a relay: program lines pass on their chain identity through a short range of middle layers. Frozen heads read progressively further up the chain, and removing parent-line
attention stops the relay. A frozen-model measurement locates the last useful intervention layer within tolerance in three of four held-out models. Task-specific LoRAs also improve MuSiQue. Default answers therefore understate the computation accessible through a tiny edit. Code and an interactive demo are available at https://lunamos.github.io/stop-thinking-too-early/
Read the full abstract ▾
Artificial Intelligence
MemEvo: Automatic Discovery of Streaming Video Memory Mechanisms
Guohong Liu, Jialei Ye, Shanhui Zhao et al. (+2 more)
Query-agnostic streaming
video understanding requires
vision-
language models to continuously compress an indefinitely growing visual stream into a bounded
memory before future queries are known. The performance depends critically on the
memory mechanism–what observations to preserve, how to represent and consolidate them, and what information to retrieve when a query eventually arrives. Rather than designing a single
memory architecture by hand, we formulate
memory design as a search problem over executable
memory programs. We introduce a lightweight domain-specific
language that expresses
memory mechanisms through structured primitives for representation, admission, retention, consolidation, budgeting, and
retrieval, while enforcing causal and bounded-
memory constraints. Although structured, the derived program space remains large and contains heterogeneous, conditionally dependent design choices whose effects can only be assessed via downstream execution. We therefore propose MemEvo, an LLM-driven auto-research framework that uses pretrained LLM as a semantics-aware proposal model to iteratively generate and refine candidate
memory programs based on accumulated experimental feedback. At runtime, a deterministic evaluation pipeline validates and evaluates each candidate, while the underlying
vision-
language model remains frozen throughout discovery. We finally produce a training-free, bounded-
memory mechanism. Extensive experiments on StreamingBench and OVO-Bench demonstrate strong streaming
video understanding performance together with substantial context and inference efficiency.
Read the full abstract ▾
All authors Guohong Liu, Jialei Ye, Shanhui Zhao, Yunxin Liu, Yuanchun Li
Artificial Intelligence
SafeCoEvo: Co-Evolving Safety Harnesses and Guards for LLM Agents at Test-Time
Yu Cheng, Yongkang Hu, Shuaijie Ma et al. (+12 more)
LLM
agents deployed in real-world environments continually encounter new tasks and safety risks, while execution feedback typically becomes available only after each task is completed. However, existing self-evolving approaches commonly rely on multiple rounds of optimization over fixed and repeatedly accessible task distributions, fundamentally differing from test-time adaptation in real-world deployment, where only experience accumulated from past tasks can be used to improve safety decisions on future unseen tasks. To address this limitation, we propose SafeCoEvo, a test-time Harness-Guard co-
evolution framework for LLM
agent safety that enables the external safety system to continually adapt from accumulated runtime experience. SafeCoEvo jointly improves two complementary safety capabilities at different timescales: S-Harness rapidly externalizes recent runtime experience into updatable explicit safety knowledge that can promptly influence subsequent tasks, while GuardVPO internalizes accumulated runtime safety experience over a longer timescale into parametric risk-judgment capabilities. By combining short-term rapid adaptation with long-term capability consolidation, SafeCoEvo continually improves the
agent's safety capabilities, reducing the unsafe outcome rate by 10.05% while improving the task success rate by 12.15% over the strongest baseline, thereby achieving simultaneous gains in safety and task utility.
Read the full abstract ▾
All authors Yu Cheng, Yongkang Hu, Shuaijie Ma, Zhihang Lin, Weicheng Meng, Jingyang Qiao, Jiuan Zhou, Yushuo Zhang, Yihang Chen, Weilin Luo, Kun Shao, Dong Li, Zhizhong Zhang, Yuan Xie, Zhaoxia Yin
Artificial Intelligence · Cross-listed from cs.LG
Factorized Scheduling Principle: Learning Interpretable and Transferable Policies via Structured Additive Functions
Hong Je-Gal and Hyun-Suk Lee
Scheduling problems arise from repeatedly selecting one item from a set of candidates based on their states. These problems often reduce to assigning priority scores and choosing the highest-ranked item. In this work, we propose a factorized scheduling principle (FSP) framework to learn interpretable and transferable scheduling rules. The FSP framework represents system states as condition distributions and decomposes a global scheduling principle into additive univariate and pairwise components with identifiability constraints. The scheduling principle enables the framework to maintain a simple priority-based structure during deployment. This principle is learned by using a policy-based objective combined with a temporal-difference signal defined on the condition distribution. Experiments on synthetic and realistic scheduling tasks demonstrate the FSP framework's strong performance, interpretability, and zero-shot generalization across different system scales.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SD
Long-Term Memory-Guided Enhancement for Target Perception in Audio-Language Models
Zhenhong Zhou, Xuanyue Zhao, Youji Liu et al. (+4 more)
Audio large
language models (ALLMs) can reason about the content of audio recordings to perform complex tasks. However, these capabilities usually collapse in real-world environments when background noise and competing sources mix the target sound. Inspired by long-term
memory in human listening, we propose Long-Term
Memory-Guided Audio Enhancement (LTM-AE) to improve selective target perception by refining the audio representations of ALLMs without training. LTM-AE extracts representations in hidden states from separate clean reference recordings as long-term
memory for each category, guiding enhancement toward a user-specified listening target. We reconstruct incoming audio
tokens in the selected category long-term
memory and interpolate the reconstructions with the original
tokens before
language backbone decoding. This interpolation controls the influence of stored auditory experience while keeping all ALLM parameters fixed. Diagnostic readouts across twenty sound categories and three ALLMs show that LTM-AE strengthens responses to a specified target amid three interfering sources. Averaged over constrained and free-form classification, accuracy gains over raw mixtures range from 29.53 to 46.15 percentage points across multiple open source models. For
speech content recovery, LTM-AE with an additional learned
token-level gate reduces Qwen2-Audio's word error rate from 23.07% to 14.77%. This work takes an initial step toward using principles of human long-term
memory to enhance ALLMs for real-world listening. Our code is available at https://github.com/aynlp/ltm-audio-code
Read the full abstract ▾
All authors Zhenhong Zhou, Xuanyue Zhao, Youji Liu, Yuanhe Zhang, Xiaoyu Ma, Lianyu Hu, Yang Liu
Artificial Intelligence
Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments?
Michael Lee, Zhipeng Wei, Yue Dong et al. (+1 more)
Agentic systems are now being widely used to orchestrate tools and reason over long contexts. However, the improving capabilities of the large
language models powering these
agents also create new attack surfaces for indirect
prompt injection. In particular, an attacker may not need to place a complete malicious instruction in retrieved content if the
agent can reconstruct the objective from incomplete fragments distributed across a long context. In this work, we introduce adaptive long-context
prompt injection (AdaLCPI), which combines long-context fragmentation with adaptive search. AdaLCPI splits an attack objective into incomplete fragments, embeds them in external content retrieved through the
agent's tools, and uses a reconstruction cue to
prompt the
agent to combine them. It then iteratively refines the fragments and cue with OpenEvolve using graded scoring and natural-
language execution feedback from the target
agent. Empirically, AdaLCPI achieves higher attack success than strong adaptive baselines, reaching 61.4% macro-average ASR compared with 32.8% for Trojan Hippo-style and 30.0% for AgentVigil. Safety evaluations should therefore test whether
agents remain robust when harmful objectives must be reconstructed from incomplete fragments.
Read the full abstract ▾
All authors Michael Lee, Zhipeng Wei, Yue Dong, N. Benjamin Erichson
Artificial Intelligence
Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning
Zhongan Bi, Kepeng Lin, Xuanang Gao et al. (+2 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large
Vision-
Language Models (LVLMs), and perception-aware methods further encourage policies to rely on visual evidence. Yet relying on the image does not guarantee that visual claims are supported by it. Before RL training, 27.81% of the correctly answered responses of Qwen2.5-VL-7B on four
multimodal reasoning benchmarks contain at least one direct visual claim that the image does not support. Since outcome-level RL rewards each response as a whole, these claims inherit the positive credit of the correct answer. We introduce a fixed-rollout counterfactual diagnostic that re-scores the same response under an intervened image to separate Evidence-Function Sensitivity (EFS), how strongly the model's predictions change, from claim persistence, whether the model keeps supporting the same claim rather than retracting it. The diagnostic reveals Sensitivity-Persistence Decoupling (SPD): under DAPO and VPPO, EFS increases and claims become more retractable overall, yet unsupported claims become significantly more persistent, whereas GRPO raises EFS without this deterioration. We therefore propose Persistence-Aware Credit Gating (PACG), which attenuates positive credit for unusually persistent visual claims and leaves all other credit unchanged. It requires no supported/unsupported labels and adds no inference cost. On Qwen2.5-VL-7B, PACG raises the nine-
benchmark average over three seeds from 58.1% to 59.9% with DAPO and from 59.8% to 60.9% with VPPO, while making unsupported claims more retractable. The gains extend to a larger model, a newer backbone, and the accuracy of HallusionBench also improves consistently. These results suggest that visual sensitivity and claim retractability are complementary dimensions of
multimodal credit assignment.
Read the full abstract ▾
All authors Zhongan Bi, Kepeng Lin, Xuanang Gao, Yuhan Sun, Lianrun Zhang