Artificial Intelligence · Cross-listed from cs.LG
When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?
Kaifei Wang, Zikang Shan, Liwei Wang et al. (+2 more)
Momentum is a classical tool for accelerating convex optimization and a standard component of modern large
language model training. Yet recent empirical work has questioned the usefulness of first-moment momentum in reinforcement learning with verifiable rewards (RLVR). We study this question through a bandit formulation of RLVR with bias-corrected exponential moving average (EMA) momentum, where the EMA coefficient $\beta$ controls how quickly the weights on past gradients decay. When policy gradients are evaluated exactly, we prove that increasing $\beta$ within a range that prevents overshoot strictly accelerates optimization. When gradients are estimated from sampled responses, a positive expected gain can be obscured by sampling fluctuations. We show that as the learning rate $\eta$ decreases, the expected gain with fixed $\beta$ vanishes faster than these fluctuations, whereas increasing $\beta$ to keep $\eta/(1-\beta)$ within the safe regime preserves a nonvanishing gain that eventually dominates them. Bandit simulations quantitatively validate these predictions. RLVR experiments on Qwen3-4B-Base using SGD with EMA and AdamW support our principle of choosing larger coefficients in practical LLM post-training. On Qwen2.5-32B with AdamW, simply increasing $\beta_1$ improves mean validation accuracy by nearly $2$ percentage points over the conventional choice, demonstrating the principle's effectiveness at a larger model scale.
Read the full abstract ▾
All authors Kaifei Wang, Zikang Shan, Liwei Wang, Yinyu Ye, Han Zhong
Artificial Intelligence · Cross-listed from cs.CL
Cortex: Consolidating Context into Parametric Memory
Jintian Zhang, Feng He, Xinnian Liang et al. (+5 more)
Context can improve large
language model performance, but these gains may be lost after context removal. Inspired by biological systems consolidation, we formulate
Memory Consolidation (MC) as internalizing these gains while preserving non-target capabilities, connecting context-specified editing, skill learning, and behavioral steering under a common objective. We introduce a five-task-family testbed measuring consolidation and retention under single, batch, and sequential updates. We propose Cortex, which generates prospective queries to anticipate uses of context and distills context-conditioned outputs into mergeable low-rank or structural feed-forward network updates. Across 4B–120B dense and mixture-of-experts backbones, including both text-only and
multimodal models, Cortex lies on the empirical consolidation–retention Pareto frontier in most evaluated settings. After 50 sequential factual edits on a 14B model, Cortex-Structural recovers 88.9% of the context-induced gain while retaining 99.8% of original non-target performance.
Read the full abstract ▾
All authors Jintian Zhang, Feng He, Xinnian Liang, Ziwen Xu, Haoming Xu, Haojing Huang, Shuangzhi Wu, Ningyu Zhang
Artificial Intelligence
From Relevance to Validity: Zero-Shot Tool Selection via Functional and Contextual Validation
Xiaoxuan Dong, Jin Zhang, Zichen Yuan et al. (+11 more)
Large
language model (LLM)
agents increasingly face tasks from unseen domains and execution contexts, making zero-shot tool selection essential for identifying tools that can satisfy task requirements. Existing dominant methods mainly rely on relevance ranking, but semantic matching alone cannot ensure that a tool is functionally compatible or remains applicable in a new context. We preliminarily identify a transfer-validity gap, where semantically relevant tools may still fail to satisfy functional or contextual requirements after transfer. To address this gap, we propose \methodname{}, a progressive validation framework that complements semantic relevance with endpoint-grounded functional compatibility and task-conditioned contextual evidence. Across cross-domain and cross-
dataset transfer, \methodname{} consistently improves capability ranking, with larger gains under functional or contextual ambiguity. Ablation results further confirm the complementary contributions of functional and contextual validation.
Read the full abstract ▾
All authors Xiaoxuan Dong, Jin Zhang, Zichen Yuan, Yue Zhou, Zihao Rao, Dexu Yu, Tingyong WU, Wenhao Deng, Junchen Fu, Pengwei Sun, Yang Ran, Bo Lan, Yongxin Ni, Youhua Li
Artificial Intelligence · Cross-listed from cs.RO
Human Demonstrations as Post-Training Data for Any Robot Body: Retargeting the Object, Not the Hand
Nagasrikanth Kallakuri
Human demonstrations are today used largely for pretraining
vision-
language-action models, with little ability to use the same data for target-embodiment or task-oriented training, which scalable deployment of
robots in industrial and general environments depends on. A human demonstration recorded on a simple and scalable device such as a mobile phone can serve as post-training data for a
robot whose body differs in structure and capability from the human who recorded it. This is made possible not by retargeting the hand's motion but the object's. A capture yields a body-free record: the object's pose over time, its contact intervals, and a reconstructed 3D object with provenance. Any
robot, given as a URDF and a short descriptor, is solved against that record and scored in simulation against the demonstrated object motion as ground truth: the human-to-simulation leg of human-to-simulation-to-
robot transfer, measured per
robot. Each (
robot, episode) pair gets one outcome: pass, failure with a named mode, or refusal, with its reason, before any motion. The method runs as a working system over a growing collection; the set analysed here is a sample of it: 42 captures solved on 8
robots, 157 pairs, of which 31.2% pass, 52.2% fail, 14.0% are refused and 2.5% are unauditable (kinematic feasibility, +/-50 mm / +/-15 deg, in simulation). Refusals held up: every pair the checks refuse on stored evidence yet ran anyway failed (13 of 13; 5 on a body whose public model welds its hand). A description file could not be shown to screen a
robot: its outcomes were barely more predictable than guessing. No policy has yet been trained on the record. The record decides what a
robot may post-train on; whether the policy then works on the real
robot is the simulation-to-
robot question, answered downstream.
Read the full abstract ▾
Artificial Intelligence
Lifted State Policy Optimization: Internalizing Self-Correction through Transition Geometry
Ren Zhuang, Ben Wang and Shuifa Sun
Preference optimization ranks final answers and trajectory RL maximizes aggregate returns, yet neither credits the local change that turns a flawed
reasoning state into a better one. We introduce Lifted State Policy Optimization (LSPO), which establishes this finite, cost-adjusted transition as the primary unit of learning. Each textual answer is augmented with a learned continuous coordinate, forming a lifted state that an energy function evaluates and orders. A shaped reward drives energy descent net of edit and step costs, and an internalization loss absorbs selected low-energy answers into the direct generator. On Qwen3.5-27B, the resulting single-pass policy reaches 71.8% aggregate accuracy across mathematics, science, code, and logic
benchmarks, matching test-time reflection at 71.9% with 28% lower latency and 25.8% fewer
tokens. The pattern holds across dense and mixture-of-experts backbones, and matched controls confirm that transition credit and the lifted coordinate each contribute, as does answer internalization. Our code is available at https://github.com/Venomrko/lspo.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Non-asymptotic Convergence of Average-reward Q-learning with Options
Kintan Saha, Ahana Deb, Anders Jonsson et al. (+1 more)
We study the finite-sample convergence of Q-learning
algorithms for the average-reward formulation of Reinforcement Learning (RL) with options. While options allow temporarily extended and hierarchical actions, the average-reward formulation incorporates continual tasks beyond discounted and episodic RL. We establish the first $\tilde{O}(1/\text{iterations})$ convergence guarantees, in the mean-square sense, for average-reward option Q-learning
algorithms in both the inter- and intra-option settings. First, we prove that both inter- and intra-option Bellman operators satisfy multi-step span contractions under unichain assumptions. Then, we propose synchronous
estimators of these multi-step option Bellman operators leading to two variants of option Q-learning. Finally, we extend the
stochastic approximation framework to prove convergence rates of these two
algorithms under the minimal assumptions used in the literature.
Read the full abstract ▾
All authors Kintan Saha, Ahana Deb, Anders Jonsson, Debabrota Basu
Artificial Intelligence · Cross-listed from cs.SE
Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents
Yuyao Ge, Shenghua Liu, Peizhong Gao et al. (+8 more)
Agents driven by large
language models are increasingly deployed on long-horizon tasks, where they use tools and modify real workspaces. How well they perform depends as much on the harness around the model as on the model itself. Existing
benchmarks measure downstream task success under a fixed harness, leaving harness engineering itself unexplored. We propose Doc2Harness, a two-stage
benchmark that evaluates long-horizon harness engineering directly. In Stage 1, a build model operating inside a fixed builder harness receives an interface-only contract skeleton and a development document that consolidates behavior across twelve frontier harnesses, and then constructs a complete candidate harness in an isolated container. In Stage 2, the frozen candidate harness runs in containers separate from the construction environment, one per test case, and is driven end to end by a scripted mock model. We score each test case using four observable channels, namely the requests sent to the model, the files on disk, the process exit code, and the command output; positive controls rule out vacuous passes. Across nineteen frontier models, each evaluated with three independent builds on 328 test cases, GPT-6 Astra achieves the highest pass rate at 53.05%, while the weakest model reaches 27.44%, a gap of roughly twenty-six percentage points. Even the strongest model fails nearly half of the test cases, and failures cluster around how cross-cutting functionality is split across modules. To our knowledge, Doc2Harness provides the first deterministic
benchmark for measuring long-horizon harness engineering.
Read the full abstract ▾
All authors Yuyao Ge, Shenghua Liu, Peizhong Gao, Chendi Ge, Chenxu Liu, Yiwei Wang, Lingrui Mei, Baolong Bi, Jiayu Yao, Jiafeng Guo, Xueqi Cheng
Artificial Intelligence · Cross-listed from cs.CV
Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation
Yuheng Chen, Yufan Chen, Zhuojun Cai et al. (+8 more)
Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction, and image captioning. This separation leaves scientific figures, captions, and visual semantics weakly connected, and fails to measure whether a model can produce a single page-level parse that is both structurally faithful and semantically accessible. We introduce Read-Parse-Describe, a unified
dataset for document parsing with visual element description generation. Given a page image, models are required to output an ordered list of layout elements containing bounding boxes, semantic labels, reading-order indices, recognized textual content, and alternative text descriptions for visual elements. The
benchmark is built from 2,402 documents, comprising 18,163 continuous pages, 125,218 layout elements, and 11,338 visual instances. Besides, we further propose a plug-and-play space-awareness enhancer that reuses multi-layer visual features and injects grid-aware spatial guidance into
vision-
language tokens for layout-aware generation. Experiments across end-to-end VLMs and pipeline parsers show that existing systems still struggle to jointly recover reading order and visual semantics, while our enhanced model achieves strong joint performance. These results highlight the need to move beyond isolated document parsing modules toward unified models that jointly read, parse, and describe complex documents. The new
dataset and code will be made publicly available at https://github.com/yufanchen96/RPD
Read the full abstract ▾
All authors Yuheng Chen, Yufan Chen, Zhuojun Cai, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Weijia Fan, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen
Artificial Intelligence
TreeFlex: Agents Navigating Their Own Search Trees
Dongzhuo Li, Fangzhou Cheng, Weiwei Li et al. (+8 more)
A ReAct-style
agent accumulates observations, intermediate results, and failed attempts in a single context. As this context grows, stale and off-path information consumes its limited capacity and obscures relevant evidence, making focused
reasoning increasingly difficult. We introduce TreeFlex, a framework in which the
agent's own policy decides where to search. The
agent records each intermediate state, with a short summary, as a node in a search tree maintained by the harness. The policy is a single
language model that acts in two roles. In the Reflection role, it has a global view of the tree through these summaries and picks a node to resume, together with an instruction. In the Execution role, the harness restores the path that led to that node, and the model carries out the instruction with the usual task tools until it saves a new node or proposes a solution, which Reflection then approves or sends back to another node. Through these choices, the policy decides whether to deepen the current line, branch from or revisit an earlier state, verify a proposal, or stop, without a fixed traversal order or a score for each node. We train navigation, task actions, checkpointing, and proposal review jointly with a tree-aware adaptation of GRPO, using task-level outcome rewards augmented by process rewards. After small-scale RL of a 4B model, with the same active-context limit for all methods, TreeFlex reaches 48.8% accuracy on a challenging deep-research
benchmark, versus 41.5% for flat ReAct and 42.8% for FoldAgent, and its Pass@8 is about ten points higher than either. On SWE-bench Verified, with all three systems trained with RL from the same 9B model, TreeFlex resolves 54.9% of tasks, versus 43.3% for FoldAgent and 45.2% for its step-wise variant.
Read the full abstract ▾
All authors Dongzhuo Li, Fangzhou Cheng, Weiwei Li, Kevin Han, Haixu Ma, Zhongchang Sun, Ruiyi Li, Serena Li, Xiangjun Fan, Lizhu Zhang, Yuandong Tian
Artificial Intelligence
Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chains
Simon Baur, Sumire Honda, Wojciech Samek et al. (+1 more)
Reliable uncertainty estimation is essential for deploying large
language models in high-stakes
reasoning tasks, yet most existing approaches rely on expensive sampling strategies or shallow
token-level signals that ignore the internal logical structure of model outputs. We introduce Argument Graph Uncertainty, a post-hoc framework for multiple choice question tasks, that estimates model confidence directly from the logical structure of a single chain-of-thought
reasoning trace. Our method uses either a lightweight instruction-tuned model or a simple deterministic rule to segment the
reasoning chain into propositional units, then applies a NLI model to infer relational edges and construct two complementary argument graphs over these segments. The resulting edge scores are aggregated into a Dirichlet posterior over answer options, whose concentration yields calibrated uncertainty estimates reflecting both local step-to-step coherence and global logical consistency across the
reasoning trace. Evaluated on GPQA, MMLU-Pro and TruthfulQA, our method consistently surpasses
token-based and consistency-based uncertainty baselines across multiple
reasoning models. We further demonstrate through extensive combination analyses that argument graph uncertainty captures an orthogonal signal to existing approaches, making it a complementary component in uncertainty ensembles. Crucially, our framework requires no additional calls to the evaluated model and relies entirely on small, efficient auxiliary models, producing fully interpretable, per-node uncertainty attributions at low computational cost.
Read the full abstract ▾
All authors Simon Baur, Sumire Honda, Wojciech Samek, Jackie Ma
Artificial Intelligence · Cross-listed from cs.CV
Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis
Yinzhen Huang, Jiahe Xie, Yitong Sun et al. (+2 more)
Multimodal Large
Language Models (MLLMs) are increasingly deployed in safety-critical domains, yet their susceptibility to backdoor attacks remains a formidable challenge. While existing defenses have evolved from visual-centric patch cues to unsupervised internal
attention statistics, they are fundamentally constrained by a reliance on static snapshots and attack-specific heuristics. Given the intricate interplay between dual modalities, the vast design space for
multimodal triggers allows diverse threats, such as textual injections, sub-perceptual noises, and coordinated cross-modal triggers, to easily circumvent such rigid priors, particularly when attack types are unknown in real-world applications. In this paper, we rethink backdoor detection by shifting the diagnostic paradigm from static observations to dynamic output
evolution. We observe that a successful attack inevitably hijacks the model's decoding trajectory, skewing internal modal contributions during generation while maintaining an abnormal output invariance despite input perturbations. To capture this behavior, we propose EVO-Detect, a unified detection framework that monitors the dynamic
evolution of internal signals during early decoding steps through two complementary channels:
Attention Flow and Logits Flow. The
attention branch measures how image-text
attention allocation evolves during early generation, whereas the logits branch quantifies the sensitivity of the output distribution to modal perturbations through interventional ablations. We then convert these trajectories into compact sample-level metrics and normalize them against negative-control signatures so that natural perturbations are less likely to be mistaken for backdoor evidence. Extensive evaluations show that our approach substantially outperforms state-of-the-art baselines such as BYE and TCAP across diverse attack families.
Read the full abstract ▾
All authors Yinzhen Huang, Jiahe Xie, Yitong Sun, Ruochen Zhang, Xingxing Wei
Artificial Intelligence
Reflect After Failure: Brain-Inspired Self-Evolving Agents for Embodied Spatial Intelligence
Xin Li, Xiaohuan Gu, Yifan Wang et al. (+6 more)
Vision-
Language Models (VLMs) have made substantial progress in spatial
reasoning, and self-evolving
agents further extend these capabilities to embodied environments by learning from interactions. However, current self-
evolution remains particularly challenging in embodied spatial tasks, complex and multi-step decisions from partial visual evidence making it difficult to determine why an execution fails. The failure may stem from misperception, insufficient exploration, or incorrect planning, yet execution feedback alone rarely reveals which component is responsible. As a result, simply accumulating interaction experience can reinforce ineffective strategies. To address this, we propose HiMeco, a
brain-inspired self-evolving
agent for embodied spatial intelligence, inspired by metacognitive monitoring and hippocampal
memory consolidation. Through Hippocampal-Metacognitive Reflection, HiMeco diagnoses failures in perception-exploration-planning (PEP), then compares execution attempts to derive reusable strategies and guidance for avoiding recurring errors. It also abstracts interaction programs into parameterized skills, using evidence contracts and execution-based validation to determine their retention and reuse. These strategies and skills guide subsequent interactions, with
memory updated only between rounds and model parameters kept frozen. HiMeco achieves state-of-the-art performance on ESI-Bench, and it also improves accuracy on EmbSpatial-Bench by 3.94 percentage points without
evolution.
Read the full abstract ▾
All authors Xin Li, Xiaohuan Gu, Yifan Wang, Shaobin Ling, Tianli Shi, Jingao Xu, Zhimeng Yin, Dapeng Wu, Yun Wang
Artificial Intelligence
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Xinyi Cheng, Tianfu Zhang, Jiwen Li et al. (+5 more)
Large
language models increasingly generate multiple
retrieval actions jointly, while downstream systems often provide only one reward for the resulting set. Broadcasting this reward provides a valid score-function gradient
estimator for the expected set-level objective, but does not determine a coalition-dependent within-set attribution. We introduce CASI (Credit Assignment via Set Intervention for Agentic
Retrieval), which constructs the missing counterfactual information through snapshot-grounded set intervention: generate and retrieve once, hold replayable snapshot components fixed, and evaluate alternative subsets under common conditions. Shapley aggregation converts these marginals into within-set credit, which bounded zero-sum advantage shaping incorporates into policy optimization. For m >= 4, singleton, leave-one-out, and full-set endpoints do not generally determine the Shapley allocation; under additive set values, extra coalition evaluations are redundant. In a five-action sponsored-search setting, exhaustive auction replay over 1,200 requests reveals substantial coalition dependence. Under the training-time evaluator, CASI achieves the highest median cosine similarity to a replay-defined Shapley reference among evaluated credit rules; under a sensitivity backend, CASI does not outperform Singleton. Across five independent training runs, CASI-trained policies achieve higher held-out auction-replay set value than Singleton- or Broadcast-trained policies. In a randomized seven-day production A/B test on 10% of traffic, the CASI-trained policy yields an observed 3.51% lift in the primary advertiser-value metric over production control.
Read the full abstract ▾
All authors Xinyi Cheng, Tianfu Zhang, Jiwen Li, Xuesen Zhang, Shihao Yu, Pengfei Zhu, Renhuan, Heda Wang
Artificial Intelligence · Cross-listed from cs.CL
CSWBench: Evaluating Context-Sensitive Code-Switching in Creative Writing
Zewen Jiang, Yahan Li, Chenlu Guo et al. (+3 more)
Large
language models (LLMs) have demonstrated multilingual capabilities across diverse
language generation and understanding tasks. Yet effective code-switching requires more than multilingual proficiency: models must decide when, where, and why to switch
languages in ways that fit the surrounding context. This context-sensitive ability remains insufficiently evaluated. We introduce CSWBench, a
benchmark for evaluating context-sensitive code-switching in creative writing through two complementary settings: Focused Narrative and Extended Narrative. We further propose CSWEval, a theory-informed framework with two complementary levels: event-level evaluation of Licensing, Contextual Appropriateness, and Functional Gain, and whole-text evaluation of task-specific quality—overall literary and narrative quality in Focused Narrative and quality emphasizing multilingual organization in Extended Narrative. Evaluating twelve LLMs, we find that models exhibit meaningful but uneven code-switching competence across narrative contexts. Model differences are particularly pronounced in multilingual literary integration, while multilingual-focused dimensions show stronger difficulty effects overall than the general narrative dimensions; narratively salient switches are associated with stronger contextual and functional realization; and event-level CS quality is moderately associated with whole-text performance.
Read the full abstract ▾
All authors Zewen Jiang, Yahan Li, Chenlu Guo, Diana Inkpen, Yuan Wu, Yi Chang
Artificial Intelligence · Cross-listed from cs.LG
FUSE: Fairness-aware Update Screening against Byzantine Exploitation in $q$-Fair Federated Learning
Tiantong Sun, Jinlin Liu, Xingru Wei et al. (+4 more)
Client-level fair
federated learning, in the form of $q$-fair
federated learning ($q$-FFL), agnostic
federated learning, or tilted empirical risk minimization, has emerged as an attractive solution to equalize utility across heterogeneous clients. This paradigm grants high-loss clients greater relative influence, while a central server aggregates their loss-weighted updates into a single shared model. Yet, the Byzantine robustness of $q$-fair
federated learning has been largely neglected, leaving the server unable to tell legitimate
fairness-induced amplification from malicious update scaling. To address this critical gap, we formalize the problem of Byzantine-robust $q$-fair
federated learning under an update-only interface, in which the server observes nothing but the
fairness-scaled client updates. Our analysis under shared class-conditional label skew shows that
fairness scaling preserves the nonnegative class-mixture structure of honest updates, while replacing their unit-sum constraint by a client-dependent radial scale. Moving beyond magnitude-based screening, we show that this warped geometry supplies two complementary tests, one radial and one structural, that a single norm cannot provide. We then introduce FUSE, a structure-aware robust aggregator that uses a small trusted labeled
dataset for coefficient recovery, scale and cone-consistency screening, and post-screening step-size control. Across four
benchmarks and seven Byzantine attacks, FUSE improves seed-wise worst-attack accuracy over the strongest baseline on each
benchmark by 3.9–30.1 percentage points, while largely preserving the influence of the high-loss honest clients that
fairness is meant to protect.
Read the full abstract ▾
All authors Tiantong Sun, Jinlin Liu, Xingru Wei, Zequan Fang, Wei Chen, Qiang Yang, Xiaojin Zhang
Artificial Intelligence · Cross-listed from cs.LG
Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics
Junze Zhu, Weihao Chen, Zhiyang Zhou et al. (+2 more)
Deploying a general-purpose model as an
agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision shows which action was taken but leaves the runtime semantics behind that action implicit. We introduce Harness Policy
Distillation (HPD), which makes these runtime control semantics an explicit training target alongside actions to align the harness policy. HPD replays reference trajectories through the harness to recover the runtime semantics that are implicit in ordinary trajectory data, and represents them as control documents at each model-call boundary. These documents provide privileged information for a teacher to generate actions, while harness verification ensures that the resulting targets are executable under the runtime contract. We jointly train a shared model on the runtime semantics recovered from control-state privileged information and native history, thereby internalizing these semantics into the model policy. At deployment, the trained model uses the original harness interface without requiring the control document or any additional supplement. On TRAJECT-Bench with the Pi harness, Qwen3-8B trained with HPD reaches 22.98% autonomous task success and 49.23% runtime compliance, against 3.09% and 19.90% for the unadapted model. Harness-verified teacher actions substantially improve the native deployment policy, while joint control-state supervision provides further gains in task completion and runtime compliance. HPD turns executable runtime semantics into dense supervision for learning harness-compatible actions at each decision boundary.
Read the full abstract ▾
All authors Junze Zhu, Weihao Chen, Zhiyang Zhou, Zhen Wu, Xinyu Dai
Artificial Intelligence · Cross-listed from cs.LG
Long-Horizon RL Needs Muon
Kai Ruan, Jinghao Lin, Tao Chen et al. (+3 more)
Long-horizon
agents must coordinate
reasoning, tool use, and environment feedback over extended interactions. Learning from sparse outcome rewards requires weighting trajectories of different lengths and updating the policy while rollouts are still being collected. We examine a single-rollout asynchronous recipe combining
Muon,
token-averaged loss, truncated importance sampling, and auxiliary length and tool-error penalties. Single-rollout sampling allows asynchronous learning without waiting for multiple trajectories for the same issue. We study optimizer choices and recipe components for repository-level issue resolution across two model families. The evaluated training runs show higher training success with the full recipe than with the baselines.
Read the full abstract ▾
All authors Kai Ruan, Jinghao Lin, Tao Chen, Ziqi Zhou, Qianshan Wei, Zihe Huang
Artificial Intelligence · Cross-listed from cs.CR
P2Patch: Exploit-Guided Vulnerability Repair
Mohsen Iranmanesh, Sina Moradi Sabet, Sina Marefat et al. (+7 more)
Large
language models are increasingly deployed as autonomous
agents for automated program repair, patching software vulnerabilities through multi-step
reasoning and tool use. Yet these systems typically validate a patch against the same evidence used to construct it – a static alert, sanitizer trace, published exploit, or test suite. A patch that blocks that one piece of evidence can therefore look successful while remaining vulnerable to the same vulnerability with some variation. We present P2Patch, a generative
adversarial repair system that treats each patch as a new attack target. Starting from a normalized vulnerability finding, an exploitation
agent constructs a reproducing proof of vulnerability (PoV), a patching
agent repairs the program, and the exploitation
agent then attacks the patched program for a distinct same-vulnerability exploit. Every successful exploit becomes a permanent repair requirement, and the final patch must block all PoVs accumulated during the patching process, with construction-time PoVs kept strictly separate from held-out evaluation oracles. Across 101 C/C++ and Java CVE subjects, this
adversarial loop surfaces a distinct confirmed exploit in 41% of runs after the patch has already blocked its original construction-time PoV. Against single-pass repair from the same finding using Claude Code, P2Patch raises held-out fix-closed coverage from 78.1% to 84.7% and beyond-upstream coverage from 42.1% to 62.5%. Under the same held-out oracles, it also certifies more attack paths as blocked than recent state-of-the-art agentic repair systems on overlapping subjects. These results show that
adversarial testing and independent security oracles expose repair gaps that one-shot validation misses.
Read the full abstract ▾
All authors Mohsen Iranmanesh, Sina Moradi Sabet, Sina Marefat, Ali Javidi Ghasr, Maxime Newman Nereyabagabo, Kousha Amouzesh, Jaivir Malhi, Vinci Velasco, Alex Chung, Mohammad A. Tayebi
Artificial Intelligence · Cross-listed from cs.LG
Retrieval Over Training: Similarity search-based Model Selection for Time Series Anomaly Detection
Christos Panourgias, Roberto Stanzione, Adrien Petralia et al. (+2 more)
Anomaly detection is a fundamental task for time-series analytics, with important implications for the downstream performance of many applications. Despite the large number of anomaly detection methods proposed in the literature, recent
benchmark studies have shown that no single detector performs best across highly heterogeneous time series. Therefore, a practical and scalable solution is to develop a model-selection method that, for a given time series, selects the anomaly detector most likely to perform well. Nevertheless, the model selection approaches proposed in the literature suffer from a significant drop in accuracy when applied in Out-of-Distribution (OOD) settings. In this paper, we tackle the aforementioned limitation and propose RAMSAD, a
retrieval-based framework for model selection in time-series anomaly detection. Given a new time series, our method queries a knowledge base of previously observed instances, retrieves the top-k most similar series, and transfers detector recommendations from their performance profiles. The framework can operate with standard similarity measures as well as
embedding-based representations. Our results show that RAMSAD outperforms existing baselines in In-Distribution (ID) settings and is the most robust selector in OOD, where, when selecting a small subset of detectors, it matches or exceeds ensembles that run the entire detector pool, at a fraction of their cost. Overall, we demonstrate that similarity-based
retrieval constitutes a strong and efficient foundation for model selection in time-series anomaly detection.
Read the full abstract ▾
All authors Christos Panourgias, Roberto Stanzione, Adrien Petralia, Themis Palpanas, Paul Boniol
Artificial Intelligence · Cross-listed from cs.CV
PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
Siqi Yang, Yimu Wang, Ziqi Cai et al. (+6 more)
A multiplayer world model must predict how the independent actions of several players change a shared environment, including changes outside a player's current field of view. A point-of-view (POV)
video captures such changes only after they enter its field of view, whereas panoramic
video records the surroundings in every viewing direction and has served as an intermediate representation of the world state in recent world models. Learning this task therefore requires synchronized POV and panoramic
videos of interacting players together with their actions, yet no existing
dataset provides this combination in Minecraft, an established testbed in which players act independently in a shared, modifiable 3D world. We present PAWA, a
dataset and framework for action-conditioned panoramic multiplayer world modeling. The
dataset pairs each player's POV
video with world-stabilized panoramic
video and aligned actions. Its distributed pipeline decouples CPU gameplay capture from GPU rendering, scales beyond 64 simultaneous players, and re-renders each recorded session under new observation configurations without replaying the gameplay. On this
dataset, we develop a dual-tower model that jointly predicts POV and panoramic
video and couples the two streams through cross-
attention that aligns
tokens observing the same direction in a shared world frame. We further introduce a quadratic-phase player encoding that provides many more player codes than existing rotary codes at the same
embedding dimension. Experiments show that PAWA outperforms state-of-the-art multiplayer world models in POV prediction quality while also producing high-fidelity panoramic predictions. The
dataset and collection pipeline are open source to support further research on multiplayer world models.
Read the full abstract ▾
All authors Siqi Yang, Yimu Wang, Ziqi Cai, Chengxuan Zhu, Shuchen Weng, Erwin Wu, Zhaofei Yu, Kaipeng Zhang, Boxin Shi
Artificial Intelligence
Design Space Exploration of Reinforcement Learning Configurations for Post-Legalization Placement
Foteini Oikonomou, Maria Pantazi-Kypraiou, George Goudroumanis et al. (+5 more)
Graph Neural Networks (GNNs) and Reinforcement Learning (RL) have recently been explored for VLSI placement optimization, but their effectiveness for post-legalization placement refinement remains unclear. This paper presents a systematic design space exploration of a GNN-Proximal Policy Optimization (GNN-PPO) framework across three ISPD2015
benchmarks. We evaluate 18 configurations covering reward functions, GNN architectures, action spaces, candidate-set sizes, and training strategies. We analyse the results we obtain to identify pitfalls and to assess the prospects and opportunities of the RL methods for post-legalization placement. On top of that we propose a design classification approach to guide the design decisions of RL models.
Read the full abstract ▾
All authors Foteini Oikonomou, Maria Pantazi-Kypraiou, George Goudroumanis, Pavlos Stoikos, Athanasios Tziouvaras, Christos Anagnostopoulos, Georgios Karakonstantis, George Floros
Artificial Intelligence · Cross-listed from cs.CL
Scaling Multilingual Safety and Reasoning Evaluation through Reasoning-Guided Translation
Zahraa Al Sahili, Ioannis Patras and Matthew Purver
Large
language models (LLMs) are increasingly deployed across
languages, yet their safety and
reasoning capabilities remain predominantly evaluated in English. Translating existing
benchmarks offers a scalable approach to multilingual evaluation, but preserving task-specific meaning across
languages presents significant challenges, particularly for harmful requests, mathematical
reasoning, and conversational interactions. We introduce a
reasoning-guided translation framework for constructing multilingual evaluations designed to preserve the structure and intended meaning of diverse
benchmark tasks. Our approach separates translation strategy selection from large-scale generation, combining
reasoning-guided configuration selection with task-aware validation, targeted repair, and independent quality screening. Using this framework, we develop a source-aligned evaluation collection spanning seven safety, conversational, and mathematical
reasoning resources across 11
languages. The resulting collection achieves 99.94% coverage of intended item–
language pairs in the two subsets requiring repair. Evaluations of open-source LLMs reveal distinct
language-dependent vulnerabilities, including inconsistent refusal behaviour and substantial
reasoning degradation in lower-resource
languages. These findings highlight the importance of task-sensitive
benchmark construction and systematic quality assessment for understanding multilingual model behaviour beyond English.
Read the full abstract ▾
Artificial Intelligence
Robust Counterfactuals for Markov Decision Processes under Statistical and Causal Uncertainties
Jessica Lally, Milad Kazemi and Nicola Paoletti
Counterfactual
reasoning (asking what would have happened, had a decision been different) is a powerful tool for identifying causes of failure and explaining the behaviour of AI systems in sequential decision-making tasks. In Markov Decision Processes (MDPs), two sources of uncertainty complicate counterfactual inference: statistical uncertainty in the MDP transitions when the MDP is estimated from trajectory data, and uncertainty about the unknown causal model, as there are multiple structural causal models (SCMs) compatible with each MDP. Existing work addresses these two challenges in isolation. We address both simultaneously by introducing a framework for partial counterfactual inference in MDPs that is robust to both statistical and causal uncertainty. Our approach provides: 1) closed-form analytical bounds for the counterfactual probabilities that are valid across all SCMs entailed by a given interval MDP (IMDP); 2) a method to compute minimal robust interventions to achieve a desired counterfactual outcome via pessimistic value iteration on the induced counterfactual IMDP. Since the latter is constructed using our analytical bounds, our approach remains efficient and scalable to realistic MDP environments. Experiments across multiple case studies demonstrate our method's robustness and practical utility for identifying minimal sets of actions that, if changed, would have led to improved outcomes.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.HC
What and Where in Identifying and Quantifying Conventions in Humans and AI Agents
Makoto Fukushima and Tim Puphal
Cooperative success rests in part on implicit communication. Partners converge on conventions: regularities that give an action meaning beyond its direct effect. Yet conventions are usually observed only indirectly: a score drop when partners are swapped shows that a convention existed, but not what it was or where it was used. In the card game Hanabi, the exact probability that a play fails can be computed from the hints received and the cards in view. Players who use conventions succeed more often than this prediction allows; the resulting margin is the convention gap. Off-belief learning provides a hierarchy of
agents (levels 1-5) whose first level uses no conventions by construction and whose later levels use progressively more. For humans and these
agents on one scale, we ask what the conventions are and where they are used. On what: in two independently collected corpora of human games, humans show a larger gap than every
agent level. On the conventions both populations use, however, humans do no better than level 2. The human advantage is concentrated in a situation that no
agent level learns to use: hints that point to a card other than the ones they touch. On where: conventions depend on what the player already knows about the card, not on how recently it was hinted, and they are used most in the mid-game; humans use them earlier and later in the game than the
agents. In interventions on the
agents, blocking a single high-gap situation costs little, because meaning reroutes to other situations, but blocking several together costs far more than the sum of the single costs. Thus, the convention gap turns conventions from an inferred cause of score loss into a measurable, comparable, and causally testable object.
Read the full abstract ▾
Artificial Intelligence
Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study
Isabel Jones, Matthew Edwards and Janina A. Hoffmann
Large
Language Models (LLMs) are increasingly deployed in multi-
agent environments that produce unanimous agreement, both in intended interactions that need coordination and in unintended emergent interactions. Consequently, understanding how LLMs with competing incentives reach consensus is crucial to better interpret and design LLM collectives. We introduce an evaluation, based on the Network Consensus game, to assess the mechanisms driving this process. Here, a network of
agents must agree on one of two colours to receive a payoff, but
agents are offered different payoffs for the agreed colour. We vary components such as communication access and network structure and assess six open-source models and one closed model (ChatGPT 5.6-sol). We find that open-source LLMs reach consensus less often than humans in comparable experiments and only if messages are communicated across the global network. ChatGPT 5.6-sol, however, is able to reach consensus with communication limited to its neighbours and with no communication ability. This is achieved through lower stubbornness and higher majority-following tendencies, and by providing more information about its neighbours' states in messages. However, situating
agents in a highly clustered graph reduces consensus, regardless of these traits, demonstrating that both system design and individual model traits drive collective outcomes.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds
Caixin Kang, Ke Wang, Yuchuan Wu et al. (+5 more)
Effective social interaction requires more than fluent responses:
agents must reach plans that fit the participants' needs and circumstances. Existing evaluations of social
agents mainly rely on LLM judges that rate the conversation, without checking the resulting plan. We introduce HICO-Bench, a
benchmark of 180 everyday coordination scenarios, such as resource allocation, scheduling, and authorization, in which two
agents with private information converse and then independently record the plan they agreed on. A program checks Feasibility (the two records match and satisfy both participants' private conditions), and an LLM judge checks Integrity (the conversation supports the recorded plan); passing both is a Verified Success. Each scenario has two versions that share the same public context but differ in one private condition, so the correct plan differs between them. Across thirteen LLMs, 36% of conversations rated highly for goal completion fail Verified Success. To learn from outcomes that arrive only at the end of a conversation, we propose Paired-Credit Policy Optimization (PaCPO). After each turn, PaCPO asks the
agent for a tentative plan, checks it in both versions of the scenario, and rewards the turns after which the tentative plans fit their respective versions. PaCPO raises Qwen3-8B's Verified Success from 20.4% to 28.3%.
Read the full abstract ▾
All authors Caixin Kang, Ke Wang, Yuchuan Wu, Xiaoqiang Wang, Ling Li, Bingxue Qiu, Yongbin Li, Yoichi Sato
Artificial Intelligence · Cross-listed from cs.SE
SWE-Stream: Scaling Executable Training Data for SWE Agents via Progressive Construction and Elastic Streaming
Chengyang Liu, Wen Yang, Shifeng Chen et al. (+5 more)
Software engineering
agents have advanced rapidly, but scaling their training remains constrained by the limited availability of high-quality executable tasks. Development histories contain valuable signals for constructing such tasks, yet they often require substantial reconstruction due to incomplete environments, missing tests, and inconsistent artifacts. We present SWE-Stream, a system for scaling executable SWE training data through progressive construction and elastic streaming execution. SWE-Stream progressively validates and refines task candidates, reusing existing artifacts when possible and applying targeted completion only when necessary. An elastic scheduler asynchronously coordinates construction stages under shared resource constraints, adapting execution budgets based on observed workload demands. Using development histories from 10,728 repositories across six programming
languages, SWE-Stream constructs 80,874 validated tasks and collects 118,688 successful trajectories. Fine-tuning Qwen3-30B-A3B-Thinking-2507 on these trajectories achieves resolve rates of 69.0%, 49.3%, and 45.42% on SWE-bench Verified, Multilingual, and Pro, respectively. In a controlled comparison with approximately matched training budgets, SWE-Stream achieves a 45.6% resolve rate on SWE-bench Verified, outperforming three existing data sources. Moreover, its streaming construction strategy improves efficiency, delivering $1.45\times$ the accepted-task throughput of batch construction and accepting 13.2% more tasks than fixed-limit streaming.
Read the full abstract ▾
All authors Chengyang Liu, Wen Yang, Shifeng Chen, Delai Qiu, Ji Yang, Junnan Zhu, Yining Wang, Shengping Liu
Artificial Intelligence
HEM-Med: A Hierarchical Agentic Memory Framework for Safe Multi-Center Medication Recommendation
Min Xiao, Jin Li, Shoujin Wang et al. (+3 more)
Medication recommendation (MR) is critical for clinical decision support. While existing MR methods are mostly designed for single-center learning, where only one independent hospital is involved, multi-center MR with hospitals of different scales has become increasingly demanding in practice. In multi center settings, abundant data from large medical centers can improve recommendation performance for low-resource centers. Despite a few preliminary attempts on this task, two critical gaps remain. Gap 1: Existing multi-center methods fail to disentangle transferable cross-hospital clinical experience from hospital-specific prescribing patterns, limiting clinical feasibility and physician acceptance. Gap 2: They commonly overlook multi label safety constraints of MR, including drug-drug interaction risks and local formulary feasibility, which are essential for safe real-world deployment. To address these gaps, we propose HEM-Med, a novel hierarchical agentic
memory framework for multi-center MR. To address Gap 1, we design a disentangled hierarchical
memory module that builds patient-level medication context attribution records and disentangles them into cross hospital experience
memory and hospital-specific residual
memory. To address Gap 2, we introduce a safety-aware medication set generation module, which integrates dual-level experience
retrieval with safety-aware set decoding to produce clinically relevant, safer, and locally adaptive medication recommendations. Experiments on a real-world multi-center eICU
dataset show that HEM-Med outperforms representative transfer learning, LLM-based, and MR baselines in terms of both recommendation accuracy and safety.
Read the full abstract ▾
All authors Min Xiao, Jin Li, Shoujin Wang, Yeguang Zhen, Yishuo Li, Wenpeng Lu
Artificial Intelligence
Value-Guided Repulsion for Offline Reinforcement Learning
Yixue Hao, Jiahao Zhang, Jiesong Lian et al. (+5 more)
Drifting models have emerged as a highly efficient and stable paradigm for policy design in offline reinforcement learning (RL). However, standard drift policies determine repulsion solely from
geometric relationships among generated actions and therefore cannot distinguish them by expected value. To address this limitation, we introduce Value-Guided Repulsion Q-Learning (VRQL) to integrate action value estimates into the particle interaction field. For each state, VRQL combines
geometric kernel distance with a normalized Q value scaling. This mechanism assigns greater repulsive influence to particles with lower values while strictly retaining attraction toward behavior action. Consequently, this value-guided interaction field aligns behavior regularization with policy improvement without adding an auxiliary behavior model stage. On 50 OGBench task configurations and 6 D4RL AntMaze configurations, VRQL demonstrates highly competitive performance against established baselines, suggesting that value-guided particle interaction is an effective method for offline RL.
Read the full abstract ▾
All authors Yixue Hao, Jiahao Zhang, Jiesong Lian, Xufei Guo, Xumeng Guo, Wenhui Zhang, Long Hu, Rui Wang
Artificial Intelligence · Cross-listed from cs.CL
Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs
Davide Baldelli, Robyn Kozierok, Mirko Bronzi et al. (+1 more)
Probes on activation vectors are widely used to detect misbehavior in
language models, but their success relies on the assumption that models cannot control what their activations encode. Humans can learn, through neurofeedback, to control the firing of even single
neurons, and recent studies investigated whether this ability extends to
language models, reaching conflicting conclusions. No experimental design so far disentangles competence at the task from genuine access to one's own activations. We fill this gap with in-context neurofeedback sessions, in which a "subject" model is asked to maximize (or minimize) a score derived from its activations, shown back to it at every turn. In our first experiment, the sentences are generated either by the subject or by a second "writer" model, while the score always derives from the activations of the subject. We test ten open-weight models from five families, with sizes from 9B to 32B. Some models move their own scores considerably, but no more than a writer of comparable competence moves them. Is this because the task requires no privileged access, or because the subject controls its activations while processing the sentences, whoever wrote them? To answer this, we prefill a fixed sentence as the subject's reply at every turn and ask it to increase or decrease the score. Three out of ten models show small but consistent control effects. We further ask whether models can control their activations to the point of fooling an activation monitor. We make models repeat sentences a monitor would flag, such as a deceptive answer for a deception monitor, and ask them to raise or lower the monitor's score. No model learns to lower it, but one model can raise it. In-context control of a model's own activations might not yet be a threat to activation monitors, but it exists in a few models and needs to be tracked as models evolve.
Read the full abstract ▾
All authors Davide Baldelli, Robyn Kozierok, Mirko Bronzi, Damiano Fornasiere