Artificial Intelligence
JusticeAxis: Benchmarking Legal Judgment between Rigid Rule Application and Ungrounded Discretion
Zhengkai Tu, Mingda Zhang, Zijia Wang et al. (+2 more)
A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion,
benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference-anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real-world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer-written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element
agents establish the facts and whose judge
agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under
Bayesian credible bounds. Experiments show that failure turns direction with scale: open-weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open-weight backbone to commercial level. Project resources are available at https://github.com/beita6969/JusticeAxis.
Read the full abstract ▾
All authors Zhengkai Tu, Mingda Zhang, Zijia Wang, Xiaoying Tang, Jimmy Huang
Artificial Intelligence · Cross-listed from cs.LG
Semifactual Credit-Augmented Policy Optimization
Junshu Pan, Zhizhang Fu, Shulin Huang et al. (+5 more)
Reinforcement learning with verifiable rewards (RLVR) has improved the
reasoning capabilities of large
language models (LLMs), yet their predictions remain sensitive to task-irrelevant
prompt features. We investigate this sensitivity through semifactual
prompt interventions that preserve the underlying problem and its answer. Our analysis reveals substantial variation in
token-level sensitivity and shows that suppressing high-drift
token candidates during decoding improves
reasoning accuracy without updating model weights. These findings highlight a limitation of Group Relative Policy Optimization (GRPO), which assigns the same outcome-derived advantage to every response
token and may reinforce potential spurious dependence alongside useful
reasoning. Motivated by this observation, we introduce Semifactual Credit-Augmented Policy Optimization (SCAPO), a causally inspired variant of GRPO that incorporates semifactual stability into
token-level credit assignment. SCAPO measures
token probability drift for fixed responses under semifactual interventions and uses normalized stability scores to reduce advantages for relatively unstable
tokens during early training, while granting no additional credit for stability alone. On Qwen3-4B-Base and Qwen3-1.7B-Base, SCAPO improves AIME 2024-2026 accuracy over GRPO by 5.63 and 4.17 percentage points, respectively. At both model scales, SCAPO achieves the best results on most evaluated mathematics
benchmarks and all evaluated out-of-distribution
benchmarks among the compared methods. These results suggest that semifactual stability provides an effective training signal for improving
reasoning and generalization through finer-grained credit assignment in RLVR. The code is available at https://github.com/DtYXs/SCAPO.
Read the full abstract ▾
All authors Junshu Pan, Zhizhang Fu, Shulin Huang, Yiran Ding, Zifan Cheng, Wenqi Shao, Qiaosheng Zhang, Yue Zhang
Artificial Intelligence · Cross-listed from cs.CV
ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing
Xinghao Chen, Xiangbo Gao, Jiongze Yu et al. (+2 more)
Recent
video generation is increasingly realistic and controllable, yet
video editing remains less developed, particularly for precise local edits that must preserve the original scene dynamics.
Video scene text editing replaces text on scene surfaces, such as storefront signs, whiteboards, and product labels, while preserving the surrounding content, motion, and camera dynamics. Although scene text editing is well studied for images,
video scene text editing that achieves high visual quality, temporal consistency, and edit locality remains underexplored. Existing resources offer limited paired real-
video data, and general
video-editing metrics do not directly measure whether the requested text remains correct over time. We introduce ViTeX-Bench, a
benchmark suite comprising ViTeX-
Dataset and a three-axis evaluation protocol. The
dataset contains 387 real-world 720p
videos with text-region masks and editing instructions: 230 provide reviewed, pipeline-generated paired edits for training, and 157 form a frozen evaluation split. The protocol evaluates text correctness, visual and temporal quality, and edit locality through 13 metrics, with one primary metric per axis and a Pareto comparison of their trade-offs. OCR calibration, human evaluation, and annotation-sensitivity analyses support the interpretation of these scores. Across eight baselines from four editing families, accurate text, temporal stability, and scene preservation remain difficult to achieve together. We also release ViTeX-Edit-14B, an open-source reference editor fine-tuned on the paired training split with motion-aligned glyph-
video conditioning. It achieves CharAcc 0.688, the highest mean among the evaluated
video-native editors, and the lowest comparable text-crop Warp among raw editor outputs. ViTeX-Bench provides a reproducible foundation for studying these trade-offs in
video scene text editing.
Read the full abstract ▾
All authors Xinghao Chen, Xiangbo Gao, Jiongze Yu, Yuheng Wu, Zhengzhong Tu
Artificial Intelligence
Turbo Harness: Instance-Adaptive Harness Optimization
Tunyu Zhang, Hao Wang, Kai Xu et al. (+1 more)
Automating the search for effective harnesses is an important step toward enabling
agents to recursively self-improve. Existing harness optimizations typically produce a single global harness that is applied uniformly across task instances. However, a harness that works well on average may not be optimal for every instance. We introduce Turbo Harness, a framework that can adapt a globally optimized harness to each instance by reusing information generated during the original optimization process. Specifically, Turbo Harness recycles artifacts produced during a completed global harness optimization run, and summarizes them into a structured playbook. We train a harness editor to leverage this prior optimization experience to generate instance-specific patches to the global harness. At inference time, the editor uses the instance and the playbook to construct a tailored harness in which the execution model operates. Through numerical experiments, we show that Turbo Harness consistently outperforms existing harness optimization baselines across seven
benchmarks spanning interactive
agent tasks, software engineering, and long-horizon terminal tasks.
Read the full abstract ▾
All authors Tunyu Zhang, Hao Wang, Kai Xu, Dimitris N. Metaxas
Artificial Intelligence
WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents
Ziyan Jiang, Jingbo Yang, Jiabao Ji et al. (+5 more)
As interactive 3D worlds are increasingly used to study intelligent behavior, it becomes important to develop efficient pipelines for identifying anomalies in these simulated environments, such as floating objects, traversable walls, or objects inconsistent with the surrounding scene.
Multimodal AI systems, including
vision-
language models (VLMs) and
vision-
language-action models (VLAs), have shown potential for automating this task. However, 3D world auditing is complex, requiring the close coupling of two distinct capabilities: action, to navigate the 3D world and search for anomalies systematically and efficiently; and visual
reasoning, to understand the environment and identify anomalies from
multimodal observations. It remains largely unexplored whether
multimodal agents can effectively couple these two capabilities, using visual
reasoning to identify potential anomalies while taking actions to validate them. In this paper, we introduce WorldAuditBench, a
benchmark for 3D world auditing comprising 213 anomaly tasks across 13 environments built with Unreal Engine 5 and Three.js, spanning five anomaly families. We evaluate five frontier models under a fixed exploration budget using two auditing paradigms: VLA-based exploration followed by VLM-based anomaly identification, and an end-to-end VLM
agent in which visual
reasoning directly guides action selection. Across the evaluated models and two paradigms, success rates range from 6.6% to 42.3%, substantially below human performance (83.4%). Through the task of world auditing, WorldAuditBench provides a testbed for studying how
multimodal agents couple action and visual
reasoning in interactive 3D environments, while highlighting current limitations in their ability to gather and interpret evidence during exploration.
Read the full abstract ▾
All authors Ziyan Jiang, Jingbo Yang, Jiabao Ji, Yujian Liu, Qiucheng Wu, Tommi Jaakkola, Yang Zhang, Shiyu Chang
Artificial Intelligence
Cogentic: Multi-Agent Orchestration for Automated Proof Discovery
Yang Cai, Vineet Gupta, Yanchen Jiang et al. (+4 more)
We present Cogentic, a multi-
agent harness for automated proof discovery on open research problems. While frontier
language models can generate strong mathematical ideas in a single shot, single-shot generation is often insufficient for open problems that require exploring multiple competing
conjectures, overcoming subtle technical obstructions, and retaining intermediate progress over a long horizon. Cogentic addresses these challenges through an iterative prove-verify loop in which an orchestrator allocates a population of independent provers across distinct proof directions, subjects their output to
adversarial verification by several specialized components, and promotes confirmed intermediate results into a persistent verified ledger that later rounds build on. The harness is designed to be able to solve research-level math and theoretical computer science problems. Using either Gemini 3.1 Pro or an early version of Gemini 4 Argon as the base model, Cogentic produced novel results on open problems across online learning, auction theory, and mechanism design. Each result was independently verified by domain experts and is developed in full in companion papers. We list these results, and new ones as they are verified, at https://sites.google.com/view/cogentic .
Read the full abstract ▾
All authors Yang Cai, Vineet Gupta, Yanchen Jiang, Christopher Liaw, Aranyak Mehta, Grigoris Velegkas, Di Wang
Artificial Intelligence · Cross-listed from cs.CV
MatLoom: Layered Text-to-Material Generation in a Compact Program Space
Anson Y. Lam, Shuqing Li and Michael R. Lyu
Material generation should produce not only an appearance, but also the rules that construct it. We introduce MatLoom, a compact, layer-oriented
language for text-to-material generation with pretrained
language models. Each program composes alpha-masked layers whose shared spatial expressions define coverage and physically based rendering (PBR) channels, making dependencies between patterns, color, and relief explicit. A standalone interpreter evaluates the program into material maps, while the source retains named fields and layer parameters for subsequent authoring. Without task-specific fine-tuning, our pipeline uses parser-guided repair and preview-based critique to revise material designs, then searches noise seeds while keeping each candidate's remaining source fixed. On a curated
benchmark of 141
prompts evaluated with six backbones, our best-performing configuration achieves higher mean scores than three
diffusion baselines on all four flat-layout
prompt-
alignment metrics. Its initial programs already exceed all three baselines on mean BLIPScore, before critique or seed search. Retained programs have a median length of 21 lines when pooled across backbones. In a blind four-way comparison involving 30 participants and 20
prompts, our renders receive 59.2% of choices, compared with 19.3% for the most-preferred baseline. Compact executable programs thus offer a way to generate
prompt-aligned materials while retaining their construction as part of the asset.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Scaling Laws for Looped Mixture of Experts
Yanbei Chen, Anirudh Goyal and Raghuraman Krishnamoorthi
Looped
transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacity at fixed active compute. Yet existing scaling laws model recurrence or sparsity in isolation. In this work, we introduce Loop Scaling Laws, the first scaling law to jointly model recurrence and sparsity alongside model size and data. At its core is a bounded, sparsity-conditional recurrence mapping that characterizes the effective-parameter gain from looping and how sparsity raises this gain. The laws predict the held-out loss of looped models more accurately than prior alternatives, and recover the standard dense and MoE scaling laws as special cases. Beyond prediction, the fitted laws provide a principled foundation for designing looped MoE models under compute and
memory constraints. Downstream evaluations further demonstrate the complementary benefits of the two axes: sparsity delivers ~3x active-parameter efficiency, recurrence yields ~2x total-parameter efficiency on
reasoning, and joint scaling further advances the performance frontier. As a practical extension, we show these gains hold at trillion-
token scale: at matched training compute, a looped MoE with law-derived recurrence matches a ~2x larger non-looped MoE on the
reasoning benchmarks, while enabling test-time scaling through recurrence.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents
Haoyuan Deng, Jiebin Liu, Tengxiao Zhang et al. (+3 more)
Pretrained
robot policies provide useful action priors, but long-horizon manipulation still requires coordination between semantic
reasoning and physical execution. Semantic
reasoning operates at a coarser timescale than physical interaction, while episode-level failures provide limited guidance on which system component should be revised. We propose DynaHarness, a dynamic physical harness that couples semantic
reasoning with physical governance through a shared execution contract and turns failure evidence into validated capability revisions. To be more specific, the slow
brain proposes capabilities and symbolic arguments, while the fast
brain grounds and monitors commands, refuses unresolved actions, substitutes capabilities, and requests replans when needed. The physical execution contract bounds each accepted command and records execution evidence across analytic skills, recovery skills, and the frozen VLA. Failure attribution localizes faults in these records and directs targeted revisions of reusable capabilities or execution mechanisms. Paired regression checks govern admission or rejection, closing the self-
evolution loop. On LIBERO-Pro, DynaHarness achieves 75.2% on 800 newly sampled initial states, compared with 17.5% for the frozen policy. With the same capability library, full dynamic execution reaches 74.0% versus 63.9% under nominal one-step replanning. This demonstrates the value of DynaHarness as a dynamic physical harness that governs how existing capabilities are grounded, monitored, and coordinated during execution. Our project page is at https://denghaoyuan123.github.io/Dynaharness_page/.
Read the full abstract ▾
All authors Haoyuan Deng, Jiebin Liu, Tengxiao Zhang, Langning Yan, Hongye Cao, Ziwei Wang
Artificial Intelligence
How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
Kirill Brilliantov, Alejandro Hernández-Cano and Emmanuel Abbé
Recent autonomous machine learning engineering (MLE)
agents have made significant progress on public leaderboards. Often motivated by progress stagnation over long-horizon cycles and limited Large
Language Model (LLM) primitives, modern MLE
agents are deployed on top of increasingly elaborate machinery: multi-
agent orchestrators, dedicated
retrieval subagents, and more. While such harnesses expand, the use of more primitive but improved coding
agents - where LLMs have direct access to the execution environment through read, write, and bash primitives - has received little
attention in the field. In this paper we find that, under an equal time budget and the same frontier LLM backbone, open-source state-of-the-art harnesses provide no advantages over a single session of a minimal-harness coding
agent baseline, pointing to the backbone as the primary driver for performance. Via a series of large-scale systematic ablation studies, we argue that the machinery layers become redundant in the coding
agent setting. We conclude that the effort spent elaborating hand-crafted harnesses around strong models yields poor returns for current MLE
benchmarks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.IT
CAS II: Orbits as Models: Kolmogorov's Structure Function under Symmetry
Romie Banerjee
In algorithmic statistics a string x is explained by a finite set containing it, and Kolmogorov's structure function records the smallest such model at each level of complexity. Strong models, those computable from the data by a total
algorithm, are essentially the
cells of simple partitions, so a partition of {0,1}^n can be read as a hypothesis and the
cell containing x as the model it assigns. We develop algorithmic statistics over symmetric partitions, the orbit partitions of groups acting on strings. The Galois connection between subgroups and partitions gives each ambient group G a
lattice of symmetric partitions with canonical certificates, joins and meets. A set is a
cell of a symmetric partition exactly when its setwise stabiliser in G acts transitively on it, so the resulting structure function is Kolmogorov's restricted to these G-homogeneous sets. With a symmetric sophistication, it measures which part of the regularity of x is symmetric. Under the full symmetric group,
cells recover all models and
cells of cheap partitions recover exactly the strong models, so normal and strange strings are characterised by
symmetry. For GL(n,2) the homogeneous sets are the linearly homogeneous ones, whose XOR dependencies look the same from every point. The GL structure function of a nonzero x lies in a band between C(x) - alpha and n - alpha, and both edges are attained: some
stochastic normal strings have simple structure invisible to linear
symmetry, with sophistication near 0 but GL-sophistication near C(x). We coordinatise permutation groups by a Burnside ring element (type) and a permutation (placement). In these coordinates a linear hypothesis is determined by its type up to n^2 bits, and any space of
symmetry hypotheses small enough to search is small enough to miss simple structure. This sets up learned search over orbit models, the subject of later papers in the series.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning
Tyler Skow, Shravan Chaudhari, Rama Chellappa et al. (+1 more)
Unlearning a fact in one
language does not guarantee its removal in others as changing the query or even the requested answer
language can reopen seemingly forgotten knowledge – a cross-lingual loophole. The most straightforward solution to this challenge – unlearning in all
languages – is neither scalable nor desirable as it amplifies damage to unrelated model capabilities. We introduce the task of
language budgeted multilingual unlearning where the goal is to select a subset of
languages that maximizes cross-lingual erasure. To study this task we introduce the Cross-Lingual Unlearning Tensor, an unlearning
benchmark that spans 174
language–script pairs and 25 atomic paraphrase types to examine when forgetting generalizes across linguistic expressions of the same knowledge. We further propose COVER, which selects source
languages to maximize predicted COVERage of
languages receiving no forget supervision, enabling unlearning on a
language budget. Surprisingly, we find naively selecting strong individual sources does not reliably compose into strong source sets motivating our development of COVER. At deployment COVER only requires benign calibration data and access to the frozen model. Across three model families and two disjoint forget sets, COVER reduces mean held-out residual access by 7.8–27.3% relative to uniform source selection. We find these gains extend beyond synthetic
benchmarks to real news documents in low-resource
language settings using human translated data from the Low Resource
Languages for Emergent Incidents (LORELEI) corpus.
Read the full abstract ▾
All authors Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
Artificial Intelligence
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
Yinghui He, Yapei Chang, Khushi Bhardwaj et al. (+4 more)
On-policy
distillation (OPD) is a promising approach for training
language agents, providing dense teacher supervision on student-generated trajectories. However, in multi-turn interaction, an incorrect action changes the states the student encounters later, so errors compound across turns. In preliminary experiments across three Qwen3 models (8B to 235B), we find that more than half of the failed rollouts contain a pivotal mistake, an action that moves the
agent farther from completing the task, and this mistake typically occurs early. These pivotal mistakes often remain recoverable: guiding the model for only a few turns after the pivotal turn can restore task success. We therefore propose PivotOPD, an on-policy
distillation framework that jointly trains the student to prevent pivotal mistakes and to recover from the states they create. At each pivotal mistake, a teacher model provides a gold action and then names a recovery action at each of the next few turns. Preventive
distillation uses the gold action with reverse KL to steer the student away from the pivotal mistake, while recovery
distillation uses the recovery actions with forward KL to transfer recovery behaviors that the student rarely samples. Against 13 baselines on ALFWorld, WebShop, and Search-based QA, PivotOPD achieves the strongest average performance for both Qwen3-1.7B and Qwen3-8B students, improving over the strongest baseline on ALFWorld by +5.5% with the 1.7B student. The gains also transfer to another model family on the software engineering domain, where PivotOPD raises the resolve rate of a Nemotron-3.5 student on SWE-Bench Verified by +3.2%. Project page: https://research.nvidia.com/labs/lpr/pivotopd/
Read the full abstract ▾
All authors Yinghui He, Yapei Chang, Khushi Bhardwaj, Daniele Molinari, Tugrul Konuk, Jan Kautz, Ali Hatamizadeh
Artificial Intelligence · Cross-listed from cs.LG
cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
Pranjal Aggarwal, Lawrence Keunho Jang, Sean Welleck et al. (+3 more)
Computer use
agents (CUAs), which use graphical user interfaces (GUIs) to complete tasks on a computer, have recently surpassed human performance on many standard
benchmarks, including difficult long-horizon tasks. Their capabilities are undoubtedly impressive, however, a key barrier to the widespread adoption and deployment of CUAs remains their speed and cost. Progress towards faster yet capable CUAs requires reliable evaluation of their speed, but many CUA
benchmarks currently face a reproducibility crisis.
Benchmarks are based on complex infrastructure with varying machine and container configurations that confound the evaluation of the execution speed of CUAs. Towards addressing this gap, we propose cua-speedrun, which introduces standardized infrastructure and task sets, with a focus on evaluating the speed and efficiency of CUAs. cua-speedrun uses a uniform virtual machine setup and execution pipeline, along with a common
agent interface that enables single-
agent implementations to operate seamlessly across different
benchmarks. Across four different CUA
benchmarks, we evaluate how
reasoning effort,
agent harnesses, and environment latency affect performance, speed, and cost. We find no single model family is optimal for all three; none of the open-weight models are on the frontier, and also, unintuitively, for some models increasing the
reasoning effort can speed up task completion, while faster environment input-output can slow down overall task completion time. We also demonstrate that we can effectively reduce the evaluation task set of most CUA
benchmarks without degrading overall statistical power, allowing for more efficient benchmarking and comparison. We believe cua-speedrun will enable structured progress towards fast, efficient CUAs, unlocking new real-world use cases and applications. All code, infrastructure, and analysis are available at https://cuaspeedrun.com.
Read the full abstract ▾
All authors Pranjal Aggarwal, Lawrence Keunho Jang, Sean Welleck, Daniel Fried, Ruslan Salakhutdinov, Jing Yu Koh
Artificial Intelligence
Belief-Aware Multi-Agent Path Finding under Map Uncertainty
Viraj Parimi, Shao-Hung Chan, Han Zhang et al. (+2 more)
Multi-
Agent Path Finding (MAPF) aims to find collision-free paths for multiple
agents in a shared environment. Classical MAPF assumes that all static obstacles are known in advance, but real-world environments can change unexpectedly due to fallen objects, spills, or other local disturbances. When such changes are spatially correlated, an observation can inform traversability estimates beyond the observed location. Prior approaches address uncertainty in traversability through contingent plans or replanning based on direct observations, but do not leverage this spatial dependence to infer the traversability of nearby unobserved locations. As a result, they cannot use one observation to anticipate nearby unobserved obstacles that may cause costly rerouting later. We focus on Belief-Aware MAPF, where map discrepancies are fixed during execution but initially unknown, and observations can be informative beyond the observed location. We propose Multi-
Agent Gaussian belief Inference for Coordination (MAGIC), a framework that updates a shared belief about traversability online based on
agents' observations. MAGIC uses a Gaussian Markov Random Field and Gaussian Belief Propagation to approximately infer traversability and construct detour-aware costs for standard MAPF planners. Our experiments on MAPF
benchmarks show that MAGIC reduces the executed sum of costs compared to existing approaches on 96.3% of instances, across several planner families and teams of up to 800
agents, demonstrating its applicability to large-scale MAPF problems.
Read the full abstract ▾
All authors Viraj Parimi, Shao-Hung Chan, Han Zhang, Jingkai Chen, Brian Williams
Artificial Intelligence · Cross-listed from cs.CV
ComputerSD: Online Self-Distillation from Real-Time Feedback for Computer-Use Agents
Yong Du, Tongbo Chen, Zhengxi Lu et al. (+5 more)
Online training enables computer-use
agents (CUAs) to improve through interaction with executable environments. However, existing methods primarily rely on sparse outcome rewards, which provide no supervision for intermediate actions. On-policy self-
distillation (OPSD) offers
token-level learning signals through privileged rescoring, but directly applying it to CUA online training presents two challenges: fixed guidance may become misaligned with the student's current state, and guidance-induced probability shifts may conflict with step-level correctness. We introduce ComputerSD, an online self-
distillation method for CUAs that converts real-time feedback from executed GUI transitions into guidance for policy learning. A fine-tuned GUI analyzer produces guidance and a step-level value score after each action; the guidance provides privileged context, while the score regulates the resulting OPSD signals. ComputerSD jointly optimizes
token-level OPSD and trajectory-level GRPO in a fully asynchronous training framework. On OSWorld-Verified, ComputerSD outperforms outcome-only GRPO by 1.9 and 4.1 percentage points on the general-purpose Qwen3-VL-8B-Thinking and specialized EvoCUA-8B backbones, respectively. Evaluation in out-of-distribution settings further supports the generalizability of ComputerSD. These results demonstrate the effectiveness of learning from real-time feedback through online self-
distillation for CUAs.
Read the full abstract ▾
All authors Yong Du, Tongbo Chen, Zhengxi Lu, Yizhou Liu, Bofan Chen, Tao Jiang, Wenhao Xu, Yongliang Shen
Artificial Intelligence · Cross-listed from cs.CV
EviRover: Reinforcing Agentic Perception Beyond a Glance
Kaixuan Fan, Kaituo Feng, Tianshuo Peng et al. (+4 more)
Visual perception is conventionally formulated as a one-shot prediction from a single glance at the image, under the assumption that the image content and the model's parametric knowledge suffice to resolve the query. This assumption often fails in real-world scenarios that hinge on fine-grained visual details or require knowledge-intensive and up-to-date information. We term such cases perception under insufficient evidence and formulate perception as an agentic process that can obtain information beyond a single glance. To address the absence of data for this setting, we design two dedicated data generation pipelines, yielding EviRover-SFT-5K and EviRover-RL-12K for training. We further construct EviLens, a human-verified
benchmark comprising 688 instances across five perception categories. Building on these data, we present EviRover, to our knowledge the first perception
agent explicitly trained to resolve perceptual queries through interaction, using supervised fine-tuning followed by agentic reinforcement learning. Experiments show that the 4B EviRover outperforms its backbone by 30 points on average on EviLens, reaching performance comparable to advanced proprietary models. The gains transfer beyond EviLens to WebEyes, conventional perception
benchmarks, and general
multimodal benchmarks, including a 15-point improvement on BrowseComp-VL. All code, models, and data are released.
Read the full abstract ▾
All authors Kaixuan Fan, Kaituo Feng, Tianshuo Peng, Yilei Jiang, Manyuan Zhang, Junke Wang, Xiangyu Yue
Artificial Intelligence · Cross-listed from cs.LG
PhantomEnvironments: Training LLM Agents in Fictional Worlds
Anmol Kabra, Swathi Saravana Selvam, Albert Gong et al. (+5 more)
Training LLM
agents with reinforcement learning (RL) is bottlenecked by environments, which must provide verifiable rewards, support long-horizon interaction, and scale cheaply. Existing approaches rely on costly human-curated data or on LLM-generated environments that risk
hallucinations and
benchmark contamination. We show that LLMs can instead be trained into capable search
agents using synthetic environments generated entirely by rules, whose generation requires no LLM and has zero marginal cost. We build PhantomEnvironments, multi-turn RL environments from fictional worlds, where
agents must search a corpus of templated articles to answer multi-hop questions. Despite sharing no facts with the real world, these strikingly simple environments yield
agents that transfer to real-world multi-hop search
benchmarks, often outperforming real-world training data on newer
benchmarks. Trained
agents generalize to unseen fictional universes, and Qwen models learn to scale their search budget roughly linearly with question difficulty, suggesting emergent search scaling from environment interaction alone. Ablating environment complexity reveals that hop count drives transfer more than constraints or comparisons: even the simplest rule-generated environments are a surprisingly effective, free resource for training generalizable LLM
agents.
Read the full abstract ▾
All authors Anmol Kabra, Swathi Saravana Selvam, Albert Gong, Chao Wan, Christian Belardi, Dongyoung Go, Katie Z. Luo, Kilian Q. Weinberger
Artificial Intelligence · Cross-listed from cs.CV
Learning Skills from Historical Action Trajectories: Action Experience Dictionary for World Action Models
Qi Lyu, Jiahua Dong, Hao Shen et al. (+9 more)
World Action Models (WAMs) couple visual dynamics prediction with action generation, yet they do not explicitly support the reuse of action experience across manipulation tasks. Furthermore, existing WAMs struggle to capture underlying cross-task semantic relationships that could guide target action prediction, as redundant background elements interfere with the extraction of key visual information. To address these challenges, we develop a novel Action Experience Dictionary (AED) that encodes historical physical action trajectories into shared action
embeddings to support skill reuse and model cross-task relationships. Specifically, we first aggregate historical actions to align with visual observations and retrieve action
embeddings from the AED using a pretrained action tokenizer. Subsequently, we visually condition the pooled
embeddings through cross-
attention and prepend them to noisy action
tokens, providing interaction context and action intent for prediction. To model action-related motion and reduce reliance on irrelevant background cues, we introduce a motion-aware transition loss that supervises visual feature change prediction over random temporal intervals. Experiments on simulation
benchmarks and in real-world cross-embodiment settings verify the effectiveness of our AED. The project code is available at https://github.com/JiahuaDong/AED .
Read the full abstract ▾
All authors Qi Lyu, Jiahua Dong, Hao Shen, Xudong Wang, Hongyuan Yu, Baichen Liu, Henghui Ding, Zhi Han, Nicu Sebe, Ivan Laptev, Fahad Shahbaz Khan, Salman Khan
Artificial Intelligence · Cross-listed from cs.CL
SCB: SpeechConversationBench for Evaluating Multi-Turn Reasoning in Speech-to-Speech Models
Kanpat Vesessook and Saksorn Ruangtanusak
Speech-to-
speech systems must solve tasks whose requirements emerge across conversational turns. We introduce SpeechConversationBench (SCB), a focused evaluation of spoken mathematical
reasoning using 103 sharded GSM8K problems. The framework compares the original problem delivered in one turn (full), its concatenated information shards delivered together (concat), and incremental spoken disclosure across turns (sharded). We report final-answer accuracy for four commercial
speech systems and LEGO, a proprietary
speech pipeline developed internally by the SCBX Innovation Lab team with explicit conversational context management. Relative to concat, sharded accuracy decreases by 5.0-25.3 percentage points across the four commercial systems. LEGO achieves 77.5 percent accuracy in all three conditions, compared with 76.6 percent sharded accuracy for GPT-4o Realtime. The two single-turn baselines distinguish sensitivity to problem reformulation from the additional challenges introduced by incremental spoken interaction.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories
Guangzhi Xiong, Xinyuan Zhang, Xiao Yang et al. (+14 more)
Long-term egocentric
video enables personalized AI assistants to reason about daily life. However, as
video histories grow to hundreds of hours spanning months or years, reprocessing raw clips for every query becomes computationally prohibitive.
Memory systems offer a scalable alternative by compacting
videos into text representations, but often fail on practical
benchmarks: either the
memory does not preserve key evidence, or the retriever fails to locate relevant entries due to
retrieval competition in growing search spaces. To address these challenges, we introduce MemLife, a
multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader. Without training or query-time
video access, MemLife improves over the strongest training-free baseline by 4.6–12.0% across four long-horizon
benchmarks. To further improve
memory quality, we propose MemOpt, a reinforcement learning framework that optimizes the
memory writer to produce faithful, informative, and retrievable
memories. MemOpt consistently improves MemLife by 2.7–5.0% across different
video and question distributions, with gains that generalize across writer and reader backbones and
memory systems.
Read the full abstract ▾
All authors Guangzhi Xiong, Xinyuan Zhang, Xiao Yang, Hyokun Yun, Kai Zhang, Shiun-Zu Kuo, Hyeonjeong Ha, Xilun Chen, Kai Sun, Lucas Liang, Guangqiang Dong, Ejaz Ahmed, Ahmed A Aly, Anuj Kumar, Raffay Hamid, Aidong Zhang, Xin Luna Dong
Artificial Intelligence
Learning from Research: Toward Lifelong Agent Harness Evolution
Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang et al. (+3 more)
Language agents are expected to solve increasingly complex tasks, creating a growing need for continual improvement. One promising approach is to evolve the
agent harness, the software that governs tool use,
memory management, and task execution, while keeping the underlying
language model fixed. Recent methods automate this process by using a meta coding
agent to modify the harness based on execution feedback. However, relying on that
agent's existing knowledge and observed failures can restrict exploration and make adaptation reactive. Inspired by how human experts learn from the research literature for new solutions, we introduce ScholarEvolve, a framework that automatically draws on state-of-the-art research to guide harness
evolution. ScholarEvolve organizes the harness
evolution directions into functional modules and uses topic modeling to identify distinct improvement strategies for each module. It implements these strategies and evaluates their combinations to improve task performance. Moreover, the framework is designed to incorporate new publications over time, allowing research advances to drive proactive lifelong
evolution. Experiments demonstrate improvements on AppWorld and Tau2-Bench. ScholarEvolve raises Qwen3.5-27B task goal completion from 49.6% to 63.6% on AppWorld Challenge, and raises GPT-5.4-mini pass@1 from 72.7% to 81.9% on Tau2-Bench Telecom.
Read the full abstract ▾
All authors Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang, Yaar Harari, Evgeniy Gabrilovich, Shiyu Chang
Artificial Intelligence · Cross-listed from cs.RO
PrefPI: Preference-Guided Steering into Out-of-Distribution Behaviors
Seungeun Rho, Wontaek Kim, Danfei Xu et al. (+1 more)
We present PrefPI (Preference-Guided Policy Iteration), an iterative framework for steering pretrained generative
robot policies using only relative preferences over self-generated trajectories. Unlike prior preference-learning methods that primarily sharpen modes already represented by the policy, we study steering beyond the initial effective support, where desired behaviors are rarely or never observed under the initial policy. Our key idea is to formulate preference learning as preference-conditioned generative modeling: preferred trajectories define a conditional distribution, whose density ratio with the broader behavior prior provides an implicit preference signal amplified by classifier-free guidance (CFG). Repeating this preference-conditioned modeling and guidance step yields a form of preference-guided policy iteration, turning incremental improvements toward previously inaccessible behaviors. Across
diffusion policies and the PI0.5 flow- matching VLA in simulation and the real world, PrefPI produces substantial behavioral shifts with limited feedback. In particular, PrefPI increases object transport height from 10.7 cm to 19.8 cm on real hardware with only 150 preference-labeled trajectories.
Read the full abstract ▾
All authors Seungeun Rho, Wontaek Kim, Danfei Xu, Sehoon Ha
Artificial Intelligence · Cross-listed from cs.LG
Game-Guided Skill Discovery through Self-Play for Playable Agent Control
Seungeun Rho, Jeonghwan Kim, Xue Bin Peng et al. (+1 more)
We present Game-Guided Skill Discovery (GGSD), a framework that uses self-play in games to discover motor skills that are directly playable by humans. Playable skills provide a compact abstraction for controlling embodied
agents through a small set of learned behaviors rather than low-level actions. To be effective, these skills should be semantically distinct, interpretable, and expressive; properties that existing unsupervised skill-discovery methods often fail to achieve simultaneously. GGSD achieves these desiderata by grounding skill discovery in competitive gameplay. A hierarchical
agent competes against its past selves, with a high-level policy selecting from a small discrete skill set and a skill-conditioned low-level policy learning the corresponding behaviors. After training, a human can replace the high-level policy and directly control the
agent through the same discrete skills. Despite the small number of high-level actions, skill transitions give rise to emergent combo behaviors, expanding expressivity beyond individual primitives. Across Ant, Franka-arm, and Unitree G1 environments, we show that GGSD produces human-playable skills that humans can compose to solve unseen tasks, such as Maze and CubePush, without additional training. An interactive demo is available at https://ggsd-demo.github.io.
Read the full abstract ▾
All authors Seungeun Rho, Jeonghwan Kim, Xue Bin Peng, Sehoon Ha
Artificial Intelligence · Cross-listed from cs.RO
Tactile Curiosity Drives Robot Interaction
Klemens Iten, Alexander Proshkin, Bhavya Sukhija et al. (+4 more)
Mastering
robot manipulation skills via reinforcement learning (RL) remains largely sample-inefficient. The most common RL
algorithms rely on random action sampling to discover new strategies, resulting in
agents that allocate most of their training budget to motions in free space, away from the contacts from which manipulation skills emerge. Existing intrinsic motivation methods based on model disagreement or epistemic uncertainty improve on isotropic noise, but they can also reward uncertainty in functionally irrelevant transitions, such as erratic motions in free space. In this work, we argue that tactile feedback provides a natural signal for exploration, and introduce TacEx, a framework that incorporates touch into epistemic uncertainty-driven exploration by decomposing model uncertainty across sensory modalities and directing curiosity toward the tactile channel. By anchoring curiosity to the sense of touch, TacEx drives the
robot to discover complex contact dynamics, learning to manipulate and grasp objects without task rewards or expert demonstrations during exploration. The interaction-dense
dataset collected through this tactile-driven curiosity supports offline learning of downstream pick-and-place policies without additional environment interaction. We further use tactile-driven exploration to post-train
vision-
language-action (VLA) models. Although the VLAs are initially pre-trained without tactile feedback, post-training with TacEx substantially improves downstream performance while remaining highly sample-efficient.
Read the full abstract ▾
All authors Klemens Iten, Alexander Proshkin, Bhavya Sukhija, Stelian Coros, Andreas Krause, Pieter Abbeel, Carmelo Sferrazza
Artificial Intelligence · Cross-listed from cs.CL
On the (In)effectiveness of AMR Augmentation for Large Language Models
Hoa Quynh Nhung Nguyen, Jacopo Staiano and Michael Sullivan
While Abstract Meaning Representation (AMR) has historically improved performance on a range of NLP tasks, the benefit—or lack thereof—of AMR augmentation for modern LLMs is thus far unclear. In this paper, we attempt to reproduce recent work that reported substantial downstream gains from AMR augmentation, finding that these are likely due to specific choices in the experimental settings used: using a consistent and unified protocol for hyperparameter selection, we observe that text-only baselines consistently match or exceed the performance of AMR-augmented models. To investigate this null result, we introduce a perplexity-based probe measuring the degree to which AMR provides an LLM with supplemental relational knowledge not already available to the model. We find that AMR augmentation does not help LLMs improve their understanding of relational content in the sentence, indicating that augmenting these models with AMR offers no clear benefit on downstream tasks.
Read the full abstract ▾
Artificial Intelligence
Unlearnable, or Unmeasured? On the Reliability of Difficulty Labels in RLVR
Chandak Chakma, Syed Nazmus Sakib, Nafiul Haque et al. (+1 more)
Reinforcement learning with verifiable rewards (RLVR) has become an important approach for improving
reasoning during post-training. Recent work suggests that some difficult
prompts remain resistant to learning even when they occasionally produce correct solutions. We revisit this unlearnability phenomenon and find that the affected
prompts do improve, at roughly one third of the learnable rate, while the difficulty-defined set used to study them is much less reproducible than expected. These difficulty labels are estimated from a limited number of sampled responses. Combining them across seeds can further change which
prompts are selected instead of simply reducing measurement noise. We develop a sampling-based framework for quantifying this instability and determining how much evaluation is required for difficulty assignments to reproduce reliably. We also revisit the gradient-similarity evidence proposed to explain unlearnability and show that part of the observed separation arises because difficult
prompts provide fewer correct rollouts from which their gradients can be estimated. Matching this sample count weakens the gradient difference but does not remove it. Overall, the slow-learning phenomenon survives our reanalysis, while both the
prompts used to define it and the evidence used to explain it require more careful measurement.
Read the full abstract ▾
All authors Chandak Chakma, Syed Nazmus Sakib, Nafiul Haque, Shifat E. Arman
Artificial Intelligence
Agent Error Dataset: Scaling 50,000 Error–Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
Kunlun Zhu, Xuyan Ye, Yibo Li et al. (+3 more)
An unsuccessful LLM
agent rollout contains more information than its final reward: the observations available to the
agent, the actions it chose, and the environment's responses. Reusing this experience for learning requires identifying a decision to revise and testing a concrete alternative. We introduce the
Agent Error
Dataset (AED), comprising 50,228 error-diagnosis pairs from 9,961 source tasks across 33 environments, 19 harness families, and 23 policy models in text-based
agent systems. We retain source traces and execution metadata to support cross-setting failure analysis and re-diagnosis without repeating the original rollout. Our five-stage Agentic Error-to-Training (AET) pipeline collects natural failures, generates diagnoses and proposed corrections, and checks them against recorded evidence. Where replay is supported, we compare corrections with original-action retries from the same checkpoint under matched execution settings. We then construct separate training views for diagnosis and actor recovery. Across 3,062 matched replay pairs, first-proposal corrections raise verifier pass rates from 18.4% to 51.1%, a gain of 32.7 percentage points. Using a separately frozen diagnosis release, full-diagnosis fine-tuning on 1,656 source tasks raises Qwen3-8B's exact-step agreement with internal teacher labels from 47.2% to 63.6%, averaged over three seeds on a 943-case holdout. The strongest prompted reference in this comparison scores 54.7%, and mean agreement improves at each of four increasing training-set sizes. In a single-seed comparison of actor-training recipes, action-only repair training scores 6.67 percentage points higher on WebShop-lite than success-only training.
Read the full abstract ▾
All authors Kunlun Zhu, Xuyan Ye, Yibo Li, Cheng Qian, Beibin Li, Heng Ji
Artificial Intelligence · Cross-listed from cs.CV
GateSPINE: Gated Cross-View Fusion for Lumbar Spine MRI Report Generation
Hoang Nguyen Van, Cuong Vuong Tuan, Trang Mai Xuan et al. (+3 more)
Automated report generation can ease the burden radiolo gists face when interpreting multi-sequence MRI studies. Unlike CT, MRI examinations comprise multiple sequences and imaging planes, each con tributing complementary diagnostic information. Existing methods en code a study as a single volume and combine multiple acquisitions by fixed rules. Findings visible in only one plane are thus diluted and of ten missed, lowering recall on clinical efficacy metrics, where a missed abnormality is most costly. We propose GateSPINE, a
vision-
language framework that fuses sagittal T1 and T2 volumes with a training-free operator, encodes the fused sagittal and axial volumes with two parallel 3D encoders, and decodes their combined representation into a report. Its core mechanism is a gated cross view fusion module that predicts, per feature channel and
token, how much of each view to admit, so the more informative view dominates at each spatial location. We evaluate GateSPINE on three lumbar MRI
datasets, comprising two public bench marks and a private cohort collected from Phenikaa University Hospital, using both natural
language generation (NLG) and clinical efficacy (CE) metrics. GateSPINE achieves the highest CE F1 through improved re call on all three
datasets; on SPIDER, which lacks an axial sequence, this reflects the sagittal fusion component rather than the gated cross-view mechanism, which is validated on the two cohorts with both imaging planes. GateSPINE also remains competitive on standard NLG metrics.
Read the full abstract ▾
All authors Hoang Nguyen Van, Cuong Vuong Tuan, Trang Mai Xuan, Bien Tran Van, Nam Tran Van, Thien Van Luong
Artificial Intelligence
PTNO: Training Neural Operators with Noisy Monte Carlo Estimates for Particle Transport Problems
Yubo Cao, Xi Deng, Mengqi Xia et al. (+3 more)
Particle transport under multiple scattering is central to radiative transfer and
plasma physics, yet high-fidelity Monte Carlo (MC) simulations must trace prohibitively many particles. Learning-based surrogates can amortize this cost, but typically train on expensive, well-converged MC solutions. We propose the Particle Transport Neural Operator (PTNO), a neural operator that learns particle transport surrogates directly from noisy, low-cost MC labels. Such labels pose two challenges: (1) high variance, which destabilizes standard supervised learning, and (2) a high dynamic range (HDR) spanning many orders of magnitude. For the first, we learn the solution operator from noisy labels of many configurations, amortizing MC cost and generalizing to unseen configurations. Because MC labels are unbiased, we show that the squared loss on them shares its minimizer with the loss on converged solutions, and our budget-allocation study over training scenes $M$, MC samples per render $N$, and independent renders per scene $K$ shows that many noisy scenes beat fewer converged ones. For the second, a nonlinear transform such as the logarithm biases noisy supervision. Instead, PTNO keeps labels in physical space and enforces positivity with a softplus output layer that represents small values effectively. We further train with a pointwise relative $L_2$ loss (PRelL2), the stop-gradient relative loss of HDR denoising and neural rendering, which normalizes each residual by the stop-gradient prediction instead of the noisy label. We demonstrate PTNO on
neutron transport in fusion reactors and radiative transfer in participating media. On the two neutronics tasks, PTNO is $10^4$-$10^5\times$ faster than converged MC on the same CPU and $10^3$-$10^5\times$ cheaper than MC at matched accuracy; on the two radiative-transfer tasks, MC at matched accuracy costs $0.8$-$11\times$ as much as PTNO.
Read the full abstract ▾
All authors Yubo Cao, Xi Deng, Mengqi Xia, Vignesh Gopakumar, Ander Gray, Anima Anandkumar