Multiagent Systems
Embodied Semantic Communication for Collective Autonomous Agents: A Tutorial on Representation, Wireless Delivery, and Closed-Loop Coordination
Yizheng Huang, Wensheng Lin, Lixin Li et al. (+3 more)
As autonomous systems and embodied intelligence enter the dynamic physical world, multi-
agent collaboration calls for a paradigm shift in communication design. However, existing communication paradigms overlook that
agents form action understanding from their own states, environmental observations, and collaboration relations through a process that evolves as a task unfolds. Consequently, reliable bit delivery, general semantic recovery, or single-task utility optimization alone cannot ensure that heterogeneous
agents form coordinated actions compatible with their own conditions from shared information during task execution. To address this gap, this paper proposes embodied semantic communication (ESC) as a paradigm that transforms information transmission into action-oriented semantic interaction. Specifically, ESC characterizes how an explicit communication link can encapsulate
multimodal perceptual states, intrinsic hardware capabilities, and collaborative intents into unified actionable semantic representations, thereby enabling heterogeneous receiving
agents to parse, align, and ground them in local motor control. This paper clarifies the conceptual boundary, system characteristics, and environment-constrained technical pathways of ESC. It maps the underlying mathematical tools, including semantic information theory, world models, and multi-
agent decision theory. Finally, this paper summarizes key open challenges, including measurable semantic reliability, ambiguity-triggered interaction under dynamic environments and tasks, and bandwidth-adaptive semantic transmission, outlining a roadmap for collective embodied networks.
Read the full abstract ▾
All authors Yizheng Huang, Wensheng Lin, Lixin Li, Qinghe Du, Wenchi Cheng, Zhu Han
Multiagent Systems
Bregman Consensus
Andrei N. Soklakov
Consider a community of
agents who are seeking consensus on a set of parameters. The
agents agree to use the same Bregman-type divergence to quantify disagreement between their individual estimates of the parameters but have varying confidence in each other's abilities. Each
agent is happy to revise their estimate by moving to the weighted barycenter of all individual estimates with higher weights applied to more trusted
agents. We show that such revisions naturally lead to an iterative
algorithm which converges to a unique consensus estimate of the parameters. Furthermore, since the consensus estimate is itself a barycenter with computable weights, the group emerges as a collective super-
agent with a well-formed opinion regarding the ability of each individual
agent.
Read the full abstract ▾
Multiagent Systems
Prompted Identity Degrades Cooperation in Multi-Agent LLM Systems
Xavier Del Giudice, Alessio Palma, Matteo Migliarini et al. (+2 more)
Multi-
agent LLM systems increasingly mix models from several providers, yet exposing each
agent's underlying model identity to its peers significantly impairs cooperation. We show that when
agents are aware of each other's model family, the group splits into clusters, where
agents prefer interacting with others carrying their same label, although nothing in the task rewards or asks for such a split. We argue that the label itself causes this split, which we define as $\textit{factionalism}$. We show and measure this phenomenon in two cooperative games and on a
reasoning benchmark, with nine to twenty-five
agents drawn from up to five open-weight model families. We further show that when the announced families are shuffled, or replaced by arbitrary labels, the factions still follow this information; when the label is removed, this behavior disappears. In strictly cooperative tasks, labeled groups spend on average $30\%$ more rounds and $55\%$ more
tokens to reach a decision, and their success rate drops from $96\%$ to $81\%$. The effect replicates across tasks, group sizes and model families. Withholding identity labels from the
agents is simple and effective mitigation.
Read the full abstract ▾
All authors Xavier Del Giudice, Alessio Palma, Matteo Migliarini, Fabio Galasso, Indro Spinelli
Multiagent Systems
VehicleArena: A Realistic Urban Environment for Multi-Agent Driving
Jie Yang, Jiajun Chen, Jiazheng Zhou et al. (+4 more)
Real-world embodied
agents often pursue independent objectives within a shared physical environment, where their actions can alter the conditions faced by others. Existing
benchmarks, however, typically assume shared goals or explicitly prescribed interaction protocols, leaving such emergent physical coupling underexplored. We introduce VehicleArena, a 3D urban-driving
benchmark for studying independently operating
agents in a dynamic shared world. In VehicleArena, LLM-controlled
agents must fulfill evolving passenger requests while navigating complex traffic, and each
agent's driving decisions can reshape traffic flow, delays, risks, and subsequent observations for surrounding
agents. The
benchmark provides 112 evaluation tasks spanning single-
agent and multi-
agent driving. Across nine evaluated models, the highest arrival rates reach only 65.0% on single-
agent tasks and 65.6% on multi-
agent tasks, while strong passenger-request or cabin scores do not reliably translate into successful trip completion. Moreover, in matched multi-
agent runs, every tested focal policy reduces the arrival rate of surrounding vehicles relative to the simulator's native traffic controller, revealing measurable externalities beyond the focal vehicle itself.
Read the full abstract ▾
All authors Jie Yang, Jiajun Chen, Jiazheng Zhou, Mianqiu Huang, Yining Zheng, Yuxin Wang, Xipeng Qiu
Multiagent Systems
Agentic Federated Learning: Rule-Based Client and Server Agents for Adaptive Training
Deepthy K. Bhaskar, VP Binu and B Minimol
Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it suitable for
privacy-sensitive applications such as healthcare, finance, and edge intelligence. However, conventional FL approaches rely on static client participation and fixed aggregation strategies, which limits their effectiveness under non-IID data distributions, heterogeneous client behavior, and noisy or unreliable updates. To overcome these issuess, this paper proposes an Agentic
Federated Learning (AFL) framework that integrates lightweight rule-based autonomous
agents at both client and server levels. The proposed framework introduces a Client-Side
Agent (CSA) that dynamically adapts local training parameters, controls participa- tion, and evaluates update reliability, while a Server-Side Orchestrator
Agent (SSOA) performs quality-aware client selection and adaptive aggregation. Unlike traditional FL methods, AFL enables context-aware decision-making during the training process, improving adaptability and robustness in dynamic distributed environments. Extensive experiments conducted on the CIFAR-10
dataset under IID, non-IID, and noisy-client settings demonstrate that AFL consistently outperforms standard base- lines including FedAvg and FedProx. Experimental results show improvements in classification accuracy, convergence speed, robustness against corrupted updates, and communication efficiency. Ablation studies further confirm the complementary contributions of CSA and SSOA, while statistical analysis validates the significance of the observed gains. The proposed AFL framework demonstrates that incorporating autonomous agentic
reasoning into
federated learning provides an effective and practical solution for intelligent, adaptive, and robust distributed learning systems.
Read the full abstract ▾
Multiagent Systems · Cross-listed from astro-ph.IM
Reconstructing Implicit Scientific Knowledge: Evaluating LLM Agents through End-to-End Reproduction of Astronomy
Yuehui Wang, Xinyu Qi, Guirong Xue et al. (+4 more)
The integration of large
language models (LLMs) into scientific workflows is accelerating, yet their ability to reconstruct the
reasoning underlying published research remains unexplored. Papers specify explicit procedures while leaving many methodological dependencies-data selection, calibration corrections, priors, and domain assumptions-implicit. This ambiguity complicates the evaluation of LLM-based
agents, since a failure to reproduce a result may reflect either limitations of the
agent or underspecification in the source. We present a framework that evaluates
agents through end-to-end reproduction, separating execution from verification and computational failure from methodological ambiguity. We apply it to fourteen astronomy studies: a case study from The Astrophysical Journal and thirteen papers published in Nature. Eleven of the thirteen contained an ambiguity preventing a uniquely specified reproduction path. In a controlled case study, twelve predefined paths, a 3x2x2 sensitivity analysis over sample definition, sky masking, and parallax zero-point treatment-gave estimates from 2.16 to 3.53 kpc for the same quantity, with only one recovering the published value (about 2.70 kpc). The published value was never used as an optimization target, selection criterion, or stopping condition; the matching path was found only after all twelve had run. Crucially, the decisive information (a +0.02 mas parallax zero-point correction) was already in the paper, but the
agents did not recognize its causal relevance until the analysis made the effect visible. Matching a published outcome therefore does not validate reconstruction of the underlying
reasoning, and the bottleneck is as often a failure to connect relevant information as to retrieve it. End-to-end reproduction thus serves both as a test of reproducibility and as a framework for evaluating implicit scientific knowledge in AI systems.
Read the full abstract ▾
All authors Yuehui Wang, Xinyu Qi, Guirong Xue, Cheng Wang, Yangbin Xie, Xiaoyu Tang, Cong Sun
Multiagent Systems · Cross-listed from cs.RO
Denoising Multi-Robot Trajectories
Yuhao Zhang, Keisuke Okumura, Ajay Shankar et al. (+1 more)
Multi-
robot trajectory planning is a fundamental problem in multi-
robot coordination but remains computationally challenging due to its nonconvex,
multimodal, and high-dimensional nature. This work builds upon D4orm, a dynamics-aware
diffusion-denoising framework, and develops a family of planning architectures for diverse operational requirements. Unlike conventional numerical optimization methods, D4orm employs sampling-based optimization to generate solution trajectories through massively parallel sampling, leveraging modern computing architectures such as GPUs. Its
diffusion-denoising structure iteratively optimizes deformations to candidate control trajectories, providing an efficient and versatile paradigm for generating kinodynamically feasible and conflict-free trajectories. Using D4orm as the building block for advanced planners, we present a decoupled planner for improved scalability, an online receding-horizon planner with feedback control, and a distributed planner for resource-constrained settings. Evaluations with differential-drive and holonomic
robots in 2D and 3D environments demonstrate that D4orm-based approaches find high-quality solutions faster and more reliably than other sampling-based optimization methods, such as MPPI, as well as a learned
diffusion-model-based method. We further demonstrate zero-shot deployment on ten real quadrotors with obstacles, large-scale deconfliction with 100 simulated
robots, and fully onboard distributed `lifelong' operation with six ground
robots. Overall, these results establish
diffusion denoising as a scalable and reliable framework for multi-
robot coordination. Code and
video: https://github.com/proroklab/d4orm
Read the full abstract ▾
All authors Yuhao Zhang, Keisuke Okumura, Ajay Shankar, Amanda Prorok
Multiagent Systems
Positional choice and robust collective behavior in fish schools: biohybrid experiments and modeling
Vahagn Grigoryan, Donato Romano, Cesare Stefanini et al. (+1 more)
Collective behavior of fish schools is usually modeled on the assumption that each individual follows specific rules of motion that depend on its position and velocity relative to its neighbors. Although these models reproduce many schooling patterns observed in nature, it remains unclear whether the assumed rules are realistic at the individual level. To address this question, we first analyzed a set of experiments in which a live fish interacted with four moving robotic fish in a tank, and measured the time it spent in each position relative to the
robots. We then simulated this experiment, replacing the fish with an
agent, and assessed the extent to which classical models agree with the experimental observations. An extensive exploration of the parameter space showed that these models closely matched the experimental observations, with a very specific choice of parameters. However, they were highly sensitive to the parameter values: a minor perturbation caused them to fail completely. To resolve this, we propose a new model that incorporates an additional exploratory term characterizing the
agent's positional preference at every moment. This model proved robust to small errors in the parameters while successfully replicating the experiments. Furthermore, when generalised to multiple fish, the model reproduced schooling behaviour and common schooling patterns, while also exhibiting exploratory behaviour at both the individual and the school level. This shows that social cohesion coexists with individual exploratory behaviour, and that realistic models of collective behaviour should account for both social interactions and this exploratory component.
Read the full abstract ▾
All authors Vahagn Grigoryan, Donato Romano, Cesare Stefanini, Giulia De Masi
Multiagent Systems · Cross-listed from cs.AI
Self-Adapting Group of Experts for Multi-Agent Reasoning
Mohammad Atif Quamar, Nurbek Tastan, Karthik Nandakumar et al. (+1 more)
Multi-
agent systems bring together
language model
agents with different roles to propose, review, and refine solutions. Each
agent's response depends on its model's capabilities, the
reasoning strategy defined by its system
prompt, and the information in its input context. Existing frameworks often adapt communication by changing this context while leaving individual
prompts fixed, even when a problem calls for different skills. We study whether
agents' initial responses can identify a strategy better suited to the current problem and guide its transfer to other
agents. To address this, we introduce SAGE (Self-Adapting Group of Experts), a training-free framework that uses answer agreement, prefix consistency, and reciprocal peer review to select a strategy donor. SAGE transfers the selected donor's
reasoning strategy to the other
agents while preserving their original roles. This transfer uses only the
agents' original system
prompts, without access to the problem or generated solutions. After strategy adaptation,
agents exchange responses through a dynamic, sparse directed acyclic graph that routes information from higher-scoring
agents to lower-scoring
agents. Experiments across multiple
agent backbones and
reasoning benchmarks show that SAGE achieves higher average accuracy than the evaluated baselines. Our code is available at https://github.com/atifquamar07/sage.
Read the full abstract ▾
All authors Mohammad Atif Quamar, Nurbek Tastan, Karthik Nandakumar, Junpei Komiyama
Multiagent Systems · Cross-listed from cs.LG
Persistent Partners Raise Prices Among Learning Agents
Paul-Peter Arslan, Yubin Kim and Xiao Xiao
When pricing
agents meet repeatedly on a platform, the platform decides who faces whom. We ask whether that choice moves the prices the
agents learn, and whether a rise comes with learned punishment. In a pre-registered randomised experiment in the Bertrand duopoly of Calvano et al., each
agent's price is set by a tabular Q-learning module, not by the small
language model attached to it, and we randomise whether each
agent keeps its partner, sees its rival's prices and can send messages. Keeping the same partner raises the level of profits, averaged over training, by 0.27 of the gap between competitive and monopoly profit (95% CI 0.20 to 0.35, all twenty paired runs positive), our registered primary result, and the resting price by 0.17 of the Nash-to-monopoly range (post hoc). A plain tabular learner reproduces the effect in all 25 further blocks, and there one permanent partner raises the level more than about three do (+0.23 against +0.05, exploratory). Where rival prices are hidden, the price-setting module cannot see a cut, so cannot punish it, yet the resting price rises as much and the rise lasts to the end of training, while with visible rivals it shrinks with longer training (post hoc). Where the rival is visible, a static best responder accounts for a third to a half of what a forced-deviation probe reads as punishment, on the starts where the rival can see the cut, and net of it the registered test of learned punishment is inconclusive. A test that looks only for punishment would thus miss the rise where the rival is hidden, while a check for profitable deviations flags most of those prices (post hoc). In an exploratory extension, untrained Qwen2.5 7B and 14B models under one
prompt show the effect when the rival's price is left out of the
prompt and inconsistently when it is shown, the 7B result replicating on fresh blocks, while two other model families show none.
Read the full abstract ▾
Multiagent Systems · Cross-listed from physics.soc-ph
Hybrid epidemic simulation framework coupling equation-based and individual-based models
Jaeyoung Kwak, Michael H. Lees, Chin Chun Ooi et al. (+1 more)
Mass gathering events like concerts, sports matches, and festivals bring many people into close contact within a short period, creating localized bursts of infection that can shape epidemic outcomes across an entire city. To evaluate how these transient transmission events translate into broader urban impacts, we developed a simulation model linking event-scale contact dynamics with citywide commuting networks. Using Madrid, Spain, as a case study, we compared several types of gatherings and examined how their effects changed under different levels of disease transmissibility. We found that mass gatherings consistently amplified outbreak magnitude, accelerated progression, advanced district-level arrival times, and synchronized spatial spread. Remarkably, while the initial seed size generated at the event accounted for much of this acceleration, post-event transmission conditions provided complementary predictive signal regarding invasion timing. These findings demonstrate that mitigating transmission during mass gatherings can yield downstream public health benefits by delaying broader spatial spread. More generally, this multiscale framework offers a tool to evaluate how temporary, localized contact shifts produce longer-lasting consequences for urban populations.
Read the full abstract ▾
All authors Jaeyoung Kwak, Michael H. Lees, Chin Chun Ooi, Wentong Cai
Multiagent Systems · Cross-listed from cs.AI
From Migration to Calibration: Aligning Agents with Deployment Requirements
Yaxiao Liu, Pengbo Liu, Yiwen Liu et al. (+3 more)
Deploying, migrating, or scaling an
agent changes its model, support system, application, and users. Calibration should align the
agent with destination requirements without requiring source imitation or universal superiority. We propose three standard families: basic capability and operational reachability; technical and professional precision, including terminology, units, reporting periods, program fidelity, and report dependencies; and user context covering roles, use cases, authorized values, permissions, and budgets. A versioned mapping distinguishes requirements to preserve, transform, add, or retire. Four instruments—
benchmarks, positive/negative examples, skills, and role-conditioned cost/utility—supply tunable reward channels within a fixed acceptance contract. Consultants annotate methods, branches, bounded loops, checkpoints, relative comparisons, and report refinement; a registry records applicable evidence. Adaptation revises configurations or trains a separate controller or skill while freezing the foundation executor. Independent hard gates govern qualification; RL and preference methods optimize candidates without supplying factual truth. Qualification and held-out improvement over the unchanged target are separate outcomes. Supplied intent labels permit a limited paired recount: small
prompt/retry gains coexist with regressions and failed gates. FinQA supplements match prior summaries and retain provenance conflicts. These observations motivate destination-specific evaluation but do not establish calibration efficacy. We specify matched-budget tests, clean reload, complete-denominator reporting, and dependency revalidation. This is a methodological proposal with partially reconciled pilot evidence; confirmatory validation and the greater-than-80% zero-shot task-context acceptance target remain outstanding.
Read the full abstract ▾
All authors Yaxiao Liu, Pengbo Liu, Yiwen Liu, Yihua Guan, Jiaxing Song, Xingjian Li
Multiagent Systems · Cross-listed from eess.SY
Strategically Robust Game-Theoretic Multi-Agent Trajectory Optimization
Victor L. Qin, Nicolas Lanzetti, Saverio Bolognani et al. (+1 more)
Aviation authorities worldwide expect Advanced Air Mobility (AAM) traffic management to be decentralized among service providers, requiring AAM flights to autonomously plan trajectories by predicting other flights' control inputs rather than relying on centralized coordination. Game-theoretic approaches that formulate multi-
agent collision avoidance as an exact dynamic potential game can efficiently find open-loop
equilibria, but they assume that
agents exactly follow their
equilibrium trajectories—an unrealistic assumption given uncertainties in actuation, perception, and computation. We propose a strategically robust formulation where each
agent protects against a fictitious adversary that, for each timestep, perturbs other
agents' control inputs within a bounded budget to minimize distance at that timestep. We show that, under reasonable assumptions on
agents' distance cost and robustness levels, the strategically robust game remains an exact dynamic potential game and admits a quasi-closed-form solution to the inner
adversarial problem for linear dynamics, which limits computational overhead. Experiments with up to eight
agents using logarithmic distance costs show that strategic robustness selects more robust trajectories in high-collision-risk configurations while leaving low-risk trajectories nearly unchanged, with only a modest increase in runtime.
Read the full abstract ▾
All authors Victor L. Qin, Nicolas Lanzetti, Saverio Bolognani, Hamsa Balakrishnan
Multiagent Systems
CEO Arena: Evaluating Long-Horizon Multi-Agent Decision-Making in Competitive Markets
An Yan, Yu Huo, Zhiwei Shang et al. (+2 more)
Long-horizon competition tests
agents' ability to coordinate business decisions under uncertainty and adapt to changing rival strategies. We introduce CEO Arena, a
benchmark that uses matched replacement evaluation to assess operating returns alongside an
agent's effects on rivals and the
market. Each CEO
agent is compared with a reference policy in the same company under the same economic seed, holding other
agents' identities and assignments fixed while all
agents adapt. In a shared eight-company
market spanning 500 simulated days, CEOs make sequential decisions on pricing, procurement, marketing, research and development, and service using private company information and noisy
market signals, under resource constraints and delayed feedback. We evaluate eight LLM-based CEO
agents in 27 main runs and 26 robustness runs. In the main evaluation, most
agents have negative mean returns, and private gains can accompany
market losses. Robustness analyses suggest that aggregate patterns extend beyond the original rule-based baseline; four of the 56 directed pairs show relatively stable effects.
Memory, action, and accounting traces suggest demand capture and rivals' pricing and spending responses as possible explanations. CEO Arena provides a controlled testbed for studying long-horizon
agent competition, strategic interaction, and
market externalities.
Read the full abstract ▾
All authors An Yan, Yu Huo, Zhiwei Shang, Yiran Peng, Chenglin Wu
Multiagent Systems
MASTraceBench: Diagnosing Collaboration Gains through Proposal Trajectories in LLM-Based Multi-Agent Systems
Yapeng Li, Songze Li, Shuang Yu et al. (+4 more)
LLM-based multi-
agent systems (MAS) have shown promise in complex problem solving. As MAS methods diversify, systematic evaluation becomes increasingly challenging. However, existing
benchmarks largely focus on final outcomes, leaving unclear how collaboration gains arise, are preserved, or are lost. To address this limitation, we introduce MASTraceBench, a
benchmark for diagnosing collaboration gains through proposal trajectories in MAS. Across six cooperative and competitive tasks, MASTraceBench tracks and grades proposal trajectories and provides a multi-layer metric suite covering Task Score, Collaboration Gain, proposal-trajectory indicators, and
Token Cost. Using MASTraceBench, we systematically compare representative MAS methods not only by final performance, but also by how
agent proposals evolve and are aggregated into the final answer. This analysis reveals a recurring pattern: final MAS answers rarely surpass the strongest initial proposal; interaction often lifts initially weaker proposals toward it, while strong initial proposals are seldom further improved and may regress. To reduce this risk, we propose CLEARS, which replaces whole-proposal exchange with claim-level evaluation across
agents to guide reliable synthesis. CLEARS more often preserves or improves upon the strongest initial proposal and achieves the highest Collaboration Gain on five of the six tasks.
Read the full abstract ▾
All authors Yapeng Li, Songze Li, Shuang Yu, Jing Yu, Zhixin Liu, Liqiang Wen, Tonghua Su
Multiagent Systems · Cross-listed from cs.LG
Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication
Mihir Chauhan and Aniket Bera
Rate-limited multi-
agent teams raise three questions the emergent-communication literature has answered only empirically: what an optimal message should encode, what compression costs over a horizon, and when a learned protocol is unique enough for a teammate to read. We answer them for rate-limited Dec-POMDPs, then measure how far reinforcement learning falls short of the optimum. Our
theorems fix what is achievable independently of any learner, so a gap between an engineered and a learned sender at the same bit budget is an optimization fact, not an information-theoretic one. We instantiate this on three MuJoCo arenas spanning zero, partial and rigid physical coupling, charging every condition exactly 2 bits per decision, and create the discriminating regime by closing a physical side channel within one arena, holding bodies, task and reward fixed. Communication value is governed by coupling: under rigid coupling through a shared object, no channel beats silence (+0.001 +/- 0.001, p = 0.982, n = 25), since proprioception already carries that information; without coupling, every condition solves the task; under partial coupling, the engineered 2-bit sender reaches an interquartile mean of 1.000 but the learned one reaches 0.482, indistinguishable from silence (p = 0.400, n = 25). With a shared alphabet, bandwidth cannot explain the gap. Warm-starting from an engineered receiver localizes the failure: the same channel reaches 0.857 versus 0.562 cold-started (p < 0.001), so it is neither representational nor one of maintenance; reinforcement learning fails to discover the protocol. Cross-play shows learned protocols are individually meaningful but mutually unintelligible: self-play 0.980 collapses to 0.144 across seeds, and our best constructed
alignment leaves at least 77% of that gap. All headline results use 25 seeds per arena and seven published baselines at matched rate.
Read the full abstract ▾
Multiagent Systems · Cross-listed from cs.AI
Same Winners, Different Success Rates: Evaluating How LLM Agents Recover from Failures
Dong Xu, Zhangfan Yang, Jiantao Wu et al. (+5 more)
Evaluating how LLM
agents recover from mid-task failures is central to deploying reliable agentic systems. Existing checkpoint-based
benchmarks measure recovery by comparing which action is selected as best across independent runs, a quantity known as set agreement. However, set agreement is a purely ordinal measure that records which action wins without reflecting the absolute level of performance. When all actions fail, they tie at zero reward, and independent runs produce the same tied set with high probability, creating an illusion of stability that masks near-zero recovery success. We formalize this limitation through a set-path
symmetry result, proving that for equal-cost Bernoulli actions the success probabilities (0.9, 0.8) and (0.2, 0.1) yield identical best-action-set distributions at every sample size. No procedure based solely on which action wins can distinguish these two regimes. We further prove that certifying exact population ties is impossible in finite time, and that the assignment of outcomes to checkpoints carries information beyond marginal outcome distributions. The pooled success probability is the missing scalar that resolves the ordinal ambiguity. Experiments on 864 frozen RecoveryBench episodes and two planning cohorts totaling 3,456 responses confirm the theoretical predictions. Agreement and held-out quality can move in opposite directions, and permuting checkpoint-to-action bindings changes 8 to 13 percent of
cell-level conclusions. Based on these findings, we propose reporting four diagnostic quantities (agreement, all-zero fraction, held-out success, and pooled success) that expose this failure mode with no additional data collection.
Read the full abstract ▾
All authors Dong Xu, Zhangfan Yang, Jiantao Wu, Shipeng Zhang, Zexuan Zhu, Jiangqiang Li, Jun Zhang, Junkai Ji
Multiagent Systems · Cross-listed from cs.AI
ReplayLens: Auditing Agents' Use of Outcomes
Dong Xu, Zhangfan Yang, Jiantao Wu et al. (+5 more)
When an
agent reuses logged experience, a changed decision may reflect the recorded score, the action's name, or the record's position in storage. Standard
memory evaluations do not reveal which relationship drives that change. We introduce ReplayLens, a black-box audit that changes one relationship in the stored history at a time, holds the remaining interface fixed, and measures the resulting decision. Four interventions target four relationships. Outcome reassignment swaps which scores belong to which actions. Pair transport moves intact action-score pairs to new record slots. Consistent renaming relabels actions in both history and menu. Key-slot reassignment changes both score attachment and position. A constructive separation shows why the audit is needed: two
memory writers with identical endpoint accuracy respond differently to the same replay, so conventional evaluation cannot resolve the underlying dependence. On black-box LLM interfaces, swapping scores changes decisions while moving intact pairs does not, separating score attachment from record order. A bounded-
memory study exposes ingestion-order sensitivity that endpoint comparison misses. In sequential experiment planning, altered historical scores redirect exploration and reduce final utility despite fresh measurements. A code-debugging
agent with sealed hidden tests shows the same pattern outside model selection. ReplayLens provides a relationship-level audit for deciding whether logged experience can be merged, reordered, or reindexed safely.
Read the full abstract ▾
All authors Dong Xu, Zhangfan Yang, Jiantao Wu, Shipeng Zhang, Zexuan Zhu, Jiangqiang Li, Jun Zhang, Junkai Ji