Artificial Intelligence
GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?
Kun Chen, Haorong Hong, Peizhong Gao et al. (+7 more)
Recent large
language models (LLMs) can operate as coding
agents that build complete games from natural
language requests. Game development is especially demanding because program logic, visual and audio content, interfaces, interaction and playability must function together in one executable artifact. Measuring this capability therefore requires evaluation of both game product and the development process. Existing
benchmarks often assess the game development capabilities of LLMs by evaluating the final artifact or an isolated development stage. Our analysis of complete human-
agent development trajectories identifies three stages that together span the lifecycle of game development with a coding
agent: initial game generation, bug diagnosis and repair, and optimization over multiple turns. Therefore, we introduce GameXpert-Bench, which operationalizes the three lifecycle stages as three complementary
benchmark tracks. GameGen evaluates complete game creation from a single request in an empty workspace. GameFix evaluates diagnosis and repair when defects are reported or left for the
agent to discover. GameOpt evaluates cumulative optimization through request chains seeded by real development trajectories between users and
agents. We evaluate each track using live game interaction, deterministic behavioral tests, or final product criteria with regression checks. The suite contains 97 generation tasks across 11 genres; 100 repair tasks from 50 game levels verified by humans, each with 19-27 injected bugs; and 17 optimization chains with six turns and 102 requests. Across the three tracks, current
agents are more reliable at producing playable foundations and implementing explicit requirements than at discovering defects, verifying runtime behavior, and preserving functionality across changes.
Read the full abstract ▾
All authors Kun Chen, Haorong Hong, Peizhong Gao, Jianfeng Lin, Tongxu Luo, Yuxuan Xie, Chenxu Liu, Jieling He, Zhongyuan Liu, Zeno Zeng
Artificial Intelligence
Beyond Success and Failure: Length-Aware Contrastive Learning for GUI Agents
Chengyang Gu, Le Zhang, Jingbo Zhou et al. (+6 more)
Graphical User Interface (GUI)
agents powered by
Multimodal Large
Language Models (MLLMs) have shown strong potential for automating tasks across diverse digital environments, where reinforcement learning (RL) has become a dominant training paradigm. However, widely used methods such as Group Relative Policy Optimization (GRPO) suffer from reward-gradient misalignment, leading to inefficient and unstable optimization. Recent work addresses this issue by reformulating RL with verifiable rewards (RLVR) as contrastive or classification-based objectives, which improve stability by eliminating problematic gradient behaviors. Despite this progress, existing contrastive RLVR methods rely primarily on outcome-level supervision and fail to capture fine-grained differences in trajectory quality within the same outcome category. In this paper, we propose Length-Aware Contrastive Learning for GUI
Agents (LACL-GUI), a contrastive RLVR framework that incorporates trajectory-level quality signals into policy optimization. LACL-GUI introduces structured preferences within both successful and failed trajectories, encouraging concise successful executions and differentiating failure quality based on divergence from successful trajectories, while preserving optimization stability. Experiments on GUI
agent benchmarks show that LACL-GUI provides more effective learning signals and consistently improves
agent performance over prior methods, highlighting the value of trajectory-level supervision in contrastive RLVR.
Read the full abstract ▾
All authors Chengyang Gu, Le Zhang, Jingbo Zhou, Yize Chen, Yu Shi, Siqi Bao, Zheng-Fan Wu, Hua Wu, Hui Xiong
Artificial Intelligence · Cross-listed from cs.CL
Training a Knowledge Base: Supervised Structure Learning for Agent-Curated Document Stores
Yu Pan and Hongfeng Yu
Retrieval-augmented generation treats the document store as a frozen input, and the offline pipelines that do build structure over it build it unsupervised – a whole corpus indexed at uniform effort, with no signal about which structure a question will need. We instead treat the knowledge base as a non-parametric model trained on (question, answer) pairs: a curator
agent answers a supervised question against the current store, is shown the gold answer, then edits the store. The store carries forward, and we evaluate the curated store with a test set, on two contamination-free
benchmarks: KBGym, a fictional-universe generator we release, and PhantomWiki. Generalization is probed with four question groups of decreasing overlap with the training set: the trained questions themselves, and unseen questions sharing both of their keys with training, one key, or neither. The curated store's advantage grows with overlap – from parity where no key was shared, through +0.176 F1 where both keys were, to 25% fewer actions at +0.294 F1 on the trained questions, the one
cell significant on both
benchmarks – while matching HippoRAG's gains with 1,913 links against its 196,112: per point of corpus covered, 1.5x the action saving and 2.1x the accuracy gain. Accuracy rises steadily with the share of the corpus the indexes cover, so training on more questions widens coverage, and with it the generalization.
Read the full abstract ▾
Artificial Intelligence
VisAdj: Learning Adjacency Matrices from Node-Link Images
Jiahao Xie and Guangmo Tong
Learning adjacency
matrices from node-link images is a fundamental problem for recovering structured graph information from visual observations. Existing methods typically rely on fixed KNN-based heuristics for candidate edge selection and fail to capture dependencies among edges. To overcome these limitations, we propose VisAdj, a new framework for
topology-aware adjacency prediction. VisAdj introduces an
attention-sparse neighbor sampler to adaptively select a high-recall set of candidate node pairs and performs joint edge inference using a line-graph
transformer that treats candidate edges as
tokens and explicitly models dependencies among incident edges. Extensive experiments on synthetic graphs, road networks, and vessel images demonstrate that VisAdj consistently outperforms existing baselines by clear margins.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
PatchGate: Narrowing the Verbalization Gap with Intrinsic Object Inventories in Frozen Vision-Language Models
Jihyung Ko, Eunji Jung, Hyeongsub Kim et al. (+4 more)
Reliable image captioning in
Vision-
Language Models (VLMs) requires captions to be both precise and complete, avoiding unsupported object mentions while covering visible objects. Existing training-free methods primarily address the former requirement, suppressing unsupported object words by intervening on model-predicted mentions during generation. Because they operate only on objects the model is already likely to mention, visible objects omitted from the output remain difficult to recover. We propose PatchGate, a training-free framework that extracts
prompt-free object evidence intrinsic to a frozen VLM before generation and uses it to narrow the gap between an intrinsic object set and final object mentions. In the first stage, Visual Evidence eXtraction (VEX) reads patch-level lexical evidence from the latter half of LM decoder layers and constructs an image-conditioned object set without any task
prompt. In the second stage, Visual-Evidence Inclusion-Exclusion Decoding (VIED) uses this object evidence to calibrate decoding logits, promoting evidence-supported but under-verbalized objects and suppressing weakly supported but over-verbalized objects. On AMBER, PatchGate improves both sides of object-level reliability, increasing visible-object coverage from 49.4 to 56.0 (+13.4%) and reducing object
hallucination by lowering CHAIR from 7.5 to 6.6 (-12.0%), without external detectors or fine-tuning and with one extra forward pass.
Read the full abstract ▾
All authors Jihyung Ko, Eunji Jung, Hyeongsub Kim, Ziseok Lee, Jae Won Cho, Sanghyun Jo, Kyungsu Kim
Artificial Intelligence
Hints, Critics, and Teachers: Prior Injection for Sparse-Reward RL in Vision-Language Math Reasoning
Qiqian Fu
Reinforcement learning for
vision-
language math
reasoning starves under sparse reward: on a pool of 20,830 visual-math problems where Qwen2-VL-2B answers 3.6% of rollouts correctly, 85-97% of GRPO rollout groups are entirely wrong and contribute zero gradient. We train eleven methods under identical conditions in this regime, each injecting a different prior: text (reference-solution hints), distribution (on-policy
distillation from a 7B teacher), and value (a value-pretrained critic with an MSE or HL-Gauss categorical loss). A prior helps exactly when it is delivered: the six arms whose prior effectively reaches the policy separate with no overlap from the remaining five – the no-prior baseline and four arms whose prior is teacher-capped, gated away, or lost to a mis-parameterized critic – both on the pooled in-domain metric and on cross-domain transfer (DynaMath). The central finding, however, concerns evaluation: one slice of the in-domain pool – long used as this project's general-distribution check – anti-correlates with genuine cross-domain transfer (Spearman rho = -0.74, n = 11 arms, permutation p = 0.011), while the hardest in-domain slice predicts it closely (rho = +0.89, p < 0.001). We attribute the inversion to a near-chance multiple-choice subset that rewards models for not having changed; read through it, the best cross-domain method looked mediocre and the worst looked like the champion. Among the methods, hint-guided exploration – not UFT's auxiliary loss – drives hint gains, and replacing the critic's MSE loss with HL-Gauss cross-
entropy is worth +14.4 points in-domain. All accuracies are blind-judged, with paired exact tests.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
More Computational Resources Do Not Ensure Higher Scholarly Impact: Evidence from Leading NLP Conference Papers
Shuai Chen, Tong Bao, Jitong Peng et al. (+1 more)
Computational resources are increasingly central to NLP research, but how closely reported GPU capability aligns with scholarly impact remains unclear. We analyze 13,921 ACL, EMNLP, and NAACL main-conference papers published between 2020 and 2025, using GPU resources as our operational measure of computational resources. From full texts, we extract GPU models and counts, standardize each paper's largest reported configuration into a comparable hardware-capability measure, and link these data to citation, award, topic, and institutional metadata. GPU reporting became more common but remained incomplete, while reported capability increased mainly through newer hardware generations and medium-scale multi-GPU configurations. Resource concentration substantially exceeded impact concentration: the annual top 20% of GPU-quantifiable papers accounted for 83.9%-89.9% of reported GPU capability, but only 27%-32% of citations and 20%-33% of paper awards. In adjusted models, a tenfold increase in aggregate reported GPU capability was associated with a 3.52-percentage-point increase in within-NLP topic-year citation percentile, but increased model R^2 by only 0.0042. GPU count showed more consistent positive associations with citation and award outcomes than newer hardware generation. Overall, reported GPU resources are associated with scholarly impact but provide little standalone explanation of research influence.
Read the full abstract ▾
All authors Shuai Chen, Tong Bao, Jitong Peng, Chengzhi Zhang
Artificial Intelligence · Cross-listed from cs.CR
ExplainGuard: A Zero Trust Framework for Post-Hoc Explanation Integrity Guarantees in Blackbox XAI Models
Maraz Mia, Shovan Roy, Mir Mehedi A. Pritom et al. (+1 more)
As machine learning (ML) models are increasingly deployed in high-stakes environments, explainable AI (XAI) methods like SHAP and LIME have become essential for regulatory compliance and trust. However, the current
auditing paradigm relies on an implicit "chain of trust" where third-party auditors are assumed to be trusted. Recent research demonstrates that this assumption is flawed and
adversarial auditors can manipulate XAI explanations through manipulation attacks such as output shuffling or scaffolding out-of-distribution (OOD) to conceal model biases while maintaining high prediction accuracy aiming for fairwashed explanation. In this paper, we introduce a novel defense framework, ExplainGuard, that leverages a Zero-Trust architecture (ZTA) design to be incorporated within the XAI explanation supply chain and ensures the integrity of the generated explanation. This framework would help us to replace the ambiguous default assumption of "auditor is trustworthy," with a continuous "verify-then-trust" approach. Our design architecture establishes a Policy Decision Point (PDP) that enforces three distinct pillars of verification before any explanation is released to the user: (1) asset integrity via behavioral fingerprint to detect model substitution, (2) semantic validity using axiomatic consistency checks to reject mathematically impossible explanations, and (3) feature faithfulness verification utilizing a ranking stability approach with minimal computational overhead. Finally, we evaluate how ExplainGuard can effectively neutralize state- of-the-art explanation manipulation attacks while transforming the
auditing process into a verifiable operation.
Read the full abstract ▾
All authors Maraz Mia, Shovan Roy, Mir Mehedi A. Pritom, Maanak Gupta
Artificial Intelligence · Cross-listed from cs.CV
SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering
Long Shu, Shuochen Liu, Wei Chen et al. (+4 more)
Knowledge-based Visual Question Answering (KB-VQA) aims to answer queries that necessitate
reasoning over external knowledge sources beyond the visual content. Typically, current methods fuse
multimodal features to retrieve external information, subsequently leveraging
Multimodal Large
Language Models (MLLMs) to derive answers from the retrieved evidence. However, these methods often struggle to capture structural associations within complex contexts to effectively filter noise. Furthermore, they frequently fail to ensure that the
reasoning process remains strictly faithful to the retrieved evidence. To address these challenges, we propose SAFE-G, a Structure-Aware Faithful Evidence-guided Generation framework, which enables precise evidence localization and trustworthy
reasoning. Specifically, we first employ a coarse-grained hybrid search fusing visual and textual modalities to recall candidate documents, and subsequently implement a structure-aware fine-grained graph
retrieval that captures structural dependencies to filter noise and pinpoint precise evidence. Moreover, we introduce a reinforcement learning (RL) strategy with an evidence-grounded reward that assigns credit to correct answers only when the selected evidence is correct. This strict
alignment constraint compels the model to anchor its response in the retrieved context, effectively enhancing its capability to locate evidence via
multimodal features and perform faithful
reasoning. Extensive experiments on the Encyclopedic-VQA and InfoSeek
benchmarks demonstrate that SAFE-G outperforms prior methods by a margin of 8.9% and 3.5%, substantially enhancing the overall
reasoning accuracy. Our source code is publicly available at: https://github.com/MINE-USTC/SAFE-G.
Read the full abstract ▾
All authors Long Shu, Shuochen Liu, Wei Chen, Junda Lin, Zhi Zheng, Huijun Hou, Tong Xu
Artificial Intelligence · Cross-listed from cs.CL
Lexical Coupling in GUI Element Grounding: Sentence Embeddings Track Labels across Mobile and Web
Qijia Chen and Giulio Jacucci
GUI grounding evaluations that expose UI elements as text metadata often treat high instruction-element
embedding similarity as evidence of semantic grounding. Across three mobile and web
benchmarks, we show that this interpretation is frequently confounded by visible-label recovery. Lexical baselines remain competitive at top-1, label-poor targets remain weak for text-only methods, and encoder top-1 hits are predictable from lexical rank, candidate-pool size, and label type. We evaluate each action as a same-screen ranking task, comparing five off-the-shelf single-vector encoders with lexical baselines. Encoders recover some lexical misses, but deployable fusion gains are much smaller than target-aware oracle gains. These findings show that
embedding-based evaluations can conflate visible-label recovery with semantic GUI grounding.
Embedding-based evaluations should therefore report lexical baselines, label-type stratification, and deployable-fusion diagnostics. Our released repository provides analysis scripts and detexted per-step panels: https://github.com/qijia123/lexical-coupling-release.
Read the full abstract ▾
Artificial Intelligence
HIRA: A Human-in-the-Loop Retrieval-Augmented Cascade for Document Classification in Regulated Industries
Shangxuan Tian, Yanhui Chen and Carlos Queiroz
Document classification in regulated industries is constrained by data residency, limited cold-start labels, scarce review capacity, and costly model-governance procedures. We present HIRA, a training-free, on-premises
retrieval-augmented cascade for document classification in regulated deployments that combines BM25 over OCR text, dense text
embeddings, and image-level representations through validation-calibrated weighted reciprocal-rank fusion. Confident documents are classified directly by
retrieval; uncertain or visually confusable documents are passed to a locally hosted LLM verifier, which receives the OCR text, retrieved exemplars, label descriptions, and confusion-specific terms. When the verifier remains uncertain, the document is sent to human review. Each correction is stored as a margin-weighted
retrieval exemplar and updates a Dirichlet-smoothed confusion graph, letting the system improve without updating model weights. On a private 80-class trade-finance corpus, HIRA processes the full 30,233-document production stream while requesting human correction for only 1,945 documents (6.4%), improving Macro-F1 from 0.6218 to 0.8548. On the corrected Tobacco-3482
benchmark, HIRA reaches 0.9423 Macro-F1 with a locally hosted DeepSeek-R1-Distill-Qwen-32B verifier, 17.4 percentage points above the zero-shot LLM baseline, while invoking the verifier for only about 40% of documents and reducing LLM calls by approximately 60%. With 518 human corrections (24.8% of the pool), HIRA matches the fully labelled pool oracle, in which all 2,086 pool documents are indexed with their ground-truth labels. These results show that selective human feedback and
retrieval-
memory adaptation can be a practical alternative to repeated model retraining for long-tail document classification in regulated deployments.
Read the full abstract ▾
Artificial Intelligence
Physics-Knowledge-Guided Hybrid Neural Learning for Arctic Sea Ice Concentration Evolution and Short-Range Prediction
Maqun Zhang, Feng Gao, Wankun Chen et al. (+3 more)
Accurate modeling of sea ice concentration (SIC)
evolution is essential for polar
climate assessment and short?range sea ice prediction. Numerical and data-driven approaches constitute major foundations for SIC modeling, but the former often require complex parameterizations and substantial compu?tation, whereas the latter rarely encode physical dependencies explicitly. This study presents the Physics-Informed Hybrid Ice Model (PIHIM), a differentiable data-driven hybrid ice model for daily SIC
evolution that organizes its network structure according to the physical dependencies encoded in the sea ice continuity equation and explicitly accounts for dynamical transport, ther?modynamically driven areal growth and loss, and unresolved local processes. PIHIM preserves the representation capacity of deep learning while providing a process-decomposed formulation of ice displacement, freeze-melt areal change, and local error closure. Two evaluation settings are adopted: reanalysis-forced simulation examines SIC
evolution stability under reanalysis forcing, and forecast-forced prediction assesses short-range performance un?der forecast-forced conditions, with reanalysis and observational SIC serving as verification references. Results indicate enhanced ice-edge preservation and error-growth control in reanalysis?forced simulation, while PIHIM retains measurable short-range prediction skill under forecast-forced conditions. Our code will be made publicly available after the paper is accepted.
Read the full abstract ▾
All authors Maqun Zhang, Feng Gao, Wankun Chen, Hui Yu, Yanhai Gan, Junyu Dong
Artificial Intelligence · Cross-listed from cs.CL
Evaluation Awareness in Language Models: Representation, Verbalization, and Control
Farzaneh Heidari, Amin Memarian and Guillaume Rabusseau
Both capability and safety
benchmarks rest upon the assumption that the behavior of
language models undergoing a test is informative about their behavior in deployment. This assumption can fail, should models infer that they are being evaluated and condition their response on such context. This hypothesis, termed “evaluation awareness”, has been observed in frontier and open-weight
language models alike. We provide a systematic study of this phenomenon, by probing for it across six
language models (from four families and three sizes) and three metrics. More precisely, we examine whether (i) being under evaluation is linearly represented within the models' activations space, (ii) it is verbalized in their output
tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. For the open-checkpoint Olmo models, we further test these measures at every training stage. In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC $\geq 0.7$). By contrast, these representations align only in part with verbalization: their correlations and mutual information are nonzero in some settings, yet vary substantially across models, layers, and readout choices. Nevertheless, steering along probe-derived directions can shift the verbalization scores. Finally, a comparison across the Olmo checkpoints reveals that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter—unlike the effects of steering, that grow more pronounced at every successive training stage. These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices
Riadul Islam, Joey Mule, Dhandeep Challagundla et al. (+3 more)
Event-based
vision has emerged as a promising paradigm for energy-aware artificial intelligence (AI), offering sparse, low-latency visual signals that reduce redundant data processing and support sustainable edge computing. However, the asynchronous and noise-prone nature of event streams creates challenges for conventional deep learning models, which are often too computationally intensive for low-power embedded platforms. This work presents a compact and configurable event-driven autoencoder that efficiently compresses neuromorphic data while preserving essential spatiotemporal structure for downstream inference. The architecture integrates lightweight convolutional encoding with robust performance under adaptive event thresholding and a minimal classifier head, enabling substantial reductions in computational cost without degrading recognition fidelity. Extensive evaluations on the Smart Event Face
Dataset (SEFD) and Event-Based Crossing
Dataset (EBCD) show that the proposed framework achieves competitive or superior accuracy compared to YOLOv9 while requiring up to 35.6$\times$ fewer parameters. To assess real-world sustainability, the model is deployed on resource-constrained hardware: a Raspberry Pi 4B and a NVIDIA Jetson Nano. On NVIDIA Jetson Nano, it delivers real-time throughput of 44.8 FPS. On a Raspberry Pi 4B CPU, the 50% autoencoder classifier consumes 16.19 J for the evaluated inference workload, corresponding to approximately 726.3$\times$ lower energy consumption than YOLOv9 under the same evaluation protocol. These results demonstrate the potential of compact event-driven models to advance environmentally conscious, low-power AI systems for high-speed perception in autonomous, mobile, and embedded computing environments.
Read the full abstract ▾
All authors Riadul Islam, Joey Mule, Dhandeep Challagundla, Shahmir Rizvi, Sean Carson, Rachit Saini
Artificial Intelligence
What Does CLIP Learn for Regional Geolocalization? Probing Visual Cues and Scene Configuration After Adaptation
Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh et al. (+1 more)
Large collections of street-view imagery provide rich visual information about urban environments, but extracting fine-grained geographic information from such data remains challenging. In particular, fine-grained regional geolocalization is challenging because nearby areas often share coarse geographic cues. We study regional geolocalization within a metropolitan area and ask whether pretrained CLIP features are sufficient for regional discrimination, and what visual information supports performance after adaptation. Using 9,085 street-view images from eight Greater Los Angeles regions, we compare zero-shot CLIP, frozen-encoder readouts, partial encoder updating, Low-Rank Adaptation (LoRA), and full fine-tuning. Frozen readouts remain near the 39.03% zero-shot accuracy, whereas encoder adaptation achieves 75.94-82.10%. Full fine-tuning also reduces the mean distance to the predicted region center from 12.30 km to 3.86 km. We probe these gains through semantic cue removal, appearance reduction using edge maps and blur, and scene-configuration disruption using patch scrambling. Adapted models achieve higher edge and blur accuracy and switch 42.92-45.56% of predictions after scrambling, compared with 10.79-14.60% for frozen methods. However, adaptation does not improve the fraction of performance retained after appearance reduction, while vegetation and sky remain influential. A Caltech101 control further shows that scrambling sensitivity is not unique to geolocalization. Overall, encoder adaptation substantially improves nearby-region discrimination and is associated with greater sensitivity to intact scene configuration, without evidence that coarse structure alone becomes sufficient for prediction. These conclusions concern viewpoint variation near known locations rather than geographically disjoint generalization.
Read the full abstract ▾
All authors Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh, Seon Ho Kim
Artificial Intelligence
ECHO: An Auditable Dual-Path Memory System for Long-Horizon Agents
Yu Qian, Hong Miao, Boyang Guo et al. (+5 more)
Long-horizon
agents require more than
retrieval over stored conversations: they must preserve source evidence, revise outdated knowledge, and distinguish relevant
memories from sufficient support for an answer. We present ECHO (
Embodied Context & History Orchestration), an auditable dual-path
memory system. A fast path captures immutable events and serves multi-route recall, while asynchronous workers consolidate experience into revisable state and rebuild derived indexes. Source provenance, temporal authority, and answer realization are explicit boundaries. The service supports original-source packets with evidence-use checks and disables model-based reranking by default. For the latest reported accuracy, the authors confirm mixed retrieved inputs comprising original dialogue and
memory summaries, answered by Qwen3-8B with NF4 quantization. In the latest author-supplied five-system aggregate evaluation, ECHO reports 56.27% answer accuracy, 98.04% source Hit@10, 93.58% macro source Recall@5, and 13.92%
hallucination flags, with 19.54 ms mean
retrieval latency. Its reported accuracy exceeds Mem0 by 4.53 percentage points in this snapshot. The results are descriptive aggregates; matched question-level evaluation and generation latency remain to be established. We separate these observations from earlier published experiments and engineering regressions.
Read the full abstract ▾
All authors Yu Qian, Hong Miao, Boyang Guo, Tingyi Jiang, Shan Zhao, Tianxing Le, Lintian Li, Meng Liu
Artificial Intelligence · Cross-listed from cs.SE
Architecture as Capability Equalizer for Coding Agents
Arquimedes Canedo
LLM-based coding
agents generate complete software systems from high-level descriptions, yet little is known about how the format of architecture specifications affects the quality of generated code or whether this effect depends on model capability. We present a controlled experiment comparing five informationally equivalent specification formats (informal prose, Mermaid diagrams with constraints and ADRs, OpenAPI, C4/Structurizr DSL, and TypeScript interface contracts with ArchUnit-style rules) across six models from three vendor families (Anthropic Claude, OpenAI GPT, Google Gemini). Across 90 multi-turn
agent trials, specification format shows a strong format x model interaction. On the strongest models (Sonnet 4.6, GPT-5), format barely matters (quality spread 0.17-0.92). On weaker models, format produces spreads of 0.83-2.42 points, with code-proximate formats (OpenAPI, TypeScript contracts) recovering most of the capability gap. Mid-tier models can consume more
tokens than frontier models for worse output when they enter compilation debugging loops that stronger models avoid. Self-validation rates collapse from 100% (Sonnet) to 0% (Gemini Flash) across the capability spectrum. TypeScript contracts triple API route coverage for the weakest model (33% to 100%). Structured architecture specifications serve as a capability equalizer, with value inversely proportional to model strength and the largest returns for cost-optimized deployments.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Reinforcement Learning on Benign Facts Amplifies Leakage of Memorized Private Data
Renfei Zhang and Niloofar Mireshghallah
Reinforcement learning with verifiable rewards (RLVR) is deployed to make models better at
reasoning tasks, but its side effect on what models will divulge is under studied. Here we show that RLVR on facts increases extraction of personally identifiable information (PII) the instruct model had already memorized. We first confirm that instruct models have already memorized PII but leave them latent, rarely surfacing one when asked. We then apply RL on benign factual data that contains no PII of any kind, and re-probe: a targeted probe over name->email pairs, and an untargeted free-recall
prompt that simply asks the model to list the addresses it knows. PII extraction rises sharply under both: on DeepSeek-V3.1, verbatim recall@k increases from 0.155 to 0.370, a 2.4x gain. The effect scales with model size: across three models spanning 8B to 671B parameters, absolute leakage is largest in the biggest model. Meanwhile model's
reasoning abilities and refusal rates are retained, indicating that RL selectively changes which memorized information is accessible rather than broadly altering the model. In summary, memorized private data can be made markedly more extractable by training that never touches it. This gives an adversary a route to memorized data that requires no
privacy-relevant training signal and no access to the data itself – only the ability to fine-tune on something innocuous.
Read the full abstract ▾
Artificial Intelligence
Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention
Xiaoying Song, Anirban Saha Anik, Jinyu Liu et al. (+3 more)
Correcting health misinformation in dialogue requires more than producing a factual rebuttal: users differ in what they know, what they believe, and what they need to hear, so an effective intervention often depends on first asking the right clarifying question. Yet existing methods either respond immediately or probe indiscriminately, treating clarification as either unnecessary or always beneficial. We propose Reward-Optimized Probe-and-Respond (RO-PnR), a framework that learns when asking is worth its cost. At each turn, RO-PnR chooses between probing for more information and committing to a final correction, guided by a turn-level reward that weighs the expected gain from probing against its interaction cost. To capture how user heterogeneity affects probing value, we model each simulated user with a latent state along health literacy and belief commitment. Experiments show that RO-PnR achieves the highest cost-adjusted utility across three health-misinformation
datasets and three base models, using 30% fewer turns than always-probe baselines.
Read the full abstract ▾
All authors Xiaoying Song, Anirban Saha Anik, Jinyu Liu, Qitao Tan, Geng Yuan, Lingzi Hong
Artificial Intelligence · Cross-listed from cs.DC
PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response
Yueying Li, Jiayang Chen, Yuanfan Chen et al. (+5 more)
AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response, imposing time-varying power caps. Existing LLM serving systems optimize a static energy objective or shed fixed priority tiers under load; either way, goodput collapses when the power envelope moves. An LLM pipeline is not a uniform load: compute-bound prefill loses throughput almost linearly with GPU frequency,
memory-bound answer decode sustains it down to $0.57\times$ nominal, and
reasoning's thinking phase couples KV-cache capacity to scheduling – so a cap should be steered to where each watt costs the least performance. PowerSlider does so with a new Flex SLO contract that turns bounded user slack into an optimization constraint, prefill–think–answer disaggregation exposing per-stage frequency and KV control, and a Karush–Kuhn–Tucker (KKT) online solver re-solving within 7.7 ms of every cap change, backed by a consolidated fail-safe that power-gates drained instances when DVFS bottoms out on static power. On SGLang with production traces, \sys{} sustains 78.3% online goodput at a 30% cap reduction versus 47.6% for the best of five baselines ($1.64\times$), holds latency-critical tails within $1.3\times$ of nominal (baselines: $2.3$–$6\times$, up to $12\times$), and delivers 92% mean goodput through a replayed CAISO grid-emergency day bottoming at $0.41\times$ (54% at the trough; every baseline below 7%).
Read the full abstract ▾
All authors Yueying Li, Jiayang Chen, Yuanfan Chen, Leo Han, Haoran Qiu, Esha Choukse, Rodrigo Fonseca, Udit Gupta
Artificial Intelligence
Agentic neural architecture search for clinical prediction from electronic health records: Cross-hospital knowledge transfer and computational efficiency
Deyi Li, Qi Xu, Lingyao Li et al. (+3 more)
Transformer architectures for clinical prediction from electronic health records (EHRs) often require costly manual tuning and may vary across hospitals and tasks. We introduce ATHENA (Agentic Transfer across Hospitals for EHR Neural Architecture Search), a knowledge-guided agentic neural architecture search framework for
Transformer-based EHR modeling. A weight-sharing supernet is pretrained once per hospital, allowing candidate architectures to inherit weights and be evaluated through fine-tuning rather than independent pretraining. A two-layer cross-hospital prior supplies high-performing source architecture examples and task-specific component preferences summarized by SHapley Additive exPlanations (SHAP)-based meta-regression. These priors guide a multi-
agent large
language model search using target-hospital validation feedback. Across six clinical prediction tasks at one held-out OneFlorida+ site and one external MIMIC-IV site, ATHENA significantly outperforms all four baselines in 9 of 12 site-task evaluations under a strict equal-compute comparison. In a separate comparison using a common pretrained AutoFormer supernet to isolate search strategy performance, ATHENA ranks first in 9 of 12 evaluations at a maximum search budget of 30 candidate evaluations. It also shows more consistent architecture selection across repeated searches. ATHENA supports more efficient architecture selection for
Transformer-based clinical prediction models.
Read the full abstract ▾
All authors Deyi Li, Qi Xu, Lingyao Li, Tiansheng Wang, Muxuan Liang, Mei Liu
Artificial Intelligence
From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism
Jing Liu, Yongxing Qi, Muchen Jiang et al. (+7 more)
Retrieval-Augmented Generation (RAG) grounds LLM generation on retrieved documents, but the standard terminal
retrieval stage–dense-vector similarity, optionally followed by reranking–often returns documents that share keywords with the query without containing the needed information, a failure mode that grows with the knowledge base. We trace it to a conceptual gap: similarity captures only associational relations, whereas the documents that matter are linked to the query causally. We model the terminal
retrieval stage with a causal graph grounded in Reichenbach's common cause principle: the keywords shared by the query and a retrieved document form a latent common cause A, and the document's residual keywords form a latent set B linking the document to the ideal output. Since a retrieved document is a
collider (A -> d <- B),
retrieval itself opens an associational path between the query and B, which licenses a training-free,
attention-style re-scoring rule: the cosine similarity between the query
embedding and the weighted centroid
embedding of B. Unlike causality-enhanced RAG variants that model causal relations inside the knowledge content, our graph models the causal structure of the
retrieval process itself. On a real 471-document enterprise knowledge base, the method promotes a relevant guideline from rank 6 to the top 3; on a controlled diagnostic corpus reproducing the keyword-stuffing regime, it improves the mean target rank from 2.88 to 1.25, while a trained cross-encoder reranker barely helps (2.63). Conversely, on three BEIR
benchmarks the score underperforms the similarity baseline, delineating the applicability boundary: the method guards the keyword-stuffing regime of growing proprietary knowledge bases and complements neural rerankers; a corpus-level calibration gate selects the correct regime with >= 95% reliability. A fully local testbed demonstrates deployability.
Read the full abstract ▾
All authors Jing Liu, Yongxing Qi, Muchen Jiang, Chengnan Hu, Qingqing Peng, Haoming Wang, Yuqing Wang, Yang Yu, Xu Zhang, Ting Wu
Artificial Intelligence · Cross-listed from quant-ph
Scalable quantum simulation of continuous-time generative models via tensor networks
Nathan X. Kodama, L. Andrew Wray, Sam Cochran et al. (+3 more)
Continuous-time flow and
diffusion models are widely used across many application domains, from large-scale deployment in computer
vision and
protein folding to emerging adoption for modeling
language, time series, and
quantum states. After training, inferring statistical properties from continuous-time models is costly. Wavefunction flows target this cost by recasting learned transport as
unitary evolution, whose final Born distribution approximates the target distribution. This prepares a coherent amplitude encoding (a qsample) that can be post-processed by
quantum algorithms offering a quadratic advantage over Monte Carlo sampling. We present the first numerical study of these flows, in which we represent time-dependent potentials and states as tensor networks. At spatial dimension $d=8$, storage falls by $\sim 10^7\times$ relative to the dense grid of $N^d$ points, and
evolution wall-clock time falls by $\gtrsim 10^3\times$ against a baseline extrapolated from the measured $d\le 5$ scaling. We validate our pipeline by reproducing the $O(1/\sqrt{p_{\rm rare}})$ scaling of rare-event sampling.
Read the full abstract ▾
All authors Nathan X. Kodama, L. Andrew Wray, Sam Cochran, Chad Rigetti, Shravan Veerapaneni, Michael J. Keiser