Computation and Language · Cross-listed from cs.AI
Kaixuan Fan, Kaituo Feng, Manyuan Zhang et al. (+7 more)
Agentic Reinforcement Learning (Agentic RL) has achieved notable success in enabling
agents to perform complex
reasoning and tool use. However, most methods still relies on sparse outcome-based reward for training. Such feedback fails to differentiate intermediate
reasoning quality, leading to suboptimal training results. In this paper, we introduce
Agent Reasoning Reward Model (
Agent-RRM), a multi-faceted reward model that produces structured feedback for agentic trajectories, including (1) an explicit
reasoning trace , (2) a focused critique that provides refinement guidance by highlighting
reasoning flaws, and (3) an overall score that evaluates process performance. Leveraging these signals, we systematically investigate three integration strategies: Reagent-C (text-augmented refinement), Reagent-R (reward-augmented guidance), and Reagent-U (unified feedback integration). Extensive evaluations across 12 diverse
benchmarks demonstrate that Reagent-U yields substantial performance leaps, achieving 43.7% on
GAIA and 46.2% on WebWalkerQA, validating the effectiveness of our
reasoning reward model and training schemes. Code, models, and
datasets are all released to facilitate future research.
Read the full abstract ▾
All authors Kaixuan Fan, Kaituo Feng, Manyuan Zhang, Tianshuo Peng, Zhixun Li, Yilei Jiang, Shuang Chen, Peng Pei, Xunliang Cai, Xiangyu Yue
Computation and Language
DynaWeb: Model-Based Reinforcement Learning of Web Agents
Hang Ding, Peidong Liu, Junqiao Wang et al. (+7 more)
The development of autonomous web
agents, powered by Large
Language Models (LLMs) and reinforcement learning (RL), represents a significant step towards general-purpose AI assistants. However, training these
agents is severely hampered by the challenges of interacting with the live internet, which is inefficient, costly, and fraught with risks. Model-based reinforcement learning (MBRL) offers a promising solution by learning a world model of the environment to enable simulated interaction. This paper introduces DynaWeb, a novel MBRL framework that trains web
agents through interacting with a web world model trained to predict naturalistic web page representations given
agent actions. This model serves as a synthetic web environment where an
agent policy can dream by generating vast quantities of rollout action trajectories for efficient online reinforcement learning. Beyond free policy rollouts, DynaWeb incorporates real expert trajectories from training data, which are randomly interleaved with on-policy rollouts during training to improve stability and sample efficiency. Experiments conducted on the challenging WebArena and WebVoyager
benchmarks demonstrate that DynaWeb consistently and significantly improves the performance of state-of-the-art open-source web
agent models. Our findings establish the viability of training web
agents through imagination, offering a scalable and efficient way to scale up online agentic RL.
Read the full abstract ▾
All authors Hang Ding, Peidong Liu, Junqiao Wang, Ziwei Ji, Meng Cao, Rongzhao Zhang, Lynn Ai, Eric Yang, Tianyu Shi, Lei Yu
Computation and Language
FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
Ajay Patel, Colin Raffel and Chris Callison-Burch
Due to limited supervised training data, large
language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data. To make the resulting model useful to users, it is further trained on a far smaller amount of "instruction-tuning" data comprised of supervised training examples of instructions and responses. To overcome the limited amount of supervised data, we propose a procedure that can transform the knowledge in internet-scale pre-training documents into billions of synthetic instruction and answer training pairs. The resulting
dataset, called FineInstructions, uses ~18M instruction templates created from real user-written queries and
prompts. These instruction templates are matched to and instantiated with human-written source documents from unstructured pre-training corpora. With "supervised" synthetic training data generated at this scale, an LLM can be pre-trained from scratch solely with the instruction-tuning objective, which is far more in-distribution with the expected downstream usage of LLMs (responding to user
prompts). We conduct controlled
token-for-
token training experiments and find pre-training on FineInstructions outperforms standard pre-training and other proposed synthetic pre-training techniques on standard
benchmarks measuring free-form response quality. Our resources can be found at https://huggingface.co/fineinstructions .
Read the full abstract ▾
Computation and Language
Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers
Xin Chen, Feng Jiang, Yiqian Zhang et al. (+5 more)
Reasoning-oriented Large
Language Models (LLMs) have achieved remarkable progress with Chain-of-Thought (CoT) prompting, yet they remain fundamentally limited by a blind self-thinking paradigm: performing extensive internal
reasoning even when critical information is missing or ambiguous. We propose Proactive Interactive
Reasoning (PIR), a new
reasoning paradigm that transforms LLMs from passive solvers into proactive inquirers that interleave
reasoning with clarification. Unlike existing search- or tool-based frameworks that primarily address knowledge uncertainty by querying external environments, PIR targets premise- and intent-level uncertainty through direct interaction with the user. PIR is implemented via two core components: (1) an uncertainty-aware supervised fine-tuning procedure that equips models with interactive
reasoning capability, and (2) a user-simulator-based policy optimization framework driven by a composite reward that aligns model behavior with user intent. Extensive experiments on mathematical
reasoning, code generation, and document editing demonstrate that PIR consistently outperforms strong baselines, achieving up to 32.70% higher accuracy, 22.90% higher pass rate, and 41.36 BLEU improvement, while reducing nearly half of the
reasoning computation and unnecessary interaction turns. Further reliability evaluations on factual knowledge, question answering, and missing-premise scenarios confirm the strong generalization and robustness of PIR. Model and code are publicly available at: \href{https://github.com/SUAT-AIRI/Proactive-Interactive-R1}
Read the full abstract ▾
All authors Xin Chen, Feng Jiang, Yiqian Zhang, Hardy Chen, Shuo Yan, Wenya Xie, Min Yang, Shujian Huang
Computation and Language
Toward Federated Large Language Models in Medicine: A Parameter-Efficient Framework for Privacy-Preserving, Multi-Institutional Adaptation
Anran Li, Yuanyuan Chen, Wenjun Long et al. (+16 more)
Large
language models (LLMs) are increasingly adapted for medical applications, but most are trained using data from a single institution because
privacy and governance constraints prevent multi-institutional data sharing. As a result, these models often generalize poorly across heterogeneous healthcare systems. We address this gap by introducing Fed-MedLoRA and Fed-MedLoRA+, a parameter-efficient
federated framework for collaborative LLM adaptation across healthcare institutions. Fed-MedLoRA transmits only low-rank adapters rather than full model weights, reducing communication overhead. We also evaluate a
privacy-preserving variant that applies Gaussian perturbation to transmitted adapter updates. Fed-MedLoRA+ further incorporates adaptive aggregation to better address cross-site heterogeneity in patient populations, annotation practices, and disease distributions. We evaluate the framework on clinical information extraction across five independent patient cohorts totaling 42,198 entities and 41,570 relations, and compare it with zero-shot and fine-tuned LLMs, domain-specific BERT models, and
federated baselines. Across all settings, the proposed methods consistently improve extraction performance and generalize better to heterogeneous cohorts. In a real-world case study using clinical notes from the Yale New Haven Health System, the framework demonstrates strong performance under low-resource new-site deployment. These results suggest that
federated, parameter-efficient LLM adaptation is feasible, scalable, and effective for multi-institutional clinical deployment.
Read the full abstract ▾
All authors Anran Li, Yuanyuan Chen, Wenjun Long, Yu Yin, Yan Hu, Hyunjae Kim, Weipeng Zhou, Yujia Zhou, Hongyi Peng, Yang Ren, Xuguang Ai, Zhenyue Qin, Ming Hu, Xiaoxiao Li, Han Yu, Yih-Chung Tham, Lucila Ohno-Machado, Hua Xu, Qingyu Chen
Computation and Language
ECO: Quantized Training without Full-Precision Master Weights
Mahdi Nikdan, Amir Zandieh, Dan Alistarh et al. (+1 more)
Quantization has significantly improved the compute and
memory efficiency of Large
Language Model (LLM) training. However, existing approaches still rely on accumulating their updates in high-precision: concretely, gradient updates must be applied to a high-precision weight buffer, known as $\textit{master weights}$. This buffer introduces substantial
memory overhead, particularly for Sparse Mixture of Experts (SMoE) models, where model parameters and optimizer states dominate
memory usage. To address this, we introduce the Error-Compensating Optimizer (ECO), which eliminates master weights by applying updates directly to quantized parameters. ECO quantizes weights after each step and carefully injects the resulting quantization error into the optimizer momentum, forming an error-feedback loop with no additional
memory. We prove that, under standard assumptions and a decaying learning rate, ECO converges to a constant-radius neighborhood of the optimum, while naive master-weight removal can incur an error that is inversely proportional to the learning rate. We show empirical results for pretraining small
Transformers (30-800M), a Gemma-3 1B model, and a 2.1B parameter Sparse MoE model with FP8 quantization, and fine-tuning DeepSeek-MoE-16B in INT4 precision. Throughout, ECO matches baselines with master weights up to near-lossless accuracy, significantly shifting the static
memory vs validation loss Pareto frontier.
Read the full abstract ▾
All authors Mahdi Nikdan, Amir Zandieh, Dan Alistarh, Vahab Mirrokni
Computation and Language · Cross-listed from cs.LG
GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
Chuanyang Zheng, Jiankai Sun, Yihang Gao et al. (+11 more)
The placement of normalization layers, specifically Pre-Norm and Post-Norm, remains an open question in
Transformer architecture design. In this work, we rethink these approaches through the lens of
manifold optimization, interpreting the outputs of the Feed-Forward Network (FFN) and
attention layers as update directions in optimization. Building on this perspective, we introduce GeoNorm, a novel method that replaces standard normalization with geodesic updates on the
manifold. Furthermore, analogous to learning rate schedules, we propose a layer-wise update decay for the FFN and
attention components. Comprehensive experiments demonstrate that GeoNorm consistently outperforms existing normalization methods in
Transformer models. Crucially, GeoNorm can be seamlessly integrated into standard
Transformer architectures, achieving performance improvements with negligible additional computational cost.
Read the full abstract ▾
All authors Chuanyang Zheng, Jiankai Sun, Yihang Gao, Chi Wang, Yuehao Wang, Jing Xiong, Liliang Ren, Bo Peng, Qingmei Wang, Xiaoran Shang, Mac Schwager, Anderson Schneider, Yuriy Nevmyvaka, Xiaodong Liu
Computation and Language
VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning
Yibo Wang, Yongcheng Jing, Shunyu Liu et al. (+5 more)
Long-context
reasoning has significantly empowered large
language models (LLMs) to tackle complex tasks, yet it introduces severe efficiency bottlenecks due to the computational complexity. Existing efficient approaches often rely on complex additional training or external models for compression, which limits scalability and discards critical fine-grained information. In this paper, we propose VTC-R1, a new efficient
reasoning paradigm that integrates
vision-text compression into the
reasoning process. Instead of processing lengthy textual traces, VTC-R1 renders intermediate
reasoning segments into compact images, which are iteratively fed back into
vision-
language models as "optical
memory." We construct a training
dataset based on OpenR1-Math-220K achieving 3.4x
token compression and fine-tune representative VLMs-Glyph and Qwen3-VL. Extensive experiments on
benchmarks such as MATH500, AIME25, AMC23 and GPQA-D demonstrate that VTC-R1 consistently outperforms standard long-context
reasoning. Furthermore, our approach significantly improves inference efficiency, achieving 2.7x speedup in end-to-end latency, highlighting its potential as a scalable solution for
reasoning-intensive applications. Our code is available at https://github.com/w-yibo/VTC-R1.
Read the full abstract ▾
All authors Yibo Wang, Yongcheng Jing, Shunyu Liu, Hao Guan, Rong-cheng Tu, Chengyu Wang, Jun Huang, Dacheng Tao
Computation and Language
$G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA
Yaxin Du, Junru Song, Yifan Zhou et al. (+8 more)
Retrieval-augmented generation is a practical paradigm for question answering over long documents, but it remains brittle for
multimodal reading where text, tables, and figures are interleaved across many pages. First, flat chunking breaks document-native structure and cross-modal
alignment, yielding semantic fragments that are hard to interpret in isolation. Second, even iterative
retrieval can fail in long contexts by looping on partial evidence or drifting into irrelevant sections as noise accumulates, since each step is guided only by the current snippet without a persistent global search state. We introduce $G^2$-Reader, a dual-graph system, to address both issues. It evolves a Content Graph to preserve document-native structure and cross-modal semantics, and maintains a Planning Graph, an agentic directed acyclic graph of sub-questions, to track intermediate findings and guide stepwise
navigation for evidence completion. On VisDoMBench across five
multimodal domains, $G^2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%).
Read the full abstract ▾
All authors Yaxin Du, Junru Song, Yifan Zhou, Cheng Wang, Jiahao Gu, Zimeng Chen, Menglan Chen, Wen Yao, Yang Yang, Ying Wen, Siheng Chen
Computation and Language
MasalBench: A Benchmark for Contextual and Cross-Cultural Understanding of Persian Proverbs in LLMs
Ghazal Kalhor and Behnam Bahrak
In recent years,
multilingual Large
Language Models (LLMs) have become an inseparable part of daily life, making it crucial for them to master the rules of conversational
language in order to communicate effectively with users. While previous work has evaluated LLMs' understanding of figurative
language in high-resource
languages, their performance in low-resource
languages remains underexplored. In this paper, we introduce MasalBench, a comprehensive
benchmark for assessing LLMs' contextual and cross-cultural understanding of Persian proverbs, which are a key component of conversation in this low-resource
language. We evaluate eight state-of-the-art LLMs on MasalBench and find that they perform well in identifying Persian proverbs in context, achieving accuracies above 0.90. However, their performance drops considerably when tasked with identifying equivalent English proverbs, with the best model achieving 0.79 accuracy. Our findings highlight the limitations of current LLMs in cultural knowledge and analogical
reasoning, and they provide a framework for assessing cross-cultural understanding in other low-resource
languages. MasalBench is available at https://github.com/kalhorghazal/MasalBench.
Read the full abstract ▾
Computation and Language
On the Paradoxical Interference between Instruction-Following and Task Solving
Yunjia Qi, Hao Peng, Xintong Shi et al. (+5 more)
Instruction following aims to align Large
Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed. However, we reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability. We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving. It measures task performance drop after inserting into the instruction a self-evident constraint, which is naturally met by the original successful model output and extracted from it. Experiments on current LLMs in mathematics, multi-hop QA, and code generation show that adding the self-evident constraints leads to substantial performance drops, even for advanced models such as Claude-Sonnet-4.5. We validate the generality of the interference across constraint types and scales. Furthermore, we identify common failure patterns, and by investigating the mechanisms of interference, we observe that failed cases allocate significantly more
attention to constraints compared to successful ones. Finally, we use SUSTAINSCORE to conduct an initial investigation into how distinct post-training paradigms affect the interference, presenting empirical observations on current
alignment strategies. We will release our code and data to facilitate further research
Read the full abstract ▾
All authors Yunjia Qi, Hao Peng, Xintong Shi, Amy Xin, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
Computation and Language
Leviathan: Decoupling Input and Output Representations in Language Models
Reza T. Batley and Sourav Saha
Modern
language models use a single
matrix for input
embedding and output projection. This couples two distinct objectives:
token representation and discrimination over a vocabulary. This work introduces Leviathan, a
Transformer architecture that replaces the input
embedding matrix with learned
embedding vectorization (LEV), a compact continuous mapping from
token indices to
embeddings. Leviathan's output head remains untied for a parameter increase of as low as 0.2%. Under controlled comparisons with identical
Transformer backbones, Leviathan consistently improves
language modeling performance over standard tied-
embedding baselines across a 200M-1.2B parameter regime on The Pile with gains that grow during training. At 1.2B scale, Leviathan reduces validation perplexity by 9%, requires $2.1\times$ fewer training
tokens to reach the tied baseline's final loss, and improves on all six downstream
benchmarks evaluated, including a 30% reduction in LAMBADA perplexity. Frequency-stratified analysis reveals gains to be concentrated in rare
tokens, where continuous parameterization reduces perplexity by 81%, falling to near zero for the most frequent.
Read the full abstract ▾
Computation and Language
Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models
Longxuan Yu, Yu Fu, Shaorong Zhang et al. (+4 more)
Autoregressive (AR)
language models enforce a fixed left-to-right generation order, creating a fundamental limitation when the required output structure conflicts with natural
reasoning (e.g., producing answers before explanations due to presentation or schema constraints). In such cases, AR models must commit to answers before generating intermediate
reasoning, and this rigid constraint forces premature commitment. Masked
diffusion language models (MDLMs), which iteratively refine all
tokens in parallel, offer a way to decouple computation order from output structure. We validate this capability on GSM8K, Math500, and ReasonOrderQA, a
benchmark we introduce with controlled difficulty and order-level evaluation. When
prompts request answers before
reasoning, AR models exhibit large accuracy gaps compared to standard chain-of-thought ordering (up to 67% relative drop), while MDLMs remain stable ($\leq$14% relative drop), a property we term "order robustness". Using ReasonOrderQA, we present evidence that MDLMs achieve order robustness by stabilizing simpler
tokens (e.g.,
reasoning steps) earlier in the
diffusion process than complex ones (e.g., final answers), enabling
reasoning tokens to stabilize before answer commitment. Finally, we identify failure conditions where this advantage weakens, outlining the limits required for order robustness.
Read the full abstract ▾
All authors Longxuan Yu, Yu Fu, Shaorong Zhang, Hui Liu, Mukund Varma T, Greg Ver Steeg, Yue Dong
Computation and Language
Junhao Ruan, Bei Li, Yongjing Yin et al. (+6 more)
In this work, we propose Causal Autoregressive
Diffusion (CARD), a novel framework that unifies the training efficiency of ARMs with the high-throughput inference of
diffusion models. CARD reformulates the
diffusion process within a strictly causal
attention mask, enabling dense, per-
token supervision in a single forward pass. To address the optimization instability of causal
diffusion, we introduce a soft-tailed masking schema to preserve local context and a context-aware reweighting mechanism derived from signal-to-noise principles. This design enables dynamic parallel decoding, where the model leverages KV-caching to adaptively generate variable-length
token sequences based on confidence. Empirically, CARD outperforms existing discrete
diffusion baselines while reducing training latency by 3 $\times$ compared to block
diffusion methods. Our results demonstrate that CARD achieves ARM-level data efficiency while unlocking the latency benefits of parallel generation, establishing a robust paradigm for next-generation efficient LLMs.
Read the full abstract ▾
All authors Junhao Ruan, Bei Li, Yongjing Yin, Pengcheng Huang, Xin Chen, Jingang Wang, Xunliang Cai, Tong Xiao, JingBo Zhu
Computation and Language
When Generic Prompt Improvements Hurt: Evaluation-Driven Iteration for LLM Applications
Daniel Commey
Evaluating Large
Language Model (LLM) applications differs from conventional software testing because outputs are probabilistic, semantically variable, and sensitive to
prompt and model changes. This technical report proposes the Minimum Viable Evaluation Suite (MVES), an audit-oriented structure for application-level LLM evaluation. MVES links application categories to failure modes, metrics, required artifacts, and validation evidence across general LLM applications,
retrieval-augmented systems, and agentic workflows. We pair the framework with a reproducible local evaluation harness covering structured extraction, RAG citation/content-compliance, and instruction-following checks. Using Ollama with Llama 3 8B Instruct and Qwen 2.5 7B Instruct, we evaluate five
prompt conditions over expanded 30-case-per-suite ablations. The results show that, in the tested local conditions, generic
prompt additions do not produce monotonic improvements: stronger output-contract
prompts improve strict extraction for both models, while RAG citation/content-compliance declines under some generic-rule conditions. The largest observed decline occurs for Qwen 2.5 on RAG when generic rules are appended to the user
prompt, from 26/30 to 9/30. These findings support evaluation-driven
prompt iteration:
prompt changes should be treated as potential regression risks and tested against task-specific suites before deployment. The accompanying repository contains the test suites,
prompt variants, evaluation harness, raw result logs, and scripts needed to reproduce the reported local ablations.
Read the full abstract ▾
Computation and Language
Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units
Jianhui Chen, Yuzhang Luo and Liangming Pan
While Mechanistic Interpretability has identified interpretable
circuits in LLMs, their causal origins in training data remain elusive. We introduce Mechanistic Data Attribution (MDA), a scalable framework that employs Influence Functions to trace interpretable units back to specific training samples. Through extensive experiments on the Pythia family, we causally validate that targeted intervention–removing or augmenting a small fraction of high-influence samples–significantly modulates the emergence of interpretable heads, whereas random interventions show no effect. Our analysis reveals that repetitive structural data (e.g., LaTeX, XML) acts as a mechanistic catalyst. Furthermore, we observe that interventions targeting induction head formation induce a concurrent change in the model's in-context learning (ICL) capability. This provides direct causal evidence for the long-standing hypothesis regarding the functional link between induction heads and ICL. Finally, we propose a mechanistic data augmentation pipeline that consistently accelerates
circuit convergence across model scales, providing a principled methodology for steering the developmental trajectories of LLMs.
Read the full abstract ▾
Computation and Language
Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding
Yifan Zhu, Huiqiang Rong and Haoran Luo
Large
Language Models (LLMs) often hallucinate, generating content inconsistent with the input.
Retrieval-Augmented Generation (RAG) and Reinforcement Learning with Human Feedback (RLHF) can mitigate
hallucinations but require resource-intensive
retrieval or large-scale fine-tuning. Decoding-based methods are lighter yet lack explicit
hallucination control. To address this, we present
Token-Guard, a
token-level
hallucination control method based on self-checking decoding.
Token-Guard performs internal verification at each
reasoning step to detect hallucinated
tokens before they propagate. Candidate fragments are further evaluated in a latent space with explicit
hallucination risk scoring, while iterative
pruning and regeneration dynamically correct detected errors. Experiments on HALU
datasets show
Token-Guard substantially reduces
hallucinations and improves generation accuracy, offering a scalable, modular solution for reliable LLM outputs. Our code is publicly available.
Read the full abstract ▾
Computation and Language
Jing Xiong, Hui Shen, Shansan Gong et al. (+7 more)
Knowledge
distillation transfers
reasoning capabilities from large teachers to efficient students. However,
token-level on-policy
distillation (OPD) constrains student exploration and requires teacher
token probabilities, precluding
distillation from black-box teachers that provide only text outputs. We introduce On-policy Verbal
Distillation (OVD), a framework that uses verbal scores from black-box teachers to rank student-generated sub-trajectories, retaining high-scoring ones and replacing low-scoring ones with teacher-generated continuations. We analyze when ranking induced by verbal scores can guide distribution approximation: under a density-ratio calibration condition on acceptance probabilities and bounded teacher-replacement error, we bound the approximation error between the resulting mixed trajectory distribution and a teacher-preferred target. On Web Q&A, OVD achieves 41.09% average EM with teacher feedback at inference, exceeding the strongest evaluated baseline by 5.89 percentage points. On AMC23, OVD-FR improves accuracy over RLVR by 10.0 percentage points (52.5% to 62.5%) after 600 training steps on 128 problems. Further experiments suggest that retaining student-generated prefixes helps preserve exploration and mitigate trajectory-level
entropy collapse. OVD also improves training efficiency: resampling selected suffixes rather than entire responses reduces mean per-step training time by 10.2% in the 128-problem setting. Project page: https://menik1126.github.io/ovd-project-page/.
Read the full abstract ▾
All authors Jing Xiong, Hui Shen, Shansan Gong, Yuxin Cheng, Jianghan Shen, Chaofan Tao, Haochen Tan, Haoli Bai, Lifeng Shang, Ngai Wong
Computation and Language · Cross-listed from cs.CY
Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems
Alexander Loth, Martin Kappes and Marc-Oliver Pahl
Generative AI and misinformation research have evolved since our 2024 survey. This paper presents an updated perspective, transitioning from literature review to practical countermeasures. We report on changes in the threat landscape, including improved AI-generated content through Large
Language Models (LLMs) and
multimodal systems - addressing the adverse impacts of AI on information quality. This work focuses on our practical contributions: JudgeGPT, a platform to evaluate human perception of AI-generated news, and RogueGPT, a controlled stimulus generation engine for research. Together, these tools form an experimental pipeline for studying how humans perceive and detect AI-generated misinformation. Our findings show that detection capabilities have improved, but competition between generation and detection continues. We discuss mitigation strategies such as LLM-based detection, inoculation approaches, and the dual-use nature of generative AI.
Read the full abstract ▾
Computation and Language
From Generative Modeling to Clinical Classification: A GPT-Based Architecture for EHR Notes
Fariba Afrin Irany and Sampson Akwafuo
The increasing availability of unstructured clinical narratives in electronic health records (EHRs) has created new opportunities for automated disease characterization, cohort identification, and clinical decision support. However, modeling long, domain-specific clinical text remains challenging due to limited labeled data, severe class imbalance, and the high computational cost of adapting large pretrained
language models. This study presents a GPT-based architecture for clinical text classification that adapts a pretrained decoder-only
Transformer using a selective fine-tuning strategy. Rather than updating all model parameters, the majority of the GPT-2 backbone is frozen, and training is restricted to the final
Transformer block, the final layer normalization, and a lightweight classification head. This approach substantially reduces the number of trainable parameters while preserving the representational capacity required to model complex clinical
language. The proposed method is evaluated on radiology reports from the MIMIC-IV-Note
dataset using uncertainty-aware CheXpert-style labels derived directly from report text. Experiments cover multiple problem formulations, including multi-label classification of radiographic findings, binary per-label classification under different uncertainty assumptions, and aggregate disease outcome prediction. Across varying
dataset sizes, the model exhibits stable convergence behavior and strong classification performance, particularly in settings dominated by non-mention and negated findings. Overall, the results indicate that selective fine-tuning of pretrained generative
language models provides an efficient and effective pathway for clinical text classification, enabling scalable adaptation to real-world EHR data while significantly reducing computational complexity.
Read the full abstract ▾
Computation and Language
SONIC: Segmented Optimized Nexus for Information Compression in Key-Value Caching
Hong Chen, Xiang Liu, Bo Wang et al. (+5 more)
The linear growth of Key-Value (KV) cache remains a bottleneck for multi-turn LLM deployment. Existing KV cache compression methods often fail to account for the structural properties of multi-turn dialogues, relying on heuristic eviction that risks losing critical context. We propose SONIC, a learning-based framework that compresses historical segments into compact and semantically rich Nexus
tokens. By integrating dynamic budget training, SONIC allows flexible adaptation to varying
memory constraints without retraining. Experiments show that at compression ratios of 80% and 50%, SONIC consistently outperforms baselines such as H2O and StreamingLLM on four diverse multi-turn
benchmarks. Specifically, on the widely used MTBench101
benchmark, SONIC achieves an average score improvement of 35.55% over state-of-the-art baselines, validating its effectiveness in sustaining coherent multi-turn dialogues. Furthermore, SONIC enhances deployment efficiency, accelerating the overall inference process by 50.1% compared to full-context generation.
Read the full abstract ▾
All authors Hong Chen, Xiang Liu, Bo Wang, Yuxuan Fan, Yuanlin Chu, Zongluo Li, Xiaowen Chu, Xuming Hu
Computation and Language · Cross-listed from cs.AI
Self-Compression of Chain-of-Thought via Multi-Agent Reinforcement Learning
Yiqun Chen, Jinyuan Feng, Wei Yang et al. (+9 more)
The inference overhead induced by redundant
reasoning undermines the interactive experience and severely bottlenecks the deployment of Large
Reasoning Models. Existing reinforcement learning (RL)-based solutions tackle this problem by coupling a length penalty with outcome-based rewards. This simplistic reward weighting struggles to reconcile brevity with accuracy, as enforcing brevity may compromise critical
reasoning logic. In this work, we address this limitation by proposing a multi-
agent RL framework that selectively penalizes redundant chunks, while preserving essential
reasoning logic. Our framework, Self-Compression via MARL (SCMA), instantiates redundancy detection and evaluation through two specialized
agents: a
Segmentation Agent for decomposing the
reasoning process into logical chunks, and a Scoring
Agent for quantifying the significance of each chunk. The
Segmentation and Scoring
agents collaboratively define an importance-weighted length penalty during training, incentivizing a
Reasoning Agent to prioritize essential logic without introducing inference overhead during deployment. Empirical evaluations across model scales demonstrate that SCMA reduces response length by 11.1% to 39.0% while boosting accuracy by 4.33% to 10.02%. Furthermore, ablation studies and qualitative analysis validate that the synergistic optimization within the MARL framework fosters emergent behaviors, yielding more powerful LRMs compared to vanilla RL paradigms.
Read the full abstract ▾
All authors Yiqun Chen, Jinyuan Feng, Wei Yang, Meizhi Zhong, Zhengliang Shi, Rui Li, Xiaochi Wei, Yan Gao, Yi Wu, Yao Hu, Zhiqiang Pu, Jiaxin Mao
Computation and Language · Cross-listed from cs.AI
JADE: Bridging the Strategic-Operational Gap in Dynamic Agentic RAG
Yiqun Chen, Erhan Zhang, Tianyi Hu et al. (+8 more)
The
evolution of
Retrieval-Augmented Generation (RAG) has shifted from static
retrieval pipelines to dynamic, agentic workflows where a central planner orchestrates multi-turn
reasoning. However, existing paradigms face a critical dichotomy: they either optimize modules jointly within rigid, fixed-graph architectures, or empower dynamic planning while treating executors as frozen, black-box tools. We identify that this decoupled optimization creates a “strategic-operational mismatch,” where sophisticated planning strategies fail to materialize due to unadapted local executors, often leading to negative performance gains despite increased system complexity. In this paper, we propose JADE (Joint Agentic Dynamic Execution), a unified framework for the joint optimization of planning and execution within dynamic, multi-turn workflows. By modeling the system as a cooperative multi-
agent team unified under a single shared backbone, JADE enables end-to-end learning driven by outcome-based rewards. This approach facilitates co-adaptation: the planner learns to operate within the capability boundaries of the executors, while the executors evolve to align with high-level strategic intent. Empirical results demonstrate that JADE transforms disjoint modules into a synergistic system, yielding remarkable performance improvements via joint optimization and enabling a flexible balance between efficiency and effectiveness through dynamic workflow orchestration.
Read the full abstract ▾
All authors Yiqun Chen, Erhan Zhang, Tianyi Hu, Shijie Wang, Zixuan Yang, Meizhi Zhong, Xiaochi Wei, Yan Gao, Yi Wu, Yao Hu, Jiaxin Mao
Computation and Language · Cross-listed from cs.AI
ProRAG: Process-Supervised Reinforcement Learning for Retrieval-Augmented Generation
Zhao Wang, Ziliang Zhao and Zhicheng Dou
Reinforcement learning (RL) has become a promising paradigm for optimizing
Retrieval-Augmented Generation (RAG) in complex
reasoning tasks. However, traditional outcome-based RL approaches often suffer from reward sparsity and inefficient credit assignment, as coarse-grained scalar rewards fail to identify specific erroneous steps within long-horizon trajectories. This ambiguity frequently leads to "process
hallucinations", where models reach correct answers through flawed logic or redundant
retrieval steps. Although recent process-aware approaches attempt to mitigate this via static preference learning or heuristic reward shaping, they often lack the on-policy exploration capabilities required to decouple step-level credit from global outcomes. To address these challenges, we propose ProRAG, a process-supervised reinforcement learning framework designed to integrate learned step-level supervision into the online optimization loop. Our framework consists of four stages: (1) Supervised Policy Warmup to initialize the model with a structured
reasoning format; (2) construction of an MCTS-based Process Reward Model (PRM) to quantify intermediate
reasoning quality; (3) PRM-Guided
Reasoning Refinement to align the policy with fine-grained process preferences; and (4) Process-Supervised Reinforcement Learning with a dual-granularity advantage mechanism. By aggregating step-level process rewards with global outcome signals, ProRAG provides precise feedback for every action. Extensive experiments on five multi-hop
reasoning benchmarks demonstrate that ProRAG achieves superior overall performance compared to strong outcome-based and process-aware RL baselines, particularly on complex long-horizon tasks, validating the effectiveness of fine-grained process supervision. The code and model are available at https://github.com/lilinwz/ProRAG.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoning
Shaojie Wang and Liang Zhang
Current LLM post-training methods optimize complete
reasoning trajectories through Supervised Fine-Tuning (SFT) followed by outcome-based Reinforcement Learning (RL). While effective, a closer examination reveals a fundamental gap: this approach does not align with how humans actually solve problems. Human cognition naturally decomposes problem-solving into two distinct stages: first acquiring abstract strategies (i.e., meta-knowledge) that generalize across problems, then adapting them to specific instances. In contrast, by treating complete trajectories as basic units, current methods are inherently problem-centric, entangling abstract strategies with problem-specific execution. To address this misalignment, we propose a cognitively-inspired framework that explicitly mirrors the two-stage human cognitive process. Specifically, Chain-of-Meta-Thought CoMT focuses supervised learning on abstract
reasoning patterns without specific executions, enabling acquisition of generalizable strategies. Confidence-Calibrated Reinforcement Learning (CCRL) then optimizes task adaptation via confidence-aware rewards on intermediate steps, preventing overconfident errors from cascading and improving execution reliability. Experiments across four models and ten
benchmarks show 2.10% and 3.86% improvements in-distribution and out-of-distribution respectively over standard methods, while remaining highly robust to variations in teacher model selection, optimization methods, and symbolic perturbations.
Read the full abstract ▾
Computation and Language
Learn-to-Distance: Distance Learning for Detecting LLM-Generated Text
Hongyi Zhou, Jin Zhu, Kai Ye et al. (+3 more)
Modern large
language models (LLMs) such as GPT, Claude, and Gemini have transformed the way we learn, work, and communicate. Yet, their ability to produce highly human-like text raises serious concerns about misinformation and academic integrity, making it an urgent need for reliable
algorithms to detect LLM-generated content. In this paper, we start by presenting a
geometric approach to demystify rewrite-based detection
algorithms, revealing their underlying rationale and demonstrating their generalization ability. Building on this insight, we introduce a novel rewrite-based detection
algorithm that adaptively learns the distance between the original and rewritten text. Theoretically, we demonstrate that employing an adaptively learned distance function is more effective for detection than using a fixed distance. Empirically, we conduct extensive experiments with over 100 settings, and find that our approach demonstrates superior performance over baseline
algorithms in the majority of scenarios. In particular, it achieves relative improvements from 54.3% to 75.4% over the strongest baseline across different target LLMs (e.g., GPT, Claude, and Gemini). A python implementation of our proposal is publicly available at https://github.com/Mamba413/L2D.
Read the full abstract ▾
All authors Hongyi Zhou, Jin Zhu, Kai Ye, Ying Yang, Erhan Xu, Chengchun Shi
Computation and Language
Embodied Task Planning via Graph-Informed Action Generation with Large Language Models
Xiang Li, Ning Yan and Masood Mortazavi
While Large
Language Models (LLMs) have demonstrated strong zero-shot
reasoning capabilities, their deployment as
embodied agents still faces fundamental challenges in long-horizon planning. Unlike open-ended text generation,
embodied agents must decompose high-level intents into actionable sub-goals while adhering to the constraints of a dynamic environment. Standard LLM planners frequently fail to maintain strategy coherence over extended horizons due to context window limitations or hallucinate state transitions that violate environment constraints. We propose GiG, a planning framework that structures
embodied agents'
memory using a Graph-in-Graph architecture. Our approach employs a Graph Neural Network (GNN) to encode environmental states into
embeddings, organizing these
embeddings into action-connected execution trace graphs within an experience
memory bank. GiG enables
retrieval of structurally-similar priors, allowing
agents to ground current decisions in relevant past structural patterns. Furthermore, we introduce a bounded lookahead module that leverages symbolic transition logic to enhance the
agent's planning capabilities through grounded action projections. We evaluate our framework on three
embodied planning
benchmarks-Robotouille Synchronous, Robotouille Asynchronous, and ALFWorld. Our method outperforms state-of-the-art baselines, achieving Pass@1 performance gains of up to 22% on Robotouille Synchronous, 37% on Asynchronous, and 15% on ALFWorld while maintaining comparable or lower computational cost.
Read the full abstract ▾
Computation and Language
Mil-SCORE: Benchmarking Long-Context Geospatial Reasoning and Planning in Large Language Models
Aadi Palnitkar, Mingyang Mao, Nicholas Waytowich et al. (+2 more)
As large
language models (LLMs) are applied to increasingly longer and more complex tasks, there is a growing need for realistic long-context
benchmarks that require selective reading and integration of heterogeneous, multi-modal information sources. This need is especially acute for geospatial planning problems, such as those found in planning for large-scale military operations, which demand fast and accurate
reasoning over maps, orders, intelligence reports, and other distributed data. To address this gap, we present MilSCORE (Military Scenario Contextual
Reasoning), to our knowledge the first scenario-level
dataset of expert-authored, multi-hop questions grounded in a complex, simulated military planning scenario used for training. MilSCORE is designed to evaluate high-stakes decision-making and planning, probing LLMs' ability to combine tactical and spatial
reasoning across multiple sources and to reason over long-horizon, geospatially rich context. The
benchmark includes a diverse set of question types across seven categories targeting both factual recall and multi-step
reasoning about constraints, strategy, and spatial analysis. We provide an evaluation protocol and report baseline results for a range of contemporary
vision-
language models. Our findings highlight substantial headroom on MilSCORE, indicating that current systems struggle with realistic, scenario-level long-context planning, and positioning MilSCORE as a challenging testbed for future work.
Read the full abstract ▾
All authors Aadi Palnitkar, Mingyang Mao, Nicholas Waytowich, Vinicius G. Goecks, Xiaomin Lin
Computation and Language · Cross-listed from cs.CY
Moral Outrage Shapes Commitments Beyond Attention: Multimodal Moral Emotions on YouTube in Korea and the US
Seongchan Park, Jaehong Kim, Hyeonseung Kim et al. (+3 more)
Understanding how media rhetoric shapes audience engagement is crucial in the
attention economy. This study examines how moral emotional framing by mainstream news channels on YouTube influences user behavior across Korea and the United States. To capture the platform's
multimodal nature, combining thumbnail images and
video titles, we develop a
multimodal moral
emotion classifier by fine tuning a
vision language model. The model is trained on human annotated
multimodal datasets in both
languages and applied to approximately 400,000
videos from major news outlets. We analyze engagement levels including views, likes, and comments, representing increasing degrees of commitment. The results show that other condemning rhetoric expressions of moral outrage that criticize others morally consistently increase all forms of engagement across cultures, with effects ranging from passive viewing to active commenting. These findings suggest that moral outrage is a particularly effective emotional strategy, attracting not only
attention but also active participation. We discuss concerns about the potential misuse of other condemning rhetoric, as such practices may deepen polarization by reinforcing in group and out group divisions. To facilitate future research and ensure reproducibility, we publicly release our Korean and English
multimodal moral
emotion classifiers.
Read the full abstract ▾
All authors Seongchan Park, Jaehong Kim, Hyeonseung Kim, Heejin Bin, Sue Moon, Wonjae Lee
Computation and Language
Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning
Bodong Du, Xuanqi Huang and Xiaomeng Li
Test-time reinforcement learning (TTRL) enables large
language models (LLMs) to self-improve on unlabeled inputs, but its effectiveness critically depends on how reward signals are estimated without ground-truth supervision. Most existing TTRL methods rely on majority voting (MV) over rollouts to produce deterministic rewards, implicitly assuming that the majority rollout provides a reliable learning signal. We show that this assumption is fragile: MV reduces the rollout distribution into a single outcome, discarding information about non-majority but correct actions candidates, and yields systematically biased reward estimates. To address this, we propose Distribution-AwareReward Estimation (DARE), which shifts reward estimation from a single majority outcome to the full empirical rollout distribution. DARE further augments this distribution-based reward with an exploration bonus and a distribution
pruning mechanism for non-majority rollout exploration and reward denoise, yielding a more informative and robust reward estimation. Extensive experiments on challenging
reasoning benchmarks show that DARE improves optimization stability and final performance over recent baselines, achieving relative improvements of 25.3% on challenging AIME 2024 and 5.3% on AMC.
Read the full abstract ▾