Computation and Language · Cross-listed from cs.LG
Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models
Nianyi Lin, Jiajie Zhang, Lei Hou et al. (+1 more)
A key challenge in applying reinforcement learning (RL) to
diffusion large
language models (dLLMs) is the intractability of their likelihood functions, which are essential for the RL objective, necessitating corresponding approximation during training. While existing methods approximate the log-likelihoods by their evidence lower bounds (ELBOs) via customized Monte Carlo (MC) sampling, they incur significant
memory overhead due to the need to retain all MC samples for the gradient computation of non-linear terms in the RL objective, and thus restrict feasible sample sizes, leading to imprecise likelihood approximations and distorted RL objective. To address this, we propose Boundary-Guided Policy Optimization (BGPO), a
memory-efficient RL
algorithm that maximizes a specially constructed lower bound of the ELBO-based objective. This lower bound is carefully designed to satisfy two key properties: (1) Linearity: it is a linear sum where each term depends only on a single MC sample, thereby enabling gradient accumulation across samples and ensuring constant
memory usage; (2) Equivalence: Both the value and gradient of this lower bound are equal to those of the ELBO-based objective in on-policy training, making it also an effective approximation for the original RL objective. These properties allow BGPO to adopt a large MC sample size, improving likelihood approximations and RL objective estimation, which in turn leads to enhanced performance. Experiments show that BGPO significantly outperforms previous RL
algorithms for dLLMs in math problem solving, code generation, and planning tasks. Our codes and models are available at \href{https://github.com/THU-KEG/BGPO}{https://github.com/THU-KEG/BGPO}.
Read the full abstract ▾
All authors Nianyi Lin, Jiajie Zhang, Lei Hou, Juanzi Li
Computation and Language · Cross-listed from cs.IR
FinVet: A Collaborative Framework of RAG and External Fact-Checking Agents for Financial Misinformation Detection
Daniel Berhane Araya and Duoduo Liao
Financial
markets face growing threats from misinformation that can trigger billions in losses in minutes. Most existing approaches lack transparency in their decision-making and provide limited attribution to credible sources. We introduce FinVet, a novel multi-
agent framework that integrates two
Retrieval-Augmented Generation (RAG) pipelines with external fact-checking through a confidence-weighted voting mechanism. FinVet employs adaptive three-tier processing that dynamically adjusts verification strategies based on
retrieval confidence, from direct metadata extraction to hybrid
reasoning to full model-based analysis. Unlike existing methods, FinVet provides evidence-backed verdicts, source attribution, confidence scores, and explicit uncertainty flags when evidence is insufficient. Experimental evaluation on the FinFact
dataset shows that FinVet achieves an F1 score of 0.85, which is a 10.4% improvement over the best individual pipeline (fact-check pipeline) and 37% improvement over standalone RAG approaches.
Read the full abstract ▾
Computation and Language
ACADREASON: Exploring the Limits of Reasoning Models with Academic Research Problems
Xin Gui, King Zhu, JinCheng Ren et al. (+17 more)
In recent years, the research focus of large
language models (LLMs) and
agents has shifted increasingly from demonstrating novel capabilities to complex
reasoning and tackling challenging tasks. However, existing evaluations focus mainly on math/code contests or general tasks, while existing multi-domain academic
benchmarks lack sufficient
reasoning depth, leaving the field without a rigorous
benchmark for high-level
reasoning. To fill this gap, we introduce the Acadreason
benchmark, designed to evaluate the ability of LLMs and
agents to acquire and reason over academic knowledge. It consists of 50 expert-annotated academic problems across five high-
reasoning domains, including computer science, economics, law, mathematics, and philosophy. All questions are sourced from top-tier publications in recent years and undergo rigorous annotation and quality control to ensure they are both challenging and answerable. We conduct systematic evaluations of over 10 mainstream LLMs and
agents. The results show that most LLMs scored below 20 points, with even the cutting-edge GPT-5 achieving only 16 points. While
agents achieved higher scores, none exceeded 40 points. This demonstrates the current capability gap between LLMs and
agents in super-intelligent academic research tasks and highlights the challenges of Acadreason.
Read the full abstract ▾
All authors Xin Gui, King Zhu, JinCheng Ren, Qianben Chen, Zekun Moore Wang, Yizhi LI, Xinpeng Liu, Xiaowan Li, Wenli Ren, Linyu Miao, Tianrui Qin, Ziqi Shu, He Zhu, Xiangru Tang, Dingfeng Shi, Jiaheng Liu, Yuchen Eleanor Jiang, Minghao Liu, Ge Zhang, Wangchunshu Zhou
Computation and Language
Advancing the Pareto Frontier of Test-Time Scaling with Multi-Path Plan Aggregation
Siheng Xiong, Ali Payani and Faramarz Fekri
Test-time scaling seeks to improve
reasoning through additional inference-time computation, commonly using solution sampling with majority voting to select the final answer. However, repeated sampling can reproduce correlated errors, limiting the accuracy gains from additional computation. We introduce Multi-Path Plan Aggregation (MPPA), which learns planning and answer selection using separate LoRA adapters on a shared frozen backbone. The planner jointly generates $K$ short plans, each guiding a separate solution generated by the frozen executor; the aggregator then scores the resulting solutions and selects the final answer. Joint planning encourages complementary approaches suited to the executor's capabilities, while learned selection uses
reasoning evidence to identify correct candidates. Alternating optimization improves planning and aggregation through supervised finetuning and reinforcement learning. Across five mathematics and science
benchmarks and three backbones, MPPA improves accuracy–
token trade-offs compared with training-free and training-based baselines. With three plans, it achieves higher average accuracy than eight directly sampled solutions with majority voting or the same learned aggregator, while reducing output
tokens by 47–54%. Evaluation with task-provided subtasks further demonstrates higher final-task accuracy than direct sampling with majority voting or learned aggregation at either the task or subtask level.
Read the full abstract ▾
Computation and Language
StoryBox: Collaborative Multi-Agent Simulation for Hybrid Bottom-Up Long-Form Story Generation Using Large Language Models
Zehao Chen, Rong Pan and Haoran Li
Human writers often begin their stories with an overarching mental scene, where they envision the interactions between characters and their environment. Inspired by this creative process, we propose a novel approach to long-form story generation, termed hybrid bottom-up long-form story generation, using multi-
agent simulations. In our method,
agents interact within a dynamic sandbox environment, where their behaviors and interactions with one another and the environment generate emergent events. These events form the foundation for the story, enabling organic character development and plot progression. Unlike traditional top-down approaches that impose rigid structures, our hybrid bottom-up approach allows for the natural unfolding of events, fostering more spontaneous and engaging storytelling. The system is capable of generating stories exceeding 10,000 words while maintaining coherence and consistency, addressing some of the key challenges faced by current story generation models. We achieve state-of-the-art performance across several metrics. This approach offers a scalable and innovative solution for creating dynamic, immersive long-form stories that evolve organically from
agent-driven interactions.
Read the full abstract ▾
Computation and Language
Xurong Xie, Zhucun Xue, Jiafu Wu et al. (+5 more)
Knowledge
distillation (KD) is a key technique for compressing large-scale
language models (LLMs), yet prevailing logit-based methods typically employ static strategies that are misaligned with the dynamic learning process of student models. These methods typically treat all
tokens indiscriminately and apply a single, fixed temperature, resulting in suboptimal knowledge transfer. To address these limitations, we propose LLM-Oriented
Token-Adaptive Knowledge
Distillation (AdaKD), a novel framework that adapts the
distillation process to the real-time learning state of each
token. AdaKD consists of two synergistic modules driven by a unified
token difficulty metric. First, our Loss-Driven Adaptive
Token Focusing (LATF) module dynamically adjusts the
distillation focus by monitoring the student's learning stability, concentrating computational resources on the most valuable
tokens at each training phase. Second, we introduce Inverse Difficulty Temperature Scaling (IDTS), a counterintuitive yet effective
token-level temperature strategy. It employs low temperatures for difficult
tokens for targeted error correction, and high temperatures for easy
tokens to encourage students to learn from the teacher's complete and smooth output distribution, thereby enhancing generalization. As a plug-and-play framework, AdaKD can consistently improve the performance of various
distillation methods on multiple model architectures and
benchmarks.
Read the full abstract ▾
All authors Xurong Xie, Zhucun Xue, Jiafu Wu, Jian Li, Yabiao Wang, Xiaobin Hu, Yong Liu, Jiangning Zhang
Computation and Language
Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
Huiyin Xue, Nafise Sadat Moosavi and Nikolaos Aletras
The success of
Transformer language models is widely credited to their dot-product
attention mechanism, which interweaves a set of key design principles: mixing information across positions (enabling multi-
token interactions), sequence-dependent activations (where
attention weights adapt to each input), a specific mathematical form (dot-product similarities plus softmax weighting), and coupling of queries and keys to evolving hidden states (grounding
attention in the current layer). However, the necessity of each of these principles remains largely untested. In this work, we systematically deconstruct
attention by designing controlled variants that selectively relax these principles, applied both uniformly across all layers and in hybrid architectures where only some layers retain standard
attention. Our empirical analysis reveals that mechanisms for mixing
tokens are indispensable, as their absence collapses models to near-random behavior, while the exact mathematical form and sequence dependency can be substantially relaxed, especially when preserved in just a subset of layers. Surprisingly, even variants that fail in isolation can achieve robust performance when interleaved with standard
attention, highlighting a cooperative effect. These findings deepen our understanding of what truly underpins
attention's effectiveness and open new avenues for simplifying
language models without sacrificing performance.
Read the full abstract ▾
Computation and Language
SemCSE-Multi: Multifaceted and Decodable Embeddings for Aspect-Specific and Interpretable Scientific Domain Mapping
Marc Brinner and Sina Zarrieß
We propose SemCSE-Multi, a novel unsupervised framework for generating multifaceted
embeddings of scientific abstracts, evaluated in the domains of invasion biology and medicine. These
embeddings capture distinct, individually specifiable aspects in isolation, thus enabling fine-grained and controllable similarity assessments as well as adaptive, user-driven visualizations of scientific domains. Our approach relies on an unsupervised procedure that produces aspect-specific summarizing sentences and trains
embedding models to map semantically related summaries to nearby positions in the
embedding space. We then distill these aspect-specific
embedding capabilities into a unified
embedding model that directly predicts multiple aspect
embeddings from a scientific abstract in a single, efficient forward pass. In addition, we introduce an
embedding decoding pipeline that decodes
embeddings back into natural
language descriptions of their associated aspects. Notably, we show that this decoding remains effective even for unoccupied regions in low-dimensional visualizations, thus offering vastly improved interpretability in user-centric settings.
Read the full abstract ▾
Computation and Language
MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
Bo Cheng, Xu Wang, Jinda Liu et al. (+2 more)
Low-Rank Adaptation (LoRA) has emerged as one of the most widely used parameter-efficient fine-tuning (PEFT) methods for adapting large
language models (LLMs) to downstream tasks. While highly effective in single-task settings, it struggles to efficiently leverage inter-task knowledge in complex multi-task learning scenarios, often requiring substantial task-specific data to achieve optimal performance. To address this limitation, we introduce MeTA-LoRA, a two-stage optimization framework that significantly improves data efficiency in multi-task adaptation. In the first stage, task-specific LoRA adapters are learned using only a few samples from each involved
dataset, enabling rapid adaptation without large-scale supervision. In the second stage, the shared LoRA adapter is updated by aggregating gradients from multiple tasks to promote knowledge transfer across tasks, further reducing data usage by leveraging common patterns. In both multi-task learning and
multilingual learning scenarios, our method matches or surpasses the performance of traditional full-data LoRA fine-tuning approaches, while using significantly less task-specific data.
Read the full abstract ▾
All authors Bo Cheng, Xu Wang, Jinda Liu, Yi Chang, Yuan Wu
Computation and Language · Cross-listed from cs.IR
REGENT: Relevance-Guided Attention for Entity-Aware Multi-Vector Neural Re-Ranking
Shubham Chatterjee
Current neural re-rankers often struggle with complex information needs and long, content-rich documents. The fundamental issue is not computational–it is intelligent content selection: identifying what matters in lengthy, multi-faceted texts. While humans naturally anchor their understanding around key entities and concepts, neural models process text within rigid
token windows, treating all interactions as equally important and missing critical semantic signals. We introduce REGENT, a neural re-ranking model that mimics human-like understanding by using entities as a "semantic skeleton" to guide
attention. REGENT integrates relevance guidance directly into the
attention mechanism, combining fine-grained lexical matching with high-level semantic
reasoning. This relevance-guided
attention enables the model to focus on conceptually important content while maintaining sensitivity to precise term matches. REGENT achieves new state-of-the-art performance in three challenging
datasets, providing up to 108% improvement over BM25 and consistently outperforming strong baselines including ColBERT and RankVicuna. To our knowledge, this is the first work to successfully integrate entity semantics directly into neural
attention, establishing a new paradigm for entity-aware information
retrieval.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking
Shubham Chatterjee and Jeff Dalton
Neural IR has advanced through two distinct paths: entity-oriented approaches leveraging knowledge graphs and multi-vector models capturing fine-grained semantics. We introduce QDER, a neural re-ranking model that unifies these approaches by integrating knowledge graph semantics into a multi-vector model. QDER's key innovation lies in its modeling of query-document relationships: rather than computing similarity scores on aggregated
embeddings, we maintain individual
token and entity representations throughout the ranking process, performing aggregation only at the final scoring stage - an approach we call "late aggregation." We first transform these fine-grained representations through learned
attention patterns, then apply carefully chosen mathematical operations for precise matches. Experiments across five standard
benchmarks show that QDER achieves significant performance gains, with improvements of 36% in nDCG@20 over the strongest baseline on TREC Robust 2004 and similar improvements on other
datasets. QDER particularly excels on difficult queries, achieving an nDCG@20 of 0.70 where traditional approaches fail completely (nDCG@20 = 0.0), setting a foundation for future work in entity-aware
retrieval.
Read the full abstract ▾
Computation and Language
Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models
Georg Ahnert, Anna-Carolina Haensch, Barbara Plank et al. (+1 more)
Many in-silico simulations of human survey responses with large
language models (LLMs) focus on generating closed-ended survey responses, whereas LLMs are typically trained to generate open-ended text instead. Previous research has used a diverse range of methods for generating closed-ended survey responses with LLMs, and a standard practice remains to be identified. In this paper, we systematically investigate the impact that various Survey Response Generation Methods have on predicted survey responses. We present the results of 32 mio. simulated survey responses across 8 Survey Response Generation Methods, 4 political attitude surveys, and 10 open-weight
language models. We find significant differences between the Survey Response Generation Methods in both individual-level and subpopulation-level
alignment. Our results show that Restricted Generation Methods perform best overall, and that
reasoning output does not consistently improve
alignment. Our work underlines the significant impact that Survey Response Generation Methods have on simulated survey responses, and we develop practical recommendations on the application of Survey Response Generation Methods.
Read the full abstract ▾
All authors Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, Markus Strohmaier
Computation and Language
LLMAtKGE: Large Language Models as Explainable Attackers against Knowledge Graph Embeddings
Ting Li, Yang Yang, Yipeng Yu et al. (+3 more)
Adversarial attacks on knowledge graph
embeddings (KGE) aim to disrupt the model's ability of link prediction by removing or inserting triples. A recent black-box method has attempted to incorporate textual and structural information to enhance attack performance. However, it is unable to generate human-readable explanations, and exhibits poor generalizability. In the past few years, large
language models (LLMs) have demonstrated powerful capabilities in text comprehension, generation, and
reasoning. In this paper, we propose LLMAtKGE, a novel LLM-based framework that selects attack targets and generates human-readable explanations. To provide the LLM with sufficient factual context under limited input constraints, we design a structured prompting scheme that explicitly formulates the attack as multiple-choice questions while incorporating KG factual evidence. To address the context-window limitation and hesitation issues, we introduce semantics-based and centrality-based filters, which compress the candidate set while preserving high recall of attack-relevant information. Furthermore, to efficiently integrate both semantic and structural information into the filter, we precompute high-order adjacency and fine-tune the LLM with a triple classification task to enhance filtering performance. Experiments on two widely used knowledge graph
datasets demonstrate that our attack outperforms the strongest black-box baselines and provides explanations via
reasoning, and showing competitive performance compared with white-box methods. Comprehensive ablation and case studies further validate its capability to generate explanations.
Read the full abstract ▾
All authors Ting Li, Yang Yang, Yipeng Yu, Liang Yao, Guoqing Chao, Ruifeng Xu
Computation and Language · Cross-listed from cs.CR
Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrails
Shuo Chen, Zhen Han, Haokun Chen et al. (+6 more)
Open-weight Large
Reasoning Models (LRMs) are approaching the capabilities of their frontier counterparts but pose significant safety concerns, as they are difficult to patch or monitor post-release. To prevent misuse,
reasoning-based safety guardrails, where models explicitly reason on safety justifications before answering, have become a promising primary defense, achieving near-perfect refusal rates on harmful queries. We show that this strong defense is alarmingly brittle and can be subverted by embarrassingly simple attacks to elicit extremely forbidden questions, such as `How to kill a man without being caught?' Specifically, we identify one systematic vulnerability from the
reasoning-then-answer mechanism: the stage-transition logic that governs when safety
reasoning begins and ends can be trivially manipulated. Based on this finding, we develop four simple yet effective red-teaming methods that systematically subvert different stages of the guardrails, either by bypassing
reasoning entirely or exploiting it to produce targeted harmful content. These methods achieve attack success rates up to 90% across five
benchmarks on multiple LRM families. Our findings reveal that
reasoning-based guardrails are necessary but not sufficient and must be paired with robust triggering mechanisms and stronger base-model
alignment. Code is in https://chenxshuo.github.io/bag-of-tricks/.
Read the full abstract ▾
All authors Shuo Chen, Zhen Han, Haokun Chen, Bailan He, Shengyun Si, Jingpei Wu, Philip Torr, Volker Tresp, Jindong Gu
Computation and Language
Culturally-Aware Conversations: A Framework & Benchmark for LLMs
Shreya Havaldar, Sunny Rai, Young-Min Cho et al. (+1 more)
Existing
benchmarks that measure cultural adaptation in LLMs are misaligned with the actual challenges these models face when interacting with users from diverse cultural backgrounds. In this work, we introduce the first framework and
benchmark designed to evaluate LLMs in realistic, multicultural conversational settings. Grounded in sociocultural theory, our framework formalizes how linguistic style - a key element of cultural communication - is shaped by situational, relational, and cultural context. We construct a
benchmark dataset based on this framework, annotated by culturally diverse raters, and propose a new set of desiderata for cross-cultural evaluation in NLP: conversational framing, stylistic sensitivity, and subjective correctness. We evaluate today's top LLMs on our
benchmark and show that these models struggle with cultural adaptation in a conversational setting.
Read the full abstract ▾
All authors Shreya Havaldar, Sunny Rai, Young-Min Cho, Lyle Ungar
Computation and Language
Invisible Languages of the LLM Universe
Saurabh Khanna and Xinxu Li
Large
Language Models are trained on massive
multilingual corpora, yet this abundance masks a profound crisis: of the world's 7,613 living
languages, approximately 2,000
languages with millions of speakers remain effectively invisible in digital ecosystems. We propose a critical framework connecting empirical measurements of
language vitality (real world demographic strength) and digitality (online presence) with postcolonial theory and epistemic injustice to explain why linguistic inequality in AI systems is not incidental but structural. Analyzing data across all documented human
languages, we identify four categories: Strongholds (33%, high vitality and digitality), Digital Echoes (6%, high digitality despite declining vitality), Fading Voices (36%, low on both dimensions), and critically, Invisible
Giants (27%, high vitality but near-zero digitality) -
languages spoken by millions yet absent from the LLM universe. We demonstrate that these patterns reflect continuities from colonial-era linguistic hierarchies to contemporary AI development, constituting digital epistemic injustice. Our analysis reveals that English dominance in AI is not a technical necessity but an artifact of power structures that systematically exclude marginalized linguistic knowledge. We conclude with implications for decolonizing
language technology and democratizing access to AI benefits.
Read the full abstract ▾
Computation and Language
Information-Preserving Reformulation of Reasoning Traces for Antidistillation
Jiayu Ding, Lei Cui, Li Dong et al. (+2 more)
Recent advances in Large
Language Models (LLMs) show that extending the length of
reasoning chains significantly improves performance on complex tasks. While revealing these
reasoning traces helps users better follow, verify, and learn from the model's problem-solving process, it also makes them highly vulnerable to unauthorized
distillation. To mitigate this risk, proprietary model providers often adopt aggressive protection strategies, such as replacing detailed
reasoning with brief summaries, which deprive users of valuable intermediate information. To address this trade-off, we propose PART, an information-preserving antidistillation reformulation of
reasoning traces. Motivated by the difference between how humans understand
reasoning traces and how LLMs exploit them for supervised fine-tuning, we design a simple but effective two-step reformulation: removing self-talk behaviors and reordering sub-conclusions. A small auxiliary model is trained to perform this reformulation, incurring minimal computational overhead. Extensive experiments demonstrate that PART consistently disrupts
distillation across student models of different sizes and types on various
reasoning benchmarks. For instance, when training on reformulated traces, even the performance of a large 32B student model decreases from 54.17 to 46.88 on AIME 2024, corresponding to a 13.5% degradation.
Read the full abstract ▾
All authors Jiayu Ding, Lei Cui, Li Dong, Nanning Zheng, Furu Wei
Computation and Language
An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
Ba-Quang Nguyen
We propose a novel neural architecture named TextGraphFuseGAT, which integrates a pretrained
transformer encoder (PhoBERT) with Graph
Attention Networks for
token-level classification tasks. The proposed model constructs a fully connected graph over the
token embeddings produced by PhoBERT, enabling the GAT layer to capture rich inter-
token dependencies beyond those modeled by sequential context alone. To further enhance contextualization, a
Transformer-style self-
attention layer is applied on top of the graph-enhanced
embeddings. The final
token representations are passed through a classification head to perform sequence labeling. We evaluate our approach on three Vietnamese
benchmark datasets: PhoNER-COVID19 for named entity recognition in the COVID-19 domain, PhoDisfluency for
speech disfluency detection, and VietMed-NER for medical-domain NER. VietMed-NER is the first Vietnamese medical spoken NER
dataset, featuring 18 entity types collected from real-world medical
speech transcripts and annotated with the BIO tagging scheme. Its specialized vocabulary and domain-specific expressions make it a challenging
benchmark for
token-level classification models. Experimental results show that our method consistently outperforms strong baselines, including
transformer-only and hybrid neural models such as BiLSTM + CNN + CRF, confirming the effectiveness of combining pretrained semantic features with graph-based relational modeling for improved
token classification across multiple domains.
Read the full abstract ▾
Computation and Language
Hallucination Detection via Internal States and Structured Reasoning Consistency in Large Language Models
Yusheng Song, Lirong Qiu, Xi Zhang et al. (+1 more)
The detection of sophisticated
hallucinations in Large
Language Models (LLMs) is hampered by a “Detection Dilemma”: methods probing internal states (Internal State Probing) excel at identifying factual inconsistencies but fail on logical fallacies, while those verifying externalized
reasoning (Chain-of-Thought Verification) show the opposite behavior. This schism creates a task-dependent blind spot: Chain-of-Thought Verification fails on fact-intensive tasks like open-domain QA where
reasoning is ungrounded, while Internal State Probing is ineffective on logic-intensive tasks like mathematical
reasoning where models are confidently wrong. We resolve this with a unified framework that bridges this critical gap. However, unification is hindered by two fundamental challenges: the Signal Scarcity Barrier, as coarse symbolic
reasoning chains lack signals directly comparable to fine-grained internal states, and the Representational
Alignment Barrier, a deep-seated mismatch between their underlying semantic spaces. To overcome these, we introduce a multi-path
reasoning mechanism to obtain more comparable, fine-grained signals, and a segment-aware temporalized cross-
attention module to adaptively fuse these now-aligned representations, pinpointing subtle dissonances. Extensive experiments on three diverse
benchmarks and two leading LLMs demonstrate that our framework consistently and significantly outperforms strong baselines. Our code is available: https://github.com/peach918/HalluDet.
Read the full abstract ▾
All authors Yusheng Song, Lirong Qiu, Xi Zhang, Zhihao Tang
Computation and Language · Cross-listed from cs.LG
ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding
Yuhang Li, Chenchen Zhang, Ruilin Lv et al. (+6 more)
While Large
Language Models (LLMs) excel at algorithmic code generation, they struggle with front-end development, where correctness is judged on rendered pixels and interaction. We present ReLook, an agentic,
vision-grounded reinforcement learning framework that empowers an
agent to close a robust generate–diagnose–refine loop by invoking a
multimodal LLM (MLLM) as a tool. During training, the
agent uses the MLLM-in-the-loop both as a visual critic–scoring code with screenshots–and as a source of actionable,
vision-grounded feedback; a strict zero-reward rule for invalid renders anchors renderability and prevents reward hacking. To prevent behavioral collapse, we introduce Forced Optimization, a strict acceptance rule that admits only improving revisions, yielding monotonically better trajectories. At inference, we decouple the critic and run a lightweight, critic-free self-edit cycle, keeping latency comparable to base decoding while retaining most of the gains. Across three widely used
benchmarks, ReLook consistently outperforms strong baselines in
vision-grounded front-end code generation, highlighting the benefits of agentic perception, visual rewards, and training-inference decoupling.
Read the full abstract ▾
All authors Yuhang Li, Chenchen Zhang, Ruilin Lv, Ao Liu, Ken Deng, Yuanxing Zhang, Jiaheng Liu, Wiggin Zhou, Bo Zhou
Computation and Language
Investigating Large Language Models' Linguistic Abilities for Text Preprocessing
Marco Braga, Gian Carlo Milanese and Gabriella Pasi
Text preprocessing is a fundamental component of Natural
Language Processing, involving techniques such as stopword removal, stemming, and lemmatization to prepare text as input for further processing and analysis. Despite the context-dependent nature of the above techniques, traditional methods usually ignore contextual information. In this paper, we investigate the idea of using Large
Language Models (LLMs) to perform various preprocessing tasks, due to their ability to take context into account without requiring extensive
language-specific annotated resources. Through a comprehensive evaluation on web-sourced data, we compare LLM-based preprocessing (specifically stopword removal, lemmatization and stemming) to traditional
algorithms across multiple text classification tasks in six European
languages. Our analysis indicates that LLMs are capable of replicating traditional stopword removal, lemmatization, and stemming methods with accuracies reaching 97%, 82%, and 74%, respectively. Additionally, we show that ML
algorithms trained on texts preprocessed by LLMs achieve an improvement of up to 6% with respect to the $F_1$ measure compared to traditional techniques. Our code,
prompts, and results are publicly available at https://github.com/GianCarloMilanese/llm_pipeline_wi-iat.
Read the full abstract ▾
Computation and Language
GenCNER: A Generative Framework for Continual Named Entity Recognition
Yawen Yang, Fukun Ma, Shiao Meng et al. (+2 more)
Traditional named entity recognition (NER) aims to identify text mentions into pre-defined entity types. Continual Named Entity Recognition (CNER) is introduced since entity categories are continuously increasing in various real-world scenarios. However, existing continual learning (CL) methods for NER face challenges of catastrophic forgetting and semantic shift of non-entity type. In this paper, we propose GenCNER, a simple but effective Generative framework for CNER to mitigate the above drawbacks. Specifically, we skillfully convert the CNER task into sustained entity triplet sequence generation problem and utilize a powerful pre-trained seq2seq model to solve it. Additionally, we design a type-specific confidence-based pseudo labeling strategy along with knowledge
distillation (KD) to preserve learned knowledge and alleviate the impact of label noise at the triplet level. Experimental results on two
benchmark datasets show that our framework outperforms previous state-of-the-art methods in multiple CNER settings, and achieves the smallest gap compared with non-CL results.
Read the full abstract ▾
All authors Yawen Yang, Fukun Ma, Shiao Meng, Aiwei Liu, Lijie Wen
Computation and Language
Who are you, ChatGPT? Personality and Demographic Style in LLM-Generated Content
Dana Sotto Porat and Ella Rabinovich
Generative large
language models (LLMs) have become central to everyday life, producing human-like text across diverse domains. A growing body of research investigates whether these models also exhibit personality- and demographic-like characteristics in their
language. In this work, we introduce a novel, data-driven methodology for assessing LLM personality without relying on self-report questionnaires, applying instead automatic personality and gender classifiers to model replies on open-ended questions collected from Reddit. Comparing six widely used models to human-authored responses, we find that LLMs systematically express higher Agreeableness and lower Neuroticism, reflecting cooperative and stable conversational tendencies. Gendered
language patterns in model text broadly resemble those of human writers, though with reduced variation, echoing prior findings on automated
agents. We contribute a new
dataset of human and model responses, along with large-scale comparative analyses, shedding new light on the topic of personality and demographic patterns of generative AI.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SI
Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation
Jiaying Wu, Zihang Fu, Haonan Wang et al. (+4 more)
Community Notes, the crowd-sourced misinformation governance system on X (formerly Twitter), allows users to flag misleading posts, attach contextual notes, and rate the notes' helpfulness. However, our empirical analysis of 30.8K health-related notes reveals substantial latency, with a median delay of 17.6 hours before notes receive a helpfulness status. To improve responsiveness during real-world misinformation surges, we propose CrowdNotes+, a unified LLM-based framework that augments Community Notes for faster and more reliable health misinformation governance. CrowdNotes+ integrates two modes: (1) evidence-grounded note augmentation and (2) utility-guided note automation, supported by a hierarchical three-stage evaluation of relevance, correctness, and helpfulness. We instantiate the framework with HealthNotes, a
benchmark of 1.2K health notes annotated for helpfulness, and a fine-tuned helpfulness judge. Our analysis first uncovers a key loophole in current crowd-sourced governance: voters frequently conflate stylistic fluency with factual accuracy. Addressing this via our hierarchical evaluation, experiments across 15 representative LLMs demonstrate that CrowdNotes+ significantly outperforms human contributors in note correctness, helpfulness, and evidence utility.
Read the full abstract ▾
All authors Jiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li, Jiafeng Guo, Preslav Nakov, Min-Yen Kan
Computation and Language
Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification
Stefan Krsteski, Giuseppe Russo, Serina Chang et al. (+2 more)
Surveys provide valuable insights into public opinion and behavior, but their execution is costly and slow. Large
language models (LLMs) have been proposed as a scalable, low-cost substitute for human respondents, but their outputs are often biased and yield invalid estimates. We study the interplay between synthesis methods that use LLMs to generate survey responses and rectification methods that debias population estimates, and explore how human responses are best allocated between them. Using two panel surveys with questions on nutrition, politics, and economics, we find that synthesis alone introduces substantial bias (24-86%), whereas combining it with rectification reduces bias below 5% and increases effective sample size by up to 14%. Overall, we challenge the common practice of using all human responses for fine-tuning, showing that under a fixed budget, allocating most to rectification results in far more effective estimation.
Read the full abstract ▾
All authors Stefan Krsteski, Giuseppe Russo, Serina Chang, Robert West, Kristina Gligorić
Computation and Language
KnowRL: Teaching Language Models to Know What They Know
Sahil Kale and Devendra Singh Dhami
Truly reliable AI requires more than simply scaling up knowledge; it demands the ability to know what it knows and when it does not. Yet recent research shows that even the best LLMs misjudge their own competence in more than one in five cases, making any response born of such internal uncertainty impossible to fully trust. Inspired by self-improvement reinforcement learning techniques that require minimal data, we present a simple but powerful framework KnowRL that strengthens a model's internal understanding of its own feasibility boundaries, enabling safer and more responsible behaviour. Our framework combines two components: (i) introspection, where the model generates and classifies tasks it judges feasible or infeasible, and (ii) consensus-based rewarding, where stability of self-knowledge assessment is reinforced through internal agreement. By using internally generated data, this design strengthens consistency in self-knowledge and entirely avoids costly external supervision. In experiments on LLaMA-3.1-8B and Qwen-2.5-7B, KnowRL steadily improved self-knowledge, validated by both intrinsic self-consistency and extrinsic benchmarking. With nothing more than a small seed set and no external supervision, our method drove gains as high as 28% in accuracy and 12% in F1, outperforming baselines in just a few iterations. Our framework essentially unlocks the untapped capacity of LLMs to self-improve their knowledge awareness, opening the door to reliable, more accountable AI and safer deployment in critical applications. Owing to its simplicity and independence from external effort, we encourage applying this reliability-enhancing process to all future models.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
DocReward: A Document Reward Model for Structuring and Stylizing
Junpeng Liu, Yuzhong Zhao, Bowen Cao et al. (+17 more)
Recent agentic workflows automate professional document generation but focus narrowly on textual quality, overlooking structural and stylistic professionalism, which is equally critical for readability. This gap stems mainly from a lack of effective reward models capable of guiding
agents toward producing documents with high structural and stylistic professionalism. We introduce DocReward, a document reward model that evaluates documents based on their structure and style. To achieve this, we propose a textual-quality-agnostic framework that ensures assessments are not confounded by content quality, and construct DocPair, a
dataset of 117K paired documents covering 32 domains and 267 types. Each pair shares identical content but differs in structural and stylistic professionalism. DocReward is trained using the Bradley-Terry loss. On a manually annotated
benchmark, DocReward outperforms GPT-5 by 14.6 percentage points in the same setting. Reinforcement learning experiments further show that DocReward effectively guides
agents toward generating documents with consistently higher structural and stylistic professionalism, highlighting its practical utility.
Read the full abstract ▾
All authors Junpeng Liu, Yuzhong Zhao, Bowen Cao, Jiayu Ding, Yilin Jia, Tengchao Lv, Yupan Huang, Wenshan Wu, Shaohan Huang, Nan Yang, Li Dong, Lei Cui, Tao Ge, Xun Wang, Huitian Jiao, Sun Mao, FNU Kartik, Si-Qing Chen, Wai Lam, Furu Wei
Computation and Language
Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
Zirui Song, Yuan Huang, Junchang Liu et al. (+7 more)
Social deduction games like Werewolf combine
language,
reasoning, and strategy, providing a testbed for studying natural
language and social intelligence. However, most studies reduce the game to LLM-based self-play, yielding templated utterances and anecdotal cases that overlook the richness of social gameplay. Evaluation further relies on coarse metrics such as survival time or subjective scoring due to the lack of quality reference data. To address these gaps, we curate a high-quality, human-verified
multimodal Werewolf
dataset containing over 100 hours of
video, 32.4M utterance
tokens, and 15 rule variants. Based on this
dataset, we propose a novel strategy-
alignment evaluation that leverages the winning faction's strategies as ground truth in two stages: 1)
Speech evaluation, formulated as multiple-choice-style tasks that assess whether the model can adopt appropriate stances across five dimensions of social ability; and 2) Decision evaluation, which assesses the model's voting choices and opponent-role inferences. This framework enables a fine-grained evaluation of models' linguistic and
reasoning capabilities, while capturing their ability to generate strategically coherent gameplay. Our experiments show that state-of-the-art LLMs show diverse performance, with roughly half remain below 0.50, revealing clear gaps in deception and counterfactual
reasoning. We hope our
dataset further inspires research on
language,
reasoning, and strategy in multi-
agent interaction.
Read the full abstract ▾
All authors Zirui Song, Yuan Huang, Junchang Liu, Haozhe Luo, Chenxi Wang, Lang Gao, Zixiang Xu, Mingfei Han, Xiaojun Chang, Xiuying Chen
Computation and Language
Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning
Dean L. Slack and Noura Al Moubayed
Although large
language models excel across many tasks, they can memorise training data and thereby expose private or copyrighted text. Most defences target the pre-training stage, leaving memorisation during fine-tuning, especially for domain adaptation and instruction tuning, poorly understood. We fine-tune Pythia, Llama3, and Mistral models spanning 1.4B-70B parameters on common evaluation
datasets and track verbatim memorisation throughout training. We find that memorisation increases dramatically in the first few epochs, often significantly before either validation perplexity or evaluation performance is optimised. We use a simple but effective n-gram memorisation score which reliably precedes verbatim memorisation; using it as an early-stopping criterion mitigates memorisation with minimal performance loss. Further, we introduce an n-gram-aware loss regulariser and show that it reduces memorisation across all model families tested by up to 40% while minimising evaluation performance trade-offs when compared to an existing memorisation mitigation strategy. These results yield practical, scalable insights into memorisation dynamics during
language model fine-tuning.
Read the full abstract ▾
Computation and Language
Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
Wenhan Ma, Hailin Zhang, Liang Zhao et al. (+4 more)
Reinforcement learning (RL) has emerged as a crucial approach for enhancing the capabilities of large
language models. However, in Mixture-of-Experts (MoE) models, the routing mechanism often introduces instability, even leading to catastrophic RL training collapse. We analyze the training-inference consistency of MoE models and identify a notable discrepancy in routing behaviors between the two phases. Moreover, even under identical conditions, the routing framework can yield divergent expert selections across repeated forward passes. To address this foundational inconsistency, we propose Rollout Routing Replay (R3), a method that records routing distributions from the inference engine and replays them during training. R3 significantly reduces training-inference policy KL divergence and mitigates extreme discrepancies without compromising training speed. Extensive experiments on various settings confirm that R3 succeeds in stabilizing RL training, preventing collapse and outperforming methods such as GSPO and TIS. We believe this work can offer a new solution for stabilizing RL in MoE models.
Read the full abstract ▾
All authors Wenhan Ma, Hailin Zhang, Liang Zhao, Yifan Song, Yudong Wang, Zhifang Sui, Fuli Luo