Computation and Language · Cross-listed from cs.AI
HugAgent: A Human Simulation Benchmark for Individual-Level Reasoning
Chance Jiajie Li, Zhenze Mo, Yuhan Tang et al. (+11 more)
Simulating human
reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large
language models now approximate human responses at scale, they remain tuned to population-level consensus, often erasing the individuality of
reasoning styles and belief trajectories. To advance the
vision of more human-like
reasoning in machines, we introduce HugAgent (HUman-Grounded
AGENT Benchmark), which rethinks human
reasoning simulation along three dimensions: (i) from averaged to individualized
reasoning, (ii) from behavioral mimicry to cognitive
alignment, and (iii) from vignette-based to open-ended data. The
benchmark evaluates whether a model can predict a specific person's behavioral responses and the underlying
reasoning dynamics in out-of-distribution scenarios, given partial evidence of their prior views. HugAgent combines structured questionnaires with semi-structured think-aloud interviews to collect ecologically valid belief states, belief updates, and
reasoning traces from human participants. Our experiments reveal a clear asymmetry: models recover a person's belief state from their own context reasonably well, but struggle to predict belief updates under intervention. Cross-person and cross-domain controls trace this gap to associative matching within a topic rather than identity-consistent
reasoning, suggesting that progress requires better-calibrated change detection, not simply more context. We scope the
benchmark to self-reported belief
reasoning in three policy domains: healthcare, surveillance, and zoning. The
benchmark, along with its complete data collection pipeline and companion chatbot, is open-sourced as HugAgent (https://github.com/jajamoa/HugAgent) and TraceYourThinking (https://github.com/jajamoa/trace-your-thinking).
Read the full abstract ▾
All authors Chance Jiajie Li, Zhenze Mo, Yuhan Tang, Ao Qu, Jiayi Wu, Kaiya Ivy Zhao, Yulu Gan, Jie Fan, Jiangbo Yu, Hang Jiang, Paul Pu Liang, Jinhua Zhao, Luis Alberto Alonso Pastor, Kent Larson
Computation and Language
FarsiMCQGen: a Persian Multiple-choice Question Generation Framework
Mohammad Heydari Rad, Rezvan Afari and Saeedeh Momtazi
Multiple-choice questions (MCQs) are commonly used in educational testing, as they offer an efficient means of evaluating learners' knowledge. However, generating high-quality MCQs, particularly in low-resource
languages such as Persian, remains a significant challenge. This paper introduces FarsiMCQGen, an innovative approach for generating Persian-
language MCQs. Our methodology combines candidate generation, filtering, and ranking techniques to build a model that generates answer choices resembling those in real MCQs. We leverage advanced methods, including
Transformers and knowledge graphs, integrated with rule-based approaches to craft credible distractors that challenge test-takers. Our work is based on data from Wikipedia, which includes general knowledge questions. Furthermore, this study introduces a novel Persian MCQ
dataset comprising 10,289 questions. This
dataset is evaluated by different state-of-the-art large
language models (LLMs). Our results demonstrate the effectiveness of our model and the quality of the generated
dataset, which has the potential to inspire further research on MCQs.
Read the full abstract ▾
Computation and Language
Iterative Topic Taxonomy Induction with LLMs: A Case Study of Electoral Advertising
Alexander Brady and Tunazzina Islam
Social media platforms play a pivotal role in shaping political discourse, but the scale and rapid
evolution of online content make systematic analysis difficult. We introduce an end-to-end framework for inducing an interpretable topic taxonomy from unlabeled text corpora. The framework combines
embedding-based clustering with iterative large
language model (LLM) inference to construct a topic taxonomy without requiring predefined labels or seed topics. It first synthesizes candidate topics from document clusters and then uses the resulting taxonomy to assign consistent topic labels across clusters. We evaluate the approach through a case study of political advertising ahead of the 2024 U.S. presidential election. We use the induced taxonomy to support downstream analyses of issue prevalence, moral framing, advertising spend, and demographic exposure patterns. These results suggest that iterative taxonomy construction can provide a scalable and interpretable approach to organizing large unlabeled text corpora while supporting substantive downstream analysis.
Read the full abstract ▾
Computation and Language
Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarks
Kirill Semenov and Rico Sennrich
For
multilingual factual knowledge assessment of LLMs,
benchmarks such as MLAMA use template translations that do not take into account the grammatical and semantic information of the named entities inserted in the sentence. This leads to numerous instances of ungrammaticality or wrong wording of the final
prompts, which complicates the interpretation of scores, especially for
languages that have a rich morphological inventory. In this work, we sample 4 Slavic
languages from the MLAMA
dataset and compare the knowledge
retrieval scores between the initial (templated) MLAMA
dataset and its sentence-level translations made by Google Translate and ChatGPT. We observe a significant increase in knowledge
retrieval scores, and provide a qualitative analysis for possible reasons behind it. We also make an additional analysis of 5 more
languages from different families and see similar patterns. Therefore, we encourage the community to control the grammaticality of highly
multilingual datasets for higher and more interpretable results, which is well approximated by whole sentence translation with neural MT or LLM systems. The
dataset and all related code is published at the Github repository: https://github.com/ZurichNLP/Fluent-mLAMA.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
Shih-Yang Liu, Xin Dong, Ximing Lu et al. (+9 more)
Reasoning language models such as OpenAI-o1, DeepSeek-R1, and Qwen achieve strong performance via extended chains of thought but often generate unnecessarily long outputs. Maximizing intelligence per
token–accuracy relative to response length–remains an open problem. We revisit reinforcement learning (RL) with the simplest length penalty–truncation–and show that accuracy degradation arises not from the lack of sophisticated penalties but from inadequate RL optimization. We identify three key challenges: (i) large bias in advantage estimation, (ii)
entropy collapse, and (iii) sparse reward signal. We address them with Doing Length pEnalty Right (DLER), a training recipe combining batch-wise reward normalization, higher clipping, dynamic sampling, and a simple truncation length penalty. DLER achieves state-of-the-art accuracy–efficiency trade-offs, cutting output length by over 70 percent while surpassing all previous baseline accuracy. It also improves test-time scaling: compared to DeepSeek-R1-7B, DLER-7B generates multiple concise responses in parallel with 28 percent higher accuracy and lower latency. We further introduce Difficulty-Aware DLER, which adaptively tightens truncation on easier questions for additional efficiency gains. We also propose an update-selective merging method that preserves baseline accuracy while retaining the concise
reasoning ability of the DLER model, which is useful for scenarios where RL training data is scarce.
Read the full abstract ▾
All authors Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Yejin Choi, Jan Kautz, Pavlo Molchanov
Computation and Language
Continual Learning via Sparse Memory Finetuning
Jessy Lin, Luke Zettlemoyer, Gargi Ghosh et al. (+4 more)
Modern
language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities. Motivated by the intuition that mitigating forgetting is challenging because trainable parameters are shared across all tasks, we investigate whether sparse parameter updates can enable learning without catastrophic forgetting. We introduce sparse
memory finetuning, leveraging
memory layer models (Berges et al., 2024), which are sparsely updated by design. By updating only the
memory slots that are highly activated by a new piece of knowledge relative to usage on pretraining data, we reduce interference between new knowledge and the model's existing capabilities. We evaluate learning and forgetting compared to full finetuning and parameter-efficient finetuning with LoRA on two question answering tasks. We find that sparse
memory finetuning learns new knowledge while exhibiting substantially less forgetting: while NaturalQuestions F1 drops by 89% after full finetuning on new facts and 71% with LoRA, sparse
memory finetuning yields only an 11% drop with the same level of new knowledge acquisition. Our results suggest sparsity in
memory layers offers a promising path toward continual learning in large
language models.
Read the full abstract ▾
All authors Jessy Lin, Luke Zettlemoyer, Gargi Ghosh, Wen-Tau Yih, Aram Markosyan, Vincent-Pierre Berges, Barlas Oğuz
Computation and Language
A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labelling
Shiyu Ji, Farnoosh Hashemi, Joice Chen et al. (+9 more)
Rhetorical strategies are central to persuasive communication, from political discourse and marketing to legal argumentation. However, analysis of rhetorical strategies has been limited by reliance on human annotation, which is costly, inconsistent, difficult to scale. Their associated
datasets are often limited to specific topics and strategies, posing challenges for robust model development. We propose a novel framework that leverages large
language models (LLMs) to automatically generate and label synthetic debate data based on a four-part rhetorical typology (causal, empirical, emotional, moral). We fine-tune
transformer-based classifiers on this LLM-labeled
dataset and validate its performance against human-labeled data on this
dataset and on multiple external corpora. Our model achieves high performance and strong generalization across topical domains. We illustrate two applications with the fine-tuned model: (1) the improvement in persuasiveness prediction from incorporating rhetorical strategy labels, and (2) analyzing temporal and partisan shifts in rhetorical strategies in U.S. Presidential debates (1960-2020), revealing increased use of affective over cognitive argument in U.S. Presidential debates.
Read the full abstract ▾
All authors Shiyu Ji, Farnoosh Hashemi, Joice Chen, Juanwen Pan, Weicheng Ma, Hefan Zhang, Sophia Pan, Ming Cheng, Shubham Mohole, Saeed Hassanpour, Soroush Vosoughi, Michael Macy
Computation and Language · Cross-listed from cs.LG
Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
Samuel Paech, Allen Roush, Judah Goldfeder et al. (+1 more)
Widespread LLM adoption has introduced characteristic repetitive phraseology, termed "slop," which degrades output quality and makes AI-generated text immediately recognizable. We present Antislop, a comprehensive framework providing tools to both detect and eliminate these overused patterns. Our approach combines three innovations: (1) The Antislop Sampler, which uses backtracking to suppress unwanted strings at inference time without destroying vocabulary; (2) An automated pipeline that profiles model-specific slop against human baselines and generates training data; (3) Final
Token Preference Optimization (FTPO), a novel fine-tuning method that operates on individual
tokens, surgically adjusting logits wherever a banned pattern has appeared in an inference trace. We demonstrate that some slop patterns appear over 1,000x more frequently in LLM output than human text. The Antislop Sampler successfully suppresses 8,000+ patterns while maintaining quality, whereas
token banning becomes unusable at just 2,000. Most importantly, FTPO achieves 90% slop reduction while maintaining or improving performance in cross-domain evals including GSM8K, MMLU, and creative writing tasks. In contrast, DPO suffers significant degradation in writing quality and lexical diversity despite achieving weaker suppression. We release all code and results under MIT license: https://github.com/sam-paech/auto-antislop.
Read the full abstract ▾
All authors Samuel Paech, Allen Roush, Judah Goldfeder, Ravid Shwartz-Ziv
Computation and Language · Cross-listed from cs.LG
Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective
Shiqi Chen, Tongyao Zhu, Zian Wang et al. (+8 more)
Large
Language Models (LLMs) as
agents often fail to improve in new environments. We identify and characterize a failure mode we call exploration collapse: under reinforcement learning (RL) in environments whose states are unfamiliar to the policy, Pass@k, the probability that at least one of k sampled trajectories succeeds, drops markedly over training even as Pass@1 edges up, revealing increasingly brittle exploration; environments closer to the pretraining distribution show no such decline. We trace this collapse to weak grounding in environment states and dynamics, and study a simple remedy: explicitly teaching the
agent to estimate the current state and predict its transitions before optimizing for reward. We instantiate it as SPA, an explore-then-exploit recipe that cold-starts the policy with a Self-Experience supervised finetuning (SFT) stage, collecting the model's own interaction trajectories and supervising state and next-state prediction, and then runs standard RL. The resulting world model serves as a grounded initialization for RL rather than an inference-time planner. Across unseen environments, SPA consistently and substantially improves over vanilla RL: for example, it raises the Sokoban success rate from 25.6% to 59.8% on Qwen2.5-1.5B-Instruct, letting sub-3B models surpass a 20B baseline on these tasks. Controlled studies indicate that the gains track four factors: grounded state representations, explicit transition modeling, self-experience trajectories from a sufficiently strong exploration policy, and adequate coverage of transition data.
Read the full abstract ▾
All authors Shiqi Chen, Tongyao Zhu, Zian Wang, Jinghan Zhang, Kangrui Wang, Ruochen Zhou, Siyang Gao, Teng Xiao, Yee Whye Teh, Junxian He, Manling Li
Computation and Language · Cross-listed from cs.CV
Composition-Grounded Data Synthesis for Visual Reasoning
Xinyi Gu, Jiayuan Mao, Zhang-Wei Hong et al. (+5 more)
Pretrained multi-modal large
language models (MLLMs) demonstrate strong performance on diverse
multimodal tasks, but remain limited in
reasoning capabilities for domains where annotations are difficult to collect. In this work, we focus on artificial image domains such as charts, rendered documents, and webpages, which are abundant in practice yet lack large-scale human annotated
reasoning datasets. We introduce COGS (COmposition-Grounded data Synthesis), a data-efficient framework for equipping MLLMs with advanced
reasoning abilities from a small set of seed questions. The key idea is to decompose each seed question into primitive perception and
reasoning factors, which can then be systematically recomposed with new images to generate large collections of synthetic question-answer pairs. Each generated question is paired with subquestions and intermediate answers, enabling reinforcement learning with factor-level process rewards. Experiments on chart
reasoning show that COGS substantially improves performance on unseen questions, with the largest gains on
reasoning-heavy and compositional questions. Moreover, training with a factor-level mixture of different seed data yields better transfer across multiple
datasets, suggesting that COGS induces generalizable capabilities rather than
dataset-specific overfitting. We further demonstrate that the framework extends beyond charts to other domains such as webpages.
Read the full abstract ▾
All authors Xinyi Gu, Jiayuan Mao, Zhang-Wei Hong, Zhuoran Yu, Pengyuan Li, Dhiraj Joshi, Rogerio Feris, Zexue He
Computation and Language · Cross-listed from stat.ML
The Coverage Principle: How Pre-Training Enables Post-Training
Fan Chen, Audrey Huang, Noah Golowich et al. (+5 more)
Language models demonstrate remarkable abilities when pre-trained on large text corpora and fine-tuned for specific tasks, but how and why pre-training shapes the success of the final model remains poorly understood. Notably, although pre-training success is often quantified by cross-
entropy loss, cross-
entropy can be a poor predictor of downstream performance. Instead, we provide a theoretical perspective on this relationship through the lens of coverage, which quantifies the probability mass the pre-trained model places on high-quality responses and which is necessary and sufficient for post-training and test-time scaling methods such as Best-of-N to succeed. Our main results develop an understanding of the coverage principle, a phenomenon whereby next-
token prediction (more generally, maximum likelihood) implicitly optimizes toward a model with good coverage. In particular, we uncover a mechanism that explains the power of coverage in predicting downstream performance: coverage generalizes faster than cross-
entropy, avoiding spurious dependence on problem-dependent parameters such as the sequence length. We also study practical algorithmic interventions with provable benefits for improving coverage, including (i) model/checkpoint selection procedures, (ii) gradient normalization schemes, and (iii) test-time decoding strategies.
Read the full abstract ▾
All authors Fan Chen, Audrey Huang, Noah Golowich, Sadhika Malladi, Adam Block, Jordan T. Ash, Akshay Krishnamurthy, Dylan J. Foster
Computation and Language · Cross-listed from cs.CV
DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage Mitigation in Text-to-Image Models
Mor Ventura, Michael Toker, Or Patashnik et al. (+2 more)
Text-to-Image (T2I) models have advanced rapidly, yet they remain vulnerable to semantic leakage, the unintended transfer of semantically related features between distinct entities. Existing mitigation strategies are often optimization-based or dependent on external inputs. We introduce DeLeaker, a lightweight, optimization-free inference-time approach that mitigates leakage by directly intervening on the model's
attention maps. Throughout the
diffusion process, DeLeaker dynamically reweights
attention maps to suppress excessive cross-entity interactions while strengthening the identity of each entity. To support systematic evaluation, we introduce SLIM (Semantic Leakage in IMages), the first
dataset dedicated to semantic leakage, comprising 1,130 human-verified samples spanning diverse scenarios, together with a novel automatic evaluation framework. Experiments demonstrate that DeLeaker consistently outperforms all baselines, even when they are provided with external information, achieving effective leakage mitigation without compromising fidelity or quality. These results underscore the value of
attention control and pave the way for more semantically precise T2I models.
Read the full abstract ▾
All authors Mor Ventura, Michael Toker, Or Patashnik, Yonatan Belinkov, Roi Reichart
Computation and Language
Can generative AI figure out figurative language? The influence of idioms on essay scoring by ChatGPT, Gemini, and Deepseek
Enis Oğuz
The developments in Generative AI technologies have paved the way for numerous innovations in different fields. Recently, Generative AI has been proposed as a competitor to AES systems in evaluating student essays automatically. Considering the potential limitations of AI in processing idioms, this study assessed the scoring performances of Generative AI models for essays with and without idioms by incorporating insights from Corpus Linguistics and Computational Linguistics. Two equal essay lists were created from 348 student essays taken from a corpus: one with multiple idioms present in each essay and another with no idioms in essays. Three Generative AI models (ChatGPT, Gemini, and Deepseek) were asked to score all essays in both lists three times, using the same rubric used by human raters in assigning essay scores. The results revealed excellent consistency for all models, but Gemini outperformed its competitors in interrater reliability with human raters. There was also no detectable bias for any demographic group in AI assessment. For essays with multiple idioms, Gemini followed a the most similar pattern to human raters. While the models in the study demonstrated potential for a hybrid approach, Gemini was the best candidate for the task due to its ability to handle figurative
language and showed promise for handling essay-scoring tasks alone in the future.
Read the full abstract ▾
Computation and Language
Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective
Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai et al. (+8 more)
Large
language models (LLMs) have achieved impressive results across a range of natural
language processing tasks, but their potential to generate harmful content has raised serious safety concerns. Current toxicity detectors primarily rely on single-label
benchmarks, which cannot adequately capture the inherently ambiguous and multi-dimensional nature of real-world toxic
prompts. This limitation results in biased evaluations, including missed toxic detections and false positives, undermining the reliability of existing detectors. Additionally, gathering comprehensive multi-label annotations across fine-grained toxicity categories is prohibitively costly, further hindering effective evaluation and development. To tackle these issues, we introduce three novel multi-label
benchmarks for toxicity detection: Q-A-MLL, R-A-MLL, and H-X-MLL, derived from public toxicity
datasets and annotated according to a detailed 15-category taxonomy. We further provide a theoretical proof that, on our released
datasets, training with pseudo-labels yields better performance than directly learning from single-label supervision. In addition, we develop a pseudo-label-based toxicity detection method. Extensive experimental results show that our approach significantly surpasses advanced baselines, including GPT-4o and DeepSeek, thus enabling more accurate and reliable evaluation of multi-label toxicity in LLM-generated content.
Read the full abstract ▾
All authors Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai, Ming-Kun Xie, Biao Liu, Changwei Wang, Lei Feng, Yuheng Jia, Gang Niu, Masashi Sugiyama, Xin Geng
Computation and Language · Cross-listed from cs.AI
Compositional Machine Design as Program Synthesis with LLMs
Wenqian Zhang, Yangyi Huang, Weiyang Liu et al. (+1 more)
Large
language models (LLMs) have shown strong abilities in writing and revising programs, yet many program-synthesis
benchmarks still evaluate programs in symbolic or digital environments. We introduce compositional machine design, a physically grounded form of program synthesis where machines are written as programs that compose standardized parts, and success is determined by simulated physical behavior. To study this problem, we present BesiegeField, a testbed built on the machine-building game Besiege. In BesiegeField, LLM
agents generate machine programs from textual functional demands, execute the resulting machines in simulation, and receive rewards and state feedback. We
benchmark LLM
agents across representative machine-design tasks under single-
agent generation, iterative editing, and hierarchical workflows. Strong models recover task-relevant structures and sometimes achieve nontrivial physical performance, but often struggle with spatially precise assembly, mechanism-level planning, and translating feedback into useful structural edits. We further finetune Qwen2.5-14B, an open-source LLM, with reinforcement learning from simulation-derived rewards. We find that, under a fixed generation budget, RL improves the best machine discovered. We additionally evaluate human performance to provide a reference point for task difficulty. These results establish compositional machine design as a testbed for studying LLM
agents that synthesize executable machine programs and improve them through physical feedback.
Read the full abstract ▾
All authors Wenqian Zhang, Yangyi Huang, Weiyang Liu, Zhen Liu
Computation and Language
Attention Is All You Need for KV Cache in Diffusion LLMs
Quan Nguyen-Tri, Mukul Ranjan and Zhiqiang Shen
This work studies how to adaptively recompute key-value (KV) caches for
diffusion large
language models (DLMs) to maximize prediction accuracy while minimizing decoding latency. Prior methods' decoders recompute QKV for all
tokens at every denoising step and layer, despite KV states changing little across most steps, especially in shallow layers, leading to substantial redundancy. We make three observations: (1) distant ${\bf MASK}$
tokens primarily act as a length-bias and can be cached block-wise beyond the active prediction window; (2) KV dynamics increase with depth, suggesting that selective refresh starting from deeper layers is sufficient; and (3) the most-attended
token exhibits the smallest KV drift, providing a conservative lower bound on cache change for other
tokens. Building on these, we propose ${\bf Elastic-Cache}$, a training-free, architecture-agnostic strategy that jointly decides ${when}$ to refresh (via an
attention-aware drift test on the most-attended
token) and ${where}$ to refresh (via a depth-aware schedule that recomputes from a chosen layer onward while reusing shallow-layer caches and off-window MASK caches). Unlike fixed-period schemes, Elastic-Cache performs adaptive, layer-aware cache updates for
diffusion LLMs, reducing redundant computation and accelerating decoding with negligible loss in generation quality. Experiments on LLaDA-Instruct, LLaDA-1.5, and LLaDA-V across mathematical
reasoning and code generation tasks demonstrate consistent speedups: $8.7\times$ on GSM8K (256
tokens), and $45.1\times$ on longer sequences, while consistently maintaining higher accuracy than the baseline. Our method achieves significantly higher throughput ($6.8\times$ on GSM8K) than existing confidence-based approaches while preserving generation quality, enabling practical deployment of
diffusion LLMs.
Read the full abstract ▾
Computation and Language
TokDrift: When LLM Speaks in Subwords but Code Speaks in Grammar
Yinxi Li, Yuntian Deng and Pengyu Nie
Large
language models (LLMs) for code rely on subword tokenizers, such as byte-pair encoding (BPE), learned from mixed natural
language text and programming
language code but driven by statistics rather than grammar. As a result, semantically identical code snippets can be tokenized differently depending on superficial factors such as whitespace or identifier naming. To measure the impact of this misalignment, we introduce TokDrift, a framework that applies semantic-preserving rewrite rules to create code variants differing only in tokenization. Across nine code LLMs, including large ones with over 30B parameters, even minor formatting changes can cause substantial shifts in model behavior. Layer-wise analysis shows that the issue originates in early
embeddings, where subword
segmentation fails to capture grammar
token boundaries. Our findings identify misaligned tokenization as a hidden obstacle to reliable code understanding and generation, highlighting the need for grammar-aware tokenization for future code LLMs.
Read the full abstract ▾
Computation and Language
LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training
Yiming Wang, Da Yin, Yuedong Cui et al. (+8 more)
Digital
agents require diverse, large-scale UI trajectories to generalize across real-world tasks, yet collecting such data is prohibitively expensive in both human annotation, infra and engineering perspectives. To this end, we introduce $\textbf{UI-Simulator}$, a scalable paradigm that generates structured UI states and transitions to synthesize training trajectories at scale. Our paradigm integrates a digital world simulator for diverse UI states, a guided rollout process for coherent exploration, and a trajectory wrapper that produces high-quality and diverse trajectories for
agent training. We further propose $\textbf{UI-Simulator-Grow}$, a targeted scaling strategy that enables more rapid and data-efficient scaling by prioritizing high-impact tasks and synthesizes informative trajectory variants. Experiments on WebArena and AndroidWorld show that UI-Simulator rivals or surpasses open-source
agents trained on real UIs with significantly better robustness, despite using weaker teacher models. Moreover, UI-Simulator-Grow matches the performance of Llama-3-70B-Instruct using only Llama-3-8B-Instruct as the base model, highlighting the potential of targeted synthesis scaling paradigm to continuously and efficiently enhance the digital
agents.
Read the full abstract ▾
All authors Yiming Wang, Da Yin, Yuedong Cui, Ruichen Zheng, Zhiqian Li, Zongyu Lin, Di Wu, Xueqing Wu, Chenchen Ye, Yu Zhou, Kai-Wei Chang
Computation and Language
Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents
Guoqing Wang, Sunhao Dai, Guangze Ye et al. (+5 more)
Large
language model (LLM)-based
agents are increasingly trained with reinforcement learning (RL) to enhance their ability to interact with external environments through tool use, particularly in search-based settings that require multi-turn
reasoning and knowledge acquisition. However, existing approaches typically rely on outcome-based rewards that are only provided exclusively upon generating the final answer. This reward sparsity becomes particularly problematic in multi-turn settings, where long trajectories exacerbate three critical issues: (i) advantage collapse, where all rollouts receive identical rewards and provide no useful learning signals; (ii) lack of fine-grained credit assignment, where the correctness of intermediate turns is obscured, especially in long-horizon tasks; and (iii) poor sample efficiency, where each rollout yields only a single outcome signal, leading to low data utilization. In this paper, we propose Information Gain-based Policy Optimization (IGPO), a simple yet effective RL framework that provides dense and intrinsic supervision for multi-turn
agent training. IGPO models each interaction turn as an incremental process of acquiring information about the ground truth, and defines turn-level rewards as the marginal increase in the policy's probability of producing the correct answer. Unlike prior process-level reward approaches that depend on external reward models or costly Monte Carlo estimation, IGPO derives intrinsic rewards directly from the model's own belief updates. These intrinsic turn-level rewards are combined with outcome-level supervision to form dense reward signals. Extensive experiments on both in-domain and out-of-domain
benchmarks demonstrate that IGPO consistently outperforms strong baselines in multi-turn scenarios, achieving higher accuracy and improved data efficiency. Our code is available at https://github.com/GuoqingWang1/IGPO.
Read the full abstract ▾
All authors Guoqing Wang, Sunhao Dai, Guangze Ye, Zeyu Gan, Wei Yao, Yong Deng, Xiaofeng Wu, Zhenzhe Ying
Computation and Language · Cross-listed from cs.LG
Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models
Jonas Geiping, Xinyu Yang and Guinan Su
Language models with recurrent depth, also referred to as universal or looped when considering
transformers, are defined by the capacity to increase their computation through the repetition of layers. Recent efforts in pretraining have demonstrated that these architectures can scale to modern
language modeling tasks while exhibiting advantages in
reasoning tasks. In this work, we examine the relationship between recurrent-depth models and
diffusion language models. Building on their similarities, we develop a new
diffusion forcing sampler for these models to accelerate generation. The sampler advances by decoding new
tokens at every forward pass of the model, while the latent states of these
tokens can be further refined in parallel through recurrence. Theoretically, generation with our sampler is strictly more expressive than the baseline autoregressive generation using the same time budget on modern hardware. Moreover, this sampler, based on principles from
diffusion literature, can be directly applied to existing 3.5B recurrent-depth
transformers without any tuning, leading to up to a 5x speedup. Consequently, our findings not only provide an efficient mechanism for parallelizing the extra computation in recurrent-depth models at inference, but also suggest that such models can be naturally viewed as strong continuous, though causal,
diffusion language models.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning
Weikang Shi, Aldrich Yu, Rongyao Fang et al. (+11 more)
While Large
Language Models (LLMs) have excelled in textual
reasoning, they struggle with mathematical domains like geometry that intrinsically rely on visual aids. Existing approaches to Visual Chain-of-Thought (VCoT) are often limited by rigid external tools or fail to generate the high-fidelity, strategically-timed diagrams necessary for complex problem-solving. To bridge this gap, we introduce MathCanvas, a comprehensive framework designed to endow unified Large
Multimodal Models (LMMs) with intrinsic VCoT capabilities for mathematics. Our approach consists of two phases. First, a Visual Manipulation stage pre-trains the model on a novel 15.2M-pair corpus, comprising 10M caption-to-diagram pairs (MathCanvas-Imagen) and 5.2M step-by-step editing trajectories (MathCanvas-Edit), to master diagram generation and editing. Second, a Strategic Visual-Aided
Reasoning stage fine-tunes the model on MathCanvas-Instruct, a new 219K-example
dataset of interleaved visual-textual
reasoning paths, teaching it when and how to leverage visual aids. To facilitate rigorous evaluation, we introduce MathCanvas-Bench, a challenging
benchmark with 3K problems that require models to produce interleaved visual-textual solutions. Our model, BAGEL-Canvas, trained under this framework, achieves an 86% relative improvement over strong LMM baselines on MathCanvas-Bench, demonstrating excellent generalization to other public math
benchmarks. Our work provides a complete toolkit-framework,
datasets, and
benchmark-to unlock complex, human-like visual-aided
reasoning in LMMs. Project Page: https://mathcanvas.github.io/
Read the full abstract ▾
All authors Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, Linjiang Huang, Si Liu, Rui Liu, Hongsheng Li
Computation and Language
DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation
Yu Zhou, Sohyun An, Haikang Deng et al. (+5 more)
Contact
languages like English exhibit rich regional variations in the form of dialects, which are often used by dialect speakers interacting with generative models. However, can
multimodal generative models effectively produce content given dialectal textual input? In this work, we study this question by constructing a new large-scale
benchmark spanning six common English dialects. We work with dialect speakers to collect and verify over 4200 unique
prompts and evaluate on 17 image and
video generative models. Our automatic and human evaluation results show that current state-of-the-art
multimodal generative models exhibit 32.26% to 48.17% performance degradation when a single dialect word is used in the
prompt. Common mitigation methods such as fine-tuning and
prompt rewriting can only improve dialect performance by small margins (< 7%), while potentially incurring significant performance degradation in Standard American English (SAE). To this end, we design a general encoder-based mitigation strategy for
multimodal generative models. Our method teaches the model to recognize new dialect features while preserving SAE performance. Experiments on models such as Stable
Diffusion 1.5 show that our method is able to simultaneously raise performance on five dialects to be on par with SAE (+34.4%), while incurring near zero cost to SAE performance.
Read the full abstract ▾
All authors Yu Zhou, Sohyun An, Haikang Deng, Da Yin, Clark Peng, Cho-Jui Hsieh, Kai-Wei Chang, Nanyun Peng
Computation and Language
MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics
Yuxing Lu, Xukai Zhao, J. Ben Tamo et al. (+6 more)
Large
Language Models (LLMs) have demonstrated remarkable capabilities on general text; however, their proficiency in specialized scientific domains that require deep, interconnected knowledge remains largely uncharacterized. Metabolomics presents unique challenges with its complex biochemical pathways, heterogeneous identifier systems, and fragmented databases. To systematically evaluate LLM capabilities in this domain, we introduce MetaBench, the first
benchmark for metabolomics assessment. Curated from authoritative public resources, MetaBench evaluates five capabilities essential for metabolomics research: knowledge, understanding, grounding,
reasoning, and research. Our evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks: while models perform well on text generation tasks, cross-database identifier grounding remains challenging even with
retrieval augmentation. Model performance also decreases on long-tail metabolites with sparse annotations. With MetaBench, we provide essential infrastructure for developing and evaluating metabolomics AI systems, enabling systematic progress toward reliable computational tools for metabolomics research.
Read the full abstract ▾
All authors Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang, May D. Wang
Computation and Language
LaSeR: Reinforcement Learning with Last-Token Self-Rewarding
Wenkai Yang, Weijie Liu, Ruobing Xie et al. (+4 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a core paradigm for enhancing the
reasoning capabilities of Large
Language Models (LLMs). To address the lack of verification signals at test time, prior studies incorporate the training of model's self-verification capability into the standard RLVR process, thereby unifying
reasoning and verification capabilities within a single LLM. However, previous practice requires the LLM to sequentially generate solutions and self-verifications using two separate
prompt templates, which significantly reduces efficiency. In this work, we theoretically reveal that the closed-form solution to the RL objective of self-verification can be reduced to a remarkably simple form: the true
reasoning reward of a solution is equal to its last-
token self-rewarding score, which is computed as the difference between the policy model's next-
token log-probability assigned to any pre-specified
token at the solution's last
token and a pre-calculated constant, scaled by the KL coefficient. Based on this insight, we propose
LaSeR (Reinforcement Learning with Last-
Token Self-Rewarding), an
algorithm that simply augments the original RLVR loss with a MSE loss that aligns the last-
token self-rewarding scores with verifier-based
reasoning rewards, jointly optimizing the
reasoning and self-rewarding capabilities of LLMs. The optimized self-rewarding scores can be utilized in both training and testing to enhance model performance. Notably, our
algorithm derives these scores from the predicted next-
token probability distribution of the last
token immediately after generation, incurring only the minimal extra cost of one additional
token inference. Experiments show that our method not only improves the model's
reasoning performance but also equips it with remarkable self-rewarding capability, thereby boosting its inference-time scaling performance.
Read the full abstract ▾
All authors Wenkai Yang, Weijie Liu, Ruobing Xie, Yiju Guo, Lulu Wu, Saiyong Yang, Yankai Lin
Computation and Language
AI-Powered Early Diagnosis of Mental Health Disorders from Real-World Clinical Conversations
Jianfeng Zhu, Julina Maharjan, Xinyu Li et al. (+2 more)
Mental health disorders remain among the leading cause of disability worldwide, yet conditions such as depression, anxiety, and Post-Traumatic Stress Disorder (PTSD) are frequently underdiagnosed or misdiagnosed due to subjective assessments, limited clinical resources, and stigma and low awareness. In primary care settings, studies show that providers misidentify depression or anxiety in over 60% of cases, highlighting the urgent need for scalable, accessible, and context-aware diagnostic tools that can support early detection and intervention. In this study, we evaluate the effectiveness of machine learning models for mental health screening using a unique
dataset of 553 real-world, semistructured interviews, each paried with ground-truth diagnoses for major depressive episodes (MDE), anxiety disorders, and PTSD. We
benchmark multiple model classes, including zero-shot prompting with GPT-4.1 Mini and MetaLLaMA, as well as fine-tuned RoBERTa models using LowRank Adaptation (LoRA). Our models achieve over 80% accuracy across diagnostic categories, with especially strongperformance on PTSD (up to 89% accuracy and 98% recall). We also find that using shorter context, focused context segments improves recall, suggesting that focused narrative cues enhance detection sensitivity. LoRA fine-tuning proves both efficient and effective, with lower-rank configurations (e.g., rank 8 and 16) maintaining competitive performance across evaluation metrics. Our results demonstrate that LLM-based models can offer substantial improvements over traditional self-report screening tools, providing a path toward low-barrier, AI-powerd early diagnosis. This work lays the groundwork for integrating machine learning into real-world clinical workflows, particularly in low-resource or high-stigma environments where access to timely mental health care is most limited.
Read the full abstract ▾
All authors Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G. Coifman, Ruoming Jin
Computation and Language · Cross-listed from cs.LG
Circuit Insights: Towards Interpretability Beyond Activations
Elena Golimblevskaia, Aakriti Jain, Bruno Puri et al. (+3 more)
The fields of explainable AI and mechanistic interpretability aim to uncover the internal structure of neural networks, with
circuit discovery as a central tool for understanding model computations. Existing approaches, however, rely on manual inspection and remain limited to toy tasks. Automated interpretability offers scalability by analyzing isolated features and their activations, but it often misses interactions between features and depends strongly on external LLMs and
dataset quality. Transcoders have recently made it possible to separate feature attributions into input-dependent and input-invariant components, providing a foundation for more systematic
circuit analysis. Building on this, we propose WeightLens and CircuitLens, two complementary methods that go beyond activation-based analysis. WeightLens interprets features directly from their learned weights, removing the need for explainer models or
datasets while matching or exceeding the performance of existing methods on context-independent features. CircuitLens captures how feature activations arise from interactions between components, revealing
circuit-level dynamics that activation-only approaches cannot identify. Together, these methods increase interpretability robustness and enhance scalable mechanistic analysis of
circuits while maintaining efficiency and quality.
Read the full abstract ▾
All authors Elena Golimblevskaia, Aakriti Jain, Bruno Puri, Ammar Ibrahim, Wojciech Samek, Sebastian Lapuschkin
Computation and Language · Cross-listed from cs.AI
False Fixed Points: Kantian Feedback, Stable Miscalibration, and Representational Compression in LLMs
Akira Okutomi
High-confidence errors in large
language models are often treated as fragile failures. We study an alternative: some errors may be false fixed points, locally stable, internally coherent, and confidently wrong. This separates robustness from truth-tracking. We develop the separation through a Kantian commitment-gate framing and a minimal linear feedback model in which stability and correctness can diverge. Across three open-weight models, overconfident wrong items are not systematically more locally fragile than confidently correct items under our hidden-state sensitivity probes. Abstention-aware self-critique reduces overconfident wrong commitments by sacrificing coverage, and C3-R, a rule-based explicit feedback gate, sharpens that tradeoff rather than eliminating it. These results motivate, but do not establish, high signal-to-noise (high-SNR) inertia and representational compression as possible mechanisms for stable miscalibration.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
TRI-DEP: A Trimodal Comparative Study for Depression Detection Using Speech, Text, and EEG
Annisaa Fitri Nurfidausi, Eleonora Mancini and Paolo Torroni
Depression is a widespread mental health disorder, yet its automatic detection remains challenging. Prior work has explored unimodal and
multimodal approaches, with
multimodal systems showing promise by leveraging complementary signals. However, existing studies are limited in scope, lack systematic comparisons of features, and suffer from inconsistent evaluation protocols. We address these gaps by systematically exploring feature representations and modelling strategies across
EEG, together with
speech and text. We evaluate handcrafted features versus pre-trained
embeddings, assess the effectiveness of different neural encoders, compare unimodal, bimodal, and trimodal configurations, and analyse fusion strategies with
attention to the role of
EEG. Consistent subject-independent splits are applied to ensure robust, reproducible benchmarking. Our results show that (i) the combination of
EEG,
speech and text modalities enhances
multimodal detection, (ii) pretrained
embeddings outperform handcrafted features, and (iii) carefully designed trimodal models achieve state-of-the-art performance. Our work lays the groundwork for future research in
multimodal depression detection.
Read the full abstract ▾
Computation and Language
Predicting Task Performance with Context-aware Scaling Laws
Kyle Montgomery, David Park, Jianhong Tu et al. (+4 more)
Scaling laws have transformed our understanding of large
language models by linking upstream metrics like cross-
entropy loss to design factors such as model size, training data, and compute. However, these conventional laws fail to capture downstream task performance, where context plays a critical role. In this work, we propose a straightforward, interpretable framework that jointly models downstream performance as a function of the training compute and the provided context. We empirically validate our framework by fitting it on the observed downstream performance of extended-context variants of Llama-2-7B and Llama-2-13B across 65,500 unique instances spanning three tasks: arithmetic
reasoning, common sense
reasoning, and machine translation. Our results demonstrate that our framework accurately models in-distribution downstream performance, generalizes across three orders of magnitude in training compute, and reliably extrapolates performance as the amount of context increases. These findings offer valuable insights into the interplay between training compute and context utilization, providing guidance for designing more efficient long-context LLMs for diverse downstream tasks. Our code is available at https://github.com/wang-research-lab/context-scaling.
Read the full abstract ▾
All authors Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang
Computation and Language
Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation
Xujun Peng, Anoop Kumar, Jingyu Wu et al. (+2 more)
Retrieval-Augmented Generation (RAG) systems leverage Large
Language Models (LLMs) to generate accurate and reliable responses that are grounded in retrieved context. However, LLMs often generate inconsistent outputs for semantically equivalent inputs, a problem compounded by the scarcity of consistency-focused training data and the limitations of current fine-tuning techniques in enhancing output consistency. We propose a new approach combining systematic synthetic data generation, triplet loss for better
embeddings, and a novel layer-wise model merging approach. Using consistency-aware weights derived from intermediate layer activations, our method effectively integrates knowledge from specialized models. Experimental results how that our merged model significantly enhances output consistency, achieving a ~47.5% improvement in response similarity over the baseline, thus offering a practical solution for increasing the reliability of an industrial RAG system.
Read the full abstract ▾
All authors Xujun Peng, Anoop Kumar, Jingyu Wu, Parker Glenn, Daben Liu