Computation and Language
Ensembling Large Language Models to Characterize Affective Dynamics in Student-AI Tutor Dialogues
Chenyu Zhang, Sharifa Alghowinem and Cynthia Breazeal
While recent studies have examined the leaning impact of large
language model (LLM) in educational contexts, the affective dynamics of LLM-mediated tutoring remain insufficiently understood. This work introduces the first ensemble-LLM framework for large-scale affect sensing in tutoring dialogues, advancing the conversation on responsible pathways for integrating generative AI into education by attending to learners' evolving affective states. To achieve this, we analyzed two semesters' worth of 16,986 conversational turns exchanged between PyTutor, an LLM-powered AI tutor, and 261 undergraduate learners across three U.S. institutions. To investigate learners' emotional experiences, we generate zero-shot affect annotations from three frontier LLMs (Gemini, GPT-4o, Claude), including scalar ratings of valence, arousal, and learning-helpfulness, along with free-text
emotion labels. These estimates are fused through rank-weighted intra-model pooling and plurality consensus across models to produce robust
emotion profiles. Our analysis shows that during interaction with the AI tutor, students typically report mildly positive affect and moderate arousal. Yet learning is not uniformly smooth: confusion and curiosity are frequent companions to problem solving, and frustration, while less common, still surfaces in ways that can derail progress. Emotional states are short-lived–positive moments last slightly longer than neutral or negative ones, but they are fragile and easily disrupted. Encouragingly, negative
emotions often resolve quickly, sometimes rebounding directly into positive states. Neutral moments frequently act as turning points, more often steering students upward than downward, suggesting opportunities for tutors to intervene at precisely these junctures.
Read the full abstract ▾
Computation and Language
ShishuLM : Achieving Optimal and Efficient Parameterization with Low Attention Transformer Models
Shivanshu Kumar and Gopalakrishnan Srinivasan
While the
transformer architecture has achieved state-of-the-art performance on natural
language processing tasks, these models impose substantial
memory and computational overhead. Recent research has identified significant architectural redundancies within these models, particularly in the
attention sub-layers in the top layers, presenting opportunities for optimization without compromising performance. Taking insights from research on inference-time layer
pruning and depth-dependent computation in
language models, we introduce an efficient
language model architecture referred to as ShishuLM. By replacing full decoder layers at the top of the model with MLP-only blocks, we achieve up to 10-60% improvement in generation latency and 1.3 -5 $\times$ gain in throughput. Upon further sharing parameters across adjacent MLP-only layers of ShishuLM, we obtain up to 20% savings in
memory with minimal degradation in performance. Our findings provide insights towards building more efficient
language modeling architectures from a pre-training standpoint by leveraging how information flows in
transformers.
Read the full abstract ▾
Computation and Language
FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs
Yingjia Wan, Haochen Tan, Xiao Zhu et al. (+9 more)
Evaluating the factuality of long-form generations from Large
Language Models (LLMs) remains challenging due to efficiency bottlenecks and reliability concerns. Prior efforts attempt this by decomposing text into claims, searching for evidence, and verifying claims, but suffer from critical drawbacks: (1) inefficiency due to overcomplicated pipeline components, and (2) ineffectiveness stemming from inaccurate claim sets and insufficient evidence. To address these limitations, we propose FaStfact, an evaluation framework that achieves the highest
alignment with human evaluation and time/
token efficiency among existing baselines. FaStfact first employs chunk-level claim extraction integrated with confidence-based pre-verification, significantly reducing the time and
token cost while ensuring reliability. For searching and verification, it collects document-level evidence from crawled web-pages and selectively retrieves it during verification. Extensive experiments based on an annotated
benchmark FaStfact-Bench demonstrate the reliability of FaStfact in both efficiently and effectively evaluating long-form factuality. Code,
benchmark data, and annotation interface tool are available at https://github.com/Yingjia-Wan/FaStfact.
Read the full abstract ▾
All authors Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo
Computation and Language
A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning
Qianben Chen, Jingyi Cao, Jiayu Zhang et al. (+12 more)
Large
language models split into two families:
reasoning-centric LLMs, which strengthen internal chain-of-thought
reasoning but cannot invoke external tools, and agentic LLMs, which learn to interact with environments and leverage tools but often lag in deep
reasoning. This divide arises from fundamentally different training objectives, leading to mismatched strengths and inefficiency on simple queries, where both families tend to overthink or over-call tools. In this work, we present Adaptive
Agent Foundation Model (A$^2$FM), a unified framework that follows a route-then-align principle: the model first learns task-aware routing and then aligns mode-specific trajectories under a shared backbone. To address the inefficiency gap, we introduce a third mode-instant-that handles simple queries directly, preventing unnecessary
reasoning or tool calls while complementing the agentic and
reasoning modes. To jointly enhance accuracy and efficiency, we propose Adaptive Policy Optimization (APO), which enforces adaptive sampling across modes and applies a cost-regularized reward. On the 32B scale, A$^2$FM achieves 13.4% on BrowseComp, 70.4% on AIME25, and 16.7% on HLE, setting new SOTA among comparable models and performing competitively with frontier LLMs across agentic,
reasoning, and general
benchmarks. Notably, the adaptive execution achieves a cost of pass of only $0.00487 per correct answer-cutting cost by 45.2% relative to
reasoning and 33.5% relative to agentic, thus delivering substantially higher cost efficiency while maintaining comparable accuracy.
Read the full abstract ▾
All authors Qianben Chen, Jingyi Cao, Jiayu Zhang, Tianrui Qin, Xiaowan Li, King Zhu, Dingfeng Shi, He Zhu, Minghao Liu, Xiaobo Liang, Xin Gui, Ge Zhang, Jian Yang, Yuchen Eleanor Jiang, Wangchunshu Zhou
Computation and Language
Repurposing Annotation Guidelines to Instruct LLM Annotators: A Case Study
Kon Woo Kim, Rezarta Islamaj, Jin-Dong Kim et al. (+2 more)
This study investigates how existing annotation guidelines can be repurposed to instruct large
language model (LLM) annotators for text annotation tasks. Traditional guidelines are written for human annotators who internalize training, while LLMs require explicit, structured instructions. We propose a moderation-oriented guideline repurposing method that transforms guidelines into clear directives for LLMs through an LLM moderation process. Using the NCBI Disease Corpus as a case study, our experiments show that repurposed guidelines can effectively guide LLM annotators, while revealing several practical challenges. The results highlight the potential of this workflow to support scalable and cost-effective refinement of annotation guidelines and automated annotation.
Read the full abstract ▾
All authors Kon Woo Kim, Rezarta Islamaj, Jin-Dong Kim, Florian Boudin, Akiko Aizawa
Computation and Language
Generate Logical Equivalence Questions
Xinyu Wang, Haoming Yu, Yicheng Yang et al. (+1 more)
Academic dishonesty is met with zero tolerance in higher education, yet plagiarism has become increasingly prevalent in the era of online teaching and learning. Automatic Question Generation (AQG) presents a potential solution to mitigate copying by creating unique questions for each student. Additionally, AQG can provide a vast array of practice questions. Our AQG focuses on generating logical equivalence questions for Discrete Mathematics, a foundational course for first-year computer science students. A literature review reveals that existing AQGs for this type of question generate all propositions that meet user-defined constraints, resulting in inefficiencies and a lack of uniform question difficulty. To address this, we propose a new approach that defines logical equivalence questions using a formal
language, translates this
language into two sets of generation rules, and develops a linear-time
algorithm for question generation. We evaluated our AQG through two experiments. The first involved a group of students completing questions generated by our system. Statistical analysis shows that the accuracy of these questions is comparable to that of textbook questions. The second experiment assessed the number of steps required to solve our generated questions, textbook questions, and those generated by multiple large
language models. The results indicated that the difficulty of our questions was similar to that of textbook questions, confirming the quality of our AQG.
Read the full abstract ▾
All authors Xinyu Wang, Haoming Yu, Yicheng Yang, Zhiyuan Li
Computation and Language · Cross-listed from cs.SD
UALM: Unified Audio Language Model for Understanding, Generation and Reasoning
Jinchuan Tian, Sang-gil Lee, Zhifeng Kong et al. (+11 more)
Recent advances in the audio
language modeling (ALM) domain tackle audio understanding and text-to-audio generation as separate tasks. Very few studies attempt to unify these tasks – an essential step toward advanced
multimodal reasoning. This paper introduces U}nified Audio
Language Model (UALM), which aims to unify audio understanding, text-to-audio generation, and
multimodal reasoning in a single model. To achieve this goal, we first present UALM-Gen, a text-to-audio
language model that directly predicts audio
tokens and is comparable to state-of-the-art
diffusion-based models. We then demonstrate, using proper data blending, training recipes, and inference techniques, that our single UALM model matches the quality of state-of-the-art specialized models in audio understanding, text-to-audio generation, and text
reasoning. Furthermore, we present UALM-Reason, a
multimodal reasoning model that utilizes both text and audio in the intermediate thinking steps to facilitate complex generation tasks. To our knowledge, this is the first demonstration in audio research of cross-modal generative
reasoning, with its effectiveness confirmed by subjective evaluations.
Read the full abstract ▾
All authors Jinchuan Tian, Sang-gil Lee, Zhifeng Kong, Sreyan Ghosh, Arushi Goel, Chao-Han Huck Yang, Wenliang Dai, Zihan Liu, Hanrong Ye, Shinji Watanabe, Mohammad Shoeybi, Bryan Catanzaro, Rafael Valle, Wei Ping
Computation and Language
SAGE: A Top-Down Bottom-Up Knowledge-Grounded User Simulator for Multi-turn AGent Evaluation
Ryan Shea, Yunan Lu, Liang Qiu et al. (+1 more)
Evaluating multi-turn interactive
agents is challenging due to the need for human assessment. Evaluation with simulated users has been introduced as an alternative, however existing approaches typically model generic users and overlook the domain-specific principles required to capture realistic behavior. We propose SAGE, a novel user Simulation framework for multi-turn
AGent Evaluation that integrates knowledge from business contexts. SAGE incorporates top-down knowledge rooted in business logic, such as ideal customer profiles, grounding user behavior in realistic customer personas. We further integrate bottom-up knowledge taken from business
agent infrastructure (e.g., product catalogs, FAQs, and knowledge bases), allowing the simulator to generate interactions that reflect users' information needs and expectations in a company's target
market. Through empirical evaluation, we find that this approach produces interactions that are more realistic and diverse, while also identifying up to 33% more
agent errors, highlighting its effectiveness as an evaluation tool to support bug-finding and iterative
agent improvement.
Read the full abstract ▾
All authors Ryan Shea, Yunan Lu, Liang Qiu, Zhou Yu
Computation and Language
Conjecturing: An Overlooked Step in Formal Mathematical Reasoning
Jasivan Alex Sivakumar, Philipp Borchert, Ronald Cardenas et al. (+1 more)
Autoformalisation, the task of expressing informal mathematical statements in formal
language, is often viewed as a direct translation process. This, however, disregards a critical preceding step: conjecturing. Many mathematical problems cannot be formalised directly without first conjecturing a conclusion such as an explicit answer, or a specific bound. Since Large
Language Models (LLMs) already struggle with autoformalisation, and the evaluation of their conjecturing ability is limited and often entangled within autoformalisation or proof, it is particularly challenging to understand its effect. To address this gap, we augment existing
datasets to create ConjectureBench, and redesign the evaluation framework and metric specifically to measure the conjecturing capabilities of LLMs both as a distinct task and within the autoformalisation pipeline. Our evaluation of foundational models, including GPT-4.1 and DeepSeek-V3.1, reveals that their autoformalisation performance is substantially overestimated when the
conjecture is accounted for during evaluation. However, the
conjecture should not be assumed to be provided. We design an inference-time method, Lean-FIRe to improve conjecturing and autoformalisation, which, to the best of our knowledge, achieves the first successful end-to-end autoformalisation of 13 PutnamBench problems with GPT-4.1 and 7 with DeepSeek-V3.1. We demonstrate that while LLMs possess the requisite knowledge to generate accurate
conjectures, improving autoformalisation performance requires treating conjecturing as an independent task, and investigating further how to correctly integrate it within autoformalisation. Finally, we provide forward-looking guidance to steer future research toward improving conjecturing, an overlooked step of formal mathematical
reasoning.
Read the full abstract ▾
All authors Jasivan Alex Sivakumar, Philipp Borchert, Ronald Cardenas, Gerasimos Lampouras
Computation and Language · Cross-listed from cs.AI
Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
Sayash Kapoor, Benedikt Stroebl, Peter Kirgis et al. (+28 more)
AI
agents have been developed for complex real-world tasks from coding to customer service. But AI
agent evaluations suffer from many challenges that undermine our understanding of how well
agents really work. We introduce the Holistic
Agent Leaderboard (HAL) to address these challenges. We make three main contributions. First, we provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of VMs, reducing evaluation time from weeks to hours while eliminating common implementation bugs. Second, we conduct three-dimensional analysis spanning models, scaffolds, and
benchmarks. We validate the harness by conducting 21,730
agent rollouts across 9 models and 9
benchmarks in coding, web
navigation, science, and customer service with a total cost of about $40,000. Our analysis reveals surprising insights, such as higher
reasoning effort reducing accuracy in the majority of runs. Third, we use LLM-aided log inspection to uncover previously unreported behaviors, such as searching for the
benchmark on HuggingFace instead of solving a task, or misusing credit cards in flight booking tasks. We share all
agent logs, comprising 2.5B
tokens of
language model calls, to incentivize further research into
agent behavior. By standardizing how the field evaluates
agents and addressing common pitfalls in
agent evaluation, we hope to shift the focus from
agents that ace
benchmarks to
agents that work reliably in the real world.
Read the full abstract ▾
All authors Sayash Kapoor, Benedikt Stroebl, Peter Kirgis, Nitya Nadgir, Zachary S Siegel, Boyi Wei, Tianci Xue, Ziru Chen, Felix Chen, Saiteja Utpala, Franck Ndzomga, Dheeraj Oruganty, Sophie Luskin, Kangheng Liu, Botao Yu, Amit Arora, Dongyoon Hahm, Harsh Trivedi, Huan Sun, Juyong Lee, Tengjun Jin, Yifan Mai, Yifei Zhou, Yuxuan Zhu, Rishi Bommasani, Daniel Kang, Dawn Song, Peter Henderson, Yu Su, Percy Liang, Arvind Narayanan
Computation and Language
Scaling Long-Horizon LLM Agent via Context-Folding
Weiwei Sun, Miao Lu, Zhan Ling et al. (+4 more)
Large
language model (LLM)
agents are fundamentally constrained by context length on long-horizon tasks. We introduce Context-Folding, a framework that empowers
agents to actively manage their working context. An
agent can procedurally branch into a sub-trajectory to handle a subtask and then fold it upon completion, collapsing the intermediate steps while retaining a concise summary of the outcome. To make this behavior learnable, we develop an end-to-end reinforcement learning framework FoldGRPO with specific process rewards to encourage effective task decomposition and context management. On complex long-horizon tasks (Deep Research and SWE), our folding
agent matches or outperforms the ReAct baselines while using an active context 10$\times$ smaller and significantly outperforms models that rely on summarization-based context management.
Read the full abstract ▾
All authors Weiwei Sun, Miao Lu, Zhan Ling, Kang Liu, Xuesong Yao, Yiming Yang, Jiecao Chen
Computation and Language
Direct Multi-Token Decoding
Xuan Luo, Weizhi Wang and Xifeng Yan
Decoder-only
transformers have become the standard architecture for large
language models (LLMs) due to their strong performance. Recent studies suggest that, in pre-trained LLMs, early, middle, and late layers may serve distinct roles: Early layers focus on understanding the input context, middle layers handle task-specific processing, and late layers convert abstract representations into output
tokens. We hypothesize that once representations have been processed by the early and middle layers, the resulting hidden states may encapsulate sufficient information to support the generation of multiple
tokens using only the late layers, eliminating the need to repeatedly traverse the early and middle layers. We refer to this inference paradigm as Direct Multi-
Token Decoding (DMTD). Unlike speculative decoding, our method introduces no additional parameters, auxiliary routines, or post-generation verification. Despite being trained on a limited
dataset, a fine-tuned DMTD Qwen3-4B model has already demonstrated promising results, achieving up to a 2x speedup with only minor performance loss. Moreover, as shown in our scaling analysis, its performance is expected to further improve with larger training
datasets.
Read the full abstract ▾
Computation and Language
Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries
Gabrielle Kaili-May Liu, Bryan Li, Arman Cohan et al. (+2 more)
Real-world use cases often present RAG systems with complex queries for which relevant information is missing from the corpus or is incomplete. In these settings, RAG systems must be able to reject unanswerable, out-of-scope queries and identify failures of
retrieval and multi-hop
reasoning. Despite this, existing RAG
benchmarks rarely reflect realistic task complexity for multi-hop or out-of-scope questions, which often can be cheated via disconnected
reasoning (i.e., solved without genuine multi-hop inference) or require only simple factual recall. This limits the ability for such
benchmarks to uncover limitations of existing RAG systems. To address this gap, we present the first pipeline for automatic, difficulty-controlled creation of un$\underline{c}$heatable, $\underline{r}$ealistic, $\underline{u}$nanswerable, and $\underline{m}$ulti-hop $\underline{q}$uerie$\underline{s}$ (CRUMQs), adaptable to any corpus and domain. We use our pipeline to create CRUMQs over two popular RAG
datasets and demonstrate its effectiveness via
benchmark experiments on leading
retrieval-augmented LLMs. Results show that compared to prior RAG
benchmarks, CRUMQs are highly challenging for RAG systems and achieve up to 81.0% reduction in cheatability scores. More broadly, our pipeline offers a simple way to enhance
benchmark difficulty and drive development of more capable RAG systems.
Read the full abstract ▾
All authors Gabrielle Kaili-May Liu, Bryan Li, Arman Cohan, William Gantt Walden, Eugene Yang
Computation and Language
GRAVITY: A Framework for Personalized Text Generation via Profile-Grounded Synthetic Preferences
Priyanka Dey, Daniele Rosa, Wenqing Zheng et al. (+3 more)
Personalization in LLMs often relies on costly human feedback or interaction logs, limiting scalability and neglecting deeper user attributes. To reduce the reliance on human annotations, we introduce
GRAVITY (Generative Response with Aligned Values, Interests, and Traits of You), a framework for generating synthetic, profile-grounded preference data that captures users' interests, values, beliefs, and personality traits. By integrating demographic, cultural, and psychological frameworks – including Hofstede's cultural dimensions, Schwartz's basic values, the World Values Survey, and Big Five OCEAN traits –
GRAVITY synthesizes preference pairs to guide personalized content generation. We evaluate
GRAVITY on book descriptions for 400 Amazon users, comparing it to
prompt-based conditioning, standard fine-tuning, and naive synthetic pair generation. Profile-grounded synthetic data consistently improves generation, especially across multiple cultures (USA, Brazil, Japan, India), achieving over 4% higher preference gains across baselines, with user studies showing that
GRAVITY outputs are preferred over 86% of the time. Our results show that scenario-grounded synthetic data can capture richer user variation, reduce reliance on costly annotation, and produce more engaging, user-centered content, offering a scalable path for LLM personalization.
Read the full abstract ▾
All authors Priyanka Dey, Daniele Rosa, Wenqing Zheng, Daniel Barcklow, Jieyu Zhao, Emilio Ferrara
Computation and Language
TopoAlign: A Framework for Aligning Code to Math via Topological Decomposition
Yupei Li, Philipp Borchert and Gerasimos Lampouras
Large
Language Models (LLMs) excel at both informal and formal (e.g. Lean 4) mathematical
reasoning but still struggle with autoformalisation, the task of transforming informal into formal mathematical statements. Yet, the performance of current Math LLMs is constrained by the scarcity of large-scale corpora, particularly those containing pairs of informal and formal statements. Interestingly, the formal
languages used in autoformalisation share structural similarities with programming
languages, and code data is available at scale. However, current models trained on code do not transfer effectively to formal math, due to structural and syntactic differences between them. To address this, we propose TopoAlign, a framework that unlocks widely available code repositories as training resources for Math LLMs. TopoAlign decomposes code into docstrings, main functions, and dependency functions, and reassembles these components into analogues that structurally mirror formal statements. We train three state-of-the-art models, DeepSeek-Math, Qwen-3 and Herald, and evaluate them on the MiniF2F, Putnam, and ProofNet
benchmarks. TopoAlign provides substantial gains for DeepSeek-Math, improving performance by 17.77% on BEq@10 and 68.82% on typecheck@10, and also measurably improves Herald by 0.12% on BEq@10 and 1.09% on typecheck@10 despite introducing no new mathematical knowledge.
Read the full abstract ▾
Computation and Language
Discrepancy Detection at the Data Level: Toward Consistent Multilingual Question Answering
Lorena Calvo-Bartolomé, Valérie Aldana, Karla Cantarero et al. (+3 more)
Multilingual question answering (QA) systems must ensure factual consistency across
languages, especially for objective queries such as What is jaundice?, while also accounting for cultural variation in subjective responses. We propose MIND, a user-in-the-loop fact-checking pipeline to detect factual and cultural discrepancies in
multilingual QA knowledge bases. MIND highlights divergent answers to culturally sensitive questions (e.g., Who assists in childbirth?) that vary by region and context. We evaluate MIND on a bilingual QA system in the maternal and infant health domain and release a
dataset of bilingual questions annotated for factual and cultural inconsistencies. We further test MIND on
datasets from other domains to assess generalization. In all cases, MIND reliably identifies inconsistencies, supporting the development of more culturally aware and factually consistent QA systems.
Read the full abstract ▾
All authors Lorena Calvo-Bartolomé, Valérie Aldana, Karla Cantarero, Alonso Madroñal de Mesa, Jerónimo Arenas-García, Jordan Boyd-Graber
Computation and Language
LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
Armel Zebaze, Rachel Bawden and Benoît Sagot
Large
reasoning models (LRMs) have led to new possibilities in terms of problem-solving, through the devising of a natural
language thought process prior to answering a query. While their capabilities are well known across mathematics and coding tasks, their impact on the task of machine translation (MT) remains underexplored. In this work, we explore the benefits of the generation of intermediate
tokens when performing MT across multiple
language pairs of different levels of resourcedness and multiple setups. We find that "thinking
tokens" do not help LRMs better perform MT. This result generalizes to models fine-tuned to reason before translating using distilled chain of thought (CoT) inspired by human translators' practices. Specifically, fine-tuning a model with synthetic CoT explanations detailing how to translate step-by-step does not outperform standard input-output fine-tuning. However, constructing the intermediate
tokens by combining the outputs of modular translation-specific prompting strategies results in improvements. Our findings underscore that the contribution of intermediate
tokens during fine-tuning highly depends on the presence of translation attempts within them. More broadly, our results suggest that using a teacher to refine target translations or to expand parallel corpora is more impactful than distilling their CoT explanations into "thinking" MT models.
Read the full abstract ▾
Computation and Language
LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance
Patrick Haller, Mark Ibrahim, Polina Kirichenko et al. (+2 more)
For Large
Language Models (LLMs) to be reliable, they must learn robust knowledge that can be generally applied in diverse settings – often unlike those seen during training. Yet, extensive research has shown that LLM performance can be brittle, with models exhibiting excessive sensitivity to trivial input variations. In this work, we explore whether this brittleness is a direct result of unstable internal knowledge representations. To explore this question, we build on previous work showing that LLM representations encode statement truthfulness – i.e., true, factual statements can be easily separated from false, inaccurate ones. Specifically, we test the robustness of learned knowledge by evaluating representation separability on samples that have undergone superficial transformations to drive them out-of-distribution (OOD), such as typos or reformulations. By applying semantically-preserving perturbations, we study how separability degrades as statements become more OOD, across four LLM families, five evaluation
datasets, and three knowledge probing methods. Our results reveal that internal representations of statement truthfulness collapse as the samples' presentations become less similar to those seen during pre-training. While LLMs can often distinguish between true and false statements when they closely resemble the pre-training data, this ability is highly dependent on the statement's exact surface form. These findings offer a possible explanation for brittle
benchmark performance: LLMs may learn shallow, non-robust knowledge representations that allow for only limited generalizability. Our work presents a fundamental challenge for the utility of truthfulness probes, and more broadly, calls for further research on improving the robustness of learned knowledge representations.
Read the full abstract ▾
All authors Patrick Haller, Mark Ibrahim, Polina Kirichenko, Levent Sagun, Samuel J. Bell
Computation and Language
R-WoM: Retrieval-augmented World Model For Computer-use Agents
Kai Mei, Jiang Guo, Shuaichen Chang et al. (+4 more)
Large
Language Models (LLMs) can serve as world models to enhance
agent decision-making in digital environments by simulating future states and predicting action outcomes, potentially eliminating costly trial-and-error exploration. However, this capability is fundamentally limited by LLMs' tendency toward
hallucination and their reliance on static training knowledge, which can lead to compounding errors that inhibit long-horizon simulations. To systematically investigate whether LLMs are appropriate for world modeling, we probe two core capabilities of world models–future state prediction and reward estimation–through three tasks: next-state identification, full-procedure planning
alignment, and milestone transition recognition. Our analysis shows that while LLMs effectively capture immediate next states and identify meaningful state transitions, their performance rapidly degrades in full-procedure planning. This highlights LLMs' limitations in reliably modeling environment dynamics over long horizons. To address these limitations, we propose the
Retrieval-augmented World Model (R-WoM), which grounds LLM simulations by incorporating factual, up-to-date knowledge retrieved from external tutorials. Experiments show that R-WoM achieves relative improvements of up to 23.4% and 16.3% on the subsets of OSWorld and Webarena compared to baselines, with particular advantage in longer-horizon simulations.
Read the full abstract ▾
All authors Kai Mei, Jiang Guo, Shuaichen Chang, Mingwen Dong, Dongkyu Lee, Xing Niu, Jiarong Jiang
Computation and Language · Cross-listed from cs.CR
Deep Research Agents Brings Deeper Harm
Shuo Chen, Zonggen Li, Xingyu Jin et al. (+8 more)
We reveal that Deep Research (DR)
agents systematically expose safety risks: simply submitting harmful queries that a standalone LLM would reject outright can elicit detailed and dangerous reports from DR
agents. Empirical analysis reveals that the advantages that make DR
agents powerful unintentionally make them vulnerable: both the research role assignment (e.g., Planner) and the multi-step execution mechanism weaken
alignment of the base LLM, leading to severe safety breaches. Through linear probe analysis of internal hidden states, we demonstrate that role assignment suppresses refusal awareness by shifting representations away from safety boundaries. Besides, multi-step execution distributes harmfulness across individual steps, preventing
alignment mechanisms from being activated throughout the research process. Exploiting these vulnerabilities, we design Intent Hijack (i.e., rephrasing harmful queries as academic research) and Plan Injection (i.e., manipulating execution plans) to further examine the safety risks of DR
agents. Extensive experiments show that our methods achieve near-perfect compliance and elicit detailed, actionable reports that significantly exceed standalone LLM outputs in technical depth and applicability. These results demonstrate alarming misalignment in DR
agents and underscore the urgent need for tailored
alignment techniques. Our code is available in https://schen.app/deeper-harm.
Read the full abstract ▾
All authors Shuo Chen, Zonggen Li, Xingyu Jin, Zhen Han, Bailan He, Tong Liu, Haokun Chen, Georg Groh, Philip Torr, Volker Tresp, Jindong Gu
Computation and Language · Cross-listed from cs.LG
Balancing Synthetic Data and Replay for Enhancing Task-Specific Capabilities
Urs Spiegelhalter, Jörg K.H. Franke and Frank Hutter
Adapting
language models to new tasks through continued pretraining faces a fundamental trade-off: models must learn new capabilities while avoiding catastrophic forgetting of existing knowledge. While prior work has studied synthetic data generation techniques, the optimal replay ratios for balancing task performance and knowledge retention under computational constraints remain poorly understood. We present a comprehensive empirical study investigating the interplay between replay ratio configuration and computational budget when adapting
language models to new tasks. Using the bAbI
reasoning tasks as our target objective, we apply synthetic data generation and systematically evaluate different total
token budgets and replay ratio configurations. We analyze their effects on both task mastery and general knowledge retention. Our experiments reveal an optimal configuration that balances task-specific performance with general knowledge retention. Based on our findings, we provide empirically-grounded guidelines for selecting replay ratios based on computational budget, enabling practitioners to achieve strong task adaptation with significantly reduced training costs.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Data or Language Supervision: What Makes CLIP Better than DINO?
Yiming Liu, Yuhui Zhang, Dhruba Ghosh et al. (+2 more)
CLIP outperforms self-supervised models like DINO as
vision encoders for
vision-
language models (VLMs), but it remains unclear whether this advantage stems from CLIP's
language supervision or its much larger training data. To disentangle these factors, we pre-train CLIP and DINO under controlled settings – using the same architecture,
dataset, and training configuration – achieving similar ImageNet accuracy.
Embedding analysis shows that CLIP captures high-level semantics (e.g., object categories, text), while DINO is more responsive to low-level features like colors and styles. When integrated into VLMs and evaluated on 20 VQA
benchmarks, CLIP excels at text-intensive tasks, while DINO slightly outperforms on
vision-centric ones. Variants of
language supervision (e.g., sigmoid loss, pre-trained
language encoders) yield limited gains. Our findings provide scientific insights into
vision encoder design and its impact on VLM performance.
Read the full abstract ▾
All authors Yiming Liu, Yuhui Zhang, Dhruba Ghosh, Ludwig Schmidt, Serena Yeung-Levy
Computation and Language · Cross-listed from cs.LG
Don't Walk the Line: Boundary Guidance for Filtered Generation
Sarah Ball and Andreas Haupt
Generative models are increasingly paired with safety classifiers that filter harmful or undesirable outputs. A common strategy is to fine-tune the generator to reduce the probability of being filtered, but this can be suboptimal: it often pushes the model toward producing samples near the classifier's decision boundary, increasing both false positives and false negatives. We propose Boundary Guidance, a reinforcement learning fine-tuning method that explicitly steers generation away from the classifier's margin. On a
benchmark of jailbreak, ambiguous, and longcontext
prompts, Boundary Guidance improves both the safety and the utility of outputs, as judged by LLM-as-a-Judge evaluations. Comprehensive ablations across model scales and reward designs demonstrate the robustness of our approach.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
Task-Aware Reduction for Scalable LLM-Database Systems
Marcus Emmanuel Barnes, Taher A. Ghaleb and Safwat Hassan
Large
Language Models (LLMs) are increasingly applied to data-intensive workflows, from database querying to developer observability. Yet the effectiveness of these systems is constrained by the volume, verbosity, and noise of real-world text-rich data such as logs, telemetry, and monitoring streams. Feeding such data directly into LLMs is costly, environmentally unsustainable, and often misaligned with task objectives. Parallel efforts in LLM efficiency have focused on model- or architecture-level optimizations, but the challenge of reducing upstream input verbosity remains underexplored. In this paper, we argue for treating the
token budget of an LLM as an
attention budget and elevating task-aware text reduction as a first-class design principle for
language – data systems. We position input-side reduction not as compression, but as
attention allocation: prioritizing information most relevant to downstream tasks. We outline open research challenges for building
benchmarks, designing adaptive reduction pipelines, and integrating
token-budget–aware preprocessing into database and
retrieval systems. Our
vision is to channel scarce
attention resources toward meaningful signals in noisy, data-intensive workflows, enabling scalable, accurate, and sustainable LLM–data integration.
Read the full abstract ▾
Computation and Language
PHANTOM RECALL: When Familiar Puzzles Fool Smart Models
Souradeep Mukhopadhyay, Rishabh Baral, Nimeesh Mahajan et al. (+5 more)
Large
language models (LLMs) such as GPT, Gemini, and Claude often appear adept at solving classic logic puzzles–but how much genuine
reasoning underlies their answers? Recent evidence suggests that these models frequently rely on memorized templates rather than
reasoning from first principles. When puzzles are slightly modified, their performance collapses, revealing a striking fragility. In particular, we asked: Have LLMs addressed these issues? To what extent? How about perturbations to other puzzles? Is there a general way of reformulating the
prompt so that the models do better? To examine these things systematically, we introduce PHANTOM RECALL, a
benchmark comprising 25 well-known logic puzzles and 149 carefully designed perturbations that preserve
reasoning structure but alter superficial details and solutions. We evaluate eleven leading LLMs and identify a recurring failure mode–phantom recall–where models confidently reproduce memorized solutions or spurious rationales that no longer fit the altered scenario. To probe and mitigate this issue, we contribute three tools: (i) an automated logical-equivalence judge to detect
reasoning mismatches, (ii) a taxonomy of fine-grained
reasoning error categories, and (iii) a prompting-based mitigation framework guided by these categories. Despite near-perfect accuracy on unmodified puzzles, models significantly underperform humans on perturbed ones, exhibiting both phantom recall and over-elaboration. Our findings reveal a crucial limitation: LLMs often fail to re-reason when contextual cues shift–highlighting the gap between linguistic fluency and logical understanding.
Read the full abstract ▾
All authors Souradeep Mukhopadhyay, Rishabh Baral, Nimeesh Mahajan, Samhitha Harish, Aswin RRV, Mihir Parmar, Mutsumi Nakamura, Chitta Baral
Computation and Language
Are Large Reasoning Models Interruptible?
Tsung-Han Wu, Mihran Miroyan, David M. Chan et al. (+3 more)
Real-world applications of Large
Reasoning Models (LRMs) often require
reasoning about changing
prompts or environments. In this work, we challenge the frozen world assumption and evaluate LRM robustness under two realistic dynamic scenarios: interruptions, which test the accuracy of model responses under budget-constrained outputs, and dynamic context, which tests model adaptation to in-flight changes. Across mathematics and programming
benchmarks that require long-form
reasoning, static evaluations consistently overestimate robustness: even state-of-the-art LRMs, which achieve high accuracy in static settings, can fail unpredictably when interrupted or exposed to changing context, with performance dropping by up to 60% when updates are introduced late in the
reasoning process. Our analysis further reveals several novel failure modes, including
reasoning leakage, where models fold the
reasoning into their final answer when interrupted; panic, where under time pressure models abandon
reasoning entirely and return incorrect answers; and self-doubt, where performance degrades when trying to incorporate updated information. Project Page: http://dynamic-lm.github.io/
Read the full abstract ▾
All authors Tsung-Han Wu, Mihran Miroyan, David M. Chan, Trevor Darrell, Narges Norouzi, Joseph E. Gonzalez
Computation and Language
Demystifying Reinforcement Learning in Agentic Reasoning
Zhaochen Yu, Ling Yang, Jiaru Zou et al. (+2 more)
Recently, the emergence of agentic RL has showcased that RL could also effectively improve the agentic
reasoning ability of LLMs, yet the key design principles and optimal practices remain unclear. In this work, we conduct a comprehensive and systematic investigation to demystify reinforcement learning in agentic
reasoning from three key perspectives: data,
algorithm, and
reasoning mode. We highlight our key insights: (i) Replacing stitched synthetic trajectories with real end-to-end tool-use trajectories yields a far stronger SFT initialization; high-diversity, model-aware
datasets sustain exploration and markedly improve RL performance. (ii) Exploration-friendly techniques are crucial for agentic RL, such as clip higher, overlong reward shaping, and maintaining adequate policy
entropy could improve the training efficiency. (iii) A deliberative strategy with fewer tool calls outperforms frequent tool calls or verbose self-
reasoning, improving tool efficiency and final accuracy. Together, these simple practices consistently enhance agentic
reasoning and training efficiency, achieving strong results on challenging
benchmarks with smaller models, and establishing a practical baseline for future agentic RL research. Beyond these empirical insights, we further contribute a high-quality, real end-to-end agentic SFT
dataset along with a high-quality RL
dataset, and demonstrate the effectiveness of our insights in boosting the agentic
reasoning ability of LLMs across four challenging
benchmarks, including AIME2024/AIME2025, GPQA-Diamond, and LiveCodeBench-v6. With our recipes, 4B-sized models could also achieve superior agentic
reasoning performance compared to 32B-sized models. Code and models: https://github.com/Gen-Verse/Open-AgentRL
Read the full abstract ▾
All authors Zhaochen Yu, Ling Yang, Jiaru Zou, Shuicheng Yan, Mengdi Wang
Computation and Language · Cross-listed from cs.LG
QeRL: Beyond Efficiency – Quantization-enhanced Reinforcement Learning for LLMs
Wei Huang, Yi Ge, Shuai Yang et al. (+11 more)
We propose QeRL, a Quantization-enhanced Reinforcement Learning framework for large
language models (LLMs). While RL is essential for LLMs'
reasoning capabilities, it is resource-intensive, requiring substantial GPU
memory and long rollout durations. QeRL addresses these issues by combining NVFP4 quantization with Low-Rank Adaptation (LoRA), accelerating rollout phase of RL while reducing
memory overhead. Beyond efficiency, our findings show that quantization noise increases policy
entropy, enhancing exploration, and enabling the discovery of better strategies during RL. To further optimize exploration, QeRL introduces an Adaptive Quantization Noise (AQN) mechanism, which dynamically adjusts noise during training. Experiments demonstrate that QeRL delivers over 1.5 times speedup in the rollout phase. Moreover, this is the first framework to enable RL training of a 32B LLM on a single H100 80GB GPU, while delivering overall speedups for RL training. It also achieves faster reward growth and higher final accuracy than 16-bit LoRA and QLoRA, while matching the performance of full-parameter fine-tuning on mathematical
benchmarks such as GSM8K (90.8%) and MATH 500 (77.4%) in the 7B model. These results establish QeRL as an efficient and effective framework for RL training in LLMs.
Read the full abstract ▾
All authors Wei Huang, Yi Ge, Shuai Yang, Yicheng Xiao, Huizi Mao, Yujun Lin, Hanrong Ye, Sifei Liu, Ka Chun Cheung, Hongxu Yin, Yao Lu, Xiaojuan Qi, Song Han, Yukang Chen
Computation and Language
Lingfei Qian, Xueqing Peng, Yan Wang et al. (+14 more)
Although Large
Language Model (LLM)-based
agents are increasingly used in financial trading, it remains unclear whether they can reason and adapt in live
markets, as most studies test models instead of
agents, cover limited periods and assets, and rely on unverified data. To address these gaps, we introduce
Agent Market Arena (AMA), the first lifelong, real-time
benchmark for evaluating LLM-based trading
agents across multiple
markets. AMA integrates verified trading data, expert-checked news, and diverse
agent architectures within a unified trading framework, enabling fair and continuous comparison under real conditions. It implements four
agents, including InvestorAgent as a single-
agent baseline, TradeAgent and HedgeFundAgent with different risk styles, and DeepFundAgent with
memory-based
reasoning, and evaluates them across GPT-4o, GPT-4.1, Claude-3.5-haiku, Claude-sonnet-4, and Gemini-2.0-flash. Live experiments on both cryptocurrency and stock
markets demonstrate that
agent frameworks display markedly distinct behavioral patterns, spanning from aggressive risk-taking to conservative decision-making, whereas model backbones contribute less to outcome variation. AMA thus establishes a foundation for rigorous, reproducible, and continuously evolving evaluation of financial
reasoning and trading intelligence in LLM-based
agents.
Read the full abstract ▾
All authors Lingfei Qian, Xueqing Peng, Yan Wang, Vincent Jim Zhang, Huan He, Hanley Smith, Yi Han, Yueru He, Haohang Li, Yupeng Cao, Yangyang Yu, Alejandro Lopez-Lira, Peng Lu, Jian-Yun Nie, Guojun Xiong, Jimin Huang, Sophia Ananiadou
Computation and Language
Scaling Language-Centric Omnimodal Representation Learning
Chenghao Xiao, Hou Pong Chan, Hao Zhang et al. (+3 more)
Recent
multimodal embedding approaches leveraging
multimodal large
language models (MLLMs) fine-tuned with contrastive learning (CL) have shown promising results, yet the underlying reasons behind their superiority remain underexplored. This work argues that a crucial advantage of MLLM-based approaches stems from implicit cross-modal
alignment achieved during generative pretraining, where the
language decoder learns to exploit
multimodal signals within a shared representation space for generating unimodal outputs. Through analysis of anisotropy and kernel similarity structure, we empirically confirm that latent
alignment emerges within MLLM representations, allowing CL to serve as a lightweight refinement stage. Leveraging this insight, we propose a
Language-Centric Omnimodal
Embedding framework, termed LCO-Emb. Extensive experiments across diverse backbones and
benchmarks demonstrate its effectiveness, achieving state-of-the-art performance across modalities. Furthermore, we identify a Generation-Representation Scaling Law (GRSL), showing that the representational capabilities gained through contrastive refinement scales positively with the MLLM's generative capabilities. This suggests that improving generative abilities evolves as an effective paradigm for enhancing representation quality. We provide a theoretical explanation of GRSL, which formally links the MLLM's generative quality to the upper bound on its representation performance, and validate it on a challenging, low-resource visual-document
retrieval task, showing that continual generative pretraining before CL can further enhance the potential of a model's
embedding capabilities. Codes, models, and resources are available at https://github.com/LCO-
Embedding/LCO-
Embedding.
Read the full abstract ▾
All authors Chenghao Xiao, Hou Pong Chan, Hao Zhang, Weiwen Xu, Mahani Aljunied, Yu Rong