Computation and Language
Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
Raha Askari, Sina Zarrieß, Özge Alacam et al. (+1 more)
Implicit meanings are integral to human communication, making it essential for
language models to be capable of identifying and interpreting them. Grice (1975) proposed a set of conversational maxims that guide cooperative dialogue, noting that speakers may deliberately violate these principles to express meanings beyond literal words, and that listeners, in turn, recognize such violations to draw pragmatic inferences. Building on Surian et al. (1996)'s study of children's sensitivity to violations of Gricean maxims, we introduce a novel
benchmark to test whether
language models pretrained on less than 10M and less than 100M
tokens can distinguish maxim-adhering from maxim-violating utterances. We compare these BabyLMs across five maxims and situate their performance relative to children and a Large
Language Model (LLM) pretrained on 3T
tokens. We find that overall, models trained on less than 100M
tokens outperform those trained on less than 10M, yet fall short of child-level and LLM competence. Our results suggest that modest data increases improve some aspects of pragmatic behavior, leading to finer-grained differentiation between pragmatic dimensions.
Read the full abstract ▾
All authors Raha Askari, Sina Zarrieß, Özge Alacam, Judith Sieker
Computation and Language
ModernBERT + ColBERT: Enhancing biomedical RAG through an advanced re-ranking retriever
Eduardo Martínez Rivera and Filippo Menolascina
Retrieval-Augmented Generation (RAG) is a powerful technique for enriching Large
Language Models (LLMs) with external knowledge, allowing for factually grounded responses, a critical requirement in high-stakes domains such as healthcare. However, the efficacy of RAG systems is fundamentally restricted by the performance of their
retrieval module, since irrelevant or semantically misaligned documents directly compromise the accuracy of the final generated response. General-purpose dense retrievers can struggle with the nuanced
language of specialised domains, while the high accuracy of in-domain models is often achieved at prohibitive computational costs. In this work, we aim to address this trade-off by developing and evaluating a two-stage
retrieval architecture that combines a lightweight ModernBERT bidirectional encoder for efficient initial candidate
retrieval with a ColBERTv2 late-interaction model for fine-grained re-ranking. We conduct comprehensive evaluations of our retriever module performance and RAG system performance in the biomedical context, fine-tuning the IR module using 10k question-passage pairs from PubMedQA. Our analysis of the retriever module confirmed the positive impact of the ColBERT re-ranker, which improved Recall@3 by up to 4.2 percentage points compared to its retrieve-only counterpart. When integrated into the biomedical RAG, our IR module leads to a state-of-the-art average accuracy of 0.4448 on the five tasks of the MIRAGE question-answering
benchmark, outperforming strong baselines such as MedCPT (0.4436). Our ablation studies reveal that this performance is critically dependent on a joint fine-tuning process that aligns the retriever and re-ranker; otherwise, the re-ranker might degrade the performance.
Read the full abstract ▾
Computation and Language
A Low-Resource Speech-Driven NLP Pipeline for Sinhala Dyslexia Assistance
Peshala Perera and Deshan Sumanathilaka
Dyslexia in adults remains an under-researched and under-served area, particularly in non-English-speaking contexts, despite its significant impact on personal and professional lives. This work addresses that gap by focusing on Sinhala, a low-resource
language with limited tools for linguistic accessibility. We present an assistive system explicitly designed for Sinhala-speaking adults with dyslexia. The system integrates Whisper for
speech-to-text conversion, SinBERT, an open-sourced fine-tuned BERT model trained for Sinhala to identify common dyslexic errors, and a combined mT5 and Mistral-based model to generate corrected text. Finally, the output is converted back to
speech using gTTS, creating a complete
multimodal feedback loop. Despite the challenges posed by limited Sinhala-
language datasets, the system achieves 0.66 transcription accuracy and 0.7 correction accuracy with 0.65 overall system accuracy. These results demonstrate both the feasibility and effectiveness of the approach. Ultimately, this work highlights the importance of inclusive Natural
Language Processing (NLP) technologies in underrepresented
languages and showcases a practical
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mamba
Baher Mohammad, Magauiya Zhussip and Stamatios Lefkimmiatis
We introduce MAVE (Mamba with Cross-
Attention for Voice Editing and Synthesis), a novel autoregressive architecture for text-conditioned voice editing and high-fidelity text-to-
speech (TTS) synthesis, built on a cross-attentive Mamba backbone. MAVE achieves state-of-the-art performance in
speech editing and very competitive results in zero-shot TTS, while not being explicitly trained on the latter task, outperforming leading autoregressive and
diffusion models on diverse, real-world audio. By integrating Mamba for efficient audio sequence modeling with cross-
attention for precise text-acoustic
alignment, MAVE enables context-aware voice editing with exceptional naturalness and speaker consistency. In pairwise human evaluations on a random 40-sample subset of the RealEdit
benchmark (400 judgments), 57.2% of listeners rated MAVE - edited
speech as perceptually equal to the original, while 24.8% prefered the original and 18.0% MAVE - demonstrating that in the majority of cases edits are indistinguishable from the source. MAVE compares favorably with VoiceCraft and FluentSpeech both on pairwise comparisons and standalone mean opinion score (MOS) evaluations. For zero-shot TTS, MAVE exceeds VoiceCraft in both speaker similarity and naturalness, without requiring multiple inference runs or post-processing. Remarkably, these quality gains come with a significantly lower
memory cost and approximately the same latency: MAVE requires ~6x less
memory than VoiceCraft during inference on utterances from the RealEdit database (mean duration: 6.21s, A100, FP16, batch size 1). Our results demonstrate that MAVE establishes a new standard for flexible, high-fidelity voice editing and synthesis through the synergistic integration of structured state-space modeling and cross-modal
attention.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs
Ivo Petrov, Jasper Dekoninck and Martin Vechev
Large
language models (LLMs) have recently shown strong performance on mathematical
benchmarks. At the same time, they are prone to
hallucination and sycophancy, often providing convincing but flawed proofs for incorrect mathematical statements provided by users. This significantly limits the applicability of LLMs in
theorem proving, as verification of these flawed proofs must be done manually by expert mathematicians. However, existing
benchmarks that measure sycophancy in mathematics are limited: they focus solely on final-answer problems, rely on very simple and often contaminated
datasets, and construct
benchmark samples using synthetic modifications that create ill-posed questions rather than well-posed questions that are demonstrably false. To address these issues, we introduce BrokenMath, the first
benchmark for evaluating sycophantic behavior in LLMs within the context of natural
language theorem proving. BrokenMath is built from advanced 2025 competition problems, which are perturbed with an LLM to produce false statements and subsequently refined through expert review. Using an LLM-as-a-judge framework, we evaluate state-of-the-art LLMs and agentic systems and find that sycophancy is widespread, with the best model, GPT-5, producing sycophantic answers 29% of the time. We further investigate several mitigation strategies, including test-time interventions and supervised fine-tuning on curated sycophantic examples. These approaches substantially reduce, but do not eliminate, sycophantic behavior.
Read the full abstract ▾
Computation and Language
JSON Whisperer: Efficient JSON Editing with LLMs
Sarel Duanis, Asnat Greenstein-Messica and Eliya Habba
Large
language models (LLMs) can modify JSON documents through natural
language commands, but current approaches regenerate entire structures for each edit, resulting in computational inefficiency. We present JSON Whisperer, a framework that enables LLMs to generate RFC 6902 diff patches-expressing only the necessary modifications-rather than complete documents. We identify two key challenges in patch-based editing: (1) LLMs often miss related updates when generating isolated patches, and (2) array manipulations require tracking index shifts across operations, which LLMs handle poorly. To address these issues, we introduce EASE (Explicitly Addressed Sequence Encoding), which transforms arrays into dictionaries with stable keys, eliminating index arithmetic complexities. Our evaluation shows that patch generation with EASE reduces
token usage by 31% while maintaining edit quality within 5% of full regeneration with particular gains for complex instructions and list manipulations. The
dataset is available at: https://github.com/emnlp2025/JSON-Whisperer/
Read the full abstract ▾
Computation and Language · Cross-listed from cond-mat.mtrl-sci
AtomWorld: A Benchmark for Evaluating Spatial Reasoning in Large Language Models on Crystalline Materials
Taoyuze Lv, Alexander Chen, Fengyu Xie et al. (+7 more)
Large
language models (LLMs) have shown promising potential in scientific research, enabling tasks ranging from knowledge
retrieval to property prediction. Existing science
benchmarks mainly focus on perceptual or knowledge-based tasks, largely ignoring the modelling tasks, a fundamental starting point for any real scientific research. For materials science, constructing and manipulating atomic structures is one of the most creative and least automated steps. In this work, we introduce AtomWorld, a
benchmark designed to evaluate the abilities of LLMs on structure modifications. The
benchmark includes ten fundamental actions under four widely used modelling categories, enabling verifiable evaluation metrics. We find that Claude Opus 4.6 generally performs the best. While the success rate decreases markedly with increasing modelling complexity, with particularly low success rates (below 12% for rotation) for operations involving complex spatial relations. Our results suggest that contemporary LLMs are better suited as copilots for materials structure modelling rather than fully unsupervised autonomous scientific
agents. Beyond evaluation, AtomWorld also serves as a testbed and playground for developing future structure-aware models, including reinforcement learning and agentic approaches.
Read the full abstract ▾
All authors Taoyuze Lv, Alexander Chen, Fengyu Xie, Chu Wu, Jeffrey Meng, Dongzhan Zhou, Yingheng Wang, Bram Hoex, Zhicheng Zhong, Tong Xie
Computation and Language
Multilingual Routing in Mixture-of-Experts
Lucas Bandarkar, Chenyuan Yang, Mohsen Fayyaz et al. (+2 more)
Mixture-of-Experts (MoE) architectures have become the key to scaling modern LLMs, yet little is understood about how their sparse routing dynamics respond to
multilingual data. In this work, we analyze expert routing patterns using parallel
multilingual datasets and present highly interpretable layer-wise phenomena. We find that MoE models route
tokens in
language-specific ways in the early and late decoder layers but exhibit significant cross-lingual routing
alignment in middle layers, mirroring parameter-sharing trends observed in dense LLMs. In particular, we reveal a clear, strong correlation between a model's performance in a given
language and how similarly its
tokens are routed to English in these layers. Extending beyond correlation, we explore inference-time interventions that induce higher cross-lingual routing
alignment. We introduce a method that steers the router by promoting middle-layer task experts frequently activated in English, and it successfully increases
multilingual performance. These 1-2% gains are remarkably consistent across two evaluation tasks, three models, and 15+
languages, especially given that these simple interventions override routers of extensively trained, state-of-the-art LLMs. In comparison, interventions outside of the middle layers or targeting
multilingual-specialized experts only yield performance degradation. Altogether, we present numerous findings that explain how MoEs process non-English text and demonstrate that generalization is limited by the model's ability to leverage
language-universal experts in all
languages.
Read the full abstract ▾
All authors Lucas Bandarkar, Chenyuan Yang, Mohsen Fayyaz, Junlin Hu, Nanyun Peng
Computation and Language
TiTok: Transfer Token-level Knowledge via Contrastive Excess to Transplant LoRA
Chanjoo Jung and Jaehyung Kim
Large
Language Models (LLMs) are widely applied in real world scenarios, yet fine-tuning them comes with significant computational and storage costs. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA mitigate these costs; however, the adapted parameters are dependent on the base model and cannot be transferred across different backbones. One way to address this issue is through knowledge
distillation, but its effectiveness inherently depends on training data. Recent work such as TransLoRA avoids this by generating synthetic data; nevertheless, this adds complexity since it requires training an additional discriminator model. In this paper, we propose TiTok, a new framework that enables effective LoRA Transplantation through
Token-level knowledge transfer. Specifically, TiTok captures task-relevant information through a
token-wise contrastive excess between a source model with and without LoRA. This excess highlights informative
tokens and enables selective filtering of synthetic data, all without additional models or overhead. Through experiments on three
benchmarks across multiple transfer settings, we demonstrate that TiTok is consistently effective, achieving average performance gains of +4~10% compared to baselines overall.
Read the full abstract ▾
Computation and Language
Multi-Agent Tool-Integrated Policy Optimization
Zhanfeng Mo, Xingxuan Li, Yuntao Chen et al. (+1 more)
Large
language models (LLMs) increasingly rely on multi-turn tool-integrated planning for knowledge-intensive and complex
reasoning tasks. Existing implementations typically rely on a single
agent, but they suffer from limited context length and noisy tool responses. A natural solution is to adopt a multi-
agent framework with planner- and worker-
agents to manage context. However, no existing methods support effective reinforcement learning post-training of tool-integrated multi-
agent frameworks. To address this gap, we propose Multi-
Agent Tool-Integrated Policy Optimization (MATPO), which enables distinct roles (planner and worker) to be trained within a single LLM instance using role-specific
prompts via reinforcement learning. MATPO is derived from a principled credit assignment mechanism across planner and worker rollouts. This design eliminates the need to deploy multiple LLMs, which would be
memory-intensive, while preserving the benefits of specialization. Experiments on
GAIA-text, WebWalkerQA, and FRAMES show that MATPO consistently outperforms single-
agent baselines by an average of 18.38% relative improvement in performance and exhibits greater robustness to noisy tool outputs. Our findings highlight the effectiveness of unifying multiple
agent roles within a single LLM and provide practical insights for stable and efficient multi-
agent RL training.
Read the full abstract ▾
All authors Zhanfeng Mo, Xingxuan Li, Yuntao Chen, Lidong Bing
Computation and Language
FocusMed: A Large Language Model-based Framework for Enhancing Medical Question Summarization with Focus Identification
Chao Liu, Ling Luo, Tengxiao Lv et al. (+4 more)
With the rapid development of online medical platforms, consumer health questions (CHQs) are inefficient in diagnosis due to redundant information and frequent non-professional terms. The medical question summary (MQS) task aims to transform CHQs into streamlined doctors' frequently asked questions (FAQs), but existing methods still face challenges such as poor identification of question focus and model
hallucination. This paper explores the potential of large
language models (LLMs) in the MQS task and finds that direct fine-tuning is prone to focus identification bias and generates unfaithful content. To this end, we propose an optimization framework based on core focus guidance. First, a
prompt template is designed to drive the LLMs to extract the core focus from the CHQs that is faithful to the original text. Then, a fine-tuning
dataset is constructed in combination with the original CHQ-FAQ pairs to improve the ability to identify the focus of the question. Finally, a multi-dimensional quality evaluation and selection mechanism is proposed to comprehensively improve the quality of the summary from multiple dimensions. We conduct comprehensive experiments on two widely-adopted MQS
datasets using three established evaluation metrics. The proposed framework achieves state-of-the-art performance across all measures, demonstrating a significant boost in the model's ability to identify critical focus of questions and a notable mitigation of
hallucinations. The source codes are freely available at https://github.com/DUT-LiuChao/FocusMed.
Read the full abstract ▾
All authors Chao Liu, Ling Luo, Tengxiao Lv, Huan Zhuang, Lejing Yu, Jian Wang, Hongfei Lin
Computation and Language
FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Method
Yuheng Li, Jiechao Gao, Wei Han et al. (+3 more)
Knowledge of the medical decision process, which can be modeled as medical decision trees (MDTs), is critical to building clinical decision support systems. However, current MDT construction methods rely heavily on time-consuming and laborious manual annotation. To address this challenge, we propose PI-LoRA (Path-Integrated LoRA), a novel low-rank adaptation method for automatically extracting MDTs from clinical guidelines and textbooks. We integrate gradient path information to capture synergistic effects between different modules, enabling more effective and reliable rank allocation. This framework ensures that the most critical modules receive appropriate rank allocations while less important ones are pruned, resulting in a more efficient and accurate model for extracting medical decision trees from clinical texts. Extensive experiments on medical guideline
datasets demonstrate that our PI-LoRA method significantly outperforms existing parameter-efficient fine-tuning approaches for the Text2MDT task, achieving better accuracy with substantially reduced model complexity. The proposed method achieves state-of-the-art results while maintaining a lightweight architecture, making it particularly suitable for clinical decision support systems where computational resources may be limited.
Read the full abstract ▾
All authors Yuheng Li, Jiechao Gao, Wei Han, Wenwen Ouyang, Wei Zhu, Hui Yi Leong
Computation and Language
Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study
Ayan Majumdar, Feihao Chen, Jinghui Li et al. (+1 more)
Large-scale web-scraped text corpora used to train general-purpose AI models often contain harmful demographic-targeted social biases, creating a regulatory need for data
auditing and developing scalable bias-detection methods. Although prior work has investigated biases in text
datasets and related detection methods, these studies remain narrow in scope. They typically focus on a single content type (e.g., hate
speech), cover limited demographic axes, overlook biases affecting multiple demographics simultaneously, and analyze limited techniques. Consequently, practitioners lack a holistic understanding of the strengths and limitations of recent large
language models (LLMs) for automated bias detection. In this study, we conduct a comprehensive
benchmark study on English texts to assess the ability of LLMs in detecting demographic-targeted social biases. To align with regulatory requirements, we frame bias detection as a multi-label task of detecting targeted identities using a demographic-focused taxonomy. We then systematically evaluate models across scales and techniques, including prompting, in-context learning, and fine-tuning. Using twelve
datasets spanning diverse content types and demographics, our study demonstrates the promise of fine-tuned smaller models for scalable detection. However, our analyses also expose persistent gaps across demographic axes and multi-demographic targeted biases, underscoring the need for more effective and scalable detection frameworks.
Read the full abstract ▾
All authors Ayan Majumdar, Feihao Chen, Jinghui Li, Xiaozhen Wang
Computation and Language
Contrastive Learning Using Graph Embeddings for Domain Adaptation of Language Models in the Process Industry
Anastasia Zhukova, Jonas Lührs, Christian E. Lobmüller et al. (+1 more)
Recent trends in NLP utilize knowledge graphs (KGs) to enhance pretrained
language models by incorporating additional knowledge from the graph structures to learn domain-specific terminology or relationships between documents that might otherwise be overlooked. This paper explores how SciNCL, a graph-aware neighborhood contrastive learning methodology originally designed for scientific publications, can be applied to the process industry domain, where text logs contain crucial information about daily operations and are often structured as sparse KGs. Our experiments demonstrate that
language models fine-tuned with triplets derived from graph
embeddings (GE) outperform a state-of-the-art mE5-large text encoder by 9.8-14.3% (5.45-7.96p) on the proprietary process industry text
embedding benchmark (PITEB) while having 3 times fewer parameters.
Read the full abstract ▾
All authors Anastasia Zhukova, Jonas Lührs, Christian E. Lobmüller, Bela Gipp
Computation and Language · Cross-listed from cs.LG
Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
Qizheng Zhang, Changran Hu, Shubhangi Upasani et al. (+10 more)
Large
language model (LLM) applications such as
agents and domain-specific
reasoning increasingly rely on context adaptation: modifying inputs with instructions, strategies, or evidence, rather than weight updates. Prior approaches improve usability but often suffer from brevity bias, which drops domain insights for concise summaries, and from context collapse, where iterative rewriting erodes details over time. We introduce ACE (Agentic Context Engineering), a framework that treats contexts as evolving playbooks that accumulate, refine, and organize strategies through a modular process of generation, reflection, and curation. ACE prevents collapse with structured, incremental updates that preserve detailed knowledge and scale with long-context models. Across
agent and domain-specific
benchmarks, ACE optimizes contexts both offline (e.g., system
prompts) and online (e.g.,
agent memory), consistently outperforming strong baselines: +10.6% on
agents and +8.6% on finance, while significantly reducing adaptation latency and rollout cost. Notably, ACE could adapt effectively without labeled supervision and instead by leveraging natural execution feedback. On the AppWorld leaderboard, ACE matches the top-ranked production-level
agent on the overall average and surpasses it on the harder test-challenge split, despite using a smaller open-source model. These results show that comprehensive, evolving contexts enable scalable, efficient, and self-improving LLM systems with low overhead.
Read the full abstract ▾
All authors Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, Kunle Olukotun
Computation and Language
FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-Tuning
Guochen Yan, Luyuan Xie, Qingni Shen et al. (+2 more)
The current paradigm of training large
language models (LLMs) on public available Web data is becoming unsustainable as high-quality data sources in specialized domains near exhaustion.
Federated Learning (FL) emerges as a practical solution for the next generation of AI on a decentralized Web, enabling
privacy-preserving collaborative fine-tuning on decentralized private data. While Low-Rank Adaptation (LoRA) is standard for efficient fine-tuning, its
federated application faces a critical bottleneck: communication overhead under heterogeneous network conditions. Structural redundancy in LoRA parameters increases communication costs and causes aggregation conflicts. To address this, we propose FedSRD, a Sparsify-Reconstruct-Decompose framework for communication-efficient
federated LLM fine-tuning. We introduce importance-aware sparsification to reduce the upload parameter count while preserving the structural integrity of LoRA updates. The server aggregates updates in full-rank space to mitigate conflicts, then decomposes the global update into a sparse low-rank format for broadcast, ensuring a symmetrically efficient cycle. We also propose an efficient variant, FedSRD-e, to reduce computational overhead. Experiments on 10
benchmarks show our framework significantly reduces communication costs by up to 90% while improving performance on heterogeneous client data.
Read the full abstract ▾
All authors Guochen Yan, Luyuan Xie, Qingni Shen, Yuejian Fang, Zhonghai Wu
Computation and Language
Robustness assessment of large audio language models in multiple-choice evaluation
Fernando López, Santosh Kesiraju and Jordi Luque
Recent advances in large audio
language models (LALMs) have primarily been assessed using a multiple-choice question answering (MCQA) framework. However, subtle changes, such as shifting the order of choices, result in substantially different results. Existing MCQA frameworks do not account for this variability and report a single accuracy number per
benchmark or category. We dive into the MCQA evaluation framework and conduct a systematic study spanning three
benchmarks (MMAU, MMAR and MMSU) and four models: Audio Flamingo 2, Audio Flamingo 3, Qwen2.5-Omni-7B-Instruct, and Kimi-Audio-7B-Instruct. Our findings indicate that models are sensitive not only to the ordering of choices, but also to the paraphrasing of the question and the choices. Finally, we propose a simpler evaluation protocol and metric that account for subtle variations and provide a more detailed evaluation report of LALMs within the MCQA framework.
Read the full abstract ▾
Computation and Language
Can LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference
Dang Anh, Rick Nouwen and Massimo Poesio
Our goal is to study how LLMs represent and interpret plural reference in ambiguous and unambiguous contexts. We ask the following research questions: (1) Do LLMs exhibit human-like preferences in representing plural reference? (2) Are LLMs able to detect ambiguity in plural anaphoric expressions and identify possible referents? To address these questions, we design a set of experiments, examining pronoun production using next-
token prediction tasks, pronoun interpretation, and ambiguity detection using different prompting strategies. We then assess how comparable LLMs are to humans in formulating and interpreting plural reference. We find that LLMs are sometimes aware of possible referents of ambiguous pronouns. However, they do not always follow human reference when choosing between interpretations, especially when the possible interpretation is not explicitly mentioned. In addition, they struggle to identify ambiguity without direct instruction. Our findings also reveal inconsistencies in the results across different types of experiments.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang et al. (+4 more)
Large
Language Models (LLMs) demonstrate their
reasoning ability through chain-of-thought (CoT) generation. However, LLM's autoregressive decoding may limit the ability to revisit and refine earlier
tokens in a holistic manner, which can also lead to inefficient exploration for diverse solutions. In this paper, we propose LaDiR} (Latent
Diffusion Reasoner), a novel
reasoning framework that unifies the expressiveness of continuous latent representation with the iterative refinement capabilities of latent
diffusion models for an existing LLM. We first construct a structured latent
reasoning space using a Variational Autoencoder (VAE) that encodes text
reasoning steps into blocks of thought
tokens, preserving semantic information and interpretability while offering compact but expressive representations. Subsequently, we utilize a latent
diffusion model that learns to denoise a block of latent thought
tokens with a blockwise bidirectional
attention mask, enabling longer horizon and iterative refinement with adaptive test-time compute. This design, combined with explicit diversity guidance during
diffusion inference, enables the generation of multiple diverse
reasoning trajectories that explore distinct regions of the latent space, rather than producing repetitive solutions as often occurs in standard autoregressive sampling. We conduct evaluations on a suite of mathematical
reasoning, code generation and puzzle planning
benchmarks. Empirical results show that LaDiR consistently improves accuracy, diversity, and interpretability over existing autoregressive,
diffusion-based, and latent
reasoning methods, revealing a new paradigm for text
reasoning with latent
diffusion.
Read the full abstract ▾
All authors Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Nicklas Majamaki, Navdeep Jaitly, Yi-An Ma, Lianhui Qin
Computation and Language
Fine-grained auxiliary learning for real-world product recommendation
Mario Almagro, Diego Ortego and David Jimenez
Product recommendation is the task of recovering the closest items to a given query within a large product corpora. Generally, one can determine if top-ranked products are related to the query by applying a similarity threshold; exceeding it deems the product relevant, otherwise manual revision is required. Despite being a well-known problem, the integration of these models in real-world systems is often overlooked. In particular, production systems have strong coverage requirements, i.e., a high proportion of recommendations must be automated. In this paper we propose ALC , an Auxiliary Learning strategy that boosts Coverage through learning fine-grained
embeddings. Concretely, we introduce two training objectives that leverage the hardest negatives in the batch to build discriminative training signals between positives and negatives. We validate ALC using three extreme multi-label classification approaches in two product recommendation
datasets; LF-AmazonTitles-131K and Tech and Durables (proprietary), demonstrating state-of-the-art coverage rates when combined with a recent threshold-consistent margin loss.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models
Xurui Song, Shuo Huai, JingJing Jiang et al. (+2 more)
Vision-
Language Model (VLM) driving
agents promise explainable end-to-end autonomy by first producing natural-
language reasoning and then predicting trajectory planning. However, whether planning is causally driven by this
reasoning remains a critical but unverified assumption. To investigate this, we build DriveMind, a large-scale driving Visual Question Answering corpus with plan-aligned Chain-of-Thought (CoT), automatically generated from nuPlan. Our data generation process converts sensors and annotations into structured inputs and, crucially, separates priors from to-be-reasoned signals, enabling clean information ablations. Using DriveMind, we train representative VLM
agents with Supervised Fine-Tuning and Group Relative Policy Optimization and evaluate them with nuPlan's metrics. Our results, unfortunately, indicate a consistent causal disconnect in
reasoning-planning: removing ego/
navigation priors causes large drops in planning scores, whereas removing CoT produces only minor changes.
Attention analysis further shows that planning primarily focuses on priors rather than the CoT. Based on this evidence, we propose the
Reasoning-Planning Decoupling Hypothesis, positing that the training-yielded
reasoning is an ancillary byproduct rather than a causal mediator. To enable efficient diagnosis, we introduce a novel, training-free probe that measures an
agent's reliance on priors by evaluating its planning robustness against minor input perturbations. In summary, we provide the community with a new
dataset and a diagnostic tool to evaluate the causal fidelity of future models.
Read the full abstract ▾
All authors Xurui Song, Shuo Huai, JingJing Jiang, Jiayi Kong, Jun Luo
Computation and Language · Cross-listed from cs.AI
ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering
Rachneet Kaur, Nishan Srishankar, Zhen Zeng et al. (+2 more)
Recent
multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts-those requiring precise visual interpretation rather than relying on textual shortcuts. To address this, we introduce ChartAgent, a novel agentic framework that explicitly performs visual
reasoning directly within the chart's spatial domain. Unlike textual chain-of-thought
reasoning, ChartAgent iteratively decomposes queries into visual subtasks and actively manipulates and interacts with chart images through specialized actions such as drawing annotations, cropping regions (e.g., segmenting pie slices, isolating bars), and localizing axes, using a library of chart-specific
vision tools to fulfill each subtask. This iterative
reasoning process closely mirrors human cognitive strategies for chart comprehension. ChartAgent achieves state-of-the-art accuracy on the ChartBench and ChartX
benchmarks, surpassing prior methods by up to 16.07% absolute gain overall and 17.31% on unannotated, numerically intensive queries. Furthermore, our analyses show that ChartAgent is (a) effective across diverse chart types, (b) achieves the highest scores across varying visual and
reasoning complexity levels, and (c) serves as a plug-and-play framework that boosts performance across diverse underlying LLMs. Our work is among the first to demonstrate visually grounded
reasoning for chart understanding using tool-augmented
multimodal agents.
Read the full abstract ▾
All authors Rachneet Kaur, Nishan Srishankar, Zhen Zeng, Sumitra Ganesh, Manuela Veloso
Computation and Language
GRACE: Generative Representation Learning via Contrastive Policy Optimization
Jiashuo Sun, Shixuan Liu, Zhaochen Su et al. (+6 more)
Prevailing methods for training Large
Language Models (LLMs) as text encoders rely on contrastive losses that treat the model as a black box function, discarding its generative and
reasoning capabilities in favor of static
embeddings. We introduce GRACE (Generative Representation Learning via Contrastive Policy Optimization), a novel framework that reimagines contrastive signals not as losses to be minimized, but as rewards that guide a generative policy. In GRACE, the LLM acts as a policy that produces explicit, human-interpretable rationales–structured natural
language explanations of its semantic understanding. These rationales are then encoded into high-quality
embeddings via mean pooling. Using policy gradient optimization, we train the model with a multi-component reward function that maximizes similarity between query positive pairs and minimizes similarity with negatives. This transforms the LLM from an opaque encoder into an interpretable
agent whose
reasoning process is transparent and inspectable. On MTEB
benchmark, GRACE yields broad cross category gains: averaged over four backbones, the supervised setting improves overall score by 11.5% over base models, and the unsupervised variant adds 6.9%, while preserving general capabilities. This work treats contrastive objectives as rewards over rationales, unifying representation learning with generation to produce stronger
embeddings and transparent rationales. The model, data and code are available at https://github.com/GasolSun36/GRACE.
Read the full abstract ▾
All authors Jiashuo Sun, Shixuan Liu, Zhaochen Su, Xianrui Zhong, Pengcheng Jiang, Bowen Jin, Peiran Li, Weijia Shi, Jiawei Han
Computation and Language · Cross-listed from cs.CR
P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs
Shuai Zhao, Xinyi Wu, Shiqian Zhao et al. (+4 more)
During fine-tuning, large
language models (LLMs) are increasingly vulnerable to data-poisoning
backdoor attacks, which compromise their reliability and trustworthiness. However, existing defense strategies suffer from limited generalization: they only work on specific attack types or task settings. In this study, we propose Poison-to-Poison (P2P), a general and effective
backdoor defense
algorithm. P2P injects benign triggers with safe alternative labels into a subset of training samples and fine-tunes the model on this re-poisoned
dataset by leveraging
prompt-based learning. This enforces the model to associate trigger-induced representations with safe outputs, thereby overriding the effects of original malicious triggers. Thanks to this robust and generalizable trigger-based fine-tuning, P2P is effective across task settings and attack types. Theoretically and empirically, we show that P2P can neutralize malicious
backdoors while preserving task performance. We conduct extensive experiments on classification, mathematical
reasoning, and summary generation tasks, involving multiple state-of-the-art LLMs. The results demonstrate that our P2P
algorithm significantly reduces the attack success rate compared with baseline models. We hope that the P2P can serve as a guideline for defending against
backdoor attacks and foster the development of a secure and trustworthy LLM community.
Read the full abstract ▾
All authors Shuai Zhao, Xinyi Wu, Shiqian Zhao, Xiaobao Wu, Zhongliang Guo, Yanhao Jia, Anh Tuan Luu
Computation and Language
GenQuest: An LLM-based Text Adventure Game for Language Learners
Qiao Wang, Adnan Labib, Robert Swier et al. (+2 more)
GenQuest is a generative text adventure game that leverages Large
Language Models (LLMs) to facilitate second
language learning through immersive, interactive storytelling. The system engages English as a Foreign
Language (EFL) learners in a collaborative "choose-your-own-adventure" style narrative, dynamically generated in response to learner choices. Game mechanics such as branching decision points and story milestones are incorporated to maintain narrative coherence while allowing learner-driven plot development. Key pedagogical features include content generation tailored to each learner's proficiency level, and a vocabulary assistant that provides in-context explanations of learner-queried text strings, ranging from words and phrases to sentences. Findings from a pilot study with university EFL students in China indicate promising vocabulary gains and positive user perceptions. Also discussed are suggestions from participants regarding the narrative length and quality, and the request for multi-modal content such as illustrations.
Read the full abstract ▾
All authors Qiao Wang, Adnan Labib, Robert Swier, Michael Hofmeyr, Zheng Yuan
Computation and Language · Cross-listed from cs.AI
Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents
Muyu He, Anand Kumar, Tsach Mackey et al. (+3 more)
Despite rapid progress in building conversational AI
agents, robustness is still largely untested. Small shifts in user behavior, such as being more impatient, incoherent, or skeptical, can cause sharp drops in
agent performance, revealing how brittle current AI
agents are. Today's
benchmarks fail to capture this fragility:
agents may perform well under standard evaluations but degrade spectacularly in more realistic and varied settings. We address this robustness testing gap by introducing TraitBasis, a lightweight, model-agnostic method for systematically stress testing AI
agents. TraitBasis learns directions in activation space corresponding to steerable user traits (e.g., impatience or incoherence), which can be controlled, scaled, composed, and applied at inference time without any fine-tuning or extra data. Using TraitBasis, we extend $\tau$-Bench to $\tau$-Trait, where user behaviors are altered via controlled trait vectors. We observe on average a 2%-30% performance degradation on $\tau$-Trait across frontier models, highlighting the lack of robustness of current AI
agents to variations in user behavior. Together, these results highlight both the critical role of robustness testing and the promise of TraitBasis as a simple, data-efficient, and compositional tool. By powering simulation-driven stress tests and training loops, TraitBasis opens the door to building AI
agents that remain reliable in the unpredictable dynamics of real-world human interactions. We have open-sourced $\tau$-Trai across four domains: airline, retail, telecom, and telehealth, so the community can systematically QA their
agents under realistic, behaviorally diverse intents and trait scenarios: https://github.com/collinear-ai/tau-trait.
Read the full abstract ▾
All authors Muyu He, Anand Kumar, Tsach Mackey, Meghana Rajeev, James Zou, Nazneen Rajani
Computation and Language
Psychological Steering in LLMs: An Evaluation of Effectiveness and Trustworthiness
Amin Banayeeanzade, Ala N. Tak, Fatemeh Bahrani et al. (+5 more)
The ability to control LLMs' emulated emotional states and personality traits is an essential step in enabling rich, human-centered interactions in socially interactive settings. We introduce PsySET, a Psychologically-informed
benchmark to evaluate LLM Steering Effectiveness and Trustworthiness across the
emotion and personality domains. Our study spans four models from different LLM families paired with various steering strategies, including prompting, fine-tuning, and representation engineering. Our results indicate that prompting is consistently effective but limited in intensity control, whereas vector injections achieve finer controllability while slightly reducing output quality. Moreover, we explore the trustworthiness of steered LLMs by assessing safety, truthfulness,
fairness, and ethics, highlighting potential side effects and behavioral shifts. Notably, we observe idiosyncratic effects; for instance, even a positive
emotion like joy can degrade robustness to
adversarial factuality, lower
privacy awareness, and increase preferential bias. Meanwhile, anger predictably elevates toxicity yet strengthens leakage resistance. Our framework establishes the first holistic evaluation of
emotion and personality steering, offering insights into its interpretability and reliability for socially interactive applications.
Read the full abstract ▾
All authors Amin Banayeeanzade, Ala N. Tak, Fatemeh Bahrani, Anahita Bolourani, Leonardo Blas, Emilio Ferrara, Jonathan Gratch, Sai Praneeth Karimireddy
Computation and Language · Cross-listed from cs.CV
MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
Soo Yong Kim, Suin Cho, Vincent-Daniel Yun et al. (+1 more)
Bridging clinical diagnostic
reasoning with AI remains a central challenge in medical imaging. We introduce MedCLM, an automated pipeline that converts detection
datasets into large-scale medical visual question answering (VQA) data with Chain-of-Thought (CoT)
reasoning by linking lesion boxes to organ
segmentation and structured rationales. These contextual signals enable medical
vision-
language models to generate question-answer pairs with step-by-step
reasoning. To utilize this data effectively, we propose an Integrated CoT-Curriculum Strategy composed of an Easy stage with explicit lesion boxes for visual grounding, a Medium stage that encourages implicit localization, and a Hard stage for weakly supervised
reasoning. Experimental results demonstrate that MedCLM attains state-of-the-art performance on several medical VQA
benchmarks, providing a scalable framework for developing clinically aligned medical
vision-
language models.
Read the full abstract ▾
All authors Soo Yong Kim, Suin Cho, Vincent-Daniel Yun, Gyeongyeon Hwang
Computation and Language
Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space
Tomas Figliolia, Nicholas Alonso, Rishi Iyer et al. (+2 more)
Multi-headed
Attention's (MHA) quadratic compute and linearly growing KV-cache make long-context
transformers expensive to train and serve. Prior works such as Grouped Query
Attention (GQA) and Multi-Latent
Attention (MLA) shrink the cache, speeding decode, but leave compute, which determines prefill and training speed, largely unchanged. We introduce Compressed Convolutional
Attention (CCA), a novel
attention method which down-projects queries, keys, and values and performs the entire
attention operation inside the shared latent space. This simple design dramatically cuts parameters, KV-cache, and FLOPs all at once by the desired compression factor. Because CCA is orthogonal to head-sharing, we combine the two to form Compressed Convolutional Grouped Query
Attention (CCGQA), which further tightens the compute-bandwidth Pareto frontier so that users can tune compression toward either FLOP or
memory limits without sacrificing quality. Experiments show that CCGQA consistently outperforms both GQA and MLA at equal KV-cache compression on dense and MoE models. Additionally, we show that CCGQA outperforms all other
attention methods on MoE models with half the KV-cache of GQA and MLA, achieving an 8x KV-cache compression with no drop in performance compared to standard MHA. CCA and CCGQA also dramatically reduce the FLOP cost of
attention which leads to substantially faster training and prefill than existing methods. On H100 GPUs, our fused CCA/CCGQA kernel reduces prefill latency by about 1.7x at a sequence length of 16k relative to MHA, and accelerates backward by about 1.3x.
Read the full abstract ▾
All authors Tomas Figliolia, Nicholas Alonso, Rishi Iyer, Quentin Anthony, Beren Millidge
Computation and Language
Mitigating Forgetting Between Supervised and Reinforcement Learning Yields Stronger Reasoners
Xiangchi Yuan, Xiang Chen, Tong Yu et al. (+4 more)
Large
Language Models (LLMs) show strong
reasoning abilities, often amplified by Chain-of-Thought (CoT) prompting and reinforcement learning (RL). Although RL
algorithms can substantially improve
reasoning, they struggle to expand
reasoning boundaries because they learn from their own
reasoning trajectories rather than acquiring external knowledge. Supervised fine-tuning (SFT) offers complementary benefits but typically requires large-scale data and risks overfitting. Recent attempts to combine SFT and RL face three main challenges: data inefficiency,
algorithm-specific designs, and catastrophic forgetting. We propose a plug-and-play framework that dynamically integrates SFT into RL by selecting challenging examples for SFT. This approach reduces SFT data requirements and remains agnostic to the choice of RL or SFT
algorithm. To mitigate catastrophic forgetting of RL-acquired skills during SFT, we select high-
entropy tokens for loss calculation and freeze parameters identified as critical for RL. Our method achieves state-of-the-art (SoTA)
reasoning performance using only 1.5% of the SFT data and 20.4% of the RL data used by prior SoTA, providing an efficient and plug-and-play solution for combining SFT and RL in
reasoning post-training.
Read the full abstract ▾
All authors Xiangchi Yuan, Xiang Chen, Tong Yu, Dachuan Shi, Can Jin, Wenke Lee, Saayan Mitra