Computation and Language
$M^2PO$: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation
Hao Wang, Linlong Xu, Heng Liu et al. (+11 more)
Aligning Large
Language Models (LLMs) with human preferences is pivotal for Machine Translation (MT), yet current approaches are often hindered by misleading reward signals. Our analysis reveals that prevailing Quality Estimation (QE) models exhibit a systematic blind spot toward partial errors, specifically partial
hallucinations and omissions, often favoring superficially fluent but unfaithful translations. To address this issue, we propose $M^2PO$ (Multi-Perspective Multi-Pair Preference Optimization), a data-centric framework for preference optimization in machine translation. First, to correct the bias toward fluency, $M^2PO$ uses a dual-perspective mechanism that decouples semantic fidelity from fluency and prioritizes faithfulness through a curriculum strategy. Second, after correcting this bias, partial errors fall between perfect and severely incorrect translations, making them difficult to learn through standard best-versus-worst comparisons. We therefore introduce a multi-pair objective that leverages the full candidate list to capture these fine-grained error signals. Experiments on WMT23, WMT24, and FLORES-200 show that $M^2PO$ enables a 9B model to outperform leading open-source baselines and achieve parity with proprietary models such as GPT-4o and Gemini-2.0-Flash, demonstrating strong potential for efficient and high-fidelity LLM-based translation. Our code and
dataset will be released.
Read the full abstract ▾
All authors Hao Wang, Linlong Xu, Heng Liu, Yangyang Liu, Xiaohu Zhao, Bo Zeng, Liangying Shao, Yichen Dong, Xinwei Wu, Jiang Zhou, Tianyu Dong, xiangxiang Zeng, Longyue Wang, Weihua Luo
Computation and Language
Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gaps
Ahmed Alzubaidi, Shaikha Alsuwaidi, Basma El Amel Boussaha et al. (+5 more)
This survey provides the first systematic review of Arabic LLM
benchmarks, analyzing 40+ evaluation
benchmarks across NLP tasks, knowledge domains, cultural understanding, and specialized capabilities. We propose a taxonomy organizing
benchmarks into four categories: Knowledge, NLP Tasks, Culture and Dialects, and Target-Specific evaluations. Our analysis reveals significant progress in
benchmark diversity while identifying critical gaps: limited temporal evaluation, insufficient multi-turn dialogue assessment, and cultural misalignment in translated
datasets. We examine three primary approaches: native collection, translation, and synthetic generation discussing their trade-offs regarding authenticity, scale, and cost. This work serves as a comprehensive reference for Arabic NLP researchers, providing insights into
benchmark methodologies, reproducibility standards, and evaluation metrics while offering recommendations for future development.
Read the full abstract ▾
All authors Ahmed Alzubaidi, Shaikha Alsuwaidi, Basma El Amel Boussaha, Leen AlQadi, Omar Alkaabi, Mohammed Alyafeai, Hamza Alobeidli, Hakim Hacid
Computation and Language · Cross-listed from cs.AI
Assessing LLM Reasoning Through Implicit Causal Chain Discovery in Climate Discourse
Liesbeth Allein, Nataly Pineda-Castañeda, Andrea Rocci et al. (+1 more)
How does a cause lead to an effect, and which intermediate causal steps explain their connection? This work scrutinizes the mechanistic causal
reasoning capabilities of large
language models (LLMs) to answer these questions through the task of implicit causal chain discovery. In a diagnostic evaluation framework, we instruct nine LLMs to generate all possible intermediate causal steps linking given cause-effect pairs in causal chain structures. These pairs are drawn from recent resources in argumentation studies featuring polarized discussion on
climate change. Our analysis reveals that LLMs vary in the number and granularity of causal steps they produce. Although they are generally self-consistent and confident about the intermediate causal connections in the generated chains, their judgments are mainly driven by associative pattern matching rather than genuine causal
reasoning. Nonetheless, human evaluations confirmed the logical coherence and integrity of the generated chains. Our baseline causal chain discovery approach, insights from our diagnostic evaluation, and
benchmark dataset with causal chains lay a solid foundation for advancing future work in implicit, mechanistic causal
reasoning in argumentation settings.
Read the full abstract ▾
All authors Liesbeth Allein, Nataly Pineda-Castañeda, Andrea Rocci, Marie-Francine Moens
Computation and Language
Investigating Lexical Change through Cross-Linguistic Colexification Patterns
Kim Gfeller, Sabine Stoll, Chundra Cathcart et al. (+1 more)
One of the most intriguing features of
language is its constant change, with ongoing shifts in how meaning is expressed. Despite decades of research, the factors that determine how and why meanings evolve remain only partly understood. Colexification – the phenomenon of expressing multiple distinct concepts using the same word form – serves as a valuable window onto the dynamics of meaning change across
languages. Here, we apply phylogenetic comparative models to dictionary data from three
language families, Austronesian, Indo-European, and Uralic, in order to shed light on the evolutionary dynamics underlying the colexification of concept pairs. We assess the effects of three predictors: associativity, borrowability, and usage frequency. Our results show that more closely related concept pairs are colexified across a larger portion of the family tree and exhibit slower rates of change. In contrast, concept pairs that are more frequent and more prone to borrowing tend to change more rapidly and are less often colexified. We also find considerable differences between the
language families under study, suggesting that areal and cultural factors may play a role.
Read the full abstract ▾
All authors Kim Gfeller, Sabine Stoll, Chundra Cathcart, Paul Widmer
Computation and Language
Doing Things with Words: Rethinking Theory of Mind Simulation in Large Language Models
Agnese Lombardi and Alessandro Lenci
Language is fundamental to human cooperation, facilitating not only the exchange of information but also the coordination of actions through shared interpretations of situational contexts. This study explores whether the Generative
Agent-Based Model (GABM) Concordia can effectively model Theory of Mind (ToM) within simulated real-world environments. Specifically, we assess whether this framework successfully simulates ToM abilities and whether GPT-4 can perform tasks by making genuine inferences from social context, rather than relying on linguistic memorization. Our findings reveal a critical limitation: GPT-4 frequently fails to select actions based on belief attribution, suggesting that apparent ToM-like abilities observed in previous studies may stem from shallow statistical associations rather than true
reasoning. Additionally, the model struggles to generate coherent causal effects from
agent actions, exposing difficulties in processing complex social interactions. These results challenge current statements about emergent ToM-like capabilities in LLMs and highlight the need for more rigorous, action-based evaluation frameworks.
Read the full abstract ▾
Computation and Language
Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment
Buwei He, Yang Liu, Zhaowei Zhang et al. (+6 more)
Large
language models (LLMs) still struggle with strategic persuasion, largely because existing approaches either neglect information asymmetry or rely on unrealistic pre-commitment assumptions. We introduce a type-induced commitment-communication mechanism that grounds
Bayesian Persuasion (BP) in natural
language dialogue without pre-commitment: the persuader narrates their potential types (e.g., honest vs. dishonest) to dynamically construct an information schema, enabling the persuadee to perform
Bayesian belief updates within the conversation itself. We implement two variants: Semi-Formal-Natural-
Language (SFNL) and Fully-Natural-
Language (FNL), evaluating them against strong baselines across multiple LLMs and human judges. BP strategies consistently outperform baselines: SFNL excels in logical credibility, while FNL shows superior robustness and emotional resonance. We verify that gains stem from genuine
Bayesian reasoning rather than superficial formatting, and we further show that supervised fine-tuning enables small models to match the persuasive performance of much larger ones.
Read the full abstract ▾
All authors Buwei He, Yang Liu, Zhaowei Zhang, Zixia Jia, Yang Yu, Huijia Wu, Zhaofeng He, Zilong Zheng, Yipeng Kang
Computation and Language
Document Intelligence in the Era of Large Language Models: A Survey
Weishi Wang, Hengchang Hu, Zhijie Zhang et al. (+3 more)
Document AI (DAI) has emerged as a vital application area, and is significantly transformed by the advent of large
language models (LLMs). While earlier approaches relied on encoder-decoder architectures, decoder-only LLMs have revolutionized DAI, bringing remarkable advancements in understanding and generation. This survey provides a comprehensive overview of DAI's
evolution, highlighting current research attempts and future prospects of LLMs in this field. We explore key advancements and challenges in
multimodal,
multilingual, and
retrieval-augmented DAI, while also suggesting future research directions, including
agent-based approaches and document-specific foundation models. This paper aims to provide a structured analysis of the state-of-the-art in DAI and its implications for both academic and practical applications.
Read the full abstract ▾
All authors Weishi Wang, Hengchang Hu, Zhijie Zhang, Zhaochen Li, Hongxin Shao, Daniel Dahlmeier
Computation and Language
D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree
Xiang Lei, Qin Li, Min Zhang et al. (+1 more)
Large
Language Models (LLMs) often exhibit factual inconsistencies and logical decay in extended, multi-turn dialogues, a challenge stemming from their reliance on static, pre-trained knowledge and an inability to reason adaptively over the dialogue history. Prevailing mitigation strategies, such as
Retrieval-Augmented Generation (RAG) and agentic working
memories, improve information recall but still engage with fundamentally static knowledge sources and follow pre-defined single
reasoning path. This hinders their ability to preserve factual and logical consistency of their responses in multi-turn dialogues while the context evolves over time. To address this issue, we propose D-SMART, a model-agnostic framework designed to maintain multi-turn dialogue consistency by enabling LLMs to build and reason over a dynamic, structured representation of the conversational context. This is achieved via two synergistic components: (1) a Dynamic Structured
Memory (DSM), which incrementally constructs and maintains an authoritative, OWL-compliant knowledge graph of the conversation; and (2) a
Reasoning Tree (RT), which executes inferences as an explicit and traceable multi-step search over the graph. As the popular-used quality score (judged by GPT-4) can overlook logical flaws, we introduce new NLI-based metrics to better measure multi-turn dialogue consistency. Comprehensive experiments on the MT-Bench-101
benchmark show that D-SMART significantly outperforms state-of-the-art baselines, elevating the dialogue consistency score by over 48% for both proprietary and open-source models, and notably improves the quality score of the latter by up to 10.1%.
Read the full abstract ▾
All authors Xiang Lei, Qin Li, Min Zhang, Min Zhang
Computation and Language
Personal Attribute Leakage in Federated Speech Models
Hamdan Al-Ali, Ali Reza Ghavamipour, Tommaso Caselli et al. (+3 more)
Federated learning is a common method for
privacy-preserving training of machine learning models. In this paper, we analyze the vulnerability of ASR models to attribute inference attacks in the
federated setting. We test a non-parametric white-box attack method under a passive threat model on three ASR models: Wav2Vec2, HuBERT, and Whisper. The attack operates solely on weight differentials without access to raw
speech from target speakers. We demonstrate attack feasibility on sensitive demographic and clinical attributes: gender, age, accent,
emotion, and dysarthria. Our findings indicate that attributes that are underrepresented or absent in the pre-training data are more vulnerable to such inference attacks. In particular, information about accents can be reliably inferred from all models. Our findings expose previously undocumented vulnerabilities in
federated ASR models and offer insights towards improved security.
Read the full abstract ▾
All authors Hamdan Al-Ali, Ali Reza Ghavamipour, Tommaso Caselli, Fatih Turkmen, Zeerak Talat, Hanan Aldarmaki
Computation and Language
Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems
Karthik Avinash, Nikhil Pareek and Rishav Hada
The increasing deployment of Large
Language Models (LLMs) across enterprise and mission-critical domains has underscored the urgent need for robust guardrailing systems that ensure safety, reliability, and compliance. Existing solutions often struggle with real-time oversight, multi-modal data handling, and explainability – limitations that hinder their adoption in regulated environments. Existing guardrails largely operate in isolation, focused on text alone making them inadequate for multi-modal, production-scale environments. We introduce Protect, natively multi-modal guardrailing model designed to operate seamlessly across text, image, and audio inputs, designed for enterprise-grade deployment. Protect integrates fine-tuned, category-specific adapters trained via Low-Rank Adaptation (LoRA) on an extensive, multi-modal
dataset covering four safety dimensions: toxicity, sexism, data
privacy, and
prompt injection. Our teacher-assisted annotation pipeline leverages
reasoning and explanation traces to generate high-fidelity, context-aware labels across modalities. Experimental results demonstrate state-of-the-art performance across all safety dimensions, surpassing existing open and proprietary models such as WildGuard, LlamaGuard-4, and GPT-4.1. Protect establishes a strong foundation for trustworthy, auditable, and production-ready safety systems capable of operating across text, image, and audio modalities.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE
Zhenyu Liu, Yunxin Li, Xuanyu Zhang et al. (+13 more)
Recent advances in unified
multimodal models indicate a clear trend towards comprehensive content generation. However, the auditory domain remains a significant challenge, with music and
speech often developed in isolation, hindering progress towards universal audio synthesis. This separation stems from inherent task conflicts and severe data imbalances, which impede the development of a truly unified audio generation model. To address this challenge, we propose UniMoE-Audio, a unified
speech and music generation model within a novel Dynamic-Capacity Mixture-of-Experts (MoE) framework. Architecturally, UniMoE-Audio introduces a Top-P routing strategy for dynamic expert number allocation, and a hybrid expert design comprising routed experts for domain-specific knowledge, shared experts for domain-agnostic features, and null experts for adaptive computation skipping. To tackle data imbalance, we introduce a three-stage training curriculum: 1) Independent Specialist Training leverages original
datasets to instill domain-specific knowledge into each "proto-expert" without interference; 2) MoE Integration and Warmup incorporates these specialists into the UniMoE-Audio architecture, warming up the gate module and shared expert using a subset of balanced
dataset; and 3) Synergistic Joint Training trains the entire model end-to-end on the fully balanced
dataset, fostering enhanced cross-domain synergy. Extensive experiments show that UniMoE-Audio not only achieves state-of-the-art performance on major
speech and music generation
benchmarks, but also demonstrates superior synergistic learning, mitigating the performance degradation typically seen in naive joint training. Our findings highlight the substantial potential of specialized MoE architecture and curated training strategies in advancing the field of universal audio generation. Homepage: https://mukioxun.github.io/Uni-MoE-site/home.html
Read the full abstract ▾
All authors Zhenyu Liu, Yunxin Li, Xuanyu Zhang, Qixun Teng, Shenyuan Jiang, Xinyu Chen, Haoyuan Shi, Jinchao Li, Qi Wang, Haolan Chen, Fanbo Meng, Mingjun Zhao, Yu Xu, Yancheng He, Baotian Hu, Min Zhang
Computation and Language
Proverbs or Pythian Oracles? Sentiments and Emotions in Greek Sayings
Katerina Korre and John Pavlopoulos
Proverbs are among the most fascinating
language phenomena that transcend cultural and linguistic boundaries. Yet, much of the global landscape of proverbs remains underexplored, as many cultures preserve their traditional wisdom within their own communities due to the oral tradition of the phenomenon. Taking advantage of the current advances in Natural
Language Processing (NLP), we focus on Greek proverbs, analyzing their sentiment and
emotion. Departing from an annotated
dataset of Greek proverbs, (1) we propose a multi-label annotation framework and
dataset that captures the emotional variability of the proverbs, (2) we up-scale to local varieties, (3) we sketch a map of Greece that provides an overview of the distribution of
emotions. Our findings show that the interpretation of proverbs is multidimensional, a property manifested through both multi-labeling and instance-level polarity. LLMs can capture and reproduce this complexity, and can therefore help us better understand the proverbial landscape of a place, as in the case of Greece, where surprise and anger compete and coexist within proverbs.
Read the full abstract ▾
Computation and Language
Taming the Fragility of KV Cache Eviction in LLM Inference
Yuan Feng, Haoyu Guo, JunLin Lv et al. (+2 more)
Large
language models have revolutionized natural
language processing, yet their deployment remains hampered by the substantial
memory and runtime overhead of the
transformer's Key-Value cache. To mitigate this, recent methods employ a scoring-aggregation framework to evict unimportant cache entries, based on the stability assumption-that a fixed subset of entries remains consistently important during generation. However, prior work has largely focused on refining importance indicators for scoring, while defaulting to mean aggregation due to a faithful trust in the stability assumption. In this work, we argue that this underlying assumption is inherently fragile, making mean aggregation highly vulnerable in extreme cases. To counter this, we propose a simple yet elegant defensive aggregation strategy: a two-step, linear-time approach that controls worst-case risk, thereby defending against extreme cases with negligible computational overhead. Embodying this strategy, we propose a novel cache eviction method, DefensiveKV and its extension, Layer-DefensiveKV, which incorporates layer-wise budget allocation. Across seven task domains (18
datasets), our methods reduce generation quality loss by 2.3x and 4.3x respectively, versus the strongest baseline under a 20% cache size. These results set new performance
benchmarks and pioneer a promising direction for optimizing cache eviction against underlying fragility through worst-case risk management. Our code is available at https://github.com/FFY0/DefensiveKV.
Read the full abstract ▾
All authors Yuan Feng, Haoyu Guo, JunLin Lv, S. Kevin Zhou, Xike Xie
Computation and Language
Embedding-Based Context-Aware Reranker
Ye Yuan, Mohammad Amin Shabani and Siqi Liu
Retrieval-Augmented Generation (RAG) systems rely on retrieving relevant evidence from a corpus to support downstream generation. The common practice of splitting a long document into multiple shorter passages enables finer-grained and targeted information
retrieval. However, it also introduces challenges when a correct
retrieval would require inference across passages, such as resolving coreference, disambiguating entities, and aggregating evidence scattered across multiple sources. Many state-of-the-art (SOTA) reranking methods, despite utilizing powerful large pretrained
language models with potentially high inference costs, still neglect the aforementioned challenges. Therefore, we propose
Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranking framework operating directly on
embeddings of retrieved passages with enhanced cross-passage understandings through the structural information of the passages and a hybrid
attention mechanism, which captures both high-level interactions across documents and low-level relationships within each document. We evaluate EBCAR against SOTA rerankers on the ConTEB
benchmark, demonstrating its effectiveness for information
retrieval requiring cross-passage inference and its advantages in both accuracy and efficiency.
Read the full abstract ▾
Computation and Language
ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering
Simon Lupart, Mohammad Aliannejadi and Evangelos Kanoulas
We present ChatR1, a
reasoning framework based on reinforcement learning (RL) for conversational question answering (CQA).
Reasoning plays an important role in CQA, where user intent evolves across dialogue turns, and utterances are often underspecified, requiring contextual interpretation, query reformulation, and dynamic coordination between
retrieval and generation. Unlike static `rewrite, retrieve, and generate' pipelines, ChatR1 interleaves search and
reasoning across turns, enabling exploratory and adaptive behaviors learned through RL. To address the challenge of sparse and delayed rewards in RL, we propose an intent-aware reward that provides turn-level feedback by aligning
retrieval and
reasoning with evolving user goals. ChatR1 demonstrates strong performance on both 3B and 7B model backbones, outperforming competitive models on five CQA
datasets, measured by different metrics (F1, BERTScore, and LLM-as-judge). We include a diverse set of CQA
datasets to cover topic shifts, evolving intents, mixed-initiative dialogues, and multi-document grounding, testing ChatR1's performance from various aspects. Ablation studies confirm the effectiveness of the intent-aware reward. Our analyses further reveal diverse
reasoning trajectories and effective use of the search tool. ChatR1 also generalizes robustly across domains, demonstrating that RL-based
reasoning enables more flexible and context-aware behavior than static CQA pipelines.
Read the full abstract ▾
Computation and Language
One-shot Style Transfer LLM log-probabilities for Authorship Attribution and Verification
Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín et al. (+1 more)
Computational stylometry studies writing style through quantitative textual patterns, enabling applications such as authorship attribution, identity linking, and plagiarism detection. Despite the relevance of
language modeling to these tasks, the pre-training of modern large
language models (LLMs) has been underutilized in authorship attribution and verification. We introduce an unsupervised framework that uses the log-probabilities of an LLM to measure style transferability between two texts. This framework takes advantage of the extensive Causal
Language Modeling (CLM) pre-training, one-shot capabilities and scale of LLMs, avoiding explicit supervision. Our methods substantially outperform prompting-based unsupervised baselines in authorship verification at similar model sizes, and is competitive with or improves contrastive baselines in most settings with sufficient model scale. We further observe strong performance across non-English
languages. The effectiveness of the proposed framework improves consistently with increasing model scale. In the case of authorship verification, we propose an additional mechanism that increases test-time computation to improve accuracy; enabling flexible trade-offs between computational cost and task performance.
Read the full abstract ▾
All authors Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín, David Camacho
Computation and Language
Cross-modal Consistency Guidance for Robust Emotion Control in Auto-Regressive TTS Models
Yizhou Peng, Yukun Ma, Chong Zhang et al. (+4 more)
While Text-to-
Speech (TTS) systems enable emotional control via natural-
language instructions, expressiveness, naturalness, and
speech quality degrade when the target
emotion conflicts with the textual semantics. We propose a Cross-modal Consistency Guided Classifier-Free Guidance (CCG-CFG) method with dynamic scales based on the degree of inconsistency between the text
emotion and the explicit
speech emotion, replacing the dropout condition with the text
emotion. We also distill the CCG-CFG guidance signal using a hard-sample mining strategy, improving the TTS model's emotional
alignment capability. Evaluations on five emotional corpora and two TTS
benchmarks show that our approaches applied to CosyVoice2 achieve up to a 12% absolute improvement in
emotion-recognition accuracy and a 10% relative improvement in subjective scores, outperforming baselines including HierSpeech++, Qwen3-TTS, and original CosyVoice2, while preserving intelligibility, naturalness, and high
speech quality.
Read the full abstract ▾
All authors Yizhou Peng, Yukun Ma, Chong Zhang, Yi-Wen Chao, Chongjia Ni, Bin Ma, Eng Siong Chng
Computation and Language
Higher Satisfaction, Lower Cost: A Technical Report on How LLMs Revolutionize Meituan's Intelligent Interaction Systems
Xuxin Cheng, Ke Zeng, Zhiquan Cao et al. (+65 more)
Enhancing customer experience is essential for business success, particularly as service demands grow in scale and complexity. Generative artificial intelligence and Large
Language Models (LLMs) have empowered intelligent interaction systems to deliver efficient, personalized, and 24/7 support. In practice, intelligent interaction systems encounter several challenges: (1) Constructing high-quality data for cold-start training is difficult, hindering self-
evolution and raising labor costs. (2) Multi-turn dialogue performance remains suboptimal due to inadequate intent understanding, rule compliance, and solution extraction. (3) Frequent
evolution of business rules affects system operability and transferability, constraining low-cost expansion and adaptability. (4) Reliance on a single LLM is insufficient in complex scenarios, where the absence of multi-
agent frameworks and effective collaboration undermines process completeness and service quality. (5) The open-domain nature of multi-turn dialogues, lacking unified golden answers, hampers quantitative evaluation and continuous optimization. To address these challenges, we introduce WOWService, an intelligent interaction system tailored for industrial applications. With the integration of LLMs and multi-
agent architectures, WOWService enables autonomous task management and collaborative problem-solving. Specifically, WOWService focuses on core modules including data construction, general capability enhancement, business scenario adaptation, multi-
agent coordination, and automated evaluation. Currently, WOWService is deployed on the Meituan App, achieving significant gains in key metrics, e.g., User Satisfaction Metric 1 (USM 1) -27.53% and User Satisfaction Metric 2 (USM 2) +25.51%, demonstrating its effectiveness in capturing user needs and advancing personalized service.
Read the full abstract ▾
All authors Xuxin Cheng, Ke Zeng, Zhiquan Cao, Linyi Dai, Wenxuan Gao, Fei Han, Ai Jian, Feng Hong, Wenxing Hu, Zihe Huang, Dejian Kong, Jia Leng, Zhuoyuan Liao, Pei Liu, Jiaye Lin, Xing Ma, Jingqing Ruan, Jiaxing Song, Xiaoyu Tan, Ruixuan Xiao, Wenhui Yu, Wenyu Zhan, Haoxing Zhang, Chao Zhou, Hao Zhou, Shaodong Zheng, Ruinian Chen, Siyuan Chen, Ziyang Chen, Yiwen Dong, Yaoyou Fan, Yangyi Fang, Yang Gan, Shiguang Guo, Qi He, Chaowen Hu, Binghui Li, Dailin Li, Xiangyu Li, Yan Li, Chengjian Liu, Xiangfeng Liu, Jiahui Lv, Qiao Ma, Jiang Pan, Cong Qin, Chenxing Sun, Wen Sun, Zhonghui Wang, Abudukelimu Wuerkaixi, Xin Yang, Fangyi Yuan, Yawen Zhu, Tianyi Zhai, Jie Zhang, Runlai Zhang, Yao Xu, Yiran Zhao, Yifan Wang, Xunliang Cai, Yangen Hu, Cao Liu, Lu Pan, Xiaoli Wang, Bo Xiao, Wenyuan Yao, Qianlin Zhou, Benchang Zhu
Computation and Language
In-Distribution Steering: Balancing Control and Coherence in Language Model Generation
Arthur Vogels, Benjamin Wong, Yann Choho et al. (+2 more)
Activation steering methods control large
language model (LLM) behavior by modifying internal activations at inference time. However, most existing activation steering methods rely on a fixed steering strength, leading to either insufficient control or unadapted intervention that degrades text plausibility and coherence. We introduce In-Distribution Steering (IDS), a novel method that adapts steering strength based on the input data distribution in representation space. IDS dynamically adjusts interventions according to how far a given input lies within the distribution, enabling adaptive intervention and generation stability during text generation. Experiments demonstrate that IDS achieves strong accuracy on classification tasks while producing coherent text without collapse, making IDS particularly well suited for real-world applications.
Read the full abstract ▾
All authors Arthur Vogels, Benjamin Wong, Yann Choho, Annabelle Blangero, Milan Bhan
Computation and Language · Cross-listed from eess.AS
Two Heads Are Better Than One: Audio-Visual Speech Error Correction with Dual Hypotheses
Sungnyun Kim, Kangwook Jang, Sungwoo Cho et al. (+3 more)
This paper introduces a new paradigm for generative error correction (GER) framework in audio-visual
speech recognition (AVSR) that reasons over modality-specific evidences directly in the
language space. Our framework, DualHyp, empowers a large
language model (LLM) to compose independent N-best hypotheses from separate automatic
speech recognition (ASR) and visual
speech recognition (VSR) models. To maximize the effectiveness of DualHyp, we further introduce RelPrompt, a noise-aware guidance mechanism that provides modality-grounded
prompts to the LLM. RelPrompt offers the temporal reliability of each modality stream, guiding the model to dynamically switch its focus between ASR and VSR hypotheses for an accurate correction. Under various corruption scenarios, our framework attains up to 57.7% error rate gain on the LRS2
benchmark over standard ASR baseline, contrary to single-stream GER approaches that achieve only 10% gain. To facilitate research within our DualHyp framework, we release the code and the
dataset comprising ASR and VSR hypotheses at https://github.com/sungnyun/dualhyp.
Read the full abstract ▾
All authors Sungnyun Kim, Kangwook Jang, Sungwoo Cho, Joon Son Chung, Hoirin Kim, Se-Young Yun
Computation and Language · Cross-listed from cs.CV
MMLongCite: A Benchmark for Evaluating Faithfulness of Long-Context Vision-Language Models
Keyan Zhou, Zecheng Tang, Lingfeng Ming et al. (+9 more)
The rapid advancement of long-context
vision language models (LCVLMs) has led to a significant expansion of their context windows. However, an extended context window does not guarantee the effective utilization of the context, posing a critical challenge for real-world applications. Current evaluations of such long-context faithfulness in
multimodal settings remain limited to short contexts. To bridge this gap, we introduce MMLongCite, the first
benchmark evaluating the faithfulness of LCVLMs via
multimodal citation generation. MMLongCite features 2,280 examples across 8 tasks and diverse modalities (image,
video, interleaved), with context lengths scaled from 16K to 128K
tokens. To test spatial localization capabilities of LCVLMs, we also introduce MMLongCite-HR, evaluating fine-grained visual grounding amidst dense pixel spaces. Through extensive benchmarking of cutting-edge LCVLMs, we provide a systematic analysis of current
multimodal citation capabilities. Our results reveal a significant discrepancy between answer correctness and citation faithfulness. We also conduct
attention pattern investigations and in-depth error analyses to reveal the underlying phenomena of failures in LCVLMs. MMLongCite establishes a rigorous foundation for diagnosing and advancing the faithfulness of LCVLMs. We hope our findings provide meaningful insights to drive further improvements in the long-context capabilities of LCVLMs.
Read the full abstract ▾
All authors Keyan Zhou, Zecheng Tang, Lingfeng Ming, Qiguang Chen, Wangjie You, Guanghao Zhou, Dan Qiao, Zheming Yang, Libo Qin, Minghui Qiu, Juntao Li, Min Zhang
Computation and Language
Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation
Zhichao Xu, Zongyu Wu, Yun Zhou et al. (+9 more)
Inspired by the success of reinforcement learning (RL) in Large
Language Model (LLM) training for domains like math and code, recent work has begun training LLMs to dynamically plan, query, and reason with search engines as tools – a paradigm increasingly referred to as agentic search. Although these methods achieve performance improvement across popular short-form QA
benchmarks, many prioritize final answer correctness while overlooking the quality of intermediate
reasoning steps, which may lead to chain-of-thought unfaithfulness. In this paper, we first introduce a comprehensive evaluation framework for agentic search, covering three distinct faithfulness metrics: Think-Search faithfulness, Information-Think faithfulness, and Think-Answer faithfulness. Our evaluations reveal that canonical agentic search systems trained through Reinforcement Learning from Verifiable Reward (RLVR) using episode-level outcome-based reward – including Search-R1 and ReSearch – have significant room for improvement on these faithfulness dimensions. To foster faithful
reasoning in agentic search, we introduce VERITAS (Verifying Entailed
Reasoning through Intermediate Traceability in Agentic Search), a novel framework that integrates fine-grained turn-level faithfulness rewards into the reinforcement learning process. Our experiments show that models trained with \ours not only significantly improve
reasoning faithfulness, but also achieve better task performance compared to baselines trained against episode-level outcome-based reward.
Read the full abstract ▾
All authors Zhichao Xu, Zongyu Wu, Yun Zhou, Aosong Feng, Kang Zhou, Sangmin Woo, Kiran Ramnath, Yijun Tian, Xuan Qi, Weikang Qiu, Lin Lee Cheong, Haibo Ding
Computation and Language
Do You Get the Hint? Benchmarking LLMs on the Board Game Concept
Ine Gevers and Walter Daelemans
Large
language models (LLMs) have achieved striking successes on many
benchmarks, yet recent studies continue to expose fundamental weaknesses. In this paper, we introduce Concept, a simple word-guessing board game, as a
benchmark for probing abductive
reasoning. Our results show that this game, easily solved by humans (with a success rate of over 90%), is still very challenging for state-of-the-art LLMs (no model exceeds 40% success rate). Specifically, we observe that LLMs struggle with interpreting other players' strategic intents, and with correcting initial hypotheses given sequential information updates. In addition, we extend the evaluation across multiple
languages, and find that the LLM performance drops further in lower-resource
languages (Dutch, French, and Spanish) compared to English.
Read the full abstract ▾
Computation and Language
Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain
Jingmin An, Yilong Song, Ruolin Yang et al. (+7 more)
Large
Language Models (LLMs) demonstrate human-level or even superior
language abilities, effectively modeling syntactic structures, yet the specific computational modules responsible remain unclear. A key question is whether LLM behavioral capabilities stem from mechanisms akin to those in the human
brain. To address these questions, we introduce the Hierarchical Frequency Tagging Probe (HFTP), a tool that utilizes frequency-domain analysis to identify
neuron-wise components of LLMs (e.g., individual Multilayer Perceptron (MLP)
neurons) and cortical regions (via intracranial recordings) encoding syntactic structures. Our results show that models such as GPT-2, Gemma, Gemma 2, Llama 2, Llama 3.1, and GLM-4 process syntax in analogous layers, while the human
brain relies on distinct cortical regions for different syntactic levels. Representational similarity analysis reveals a stronger
alignment between LLM representations and the left hemisphere of the
brain (dominant in
language processing). Notably, upgraded models exhibit divergent trends: Gemma 2 shows greater
brain similarity than Gemma, while Llama 3.1 shows less
alignment with the
brain compared to Llama 2. These findings offer new insights into the interpretability of LLM behavioral improvements, raising questions about whether these advancements are driven by human-like or non-human-like mechanisms, and establish HFTP as a valuable tool bridging computational linguistics and cognitive neuroscience. This project is available at https://github.com/LilTiger/HFTP.
Read the full abstract ▾
All authors Jingmin An, Yilong Song, Ruolin Yang, Nai Ding, Lingxi Lu, Yuxuan Wang, Wei Wang, Chu Zhuang, Qian Wang, Fang Fang
Computation and Language · Cross-listed from cs.AI
EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems
Yufei He, Juncheng Liu, Yue Liu et al. (+5 more)
A fundamental limitation of current AI
agents is their inability to learn complex skills on the fly at test time, often behaving like "clever but clueless interns" in novel environments. This severely limits their practical utility. To systematically measure and drive progress on this challenge, we first introduce the Jericho Test-Time Learning (J-TTL)
benchmark. J-TTL is a new evaluation setup where an
agent must play the same game for several consecutive episodes, attempting to improve its performance from one episode to the next. On J-TTL, we find that existing adaptation methods like reflection,
memory, or reinforcement learning struggle. To address the challenges posed by our
benchmark, we present EvoTest, an evolutionary test-time learning framework that improves an
agent without any fine-tuning or gradients-by evolving the entire agentic system after every episode. EvoTest has two roles: the Actor
Agent, which plays the game, and the Evolver
Agent, which analyzes the episode transcript to propose a revised configuration for the next run. This configuration rewrites the
prompt, updates
memory by logging effective state-action choices, tunes hyperparameters, and learns the tool-use routines. On our J-TTL
benchmark, EvoTest consistently increases performance, outperforming not only reflection and
memory-only baselines but also more complex online fine-tuning methods. Notably, our method is the only one capable of winning two games (Detective and Library), while all baselines fail to win any.
Read the full abstract ▾
All authors Yufei He, Juncheng Liu, Yue Liu, Yibo Li, Tri Cao, Zhiyuan Hu, Xinxing Xu, Bryan Hooi
Computation and Language · Cross-listed from cs.AI
Personalized Learning Path Planning with Goal-Driven Learner State Modeling
Joy Jia Yin Lim, Ye He, Jifan Yu et al. (+7 more)
Personalized Learning Path Planning (PLPP) aims to design adaptive learning paths that align with individual goals. While large
language models (LLMs) show potential in personalizing learning experiences, existing approaches often lack mechanisms for goal-aligned planning. We introduce Pxplore, a novel framework for PLPP that integrates a reinforcement-based training paradigm and an LLM-driven educational architecture. We design a structured learner state model and an automated reward function that transforms abstract objectives into computable signals. We train the policy combining supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), and deploy it within a real-world learning platform. Extensive experiments validate Pxplore's effectiveness in producing coherent, personalized, and goal-driven learning paths. We release our code and
dataset at https://github.com/Pxplore/pxplore-algo.
Read the full abstract ▾
All authors Joy Jia Yin Lim, Ye He, Jifan Yu, Xin Cong, Daniel Zhang-Li, Zhiyuan Liu, Huiqin Liu, Lei Hou, Juanzi Li, Bin Xu
Computation and Language
A fully automated and scalable Parallel Data Augmentation for Low Resource Languages using Image and Text Analytics
Prawaal Sharma, Navneet Goyal, Poonam Goyal et al. (+1 more)
Linguistic diversity across the world creates a disparity with the availability of good quality digital
language resources thereby restricting the technological benefits to majority of human population. The lack or absence of data resources makes it difficult to perform NLP tasks for low-resource
languages. This paper presents a novel scalable and fully automated methodology to extract bilingual parallel corpora from newspaper articles using image and text analytics. We validate our approach by building parallel data corpus for two different
language combinations and demonstrate the value of this
dataset through a downstream task of machine translation and improve over the current baseline by close to 3 BLEU points.
Read the full abstract ▾
All authors Prawaal Sharma, Navneet Goyal, Poonam Goyal, Vishnupriyan R
Computation and Language
LLM-Guided Synthetic Augmentation (LGSA) for Mitigating Bias in AI Systems
Sai Suhruth Reddy Karri, Yashwanth Sai Nallapuneni, Laxmi Narasimha Reddy Mallireddy et al. (+1 more)
Bias in AI systems, especially those relying on natural
language data, raises ethical and practical concerns. Underrepresentation of certain groups often leads to uneven performance across demographics. Traditional
fairness methods, such as pre-processing, in-processing, and post-processing, depend on protected-attribute labels, involve accuracy-
fairness trade-offs, and may not generalize across
datasets. To address these challenges, we propose LLM-Guided Synthetic Augmentation (LGSA), which uses large
language models to generate counterfactual examples for underrepresented groups while preserving label integrity. We evaluated LGSA on a controlled
dataset of short English sentences with gendered pronouns, professions, and binary classification labels. Structured
prompts were used to produce gender-swapped paraphrases, followed by quality control including semantic similarity checks, attribute verification, toxicity screening, and human spot checks. The augmented
dataset expanded training coverage and was used to train a classifier under consistent conditions. Results show that LGSA reduces performance disparities without compromising accuracy. The baseline model achieved 96.7 percent accuracy with a 7.2 percent gender bias gap. Simple swap augmentation reduced the gap to 0.7 percent but lowered accuracy to 95.6 percent. LGSA achieved 99.1 percent accuracy with a 1.9 percent bias gap, improving performance on female-labeled examples. These findings demonstrate that LGSA is an effective strategy for bias mitigation, enhancing subgroup balance while maintaining high task accuracy and label fidelity.
Read the full abstract ▾
All authors Sai Suhruth Reddy Karri, Yashwanth Sai Nallapuneni, Laxmi Narasimha Reddy Mallireddy, Gopichand G
Computation and Language
Text Anomaly Detection with Simplified Isolation Kernel
Yang Cao, Sikun Yang, Yujiu Yang et al. (+2 more)
Two-step approaches combining pre-trained large
language model
embeddings and anomaly detectors demonstrate strong performance in text anomaly detection by leveraging rich semantic representations. However, high-dimensional dense
embeddings extracted by large
language models pose challenges due to substantial
memory requirements and high computation time. To address this challenge, we introduce the Simplified Isolation Kernel (SIK), which maps high-dimensional dense
embeddings to lower-dimensional sparse representations while preserving crucial anomaly characteristics. SIK has linear time complexity and significantly reduces space complexity through its innovative boundary-focused feature mapping. Experiments across 7
datasets demonstrate that SIK achieves better detection performance than 11 state-of-the-art (SOTA) anomaly detection
algorithms while maintaining computational efficiency and low
memory cost. All code and demonstrations are available at https://github.com/charles-cao/SIK.
Read the full abstract ▾
All authors Yang Cao, Sikun Yang, Yujiu Yang, Lianyong Qi, Ming Liu
Computation and Language
StressTransfer: Stress-Aware Speech-to-Speech Translation with Emphasis Preservation
Xi Chen, Yuchen Song and Satoshi Nakamura
We propose a stress-aware
speech-to-
speech translation (S2ST) system that preserves word-level emphasis by leveraging LLMs for cross-lingual emphasis conversion. Our method translates source-
language stress into target-
language tags that guide a controllable TTS model. To overcome data scarcity, we developed a pipeline to automatically generate aligned training data and introduce the "LLM-as-Judge" for evaluation. Experiments show our approach substantially outperforms baselines in preserving emphasis while maintaining comparable translation quality, speaker intent, and naturalness. Our work highlights the importance of prosody in translation and provides an effective, data-efficient solution for preserving paralinguistic cues in S2ST.
Read the full abstract ▾