Computation and Language · Cross-listed from cs.IR
FUSE : Failure-aware Usage of Subagent Evidence for MultiModal Search and Recommendation
Tushar Vatsa, Vibha Belavadi, Priya Shanmugasundaram et al. (+2 more)
Multimodal creative assistants decompose user goals and route tasks to subagents for layout, styling,
retrieval, and generation.
Retrieval quality is pivotal, yet failures can arise at several stages: understanding user intent, choosing content types, finding candidates (recall), or ranking results. Meanwhile, sending and processing images is costly, making naive
multimodal approaches impractical. We present FUSE: Failure-aware Usage of Subagent Evidence for
MultiModal Search and Recommendation. FUSE replaces most raw-image prompting with a compact Grounded Design Representation (GDR): a selection aware JSON of canvas elements (image, text, shape, icon,
video, logo), structure, styles, salient colors, and user selection provided by the Planner team. FUSE implements seven context budgeting strategies: comprehensive baseline prompting, context compression, chain-of-thought
reasoning, mini-shot optimization,
retrieval-augmented context, two-stage processing, and zero-shot minimalism. Finally, a pipeline attribution layer monitors system performance by converting subagent signals into simple checks: intent
alignment, content-type/routing sanity, recall health (e.g., zero-hit and top-match strength), and ranking displacement analysis. We evaluate the seven context budgeting variants across 788 evaluation queries from diverse users and design templates (refer Figure 3). Our systematic evaluation reveals that Context Compression achieves optimal performance across all pipeline stages, with 93.3% intent accuracy, 86.8% routing success(with fallbacks), 99.4% recall, and 88.5% NDCG@5. This approach demonstrates that strategic context summarization outperforms both comprehensive and minimal contextualization strategies.
Read the full abstract ▾
All authors Tushar Vatsa, Vibha Belavadi, Priya Shanmugasundaram, Suhas Suresha, Dewang Sultania
Computation and Language
Cacheback: Speculative Decoding With Nothing But Cache
Zhiyao Ma, In Gim and Lin Zhong
We present Cacheback Decoding, a training-free and model-agnostic speculative decoding method that exploits the locality in
language to accelerate Large
Language Model (LLM) inference. Cacheback leverages only Least Recently Used (LRU) cache tables of
token n-grams to generate draft sequences. Cacheback achieves state-of-the-art performance among comparable methods despite its minimalist design, and its simplicity allows easy integration into existing systems. Cacheback also shows potential for fast adaptation to new domains.
Read the full abstract ▾
Computation and Language
Semantics Meet Signals: Dual Codebook Representationl Learning for Generative Recommendation
Zheng Hui, Xiaokai Wei, Reza Shirkavand et al. (+4 more)
Generative recommendation has recently emerged as a powerful paradigm that unifies
retrieval and generation, representing items as discrete semantic
tokens and enabling flexible sequence modeling with autoregressive models. Despite its success, existing approaches rely on a single, uniform codebook to encode all items, overlooking the inherent imbalance between popular items rich in collaborative signals and long-tail items that depend on semantic understanding. We argue that this uniform treatment limits representational efficiency and hinders generalization. To address this, we introduce FlexCode, a popularity-aware framework that adaptively allocates a fixed
token budget between a collaborative filtering (CF) codebook and a semantic codebook. A lightweight MoE dynamically balances CF-specific precision and semantic generalization, while an
alignment and smoothness objective maintains coherence across the popularity spectrum. We perform experiments on both public and industrial-scale
datasets, showing that FlexCode consistently outperform strong baselines. FlexCode provides a new mechanism for
token representation in generative recommenders, achieving stronger accuracy and tail robustness, and offering a new perspective on balancing memorization and generalization in
token-based recommendation models.
Read the full abstract ▾
All authors Zheng Hui, Xiaokai Wei, Reza Shirkavand, Chen Wang, Weizhi Zhang, Alejandro Peláez, Michelle Gong
Computation and Language · Cross-listed from cs.LG
Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
Fengming Yu, Qingyu Meng, Haiwei Pan et al. (+1 more)
With the rapid development of deep learning, large
language models have shown strong capabilities in complex
reasoning tasks such as mathematical equation solving. However, their substantial computational and storage costs hinder practical deployment. This paper proposes a lightweight optimization method that integrates dynamic
attention head
pruning with knowledge
distillation. The approach dynamically evaluates the importance of each
attention head in the multi-head
attention mechanism using a combination of weight norms and
entropy, and prunes redundant heads in real time to reduce computational overhead. To mitigate performance degradation, knowledge
distillation transfers information from the original model to the pruned student, enabling the smaller model to preserve
reasoning ability. Experiments conducted on both Math23k and ASDiv-A verify the effectiveness of the proposed method. For example, on Math23k with a 30%
pruning ratio, parameters are reduced by 18.7%, inference speed is improved by 27.5%, FLOPs are reduced by 19.3%, and accuracy drops only 0.7% (from 84.4% to 83.7%). These results demonstrate that the method achieves substantial efficiency gains while maintaining strong
reasoning performance, providing a practical solution for efficient deployment of large
language models in mathematical
reasoning tasks.
Read the full abstract ▾
All authors Fengming Yu, Qingyu Meng, Haiwei Pan, Kejia Zhang
Computation and Language
Concept-Based Interpretability for Toxicity Detection
Samarth Garg, Divya Singh, Deeksha Varshney et al. (+1 more)
The rise of social networks has not only facilitated communication but also allowed the spread of harmful content. Although significant advances have been made in detecting toxic
language in textual data, the exploration of concept-based explanations in toxicity detection remains limited. In this study, we leverage various subtype attributes present in toxicity detection
datasets, such as obscene, threat, insult, identity attack, and sexual explicit as concepts that serve as strong indicators to identify whether
language is toxic. However, disproportionate attribution of concepts towards the target class often results in classification errors. Our work introduces an interpretability technique based on the Concept Gradient (CG) method which provides a more causal interpretation by measuring how changes in concepts directly affect the output of the model. This is an extension of traditional gradient-based methods in machine learning, which often focus solely on input features. We propose the curation of Targeted Lexicon Set, which captures toxic words that contribute to misclassifications in text classification models. To assess the significance of these lexicon sets in misclassification, we compute Word-Concept
Alignment (WCA) scores, which quantify the extent to which these words lead to errors due to over-attribution to toxic concepts. Finally, we introduce a lexicon-free augmentation strategy by generating toxic samples that exclude predefined toxic lexicon sets. This approach allows us to examine whether over-attribution persists when explicit lexical overlap is removed, providing insights into the model's attribution on broader toxic
language patterns.
Read the full abstract ▾
All authors Samarth Garg, Divya Singh, Deeksha Varshney, Mamta
Computation and Language
From Phonemes to Meaning: Evaluating Large Language Models on Tamil
Jeyarajalingam Varsha, Menan Velayuthan, Sumirtha Karunakaran et al. (+2 more)
Large
Language Models (LLMs) have shown strong generalization across tasks in high-resource
languages; however, their linguistic competence in low-resource and morphologically rich
languages such as Tamil remains largely unexplored. Existing
multilingual benchmarks often rely on translated English
datasets, failing to capture the linguistic and cultural nuances of the target
language. To address this gap, we introduce ILAKKANAM, the first Tamil-specific linguistic evaluation
benchmark manually curated using 820 questions from Sri Lankan school-level Tamil subject examination papers. Each question is annotated by trained linguists under five linguistic categories and a factual knowledge category, spanning Grades 1–13 to ensure broad linguistic coverage. We evaluate both closed-source and open-source LLMs using a standardized evaluation framework. Our results show that Gemini 2.5 achieves the highest overall performance, while open-source models lag behind, highlighting the gap in linguistic grounding. Category- and grade-wise analyses reveal that all models perform well on lower-grade questions but show a clear decline as linguistic complexity increases. Further, no strong correlation is observed between a model's overall performance and its ability to identify linguistic categories, suggesting that performance may be driven by exposure rather than genuine understanding.
Read the full abstract ▾
All authors Jeyarajalingam Varsha, Menan Velayuthan, Sumirtha Karunakaran, Rasan Nivethiga, Kengatharaiyer Sarveswaran
Computation and Language
Don't Think of the White Bear: Ironic Negation in Transformer Models Under Cognitive Load
Logan Mann, Nayan Saxena, Sarah Tandon et al. (+3 more)
Negation instructions such as 'do not mention $X$' can paradoxically increase the accessibility of $X$ in human thought, a phenomenon known as ironic rebound. Large
language models (LLMs) face the same challenge: suppressing a concept requires internally activating it, which may
prime rebound instead of avoidance. We investigated this tension with two experiments. (1) Load & content: after a negation instruction, we vary distractor text (semantic, syntactic, repetition) and measure rebound strength. (2) Polarity separation: We test whether models distinguish neutral from negative framings of the same concept and whether this separation predicts rebound persistence. Results show that rebound consistently arises immediately after negation and intensifies with longer or semantic distractors, while repetition supports suppression. Stronger polarity separation correlates with more persistent rebound. Together, these findings, complemented by a
circuit tracing analysis that identifies sparse middle-layer
attention heads amplifying forbidden
tokens while early layers suppress, link cognitive predictions of ironic rebound with mechanistic insights into long-context interference. To support future work, we release ReboundBench, a
dataset of $5,000$ systematically varied negation
prompts designed to probe rebound in LLMs.
Read the full abstract ▾
All authors Logan Mann, Nayan Saxena, Sarah Tandon, Chenhao Sun, Savar Toteja, Kevin Zhu
Computation and Language · Cross-listed from eess.AS
VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing
Zhisheng Zheng, Puyuan Peng, Anuj Diwan et al. (+5 more)
We introduce VoiceCraft-X, an autoregressive neural codec
language model which unifies
multilingual speech editing and zero-shot Text-to-
Speech (TTS) synthesis across 11
languages: English, Mandarin, Korean, Japanese, Spanish, French, German, Dutch, Italian, Portuguese, and Polish. VoiceCraft-X utilizes the Qwen3 large
language model for phoneme-free cross-lingual text processing and a novel
token reordering mechanism with time-aligned text and
speech tokens to handle both tasks as a single sequence generation problem. The model generates high-quality, natural-sounding
speech, seamlessly creating new audio or editing existing recordings within one framework. VoiceCraft-X shows robust performance in diverse linguistic settings, even with limited per-
language data, underscoring the power of unified autoregressive approaches for advancing complex, real-world
multilingual speech applications. Audio samples are available at https://zhishengzheng.com/voicecraft-x/.
Read the full abstract ▾
All authors Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, David Harwath
Computation and Language
Do LLMs and Humans Find the Same Questions Difficult? A Case Study on Japanese Quiz Answering
Naoya Sugiura, Kosuke Yamada, Yasuhiro Ogawa et al. (+2 more)
LLMs have achieved performance that surpasses humans in many NLP tasks. However, it remains unclear whether problems that are difficult for humans are also difficult for LLMs. This study investigates how the difficulty of quizzes in a buzzer setting differs between LLMs and humans. Specifically, we first collect Japanese quiz data including questions, answers, and correct response rate of humans, then prompted LLMs to answer the quizzes under several settings, and compare their correct answer rate to that of humans from two analytical perspectives. The experimental results showed that, compared to humans, LLMs struggle more with quizzes whose correct answers are not covered by Wikipedia entries, and also have difficulty with questions that require numerical answers.
Read the full abstract ▾
All authors Naoya Sugiura, Kosuke Yamada, Yasuhiro Ogawa, Katsuhiko Toyama, Ryohei Sasano
Computation and Language
AugAbEx: Bridging Abstractive and Extractive Legal Summarization
Purnima Bindal, Vikas Kumar and Vasudha Bhatnagar
Automatic summarization of legal judgments liberates law professionals from heavy cognitive burden due to the complexity of the
language, context-sensitive legal jargon, and the length of the document. Caveats of abstractive summarization for legal documents, revealed in recent studies, have impelled the development of newhybrid and extractive summarization methods, along with
datasets in the legal domain. We propose an efficient and elegant pipeline (AugAbEx) to repurpose an existing large case summarization
dataset with human-written gold-standard summaries by augmenting it with silver standard extractive summaries ensconcing experts' opinion. Availability of silver summaries along with gold standard human-written summaries will bolster development and evaluation of novel hybrid and extractive case-summarization
algorithms in the legal domain. We thoroughly scrutinize the augmented extractive summaries in structural, lexical, and semantic dimensions, within a domain specific framework, to ensure quality. Extensive experiments and statistical test on seven English legal case-summarization
datasets demonstrate that the silver standard extractive summaries produced by the proposed pipeline score well across all evaluation dimensions. Comparison of AugAbEx with two baselines and three current state-of-the-art methods reveals that it outperforms all competing methods, and the quality of the summaries produced by the proposed pipeline is superior
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
How Far Do SSL Speech Models Listen for Tone? Temporal Focus of Tone Representation under Low-resource Transfer
Minu Kim, Ji Sub Um and Hoirin Kim
Lexical tone is central to many
languages but remains underexplored in self-supervised learning (SSL)
speech models, especially beyond Mandarin. We study four
languages with complex and diverse tone systems (Burmese, Thai, Lao, and Vietnamese) to ask how far such models "listen" for tone and how transfer operates in low-resource conditions. As a baseline reference, we estimate the temporal span of tone cues: approximately 100ms (Burmese/Thai) and 180ms (Lao/Vietnamese). Probes and gradient analysis on fine-tuned SSL models reveal that tone transfer varies by downstream task: automatic
speech recognition fine-tuning aligns spans with
language-specific tone cues, while prosody- and voice-related tasks bias toward overly long spans. These findings indicate that tone transfer is shaped by downstream task, highlighting task effects on temporal focus in tone modeling.
Read the full abstract ▾
Computation and Language
Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor
Ivan Zakazov, Berke Argin, Oussama Gabouj et al. (+6 more)
Motivated by the high costs of using black-box Large
Language Models (LLMs), we introduce a novel
prompt compression paradigm, under which we use smaller LLMs to compress inputs for the larger ones. We present the first comprehensive LLM-as-a-compressor
benchmark spanning 25 open- and closed-source models, which reveals significant disparity in models' compression ability in terms of (i) preserving semantically important information (ii) following the user-provided compression rate (CR). We further improve the performance of gpt-4.1-mini, the best overall vanilla compressor, with Textgrad-based compression meta-
prompt optimization. We also identify the most promising open-source vanilla LLM - Qwen3-4B - and post-train it with a combination of supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), pursuing the dual objective of CR adherence and maximizing the downstream task performance. We call the resulting model Cmprsr and demonstrate its superiority over both extractive and vanilla abstractive compression across the entire range of compression rates on lengthy inputs from MeetingBank and LongBench as well as short
prompts from GSM8k. The latter highlights Cmprsr's generalizability across varying input lengths and domains. Moreover, Cmprsr closely follows the requested compression rate, offering fine control over the cost-quality trade-off.
Read the full abstract ▾
All authors Ivan Zakazov, Berke Argin, Oussama Gabouj, Kamel Charaf, Alexander Sharipov, Alexi Semiz, Lorenzo Drudi, Nicolas Baldwin, Robert West
Computation and Language · Cross-listed from cs.CV
D$^{3}$ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMs
Shuochen Chang, Xiaofeng Zhang, Qingyang Liu et al. (+1 more)
Diffusion-based
multimodal large
language models (
Diffusion MLLMs) have recently demonstrated impressive non-autoregressive generative capabilities across
vision-and-
language tasks. However,
Diffusion MLLMs exhibit substantially slower inference than autoregressive models: Each denoising step employs full bidirectional self-
attention over the entire sequence, resulting in cubic decoding complexity that becomes computationally impractical with thousands of visual
tokens. To address this challenge, we propose D$^{3}$ToM, a Decider-guided dynamic
token merging method that dynamically merges redundant visual
tokens at different denoising steps to accelerate inference in
Diffusion MLLMs. At each denoising step, D$^{3}$ToM uses decider
tokens-the
tokens generated in the previous denoising step-to build an importance map over all visual
tokens. Then it maintains a proportion of the most salient
tokens and merges the remainder through similarity-based aggregation. This plug-and-play module integrates into a single
transformer layer, physically shortening the visual
token sequence for all subsequent layers without altering model parameters. Moreover, D$^{3}$ToM employs a merge ratio that dynamically varies with each denoising step, aligns with the native decoding process of
Diffusion MLLMs, achieving superior performance under equivalent computational budgets. Extensive experiments show that D$^{3}$ToM accelerates inference while preserving competitive performance. The code is released at https://github.com/bcmi/D3ToM-
Diffusion-MLLM.
Read the full abstract ▾
All authors Shuochen Chang, Xiaofeng Zhang, Qingyang Liu, Li Niu
Computation and Language
ViConBERT: Context-Gloss Aligned Vietnamese Word Embedding for Polysemous and Sense-Aware Representations
Khang T. Huynh, Dung H. Nguyen and Binh T. Nguyen
Recent advances in contextualized word
embeddings have greatly improved semantic tasks such as Word Sense Disambiguation (WSD) and contextual similarity, but most progress has been limited to high-resource
languages like English. Vietnamese, in contrast, still lacks robust models and evaluation resources for fine-grained semantic understanding. In this paper, we present ViConBERT, a novel framework for learning Vietnamese contextualized
embeddings that integrates contrastive learning (SimCLR) and gloss-based
distillation to better capture word meaning. We also introduce ViConWSD, the first large-scale synthetic
dataset for evaluating semantic understanding in Vietnamese, covering both WSD and contextual similarity. Experimental results show that ViConBERT outperforms strong baselines on WSD (F1 = 0.87) and achieves competitive performance on ViCon (AP = 0.88) and ViSim-400 (Spearman's rho = 0.60), demonstrating its effectiveness in modeling both discrete senses and graded semantic relations. Our code, models, and data are available at https://github.com/tkhangg0910/ViConBERT
Read the full abstract ▾
Computation and Language
Consistency Is the Key: Detecting Hallucinations in LLM Generated Text By Checking Inconsistencies About Key Facts
Raavi Gupta, Pranav Hari Panicker, Sumit Bhatia et al. (+1 more)
Large
language models (LLMs), despite their remarkable text generation capabilities, often hallucinate and generate text that is factually incorrect and not grounded in real-world knowledge. This poses serious risks in domains like healthcare, finance, and customer support. A typical way to use LLMs is via the APIs provided by LLM vendors where there is no access to model weights or options to fine-tune the model. Existing methods to detect
hallucinations in such settings where the model access is restricted or constrained by resources typically require making multiple LLM API calls, increasing latency and API cost. We introduce CONFACTCHECK, an efficient
hallucination detection approach that does not leverage any external knowledge base and works on the simple intuition that responses to factual probes within the generated text should be consistent within a single LLM and across different LLMs. Rigorous empirical evaluation on multiple
datasets that cover both the generation of factual texts and the open generation shows that CONFACTCHECK can detect hallucinated facts efficiently using fewer resources and achieves higher accuracy scores compared to existing baselines that operate under similar conditions. Our code is available here.
Read the full abstract ▾
All authors Raavi Gupta, Pranav Hari Panicker, Sumit Bhatia, Ganesh Ramakrishnan
Computation and Language
MME-RAG: Multi-Manager-Expert Retrieval-Augmented Generation for Fine-Grained Entity Recognition in Task-Oriented Dialogues
Liang Xue, Haoyu Liu, Yajun Tian et al. (+2 more)
Fine-grained entity recognition is crucial for
reasoning and decision-making in task-oriented dialogues, yet current large
language models (LLMs) continue to face challenges in domain adaptation and
retrieval controllability. We introduce MME-RAG, a Multi-Manager-Expert
Retrieval-Augmented Generation framework that decomposes entity recognition into two coordinated stages: type-level judgment by lightweight managers and span-level extraction by specialized experts. Each expert is supported by a KeyInfo retriever that injects semantically aligned, few-shot exemplars during inference, enabling precise and domain-adaptive extraction without additional training. Experiments on CrossNER, MIT-Movie, MIT-Restaurant, and our newly constructed multi-domain customer-service
dataset demonstrate that MME-RAG performs better than recent baselines in most domains. Ablation studies further show that both the hierarchical decomposition and KeyInfo-guided
retrieval are key drivers of robustness and cross-domain generalization, establishing MME-RAG as a scalable and interpretable solution for adaptive dialogue understanding.
Read the full abstract ▾
All authors Liang Xue, Haoyu Liu, Yajun Tian, Xinyu Zhong, Yang Liu
Computation and Language
CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic
Yaocheng Zhang, Haohuan Huang, Zijun Song et al. (+4 more)
Tool-Integrated
Reasoning (TIR) with search engines enables large
language models to iteratively retrieve up-to-date external knowledge, enhancing adaptability and generalization in complex question-answering tasks. However, existing search
agent pipelines typically depend on reinforcement learning based optimization, which often suffers from sparse outcome rewards, leading to inefficient exploration and unstable training. We introduce CriticSearch, a fine-grained credit-assignment framework that supplies dense, turn-level feedback via a retrospective critic mechanism. During training, a frozen, asymmetric critique LLM retrospectively evaluates each turn using privileged information from the full trajectory and gold answers, converting these assessments into stable, dense rewards that guide policy improvement. Experimental results across diverse multi-hop
reasoning benchmarks demonstrate that CriticSearch consistently outperforms existing baselines, achieving faster convergence, improved training stability, and higher performance.
Read the full abstract ▾
All authors Yaocheng Zhang, Haohuan Huang, Zijun Song, Yuanheng Zhu, Qichao Zhang, Zijie Zhao, Dongbin Zhao
Computation and Language
Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding
Pinxue Guo, Chongruo Wu, Xinyu Zhou et al. (+7 more)
Multimodal Large
Language Models (MLLMs) have unlocked powerful cross-modal capabilities, but still significantly suffer from
hallucinations. As such, accurate detection of
hallucinations in MLLMs is imperative for ensuring their reliability in practical applications. To this end, guided by the principle of "Seeing is Believing", we introduce VBackChecker, a novel reference-free
hallucination detection framework that verifies the consistency of MLLMgenerated responses with visual inputs, by leveraging a pixellevel Grounding LLM equipped with
reasoning and referring
segmentation capabilities. This reference-free framework not only effectively handles rich-context scenarios, but also offers interpretability. To facilitate this, an innovative pipeline is accordingly designed for generating instruction-tuning data (R-Instruct), featuring rich-context descriptions, grounding masks, and hard negative samples. We further establish R^2 -HalBench, a new
hallucination benchmark for MLLMs, which, unlike previous
benchmarks, encompasses real-world, rich-context descriptions from 18 MLLMs with high-quality annotations, spanning diverse object-, attribute, and relationship-level details. VBackChecker outperforms prior complex frameworks and achieves state-of-the-art performance on R^2 -HalBench, even rivaling GPT-4o's capabilities in
hallucination detection. It also surpasses prior methods in the pixel-level grounding task, achieving over a 10% improvement. All codes, data, and models are available at https://github.com/PinxueGuo/VBackChecker.
Read the full abstract ▾
All authors Pinxue Guo, Chongruo Wu, Xinyu Zhou, Lingyi Hong, Zhaoyu Chen, Jinglun Li, Kaixun Jiang, Sen-ching Samson Cheung, Wei Zhang, Wenqiang Zhang
Computation and Language
AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing
Qingyu Zhang, Chunlei Xin, Xuanang Chen et al. (+7 more)
Goal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a significant challenge for even state-of-the-art Large
Language Models (LLMs). A lack of task-specific data often limits previous works, and direct LLM application suffers from strategic brittleness and factual
hallucination. In this paper, we first construct and release TeleSalesCorpus, the first real-world-grounded dialogue
dataset for this domain. We then propose AI-Salesman, a novel framework featuring a dual-stage architecture. For the training stage, we design a
Bayesian-supervised reinforcement learning
algorithm that learns robust sales strategies from noisy dialogues. For the inference stage, we introduce the Dynamic Outline-Guided
Agent (DOGA), which leverages a pre-built script library to provide dynamic, turn-by-turn strategic guidance. Moreover, we design a comprehensive evaluation framework that combines fine-grained metrics for key sales skills with the LLM-as-a-Judge paradigm. Experimental results demonstrate that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, showcasing its effectiveness in complex persuasive scenarios.
Read the full abstract ▾
All authors Qingyu Zhang, Chunlei Xin, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Qing Ye, Qianlong Xie, Xingxing Wang
Computation and Language
PRISM of Opinions: A Persona-Reasoned Multimodal Framework for User-centric Conversational Stance Detection
Bingbing Wang, Zhixin Bai, Zhengda Jin et al. (+6 more)
The rapid proliferation of
multimodal social media content has driven research in
Multimodal Conversational Stance Detection (MCSD), which aims to interpret users' attitudes toward specific targets within complex discussions. However, existing studies remain limited by: **1) pseudo-multimodality**, where visual cues appear only in source posts while comments are treated as text-only, misaligning with real-world
multimodal interactions; and **2) user homogeneity**, where diverse users are treated uniformly, neglecting personal traits that shape stance expression. To address these issues, we introduce **U-MStance**, the first user-centric MCSD
dataset, containing over 40k annotated comments across six real-world targets. We further propose **PRISM**, a **P**ersona-**R**easoned mult**I**modal **S**tance **M**odel for MCSD. PRISM first derives longitudinal user personas from historical posts and comments to capture individual traits, then aligns textual and visual cues within conversational context via Chain-of-Thought to bridge semantic and pragmatic gaps across modalities. Finally, a mutual task reinforcement mechanism is employed to jointly optimize stance detection and stance-aware response generation for bidirectional knowledge transfer. Experiments on U-MStance demonstrate that PRISM yields significant gains over strong baselines, underscoring the effectiveness of user-centric and context-grounded
multimodal reasoning for realistic stance understanding.
Read the full abstract ▾
All authors Bingbing Wang, Zhixin Bai, Zhengda Jin, Zihan Wang, Xintong Song, Jingjie Lin, Sixuan Li, Jing Li, Ruifeng Xu
Computation and Language
LLMLagBench: Identifying Temporal Training Boundaries in Large Language Models
Piotr Pęzik, Konrad Kaczyński, Maria Szymańska et al. (+4 more)
Large
Language Models (LLMs) are pretrained on textual data up to a specific temporal cutoff. This creates a strict knowledge boundary beyond which models cannot provide accurate information without querying external sources. More subtly, when this limitation is unknown or ignored, LLMs may inadvertently blend outdated time-sensitive information with general knowledge during
reasoning tasks, potentially compromising response accuracy. We introduce LLMLagBench, an LLM freshness
benchmark, as a systematic approach for identifying the earliest probable temporal boundaries of an LLM's training data by evaluating its knowledge of recent events. We then apply this
benchmark to evaluate a large set of LLMs, including models with both explicitly declared and undeclared training cutoffs. The reliability of the
benchmark is assessed by manual validation and comparison with publicly released information about LLM pretraining.
Read the full abstract ▾
All authors Piotr Pęzik, Konrad Kaczyński, Maria Szymańska, Filip Żarnecki, Zuzanna Deckert, Jakub Kwiatkowski, Wojciech Janowski
Computation and Language
Exploring Parameter-Efficient Fine-Tuning and Backtranslation for the WMT 25 General Translation Task
Felipe Fujita and Hideyuki Takada
In this paper, we explore the effectiveness of combining fine-tuning and backtranslation on a small Japanese corpus for neural machine translation. Starting from a baseline English{\textrightarrow}Japanese model (COMET = 0.460), we first apply backtranslation (BT) using synthetic data generated from monolingual Japanese corpora, yielding a modest increase (COMET = 0.468). Next, we fine-tune (FT) the model on a genuine small parallel
dataset drawn from diverse Japanese news and literary corpora, achieving a substantial jump to COMET = 0.589 when using Mistral 7B. Finally, we integrate both backtranslation and fine-tuning{ – }first augmenting the small
dataset with BT generated examples, then adapting via FT{ – }which further boosts performance to COMET = 0.597. These results demonstrate that, even with limited training data, the synergistic use of backtranslation and targeted fine-tuning on Japanese corpora can significantly enhance translation quality, outperforming each technique in isolation. This approach offers a lightweight yet powerful strategy for improving low-resource
language pairs.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CE
Preference Learning from Physics-Based Feedback: Tuning Language Models to Design BCC/B2 Superalloys
Satanu Ghosh, Collin Holgate, Neal R. Brodnik et al. (+4 more)
We apply preference learning to the task of
language model-guided design of novel structural
alloys. In contrast to prior work that focuses on generating stable inorganic
crystals, our approach targets the synthesizeability of a specific structural class: BCC/B2 superalloys, an underexplored family of materials with potential applications in extreme environments. Using three open-weight models (LLaMA-3.1, Gemma-2, and OLMo-2), we demonstrate that
language models can be optimized for multiple design objectives using a single, unified reward signal through Direct Preference Optimization (DPO). Unlike prior approaches that rely on heuristic or human-in-the-loop feedback (costly), our reward signal is derived from thermodynamic phase calculations, offering a scientifically grounded criterion for model tuning. To our knowledge, this is the first demonstration of preference-tuning a
language model using physics-grounded feedback for structural
alloy design. The resulting framework is general and extensible, providing a path forward for intelligent design-space exploration across a range of physical science domains.
Read the full abstract ▾
All authors Satanu Ghosh, Collin Holgate, Neal R. Brodnik, Doug Downey, Samantha Daly, Tresa M. Pollock, Samuel Carton
Computation and Language
CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
Truong Vo and Sanmi Koyejo
Large
language models (LLMs) are increasingly deployed in culturally diverse environments, yet existing evaluations of cultural competence remain limited. Existing methods focus on de-contextualized correctness or forced-choice judgments, overlooking the need for cultural understanding and
reasoning required for appropriate responses. To address this gap, we introduce a set of
benchmarks that, instead of directly probing abstract norms or isolated statements, present models with realistic situational contexts that require culturally grounded
reasoning. In addition to the standard Exact Match metric, we introduce four complementary metrics (Coverage, Specificity, Connotation, and Coherence) to capture different dimensions of model's response quality. Empirical analysis across frontier models reveals that thin evaluation systematically overestimates cultural competence and produces unstable assessments with high variance. In contrast, thick evaluation exposes differences in
reasoning depth, reduces variance, and provides more stable, interpretable signals of cultural understanding.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CY
Leveraging Large Language Models for Career Mobility Analysis: A Study of Gender, Race, and Job Change Using U.S. Online Resume Profiles
Palakorn Achananuparp, Ye Xu, Yao Lu et al. (+2 more)
We present a large-scale analysis of career mobility of college-educated U.S. workers using online resume profiles to investigate how gender, race, and job change options are associated with upward mobility. This study addresses key research questions of how the job changes affect their upward career mobility, and how the outcomes of upward career mobility differ by gender and race. We address data challenges – such as missing demographic attributes, missing wage data, and noisy occupation labels – through various data processing and Artificial Intelligence (AI) methods. In particular, we develop a large
language models (LLMs) based occupation classification method known as FewSOC that achieves accuracy significantly higher than the original occupation labels in the resume
dataset. Analysis of 228,710 career trajectories reveals that intra-firm occupation change has been found to facilitate upward mobility most strongly, followed by inter-firm occupation change and inter-firm lateral move. Women and Black college graduates experience significantly lower returns from job changes than men and White peers. Multilevel sensitivity analyses confirm that these disparities are robust to cluster-level heterogeneity and reveal additional intersectional patterns.
Read the full abstract ▾
All authors Palakorn Achananuparp, Ye Xu, Yao Lu, Xavier Jayaraj Siddarth Ashok, Ee-Peng Lim
Computation and Language
Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations
Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan et al. (+4 more)
Explanations are often promoted as tools for transparency, but they can also foster confirmation bias; users may assume
reasoning is correct whenever outputs appear acceptable. We study this double-edged role of Chain-of-Thought (CoT) explanations in
multimodal moral scenarios by systematically perturbing
reasoning chains and manipulating delivery tones. Specifically, we analyze
reasoning errors in
vision language models (VLMs) and how they impact user trust and the ability to detect errors. Our findings reveal two key effects: (1) users often equate trust with outcome agreement, sustaining reliance even when
reasoning is flawed, and (2) the confident tone suppresses error detection while maintaining reliance, showing that delivery styles can override correctness. These results highlight how CoT explanations can simultaneously clarify and mislead, underscoring the need for NLP systems to provide explanations that encourage scrutiny and critical thinking rather than blind trust. All code will be released publicly.
Read the full abstract ▾
All authors Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan, Mingxi Yan, Gunhee Kim, Maarten Sap, Motahhare Eslami
Computation and Language
A Reasoning Paradigm for Named Entity Recognition
Hui Huang, Yanping Chen, Ruizhang Huang et al. (+2 more)
Generative LLMs typically improve Named Entity Recognition (NER) performance through instruction tuning. They excel at generating entities by semantic pattern matching but lack an explicit, verifiable
reasoning mechanism. This "cognitive shortcutting" leads to suboptimal performance and brittle generalization, especially in zero-shot and lowresource scenarios where
reasoning from limited contextual cues is crucial. To address this issue, a
reasoning framework is proposed for NER, which shifts the extraction paradigm from implicit pattern matching to explicit
reasoning. This framework consists of three stages: Chain of Thought (CoT) generation, CoT tuning, and
reasoning enhancement. First, a
dataset annotated with NER-oriented CoTs is generated, which contain task-relevant
reasoning chains. Then, they are used to tune the NER model to generate coherent rationales before deriving the final answer. Finally, a
reasoning enhancement stage is implemented to optimize the
reasoning process using a comprehensive reward signal. This stage ensures explicit and verifiable extractions. Experiments show that ReasoningNER demonstrates impressive cognitive ability in the NER task, achieving competitive performance. In zero-shot settings, it achieves state-of-the-art (SOTA) performance, outperforming GPT-4 by 12.3 percentage points on the F1 score. Analytical results also demonstrate its great potential to advance research in reasoningoriented information extraction. Our codes are available at https://github.com/HuiResearch/ReasoningIE.
Read the full abstract ▾
All authors Hui Huang, Yanping Chen, Ruizhang Huang, Chuan Lin, Yongbin Qin
Computation and Language
Steven Cao, Gregory Valiant and Percy Liang
We study the problem of
entropy calibration, which asks whether a
language model's
entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with
entropy per step increasing as generations grow longer, due to error accumulation. To calibrate the model and improve text quality, it has become standard practice to truncate the distribution, but this approach reduces output diversity, which we would like to avoid. Therefore, in this paper, we ask: does miscalibration improve automatically with scale, and if not, is it theoretically possible to calibrate without tradeoffs? To build intuition, we first study a simplified theoretical setting to characterize the scaling behavior of miscalibration with respect to
dataset size. We find that the rate of scaling depends on the power law exponent of the data distribution – in particular, for a power law exponent close to 1, the scaling exponent is close to 0, meaning that miscalibration improves very slowly with scale. Next, we measure miscalibration empirically in
language models ranging from 0.5B to 70B parameters. We find that the observed scaling behavior is similar to what is predicted theoretically: our fitted scaling exponents for text are close to 0, meaning that larger models accumulate error at a similar rate as smaller ones. This scaling (or, lack thereof) provides one explanation for why we sample from larger models with similar amounts of truncation as smaller models, even though the larger models are of higher quality. However, truncation is not a satisfying solution because it comes at the cost of increased log loss. In theory, is it even possible to reduce
entropy while preserving log loss? We prove that it is possible, if we assume access to a black box which can fit models to predict the future
entropy of text.
Read the full abstract ▾