Computation and Language
MajinBook: An open catalogue of digitally mediated world literature
Antoine Mazières and Thierry Poibeau
This data paper introduces MajinBook, an open catalogue designed to facilitate the use of shadow libraries-such as Library Genesis and Z-Library-for computational social science and cultural analytics. By linking metadata from these vast, crowd-sourced archives with structured bibliographic data from Goodreads, we create a high-precision corpus of over 539,000 references to digitally mediated English-
language books. Spanning three centuries and reflecting a contemporary selection bias, these entries are enriched with first publication dates, genres, and popularity metrics like ratings and reviews. Our methodology prioritises natively digital EPUB files to ensure machine-readable quality, while addressing biases in traditional corpora like HathiTrust, and includes secondary
datasets for French, German, and Spanish. We evaluate the linkage strategy for accuracy, release all underlying data openly, and discuss the project's legal permissibility under EU and US frameworks for text and data mining in research.
Read the full abstract ▾
Computation and Language
Studies with impossible languages falsify LMs as models of human language
Jeffrey S. Bowers and Jeff Mitchell
According to Futrell and Mahowald [arXiv:2501.17047], both infants and
language models (LMs) find attested
languages easier to learn than impossible
languages that have unnatural structures. We review the literature and show that LMs often learn attested and many impossible
languages equally well. Difficult to learn impossible
languages are simply more complex (or random). LMs are missing human inductive biases that support
language acquisition.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
On-Device Fine-Tuning via Backprop-Free Zeroth-Order Optimization
Prabodh Katti, Houssem Sifaou, Sangwoo Park et al. (+2 more)
On-device fine-tuning is a critical capability for edge AI systems, which must support adaptation to different agentic tasks under stringent
memory constraints. Conventional backpropagation (BP)-based training requires storing layer activations and optimizer states, a demand that can be only partially alleviated through checkpointing. In edge deployments in which the model weights must reside entirely in device
memory, this overhead severely limits the maximum model size that can be deployed.
Memory-efficient zeroth-order optimization (MeZO) alleviates this bottleneck by estimating gradients using forward evaluations alone, eliminating the need for storing intermediate activations or optimizer states. This enables significantly larger models to fit within on-chip
memory, albeit at the cost of potentially longer fine-tuning wall-clock time. This paper first provides a theoretical estimate of the relative model sizes that can be accommodated under BP and MeZO training. We then numerically validate the analysis, demonstrating that MeZO exhibits accuracy advantages under on-device
memory constraints, provided sufficient wall-clock time is available for fine-tuning.
Read the full abstract ▾
All authors Prabodh Katti, Houssem Sifaou, Sangwoo Park, Bipin Rajendran, Osvaldo Simeone
Computation and Language
M-DAIGT: A Shared Task on Multi-Domain Detection of AI-Generated Text
Salima Lamsiyah, Saad Ezzini, Abdelkader El Mahdaouy et al. (+5 more)
The generation of highly fluent text by Large
Language Models (LLMs) poses a significant challenge to information integrity and academic research. In this paper, we introduce the Multi-Domain Detection of AI-Generated Text (M-DAIGT) shared task, which focuses on detecting AI-generated text across multiple domains, particularly in news articles and academic writing. M-DAIGT comprises two binary classification subtasks: News Article Detection (NAD) (Subtask 1) and Academic Writing Detection (AWD) (Subtask 2). To support this task, we developed and released a new large-scale
benchmark dataset of 30,000 samples, balanced between human-written and AI-generated texts. The AI-generated content was produced using a variety of modern LLMs (e.g., GPT-4, Claude) and diverse prompting strategies. A total of 46 unique teams registered for the shared task, of which four teams submitted final results. All four teams participated in both Subtask 1 and Subtask 2. We describe the methods employed by these participating teams and briefly discuss future directions for M-DAIGT.
Read the full abstract ▾
All authors Salima Lamsiyah, Saad Ezzini, Abdelkader El Mahdaouy, Hamza Alami, Abdessamad Benlahbib, Samir El Amrany, Salmane Chafik, Hicham Hammouchi
Computation and Language
LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models
Jian Gao, Richeng Xuan, Zhaolu Kang et al. (+9 more)
The rapid advancement of large
language models (LLMs) has not been matched by their evaluation in low-resource
languages, especially Southeast Asian
languages like Lao. To fill this gap, we introduce LaoBench, the first large-scale, high-quality, and multidimensional
benchmark for assessing LLM
language understanding and
reasoning in Lao. LaoBench contains 17,000+ expert-curated samples across three dimensions: culturally grounded knowledge application, curriculum-aligned K12 education, and bilingual translation among Lao, Chinese, and English. It includes open-source and held-out subsets, where the held-out portion enables secure black-box evaluation via a controlled service to improve
fairness and data security. We construct LaoBench with a hybrid pipeline that combines expert authoring with
agent-assisted verification, ensuring linguistic accuracy, cultural relevance, and educational validity. We evaluate diverse state-of-the-art open-source and closed-source LLMs, and find that even strong
multilingual models lag behind human experts, particularly in culturally grounded
reasoning and translation fidelity. We hope LaoBench will catalyze research on Lao and other underrepresented Southeast Asian
languages for more inclusive
multilingual evaluation.
Read the full abstract ▾
All authors Jian Gao, Richeng Xuan, Zhaolu Kang, Dingshi Liao, Wenxin Huang, Zongmou Huang, Yangdi Xu, Bowen Qin, Zheqi He, Xi Yang, Changjin Li, Yonghua Lin
Computation and Language
NOVA: An Agentic Framework for Automated Histopathology Analysis and Discovery
Anurag J. Vaidya, Felix Meissen, Daniel C. Castro et al. (+7 more)
Digitized histopathology analysis involves complex, time-intensive workflows and specialized expertise, limiting its accessibility. We introduce NOVA, an agentic framework that translates scientific queries into executable analysis pipelines by iteratively generating and running Python code. NOVA integrates 49 domain-specific tools (e.g.,
nuclei segmentation, whole-slide encoding) built on open-source software, and can also create new tools ad hoc. To evaluate such systems, we present SlideQuest, a 90-question
benchmark – verified by pathologists and biomedical scientists – spanning data processing, quantitative analysis, and hypothesis testing. Unlike prior biomedical
benchmarks focused on knowledge recall or diagnostic QA, SlideQuest demands multi-step
reasoning, iterative coding, and computational problem solving. Quantitative evaluation shows NOVA outperforms coding-
agent baselines, and a pathologist-verified case study links morphology to prognostically relevant PAM50 subtypes, demonstrating its scalable discovery potential.
Read the full abstract ▾
All authors Anurag J. Vaidya, Felix Meissen, Daniel C. Castro, Shruthi Bannur, Tristan Lazard, Drew F. K. Williamson, Faisal Mahmood, Javier Alvarez-Valle, Stephanie L. Hyland, Kenza Bouzid
Computation and Language
LAET: A Layer-wise Adaptive Ensemble Tuning Framework for Pretrained Language Models
Jawad Ibn Ahad, Muhammad Rafsan Kabir, Robin Krambroeckers et al. (+3 more)
Natural
Language Processing (NLP) has transformed the financial industry, enabling advancements in areas such as textual analysis, risk management, and
forecasting. Large
language models (LLMs) like BloombergGPT and FinMA have set new
benchmarks across various financial NLP tasks, including sentiment analysis, stock movement prediction, and credit risk assessment. Furthermore, FinMA-ES, a bilingual financial LLM, has also demonstrated strong performance using the
FLARE and
FLARE-ES
benchmarks. However, the high computational demands of these models limit the accessibility of many organizations. To address this, we propose Layer-wise Adaptive Ensemble Tuning (LAET), a novel strategy that selectively fine-tunes the most effective layers of pre-trained LLMs by analyzing hidden state representations while freezing less critical layers. LAET significantly reduces computational overhead while enhancing task-specific performance. Our approach shows strong results in financial NLP tasks, outperforming existing
benchmarks and state-of-the-art LLMs such as GPT-4, even with smaller LLMs ($\sim$3B parameters). This work bridges cutting-edge financial NLP research and real-world deployment with efficient and scalable models for financial applications.
Read the full abstract ▾
All authors Jawad Ibn Ahad, Muhammad Rafsan Kabir, Robin Krambroeckers, Sifat Momen, Nabeel Mohammed, Shafin Rahman
Computation and Language
iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference
Wei Fan, JinYi Yoon and Bo Ji
Large
Language Model (LLM)
agent systems have advanced rapidly, driven by their strong generalization in zero-shot settings. To further enhance
reasoning and accuracy on complex tasks, Multi-
Agent Debate (MAD) has emerged as a promising framework that engages multiple LLM
agents in structured debates to encourage diverse
reasoning. However, triggering MAD for every query is inefficient, as it incurs substantial computational (
token) cost and may even degrade accuracy by overturning correct single-
agent answers. To address these limitations, we propose intelligent Multi-
Agent Debate (iMAD), a
token-efficient framework that selectively triggers MAD only when it is likely to be beneficial (i.e., correcting an initially wrong answer). To achieve this goal, iMAD learns generalizable model behaviors to make accurate debate decisions. Specifically, iMAD first
prompts a single
agent to produce a structured self-critique response, from which we extract 41 interpretable linguistic and semantic features capturing hesitation cues. Then, iMAD uses a lightweight debate-decision classifier, trained using our proposed FocusCal loss, to determine whether to trigger MAD, enabling robust debate decisions without test
dataset-specific tuning. Through extensive experiments using six (visual) question answering
datasets against five competitive baselines, we have shown that iMAD significantly reduces
token usage (by up to 92%) while also improving final answer accuracy (by up to 13.5%).
Read the full abstract ▾
Computation and Language · Cross-listed from cs.HC
Building the Web for Agents: A Declarative Framework for Agent-Web Interaction
Sven Schultze, Meike Verena Kietzmann, Nils-Lucas Schönfeld et al. (+1 more)
The increasing deployment of autonomous AI
agents on the web is hampered by a fundamental misalignment:
agents must infer affordances from human-oriented user interfaces, leading to brittle, inefficient, and insecure interactions. To address this, we introduce VOIX, a web-native framework that enables websites to expose reliable, auditable, and
privacy-preserving capabilities for AI
agents through simple, declarative HTML elements. VOIX introduces <tool> and <context> tags, allowing developers to explicitly define available actions and relevant state, thereby creating a clear, machine-readable contract for
agent behavior. This approach shifts control to the website developer while preserving user
privacy by disconnecting the conversational interactions from the website. We evaluated the framework's practicality, learnability, and expressiveness in a three-day hackathon study with 16 developers. The results demonstrate that participants, regardless of prior experience, were able to rapidly build diverse and functional
agent-enabled web applications. Ultimately, this work provides a foundational mechanism for realizing the Agentic Web, enabling a future of seamless and secure human-AI collaboration on the web.
Read the full abstract ▾
All authors Sven Schultze, Meike Verena Kietzmann, Nils-Lucas Schönfeld, Ruth Stock-Homburg
Computation and Language · Cross-listed from eess.SY
Language-Aided State Estimation
Yuki Miyoshi, Masaki Inoue and Yusuke Fujimoto
Natural
language data, such as text and
speech, have become readily available through social networking services and chat platforms. By leveraging human observations expressed in natural
language, this paper addresses the problem of state estimation for physical systems, in which humans act as sensing
agents. To this end, we propose a
Language-Aided Particle Filter (LAPF), a particle filter framework that structures human observations via natural
language processing and incorporates them into the update step of the state estimation. Finally, the LAPF is applied to the water level estimation problem in an irrigation canal and its effectiveness is demonstrated.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
SQuaD: The Software Quality Dataset
Mikel Robredo, Matteo Esposito, Davide Taibi et al. (+2 more)
Software quality research increasingly relies on large-scale
datasets that measure both the product and process aspects of software systems. However, existing resources often focus on limited dimensions, such as code smells, technical debt, or refactoring activity, thereby restricting comprehensive analyses across time and quality dimensions. To address this gap, we present the Software Quality
Dataset (SQuaD), a multi-dimensional, time-aware collection of software quality metrics extracted from 450 mature open-source projects across diverse ecosystems, including Apache, Mozilla, FFmpeg, and the Linux kernel. By integrating nine state-of-the-art static analysis tools, i.e., SonarQube, CodeScene, PMD, Understand, CK, JaSoMe, RefactoringMiner, RefactoringMiner++, and PyRef, our
dataset unifies over 700 unique metrics at method, class, file, and project levels. Covering a total of 63,586 analyzed project releases, SQuaD also provides version control and issue-tracking histories, software vulnerability data (CVE/CWE), and process metrics proven to enhance Just-In-Time (JIT) defect prediction. The SQuaD enables empirical research on maintainability, technical debt, software
evolution, and quality assessment at unprecedented scale. We also outline emerging research directions, including automated
dataset updates and cross-project quality modeling to support the continuous
evolution of software analytics. The
dataset is publicly available on ZENODO (DOI: 10.5281/zenodo.17566690).
Read the full abstract ▾
All authors Mikel Robredo, Matteo Esposito, Davide Taibi, Rafael Peñaloza, Valentina Lenarduzzi
Computation and Language · Cross-listed from cs.CV
Discovering Meaningful Units with Visually Grounded Semantics from Image Captions
Melika Behjati and James Henderson
Fine-grained knowledge is crucial for
vision-
language models to obtain a better understanding of the real world. While there has been work trying to acquire this kind of knowledge in the space of
vision and
language, it has mostly focused on aligning the image patches with the
tokens on the
language side. However, image patches do not have any meaning to the human eye, and individual
tokens do not necessarily carry groundable information in the image. It is groups of
tokens which describe different aspects of the scene. In this work, we propose a model which groups the caption
tokens as part of its architecture in order to capture a fine-grained representation of the
language. We expect our representations to be at the level of objects present in the image, and therefore align our representations with the output of an image encoder trained to discover objects. We show that by learning to group the
tokens, the
vision-
language model has a better fine-grained understanding of
vision and
language. In addition, the
token groups that our model discovers are highly similar to groundable phrases in text, both qualitatively and quantitatively.
Read the full abstract ▾
Computation and Language
KGQuest: Template-Driven QA Generation from Knowledge Graphs with LLM-Based Refinement
Sania Nayab, Marco Simoni, Giulio Rossolini et al. (+1 more)
The generation of questions and answers (QA) from knowledge graphs (KG) plays a crucial role in the development and testing of educational platforms, dissemination tools, and large
language models (LLM). However, existing approaches often struggle with scalability, linguistic quality, and factual consistency. This paper presents a scalable and deterministic pipeline for generating natural
language QA from KGs, with an additional refinement step using LLMs to further enhance linguistic quality. The approach first clusters KG triplets based on their relations, creating reusable templates through natural
language rules derived from the entity types of objects and relations. A module then leverages LLMs to refine these templates, improving clarity and coherence while preserving factual accuracy. Finally, the instantiation of answer options is achieved through a selection strategy that introduces distractors from the KG. Our experiments demonstrate that this hybrid approach efficiently generates high-quality QA pairs, combining scalability with fluency and linguistic precision.
Read the full abstract ▾
All authors Sania Nayab, Marco Simoni, Giulio Rossolini, Andrea Saracino
Computation and Language
LANE: Lexical Adversarial Negative Examples for Word Sense Disambiguation
Jader Martins Camboim de Sá, Jooyoung Lee, Cédric Pruski et al. (+1 more)
Fine-grained word meaning resolution remains a critical challenge for neural
language models (NLMs) as they often overfit to global sentence representations, failing to capture local semantic details. We propose a novel
adversarial training strategy, called LANE, to address this limitation by deliberately shifting the model's learning focus to the target word. This method generates challenging negative training examples through the selective marking of alternate words in the training set. The goal is to force the model to create a greater separability between same sentences with different marked words. Experimental results on lexical semantic change detection and word sense disambiguation
benchmarks demonstrate that our approach yields more discriminative word representations, improving performance over standard contrastive learning baselines. We further provide qualitative analyses showing that the proposed negatives lead to representations that better capture subtle meaning differences even in challenging environments. Our method is model-agnostic and can be integrated into existing representation learning frameworks.
Read the full abstract ▾
All authors Jader Martins Camboim de Sá, Jooyoung Lee, Cédric Pruski, Marcos Da Silveira
Computation and Language
Adverbs Revisited: Enhancing WordNet Coverage of Adverbs with a Supersense Taxonomy
Jooyoung Lee and Jader Martins Camboim de Sá
WordNet offers rich supersense hierarchies for nouns and verbs, yet adverbs remain underdeveloped, lacking a systematic semantic classification. We introduce a linguistically grounded supersense typology for adverbs, empirically validated through annotation, that captures major semantic domains including manner, temporal, frequency, degree, domain, speaker-oriented, and subject-oriented functions. Results from a pilot annotation study demonstrate that these categories provide broad coverage of adverbs in natural text and can be reliably assigned by human annotators. Incorporating this typology extends WordNet's coverage, aligns it more closely with linguistic theory, and facilitates downstream NLP applications such as word sense disambiguation, event extraction, sentiment analysis, and discourse modeling. We present the proposed supersense categories, annotation outcomes, and directions for future work.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Dayong Liang, Xiao-Yong Wei, Linhui Xiao et al. (+1 more)
Hallucination continues to pose a major obstacle in the
reasoning capabilities of large
language models (LLMs). Although the Multi-
Agent Debate (MAD) paradigm offers a promising solution by promoting consensus among multiple
agents to enhance reliability, it relies on the unrealistic assumption that all debaters are rational and reflective, which is a condition that may not hold when
agents themselves are prone to
hallucinations. To address this gap, we introduce the Multi-
agent Undercover Gaming (MUG) protocol, inspired by social deduction games like "Who is Undercover?". MUG reframes MAD as a process of detecting "undercover"
agents (those suffering from
hallucinations) by employing
multimodal counterfactual tests. Specifically, we modify reference images to introduce counterfactual evidence and observe whether
agents can accurately identify these changes, providing ground-truth for identifying hallucinating
agents and enabling robust, crowd-powered
multimodal reasoning. MUG advances MAD protocols along three key dimensions: (1) enabling factual verification beyond statistical consensus through counterfactual testing; (2) introducing cross-evidence
reasoning via dynamically modified evidence sources instead of relying on static inputs; and (3) fostering active
reasoning, where
agents engage in probing discussions rather than passively answering questions. Collectively, these innovations offer a more reliable and effective framework for
multimodal reasoning in LLMs. The source code can be accessed at https://github.com/YongLD/MUG.git.
Read the full abstract ▾
All authors Dayong Liang, Xiao-Yong Wei, Linhui Xiao, Changmeng Zheng
Computation and Language
PRSM: A Measure to Evaluate CLIP's Robustness Against Paraphrases
Udo Schlegel, Franziska Weeber, Jian Lan et al. (+1 more)
Contrastive
Language-Image Pre-training (CLIP) is a widely used
multimodal model that aligns text and image representations through large-scale training. While it performs strongly on zero-shot and few-shot tasks, its robustness to linguistic variation, particularly paraphrasing, remains underexplored. Paraphrase robustness is essential for reliable deployment, especially in socially sensitive contexts where inconsistent representations can amplify demographic biases. In this paper, we introduce the Paraphrase Ranking Stability Metric (PRSM), a novel measure for quantifying CLIP's sensitivity to paraphrased queries. Using the Social Counterfactuals
dataset, a
benchmark designed to reveal social and demographic biases, we empirically assess CLIP's stability under paraphrastic variation, examine the interaction between paraphrase robustness and gender, and discuss implications for
fairness and equitable deployment of
multimodal systems. Our analysis reveals that robustness varies across paraphrasing strategies, with subtle yet consistent differences observed between male- and female-associated queries.
Read the full abstract ▾
All authors Udo Schlegel, Franziska Weeber, Jian Lan, Thomas Seidl
Computation and Language
Speech-Aware Long Context Pruning and Integration for Contextualized Automatic Speech Recognition
Yiming Rong, Yixin Zhang, Ziyi Wang et al. (+5 more)
Automatic
speech recognition (ASR) systems have achieved remarkable performance in common conditions but often struggle to leverage long-context information in contextualized scenarios that require domain-specific knowledge, such as conference presentations. This challenge arises primarily due to constrained model context windows and the sparsity of relevant information within extensive contextual noise. To solve this, we propose the SAP$^{2}$ method, a novel framework that dynamically prunes and integrates relevant contextual keywords in two stages. Specifically, each stage leverages our proposed
Speech-Driven
Attention-based Pooling mechanism, enabling efficient compression of context
embeddings while preserving
speech-salient information. Experimental results demonstrate state-of-the-art performance of SAP$^{2}$ on the SlideSpeech and LibriSpeech
datasets, achieving word error rates (WER) of 7.71% and 1.12%, respectively. On SlideSpeech, our method notably reduces biased keyword error rates (B-WER) by 41.1% compared to non-contextual baselines. SAP$^{2}$ also exhibits robust scalability, consistently maintaining performance under extensive contextual input conditions on both
datasets.
Read the full abstract ▾
All authors Yiming Rong, Yixin Zhang, Ziyi Wang, Deyang Jiang, Yunlong Zhao, Haoran Wu, Shiyu Zhou, Bo Xu
Computation and Language
Enhancing Meme Emotion Understanding with Multi-Level Modality Enhancement and Dual-Stage Modal Fusion
Yi Shi, Wenlong Meng, Zhenyuan Guo et al. (+2 more)
With the rapid rise of social media and Internet culture, memes have become a popular medium for expressing emotional tendencies. This has sparked growing interest in Meme
Emotion Understanding (MEU), which aims to classify the emotional intent behind memes by leveraging their
multimodal contents. While existing efforts have achieved promising results, two major challenges remain: (1) a lack of fine-grained
multimodal fusion strategies, and (2) insufficient mining of memes' implicit meanings and background knowledge. To address these challenges, we propose MemoDetector, a novel framework for advancing MEU. First, we introduce a four-step textual enhancement module that utilizes the rich knowledge and
reasoning capabilities of
Multimodal Large
Language Models (MLLMs) to progressively infer and extract implicit and contextual insights from memes. These enhanced texts significantly enrich the original meme contents and provide valuable guidance for downstream classification. Next, we design a dual-stage modal fusion strategy: the first stage performs shallow fusion on raw meme image and text, while the second stage deeply integrates the enhanced visual and textual features. This hierarchical fusion enables the model to better capture nuanced cross-modal emotional cues. Experiments on two
datasets, MET-MEME and MOOD, demonstrate that our method consistently outperforms state-of-the-art baselines. Specifically, MemoDetector improves F1 scores by 4.3% on MET-MEME and 3.4% on MOOD. Further ablation studies and in-depth analyses validate the effectiveness and robustness of our approach, highlighting its strong potential for advancing MEU. Our code is available at https://github.com/singing-cat/MemoDetector.
Read the full abstract ▾
All authors Yi Shi, Wenlong Meng, Zhenyuan Guo, Chengkun Wei, Wenzhi Chen
Computation and Language
AV-Dialog: Spoken Dialogue Models with Audio-Visual Input
Tuochao Chen, Bandhav Veluri, Hongyu Gong et al. (+1 more)
Dialogue models falter in noisy, multi-speaker environments, often producing irrelevant responses and awkward turn-taking. We present AV-Dialog, the first
multimodal dialog framework that uses both audio and visual cues to track the target speaker, predict turn-taking, and generate coherent responses. By combining acoustic tokenization with multi-task, multi-stage training on monadic, synthetic, and real audio-visual dialogue
datasets, AV-Dialog achieves robust streaming transcription, semantically grounded turn-boundary detection and accurate responses, resulting in a natural conversational flow. Experiments show that AV-Dialog outperforms audio-only models under interference, reducing transcription errors, improving turn-taking prediction, and enhancing human-rated dialogue quality. These results highlight the power of seeing as well as hearing for speaker-aware interaction, paving the way for {spoken} dialogue
agents that perform {robustly} in real-world, noisy environments.
Read the full abstract ▾
All authors Tuochao Chen, Bandhav Veluri, Hongyu Gong, Shyamnath Gollakota
Computation and Language
Analysing Personal Attacks in U.S. Presidential Debates
Ruban Goyal, Rohitash Chandra and Sonit Singh
Personal attacks have become a notable feature of U.S. presidential debates and play an important role in shaping public perception during elections. Detecting such attacks can improve transparency in political discourse and provide insights for journalists, analysts and the public. Advances in deep learning and
transformer-based models, particularly BERT and large
language models (LLMs) have created new opportunities for automated detection of harmful
language. Motivated by these developments, we present a framework for analysing personal attacks in U.S. presidential debates. Our work involves manual annotation of debate transcripts across the 2016, 2020 and 2024 election cycles, followed by statistical and
language-model based analysis. We investigate the potential of fine-tuned
transformer models alongside general-purpose LLMs to detect personal attacks in formal political
speech. This study demonstrates how task-specific adaptation of modern
language models can contribute to a deeper understanding of political communication.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
CLARITY: Contextual Linguistic Adaptation and Accent Retrieval for Dual-Bias Mitigation in Text-to-Speech Generation
Crystal Min Hui Poon, Pai Chet Ng, Xiaoxiao Miao et al. (+4 more)
Instruction-guided text-to-
speech (TTS) research has reached a maturity level where excellent
speech generation quality is possible on demand, yet two coupled biases persist in reducing perceived quality: accent bias, where models default towards dominant phonetic patterns, and linguistic bias, a misalignment in dialect-specific lexical or cultural information. These biases are interdependent and authentic accent generation requires both accent fidelity and correctly localized text. We present CLARITY (Contextual Linguistic Adaptation and
Retrieval for Inclusive TTS sYnthesis), a backbone-agnostic framework to address both biases through dual-signal optimization. Firstly, we apply contextual linguistic adaptation to localize input text to align with the target dialect. Secondly, we propose
retrieval-augmented accent prompting (RAAP) to ensure accent-consistent
speech prompts. We evaluate CLARITY on twelve varieties of English accent via both subjective and objective analysis. Results clearly indicate that CLARITY improves accent accuracy and
fairness, ensuring higher perceptual quality output\footnote{Code and audio samples are available at https://github.com/ICT-SIT/CLARITY.
Read the full abstract ▾
All authors Crystal Min Hui Poon, Pai Chet Ng, Xiaoxiao Miao, Immanuel Jun Kai Loh, Bowen Zhang, Haoyu Song, Ian Mcloughlin
Computation and Language
Sora Kadotani, Kosuke Nishida and Kyosuke Nishida
Large
language models (LLMs) can generate fluent responses, but sometimes hallucinate facts. In this paper, we investigate whether LLMs can detect their own
hallucinations. We formulate
hallucination detection as a classification task of a sentence. We propose a framework for estimating LLMs' capability of
hallucination detection and a classification method using Chain-of-Thought (CoT) to extract knowledge from their parameters. The experimental results indicated that GPT-$3.5$ Turbo with CoT detected $58.2\%$ of its own
hallucinations. We concluded that LLMs with CoT can detect
hallucinations if sufficient knowledge is contained in their parameters.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
S2D-ALIGN: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation
Jiechao Gao, Chang Liu and Yuangang Li
Radiology Report Generation (RRG) aims to automatically generate diagnostic reports from radiology images. To achieve this, existing methods have leveraged the powerful cross-modal generation capabilities of
Multimodal Large
Language Models (MLLMs), primarily focusing on optimizing cross-modal
alignment between radiographs and reports through Supervised Fine-Tuning (SFT). However, by only performing instance-level
alignment with the image-text pairs, the standard SFT paradigm fails to establish anatomically-grounded
alignment, where the templated nature of reports often leads to sub-optimal generation quality. To address this, we propose S2D-Align, a novel SFT paradigm that establishes anatomically-grounded
alignment by leveraging auxiliary signals of varying granularities. S2D-Align implements a shallow-to-deep strategy, progressively enriching the
alignment process: it begins with the coarse radiograph-report pairing, then introduces reference reports for instance-level guidance, and ultimately utilizes key phrases to ground the generation in specific anatomical details. To bridge the different
alignment stages, we introduce a
memory-based adapter that empowers feature sharing, thereby integrating coarse and fine-grained guidance. For evaluation, we conduct experiments on the public MIMIC-CXR and IU X-Ray
benchmarks, where S2D-Align achieves state-of-the-art performance compared to existing methods. Ablation studies validate the effectiveness of our multi-stage, auxiliary-guided approach, highlighting a promising direction for enhancing grounding capabilities in complex, multi-modal generation tasks.
Read the full abstract ▾
Computation and Language
Correcting Mean Bias in Text Embeddings: A Refined Renormalization with Training-Free Improvements on MMTEB
Xingyu Ren, Youran Sun and Haoyu Liang
We find that current sentence-
embedding models produce outputs with a consistent bias: every
embedding $e$ decomposes as $\tilde e + \mu$, where the mean $\mu$ is near-identical across all sentences. We study two training-free corrections – subtracting $\mu$ directly (R1), or projecting each
embedding off the mean direction (R2) – and show, via a first-order error-propagation argument, that R2 cancels the parallel component of mean-estimation error that R1 retains. Across 38 models on the Massive
Multilingual Text
Embedding Benchmark (MMTEB)~\citep{MMTEB}, R2 yields consistent classification gains (paired $\bar t = 3.31$, 29 of 38 models with $t>2$, zero losses), and the per-model mean norm $\Vert\mu\Vert$ correlates with which models benefit most. A nine-method dose-response ablation on five models further reveals that mild single-direction removal helps, but full principal component analysis (PCA) whitening hurts every model we test, and that R2 and All-but-the-Top with depth one agree within $0.18$ pp downstream despite weak
geometric alignment between $\hat\mu$ and the centered top principal component.
Read the full abstract ▾
Computation and Language
Automata-Based Steering of Large Language Models for Diverse Structured Generation
Xiaokun Luan, Zeming Wei, Yihao Zhang et al. (+1 more)
Large
language models (LLMs) are increasingly tasked with generating structured outputs. While structured generation methods ensure validity, they often lack output diversity, a critical limitation that we confirm in our preliminary study. We propose a novel method to enhance diversity in automaton-based structured generation. Our approach utilizes automata traversal history to steer LLMs towards novel structural patterns. Evaluations show our method significantly improves structural and content diversity while maintaining comparable generation efficiency. Furthermore, we conduct a case study showcasing the effectiveness of our method in generating diverse test cases for testing open-source libraries.
Read the full abstract ▾
All authors Xiaokun Luan, Zeming Wei, Yihao Zhang, Meng Sun
Computation and Language
When Data is the Algorithm: A Systematic Study and Curation of Preference Optimization Datasets
Aladin Djuhera, Farhan Ahmed, Swanand Ravindra Kadhe et al. (+3 more)
Aligning large
language models (LLMs) is a central objective of post-training, often achieved through reward modeling and reinforcement learning methods. Among these, direct preference optimization (DPO) has emerged as a widely adopted technique that fine-tunes LLMs on preferred completions over less favorable ones. While most frontier LLMs do not disclose their curated preference pairs, the broader LLM community has released several open-source DPO
datasets, including TuluDPO, ORPO, UltraFeedback, HelpSteer, and Code-Preference-Pairs. However, systematic comparisons remain scarce, largely due to the high computational cost and the lack of rich quality annotations, making it difficult to understand how preferences were selected, which task types they span, and how well they reflect human judgment on a per-sample level. In this work, we present the first comprehensive, data-centric analysis of popular open-source DPO corpora. We leverage the Magpie framework to annotate each sample for task category, input quality, and preference reward, a reward-model-based signal that validates the preference order without relying on human annotations. This enables a scalable, fine-grained inspection of preference quality across
datasets, revealing structural and qualitative discrepancies in reward margins. Building on these insights, we systematically curate a new DPO mixture, UltraMix, that draws selectively from all five corpora while removing noisy or redundant samples. UltraMix is 30% smaller than the best-performing individual
dataset yet exceeds its performance across key
benchmarks. We publicly release all annotations, metadata, and our curated mixture to facilitate future research in data-centric preference optimization.
Read the full abstract ▾
All authors Aladin Djuhera, Farhan Ahmed, Swanand Ravindra Kadhe, Syed Zawad, Heiko Ludwig, Holger Boche
Computation and Language
DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains
Xiying Zhao, Zhoufutu Wen, Zhixuan Chen et al. (+20 more)
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. While these translations demand discourse-level coherence and strict terminological precision, current evaluation methods predominantly focus on segment-level accuracy and fluency. To address this limitation, we introduce DiscoX, a new
benchmark for discourse-level and expert-level Chinese-English translation. It comprises 200 professionally-curated texts from 7 domains, with an average length exceeding 1700
tokens. To evaluate performance on DiscoX, we also develop Metric-S, a reference-free system that provides fine-grained automatic assessments across accuracy, fluency, and appropriateness. Metric-S demonstrates strong consistency with human judgments, significantly outperforming existing metrics. Our experiments reveal a remarkable performance gap: even the most advanced LLMs still trail human experts on these tasks. This finding validates the difficulty of DiscoX and underscores the challenges that remain in achieving professional-grade machine translation. The proposed
benchmark and evaluation system provide a robust framework for more rigorous evaluation, facilitating future advancements in LLM-based translation.
Read the full abstract ▾
All authors Xiying Zhao, Zhoufutu Wen, Zhixuan Chen, Jingzhe Ding, Jianpeng Jiao, Shuai Li, Xi Li, Danni Liang, Shengda Long, Qianqian Liu, Xianbo Wu, Hongwan Gao, Xiang Gao, Liang Hu, Jiashuo Liu, Mengyun Liu, Weiran Shi, Chenghao Yang, Qianyu Yang, Xuanliang Zhang, Ge Zhang, Wenhao Huang, Yuwen Tang
Computation and Language
CardioEmbed: Domain-Specialized Text Embeddings for Clinical Cardiology
Richard J. Young and Alice M. Matthews
Biomedical text
embeddings have primarily been developed using research literature from PubMed, yet clinical cardiology practice relies heavily on procedural knowledge and specialized terminology found in comprehensive textbooks rather than research abstracts. This research practice gap limits the effectiveness of existing
embedding models for clinical applications incardiology. This study trained CardioEmbed, a domain-specialized
embedding model based on Qwen3-
Embedding-8B, using contrastive learning on a curated corpus of seven comprehensive cardiology textbooks totaling approximately 150,000 sentences after deduplication. The model employs InfoNCE loss with in-batch negatives and achieves 99.60%
retrieval accuracy on cardiac-specific semantic
retrieval tasks, a +15.94 percentage point improvement over MedTE, the current state-of-the-art medical
embedding model. On MTEB medical
benchmarks, the model obtained BIOSSES 0.77 Spearman and SciFact 0.61 NDCG@10, indicating competitive performance on related biomedical domains. Domain-specialized training on comprehensive clinical textbooks yields near-perfect cardiology
retrieval (99.60% Acc@1), improving over MedTE by +15.94 percentage points.
Read the full abstract ▾
Computation and Language
Evaluating Large Language Models on Rare Disease Diagnosis: A Case Study using House M.D
Arsh Gupta, Ajay Narayanan Sridhar, Bonam Mingole et al. (+1 more)
Large
language models (LLMs) have demonstrated capabilities across diverse domains, yet their performance on rare disease diagnosis from narrative medical cases remains underexplored. We introduce a novel
dataset of 176 symptom-diagnosis pairs extracted from House M.D., a medical television series validated for teaching rare disease recognition in medical education. We evaluate four state-of-the-art LLMs such as GPT 4o mini, GPT 5 mini, Gemini 2.5 Flash, and Gemini 2.5 Pro on narrative-based diagnostic
reasoning tasks. Results show significant variation in performance, ranging from 16.48% to 38.64% accuracy, with newer model generations demonstrating a 2.3 times improvement. While all models face substantial challenges with rare disease diagnosis, the observed improvement across architectures suggests promising directions for future development. Our educationally validated
benchmark establishes baseline performance metrics for narrative medical
reasoning and provides a publicly accessible evaluation framework for advancing AI-assisted diagnosis research.
Read the full abstract ▾
All authors Arsh Gupta, Ajay Narayanan Sridhar, Bonam Mingole, Amulya Yadav