Computation and Language
Segment, Embed, and Align: A Universal Recipe for Aligning Subtitles to Signing
Zifan Jiang, Youngjoon Jang, Liliane Momeni et al. (+3 more)
The goal of this work is to develop a universal approach for aligning subtitles (i.e., spoken
language text with corresponding timestamps) to continuous sign
language videos. Prior approaches typically rely on end-to-end training tied to a specific
language or
dataset, which limits their generality. In contrast, our method Segment, Embed, and Align (SEA) provides a single framework that works across multiple
languages and domains. SEA leverages two pretrained models: the first to segment a
video frame sequence into individual signs and the second to embed the
video clip of each sign into a shared latent space with text.
Alignment is subsequently performed with a lightweight dynamic programming procedure that runs efficiently on CPUs within a minute, even for hour-long episodes. SEA is flexible and can adapt to a wide range of scenarios, utilizing resources from small lexicons to large continuous corpora. Experiments on four sign
language datasets demonstrate state-of-the-art
alignment performance, highlighting the potential of SEA to generate high-quality parallel data for advancing sign
language processing. SEA's code and models are openly available.
Read the full abstract ▾
All authors Zifan Jiang, Youngjoon Jang, Liliane Momeni, Gül Varol, Sarah Ebling, Andrew Zisserman
Computation and Language
Adaptation of Embedding Models to Financial Filings via LLM Distillation
Eliot Brenner, Dominic Seyler, Manjunath Hegde et al. (+3 more)
Despite advances in generative large
language models (LLMs), practical application of specialized conversational AI
agents remains constrained by computation costs, latency requirements, and the need for precise domain-specific relevance measures. While existing
embedding models address the first two constraints, they underperform on information
retrieval in specialized domains like finance. This paper introduces a scalable pipeline that trains specialized models from an unlabeled corpus using a general purpose
retrieval embedding model as foundation. Our method yields an average of 27.7% improvement in MRR$\texttt{@}$5, 44.6% improvement in mean DCG$\texttt{@}$5 across 14 financial filing types measured over 21,800 query-document pairs, and improved NDCG on 3 of 4 document classes in FinanceBench. We adapt
retrieval embeddings (bi-encoder) for RAG, not LLM generators, using LLM-judged relevance to distill domain knowledge into a compact retriever. There are prior works which pair synthetically generated queries with real passages to directly fine-tune the
retrieval model. Our pipeline differs from these by introducing interaction between student and teacher models that interleaves
retrieval-based mining of hard positive/negative examples from the unlabeled corpus with iterative retraining of the student model's weights using these examples. Each
retrieval iteration uses the refined student model to mine the corpus for progressively harder training examples for the subsequent training iteration. The methodology provides a cost-effective solution to bridging the gap between general-purpose models and specialized domains without requiring labor-intensive human annotation.
Read the full abstract ▾
All authors Eliot Brenner, Dominic Seyler, Manjunath Hegde, Andrei Simion, Koustuv Dasgupta, Bing Xiang
Computation and Language
Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
Vala Vakilian, Zimeng Wang, Ankit Singh Rawat et al. (+1 more)
We investigate the short-context dominance hypothesis: that for most sequences, a small local prefix suffices to predict their next
tokens. Using large
language models as statistical oracles, we measure the minimum context length (MCL) needed to reproduce accurate full-context predictions across
datasets with sequences of varying lengths. For sequences with 1-7k
tokens from long-context documents, we consistently find that 75-80% require only the last 96
tokens at most. Given the dominance of short-context
tokens, we then ask whether it is possible to detect challenging long-context sequences for which a short local prefix does not suffice for prediction. We introduce a practical proxy to MCL, called Distributionally Aware MCL (DaMCL), that does not require knowledge of the actual next-
token and is compatible with sampling strategies beyond greedy decoding. Our experiments validate that simple thresholding of the metric defining DaMCL achieves high performance in detecting long vs. short context sequences. Finally, to counter the bias that short-context dominance induces in LLM output distributions, we develop an intuitive decoding
algorithm that leverages our detector to identify and boost
tokens that are long-range-relevant. Across Q&A tasks and model architectures, we confirm that mitigating the bias improves performance.
Read the full abstract ▾
All authors Vala Vakilian, Zimeng Wang, Ankit Singh Rawat, Christos Thrampoulidis
Computation and Language · Cross-listed from cs.SD
Beyond Unified Models: A Service-Oriented Approach to Low Latency, Context Aware Phonemization for Real Time TTS
Mahta Fetrat, Donya Navabi, Zahra Dehghanian et al. (+2 more)
Lightweight, real-time text-to-
speech systems are crucial for accessibility. However, the most efficient TTS models often rely on lightweight phonemizers that struggle with context-dependent challenges. In contrast, more advanced phonemizers with a deeper linguistic understanding typically incur high computational costs, which prevents real-time performance. This paper examines the trade-off between phonemization quality and inference speed in G2P-aided TTS systems, introducing a practical framework to bridge this gap. We propose lightweight strategies for context-aware phonemization and a service-oriented TTS architecture that executes these modules as independent services. This design decouples heavy context-aware components from the core TTS engine, effectively breaking the latency barrier and enabling real-time use of high-quality phonemization models. Experimental results confirm that the proposed system improves pronunciation soundness and linguistic accuracy while maintaining real-time responsiveness, making it well-suited for offline and end-device TTS applications.
Read the full abstract ▾
All authors Mahta Fetrat, Donya Navabi, Zahra Dehghanian, Morteza Abolghasemi, Hamid R. Rabiee
Computation and Language
Do Generalisation Results Generalise?
Matteo Boglioni, Andrea Sgobbi, Gabriel Tavernini et al. (+3 more)
A large
language model's (LLM's) out-of-distribution (OOD) generalisation ability is crucial to its deployment. Previous work assessing LLMs' generalisation performance, however, typically focuses on a single out-of-distribution
dataset. This approach may fail to precisely evaluate the capabilities of the model, as the data shifts encountered once a model is deployed are much more diverse. In this work, we investigate whether OOD generalisation results generalise. More specifically, we evaluate a model's performance across multiple OOD testsets throughout a finetuning run; we then evaluate the partial correlation of performances across these testsets, regressing out in-domain performance. This allows us to assess how correlated are generalisation performances once in-domain performance is controlled for. Analysing OLMo2 and OPT, we observe no overarching trend in generalisation results: the existence of a positive or negative correlation between any two OOD testsets depends strongly on the specific choice of model analysed.
Read the full abstract ▾
All authors Matteo Boglioni, Andrea Sgobbi, Gabriel Tavernini, Francesco Rita, Marius Mosbach, Tiago Pimentel
Computation and Language · Cross-listed from cs.LG
Group Representational Position Encoding
Yifan Zhang, Zixiang Chen, Yifeng Liu et al. (+6 more)
We present GRAPE (Group Representational Position Encoding), a unified framework for positional encoding based on group actions. GRAPE unifies two families of mechanisms: (i) multiplicative rotations (Multiplicative GRAPE) in $\operatorname{SO}(d)$ and (ii) additive logit biases (Additive GRAPE) arising from unipotent actions in the general linear group $\mathrm{GL}$. In Multiplicative GRAPE, a position $n \in \mathbb{Z}$ (or $t \in \mathbb{R}$) acts as $\mathbf{G}(n) = \exp(n \, \omega \, \mathbf{L})$ with a rank-2 skew-symmetric generator $\mathbf{L} \in \mathbb{R}^{d \times d}$, yielding a relative, compositional, norm-preserving map with a closed-form
matrix exponential. RoPE is recovered exactly when the $d/2$ planes correspond to canonical coordinate pairs with a log-uniform spectrum. Learned commuting subspaces and compact non-commuting mixtures strictly extend this geometry to capture cross-subspace feature coupling at $O(d)$ and $O(r d)$ cost per head, respectively. In Additive GRAPE, additive logits arise from rank-1 (or low-rank) unipotent actions, recovering ALiBi and the Forgetting
Transformer (FoX) as exact special cases while preserving an exact relative law and streaming cacheability. Overall, GRAPE provides a principled design space for positional geometry in long-context models, subsuming RoPE and ALiBi as special cases. Project page: https://github.com/model-architectures/GRAPE.
Read the full abstract ▾
All authors Yifan Zhang, Zixiang Chen, Yifeng Liu, Zhen Qin, Huizhuo Yuan, Kangping Xu, Yang Yuan, Quanquan Gu, Andrew Chi-Chih Yao
Computation and Language
Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support
Raunak Jain
LLM-based
agents are increasingly deployed for expert decision support, yet human-AI teams in high-stakes settings do not yet reliably outperform the best individual. We argue this complementarity gap reflects a fundamental mismatch: current
agents are trained as answer engines, not as partners in the collaborative sensemaking through which experts actually make decisions. Sensemaking (the ability to co-construct causal explanations, surface uncertainties, and adapt goals) is the key capability that current training pipelines do not explicitly develop or evaluate. We propose Collaborative Causal Sensemaking (CCS) as a research agenda to develop this capability from the ground up, spanning new training environments that reward collaborative thinking, representations for shared human-AI mental models, and evaluation centred on trust and complementarity. Taken together, these directions shift MAS research from building oracle-like answer engines to cultivating AI teammates that co-reason with their human partners over the causal structure of shared decisions, advancing the design of effective human-AI teams.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
Nearchos Potamitis, Vansh Ramani, Har Ashish Arora et al. (+3 more)
Benchmark scores for LLM
reasoning systems are reported as single numbers, yet the same model, strategy, and task can produce meaningfully different answers and costs across repeated executions, even under greedy decoding (T = 0). This variance is not a statistical nuisance: the highest-performing strategy wins only 77% of head-to-head runs against its nearest competitor, meaning a single observed score can silently misrank systems. We introduce ReasonBench, a
benchmark suite recording 30 independent trials across 10
reasoning strategies, 12 models, and 6 tasks, treating quality and cost as distributions rather than point estimates. We find that this variance is structured rather than random: a two-component taxonomy – Global Noise, capturing cross-
benchmark unevenness, and Run Noise, capturing within-
benchmark stochasticity – reveals that strategy architecture predicts stability profiles, while models and strategies shift orthogonal aspects of the distribution. A hierarchical decomposition attributes three-quarters of score variance to
benchmark, system, and item structure, with a persistent residual that single-run evaluation silently absorbs. Finally, cost and quality decouple asymmetrically: cheap methods are structurally immune to joint cost-quality failure, while expensive methods remain exposed regardless of their accuracy. These findings establish instability as an inherent property of
reasoning systems and motivate distribution-aware evaluation as standard practice.
Read the full abstract ▾
All authors Nearchos Potamitis, Vansh Ramani, Har Ashish Arora, Dhairya Kuchhal, Lars Klein, Akhil Arora
Computation and Language
On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
Charlie Zhang, Graham Neubig and Xiang Yue
Recent reinforcement learning (RL) techniques have yielded impressive
reasoning improvements in
language models, yet it remains unclear whether post-training truly extends a model's
reasoning ability beyond what it acquires during pre-training. A central challenge is the lack of control in modern training pipelines: large-scale pre-training corpora are opaque, mid-training is often underexamined, and RL objectives interact with unknown prior knowledge in complex ways. To resolve this ambiguity, we develop a fully controlled experimental framework that isolates the causal contributions of pre-training, mid-training, and RL-based post-training. Our approach employs synthetic
reasoning tasks with explicit atomic operations, parseable step-by-step
reasoning traces, and systematic manipulation of training distributions. We evaluate models along two axes: extrapolative generalization to more complex compositions and contextual generalization across surface contexts. Using this framework, we reconcile competing views on RL's effectiveness. We show that: 1) RL produces true capability gains (pass@128) only when pre-training leaves sufficient headroom and when RL data target the model's edge of competence, tasks at the boundary that are difficult but not yet out of reach. 2) Contextual generalization requires minimal yet sufficient pre-training exposure, after which RL can reliably transfer. 3) Mid-training significantly enhances performance under fixed compute compared with RL only, demonstrating its central but underexplored role in training pipelines. 4) Process-level rewards reduce reward hacking and improve
reasoning fidelity. Together, these results clarify the interplay between pre-training, mid-training, and RL, offering a foundation for understanding and improving
reasoning LM training strategies.
Read the full abstract ▾
Computation and Language
Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives?
Karin de Langis, Püren Öncel, Ryan Peters et al. (+4 more)
Leveraging a
dataset of paired narratives, we investigate the extent to which large
language models (LLMs) can reliably separate incoherent and coherent stories. A probing study finds that LLMs' internal representations can reliably identify incoherent narratives. However, LLMs generate responses to rating questions that fail to satisfactorily separate the coherent and incoherent narratives across several
prompt variations, hinting at a gap in LLM's understanding of storytelling. The
reasoning LLMs tested do not eliminate these deficits, indicating that thought strings may not be able to fully address the discrepancy between model internal state and behavior. Additionally, we find that LLMs appear to be more sensitive to incoherence resulting from an event that violates the setting (e.g., a rainy day in the desert) than to incoherence arising from a character violating an established trait (e.g., Mary, a vegetarian, later orders a cheeseburger), suggesting that LLMs may rely more on prototypical world knowledge than building meaning-based narrative coherence. The consistent asymmetry found in our results suggests that LLMs do not have a complete grasp on narrative coherence.
Read the full abstract ▾
All authors Karin de Langis, Püren Öncel, Ryan Peters, Andrew Elfenbein, Laura Kristen Allen, Andreas Schramm, Dongyeop Kang
Computation and Language
Automated Generation of Custom MedDRA Queries Using SafeTerm Medical Map
Francois Vandenhende, Anna Georgiou, Michalis Georgiou et al. (+3 more)
In pre-
market drug safety review, grouping related adverse event terms into standardised MedDRA queries or the FDA Office of New Drugs Custom Medical Queries (OCMQs) is critical for signal detection. We present a novel quantitative artificial intelligence system that understands and processes medical terminology and automatically retrieves relevant MedDRA Preferred Terms (PTs) for a given input query, ranking them by a relevance score using multi-criteria statistical methods. The system (SafeTerm) embeds medical query terms and MedDRA PTs in a multidimensional vector space, then applies cosine similarity and extreme-value clustering to generate a ranked list of PTs. Validation was conducted against the FDA OCMQ v3.0 (104 queries), restricted to valid MedDRA PTs. Precision, recall and F1 were computed across similarity-thresholds. High recall (>95%) is achieved at moderate thresholds. Higher thresholds improve precision (up to 86%). The optimal threshold (~0.70 - 0.75) yielded recall ~50% and precision ~33%. Narrow-term PT subsets performed similarly but required slightly higher similarity thresholds. The SafeTerm AI-driven system provides a viable supplementary method for automated MedDRA query generation. A similarity threshold of ~0.60 is recommended initially, with increased thresholds for refined term selection.
Read the full abstract ▾
All authors Francois Vandenhende, Anna Georgiou, Michalis Georgiou, Theodoros Psaras, Ellie Karekla, Elena Hadjicosta
Computation and Language
HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
Sujoy Nath, Arkaprabha Basu, Sharanya Dasgupta et al. (+1 more)
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable capabilities in
vision-
language understanding tasks. While these models often produce linguistically coherent output, they often suffer from
hallucinations, generating descriptions that are factually inconsistent with the visual content, potentially leading to adverse consequences. Therefore, the assessment of
hallucinations in MLLM has become increasingly crucial in the model development process. Contemporary methodologies predominantly depend on external LLM evaluators, which are themselves susceptible to
hallucinations and may present challenges in terms of domain adaptation. In this study, we propose the hypothesis that
hallucination manifests as measurable irregularities within the internal layer dynamics of MLLMs, not merely due to distributional shifts but also in the context of layer-wise analysis of specific assumptions. By incorporating such modifications, \textsc{HalluShift++} broadens the efficacy of
hallucination detection from text-based large
language models (LLMs) to encompass
multimodal scenarios. Our codebase is available at https://github.com/C0mRD/HalluShift_Plus.
Read the full abstract ▾
All authors Sujoy Nath, Arkaprabha Basu, Sharanya Dasgupta, Swagatam Das
Computation and Language
When Large Language Models Do Not Work: Online Incivility Prediction through Graph Neural Networks
Zihan Chen and Lanyu Yu
Online incivility has emerged as a widespread and persistent problem in digital communities, imposing substantial social and psychological burdens on users. Although many platforms attempt to curb incivility through moderation and automated detection, the performance of existing approaches often remains limited in both accuracy and efficiency. To address this challenge, we propose a Graph Neural Network (GNN) framework for detecting three types of uncivil behavior (i.e., toxicity, aggression, and personal attacks) within the English Wikipedia community. Our model represents each user comment as a node, with textual similarity between comments defining the edges, allowing the network to jointly learn from both linguistic content and relational structures among comments. We also introduce a dynamically adjusted
attention mechanism that adaptively balances nodal and topological features during information aggregation. Empirical evaluations demonstrate that our proposed architecture outperforms 12 state-of-the-art Large
Language Models (LLMs) across multiple metrics while requiring significantly lower inference cost. These findings highlight the crucial role of structural context in detecting online incivility and address the limitations of text-only LLM paradigms in behavioral prediction. All
datasets and comparative outputs will be publicly available in our repository to support further research and reproducibility.
Read the full abstract ▾
Computation and Language
Bridging Code Graphs and Large Language Models for Better Code Understanding
Zeqi Chen, Zhaoyang Chu, Yi Gui et al. (+3 more)
Large
Language Models (LLMs) have demonstrated remarkable performance in code intelligence tasks such as code generation, summarization, and translation. However, their reliance on linearized
token sequences limits their ability to understand the structural semantics of programs. While prior studies have explored graphaugmented prompting and structure-aware pretraining, they either suffer from
prompt length constraints or require task-specific architectural changes that are incompatible with large-scale instructionfollowing LLMs. To address these limitations, this paper proposes CGBridge, a novel plug-and-play method that enhances LLMs with Code Graph information through an external, trainable Bridge module. CGBridge first pre-trains a code graph encoder via selfsupervised learning on a large-scale
dataset of 270K code graphs to learn structural code semantics. It then trains an external module to bridge the modality gap among code, graph, and text by aligning their semantics through cross-modal
attention mechanisms. Finally, the bridge module generates structure-informed
prompts, which are injected into a frozen LLM, and is fine-tuned for downstream code intelligence tasks. Experiments show that CGBridge achieves notable improvements over both the original model and the graphaugmented prompting method. Specifically, it yields a 16.19% and 9.12% relative gain in LLM-as-a-Judge on code summarization, and a 9.84% and 38.87% relative gain in Execution Accuracy on code translation. Moreover, CGBridge achieves over 4x faster inference than LoRA-tuned models, demonstrating both effectiveness and efficiency in structure-aware code understanding.
Read the full abstract ▾
All authors Zeqi Chen, Zhaoyang Chu, Yi Gui, Feng Guo, Yao Wan, Chuan Shi
Computation and Language
PCMind-2.1-Kaiyuan-2B Technical Report
Kairong Luo, Zhenbo Sun, Xinyu Shi et al. (+9 more)
The rapid advancement of Large
Language Models (LLMs) has resulted in a significant knowledge gap between the open-source community and industry, primarily because the latter relies on closed-source, high-quality data and training recipes. To address this, we introduce PCMind-2.1-Kaiyuan-2B, a fully open-source 2-billion-parameter model focused on improving training efficiency and effectiveness under resource constraints. Our methodology includes three key innovations: a
Quantile Data Benchmarking method for systematically comparing heterogeneous open-source
datasets and providing insights on data mixing strategies; a Strategic Selective Repetition scheme within a multi-phase paradigm to effectively leverage sparse, high-quality data; and a Multi-Domain Curriculum Training policy that orders samples by quality. Supported by a highly optimized data preprocessing pipeline and architectural modifications for FP16 stability, Kaiyuan-2B achieves performance competitive with state-of-the-art fully open-source models, demonstrating practical and scalable solutions for resource-limited pretraining. We release all assets (including model weights, data, and code) under Apache 2.0 license at https://huggingface.co/thu-pacman/PCMind-2.1-Kaiyuan-2B.
Read the full abstract ▾
All authors Kairong Luo, Zhenbo Sun, Xinyu Shi, Shengqi Chen, Bowen Yu, Yunyi Chen, Chenyi Dang, Hengtao Tao, Hui Wang, Fangming Liu, Kaifeng Lyu, Wenguang Chen
Computation and Language
Metric-Fair Prompting: Treating Similar Samples Similarly
Jing Wang, Jie Shen, Xing Niu et al. (+2 more)
We introduce Metric-Fair Prompting, a
fairness-aware prompting framework that guides large
language models (LLMs) to make decisions under metric-
fairness constraints. In the application of multiple-choice medical question answering, each {(question, option)} pair is treated as a binary instance with label $+1$ (correct) or $-1$ (incorrect). To promote {individual
fairness}~–~treating similar instances similarly~–~we compute question similarity using NLP
embeddings and solve items in joint pairs of similar questions rather than in isolation. The
prompt enforces a global decision protocol: extract decisive clinical features, map each \((\text{question}, \text{option})\) to a score $f(x)$ that acts as confidence, and impose a Lipschitz-style constraint so that similar inputs receive similar scores and, hence, consistent outputs. Evaluated on the {MedQA (US)}
benchmark, Metric-Fair Prompting is shown to improve performance over standard single-item prompting, demonstrating that
fairness-guided, confidence-oriented
reasoning can enhance LLM accuracy on high-stakes clinical multiple-choice questions.
Read the full abstract ▾
All authors Jing Wang, Jie Shen, Xing Niu, Tong Zhang, Jeremy Weiss
Computation and Language
Complementary Learning Approach for Text Classification using Large Language Models
Navid Asgari and Benjamin M. Cole
In this study, we propose a structured methodology that utilizes large
language models (LLMs) in a cost-efficient and parsimonious manner, integrating the strengths of scholars and machines while offsetting their respective weaknesses. Our methodology, facilitated through a chain of thought and few-shot learning prompting from computer science, extends best practices for co-author teams in qualitative research to human-machine teams in quantitative research. This allows humans to utilize abductive
reasoning and natural
language to interrogate not just what the machine has done but also what the human has done. Our method highlights how scholars can manage inherent weaknesses OF LLMs using careful, low-cost techniques. We demonstrate how to use the methodology to interrogate human-machine rating discrepancies for a sample of 1,934 press releases announcing pharmaceutical alliances (1990-2017).
Read the full abstract ▾
Computation and Language
A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification
Nicolas Calbucura, Jose Guillen and Valentin Barriere
This paper presents a simple method that allows to easily enhance textual pre-trained large
language models with
speech information, when fine-tuned for a specific classification task. A classical issue with the fusion of many
embeddings from audio with text is the large length of the audio sequence compared to the text one. Our method benefits from an existing
speech tokenizer trained for Audio
Speech Recognition that output long sequences of
tokens from a large vocabulary, making it difficult to integrate it at low cost in a large
language model. By applying a simple lasso-based feature selection on
multimodal Bag-of-Words representation, we retain only the most important audio
tokens for the task, and adapt the
language model to them with a self-supervised
language modeling objective, before fine-tuning it on the downstream task. We show this helps to improve the performances compared to an unimodal model, to a bigger SpeechLM or to integrating audio via a learned representation. We demonstrate its effectiveness on Argumentative Fallacy Detection and Classification tasks where audio was previously believed counterproductive, and affective computing tasks on a widely-used
dataset. We also provide an in-depth analysis of the method, showing that even a random audio
token selection helps enhancing the unimodal model. Our code is available [online](https://github.com/salocinc/EMNLP26SpeechTokLLM/).
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Models
Kassoum Sanogo and Renzo Ardiccioni
Vision-
language models (VLMs) frequently generate hallucinated content plausible but incorrect claims about image content. We propose a training-free self-correction framework enabling VLMs to iteratively refine responses through uncertainty-guided visual re-
attention. Our method combines multidimensional uncertainty quantification (
token entropy,
attention dispersion, semantic consistency, claim confidence) with
attention-guided cropping of under-explored regions. Operating entirely with frozen, pretrained VLMs, our framework requires no gradient updates. We validate our approach on the POPE and MMHAL BENCH
benchmarks using the Qwen2.5-VL-7B [23] architecture. Experimental results demonstrate that our method reduces
hallucination rates by 9.8 percentage points compared to the baseline, while improving object existence accuracy by 4.7 points on
adversarial splits. Furthermore, qualitative analysis confirms that uncertainty-guided re-
attention successfully grounds corrections in visual evidence where standard decoding fails. We validate our approach on Qwen2.5-VL-7B [23], with plans to extend validation across diverse architectures in future versions. We release our code and methodology to facilitate future research in trustworthy
multimodal systems.
Read the full abstract ▾
Computation and Language
Performance of the SafeTerm AI-Based MedDRA Query System Against Standardised MedDRA Queries
Francois Vandenhende, Anna Georgiou, Michalis Georgiou et al. (+3 more)
In pre-
market drug safety review, grouping related adverse event terms into SMQs or OCMQs is critical for signal detection. We assess the performance of SafeTerm Automated Medical Query (AMQ) on MedDRA SMQs. The AMQ is a novel quantitative artificial intelligence system that understands and processes medical terminology and automatically retrieves relevant MedDRA Preferred Terms (PTs) for a given input query, ranking them by a relevance score (0-1) using multi-criteria statistical methods. The system (SafeTerm) embeds medical query terms and MedDRA PTs in a multidimensional vector space, then applies cosine similarity, and extreme-value clustering to generate a ranked list of PTs. Validation was conducted against tier-1 SMQs (110 queries, v28.1). Precision, recall and F1 were computed at multiple similarity-thresholds, defined either manually or using an automated method. High recall (94%)) is achieved at moderate similarity thresholds, indicative of good
retrieval sensitivity. Higher thresholds filter out more terms, resulting in improved precision (up to 89%). The optimal threshold (0.70)) yielded an overall recall of (48%) and precision of (45%) across all 110 queries. Restricting to narrow-term PTs achieved slightly better performance at an increased (+0.05) similarity threshold, confirming increased relatedness of narrow versus broad terms. The automatic threshold (0.66) selection prioritizes recall (0.58) to precision (0.29). SafeTerm AMQ achieves comparable, satisfactory performance on SMQs and sanitized OCMQs. It is therefore a viable supplementary method for automated MedDRA query generation, balancing recall and precision. We recommend using suitable MedDRA PT terminology in query formulation and applying the automated threshold method to optimise recall. Increasing similarity scores allows refined, narrow terms selection.
Read the full abstract ▾
All authors Francois Vandenhende, Anna Georgiou, Michalis Georgiou, Theodoros Psaras, Ellie Karekla, Elena Hadjicosta
Computation and Language
MoCoRP: Modeling Consistent Relations between Persona and Response for Persona-based Dialogue
Kyungro Lee, Dongha Choi and Hyunju Lee
As dialogue systems become increasingly important across various domains, a key challenge in persona-based dialogue is generating engaging and context-specific interactions while ensuring the model acts with a coherent personality. However, existing persona-based dialogue
datasets lack explicit relations between persona sentences and responses, which makes it difficult for models to effectively capture persona information. To address these issues, we propose MoCoRP (Modeling Consistent Relations between Persona and Response), a framework that incorporates explicit relations into
language models. MoCoRP leverages an NLI expert to explicitly extract the NLI relations between persona sentences and responses, enabling the model to effectively incorporate appropriate persona information from the context into its responses. We applied this framework to pre-trained models like BART and further extended it to modern large
language models (LLMs) through
alignment tuning. Experimental results on the public
datasets ConvAI2 and MPChat demonstrate that MoCoRP outperforms existing baselines, achieving superior persona consistency and engaging, context-aware dialogue generation. Furthermore, our model not only excels in quantitative metrics but also shows significant improvements in qualitative aspects. These results highlight the effectiveness of explicitly modeling persona-response relations in persona-based dialogue. The source codes of MoCoRP are available at https://github.com/DMCB-GIST/MoCoRP.
Read the full abstract ▾
Computation and Language
Over-representation of phonological features in basic vocabulary doesn't replicate when controlling for spatial and phylogenetic effects
Frederic Blum
The statistical over-representation of phonological features in the basic vocabulary of
languages is often interpreted as reflecting potentially universal sound symbolic patterns. However, most of those results have not been tested explicitly for reproducibility and might be prone to biases in the study samples or models. Many studies on the topic do not adequately control for genealogical and areal dependencies between sampled
languages, casting doubts on the robustness of the results. In this study, we test the robustness of a recent study on sound symbolism of basic vocabulary concepts which analyzed 245
languages.The new sample includes data on 2864
languages from Lexibank. We modify the original model by adding statistical controls for spatial and phylogenetic dependencies between
languages. The new results show that most of the previously observed patterns are not robust, and in fact many patterns disappear completely when adding the genealogical and areal controls. A small number of patterns, however, emerges as highly stable even with the new sample. Through the new analysis, we are able to assess the distribution of sound symbolism on a larger scale than previously. The study further highlights the need for testing all universal claims on
language for robustness on various levels.
Read the full abstract ▾
Computation and Language
Minimum Bayes Risk Decoding for Error Span Detection in Reference-Free Automatic Machine Translation Evaluation
Boxuan Lyu, Haiyue Song, Hidetaka Kamigaito et al. (+5 more)
Error Span Detection (ESD) extends automatic machine translation (MT) evaluation by localizing translation errors and labeling their severity. Current generative ESD methods typically use Maximum a Posteriori (MAP) decoding, assuming that the model-estimated probabilities are perfectly correlated with similarity to the human annotation, but we often observe higher likelihood assigned to an incorrect annotation than to the human one. We instead apply Minimum Bayes Risk (MBR) decoding to generative ESD. We use a sentence- or span-level similarity function for MBR decoding, which selects candidate hypotheses based on their approximate similarity to the human annotation. Experimental results on the WMT24 Metrics Shared Task show that MBR decoding significantly improves span-level performance and generally matches or outperforms MAP at the system and sentence levels. To reduce the computational cost of MBR decoding, we further distill its decisions into a model decoded via greedy search, removing the inference-time latency bottleneck.
Read the full abstract ▾
All authors Boxuan Lyu, Haiyue Song, Hidetaka Kamigaito, Chenchen Ding, Hideki Tanaka, Masao Utiyama, Kotaro Funakoshi, Manabu Okumura
Computation and Language
SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents
Michelle Wastl, Jannis Vamvas and Rico Sennrich
Recognizing semantic differences across documents is crucial for text generation evaluation and content
alignment, especially in cross-lingual settings. However, as a standalone task, it has received little
attention. We address this by introducing SwissGov-RSD, the first naturalistic, document-level, cross-lingual
dataset for semantic difference recognition. It encompasses a total of 224 multi-parallel documents in English–German, English–French, and English–Italian with
token-level difference annotations by human annotators. We evaluate a variety of open-source and closed-source large
language models as well as encoder models across different fine-tuning settings on this new
benchmark. Our results show that current automatic approaches perform poorly compared to their performance on monolingual, sentence-level, and synthetic
benchmarks, revealing a considerable gap for both LLMs and encoder models. We make our code and
dataset publicly available.
Read the full abstract ▾
Computation and Language
Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
Xiaoran Liu, Yuerong Song, Zhigeng Liu et al. (+6 more)
Rotary Position
Embeddings (RoPE) have become a standard for encoding sequence order in Large
Language Models (LLMs) by applying rotations to query and key vectors in the complex plane. Standard implementations, however, utilize only the real component of the complex-valued dot product for
attention score calculation. This simplification discards the imaginary component, which contains valuable phase information, leading to a potential loss of relational details crucial for modeling long-context dependencies. In this paper, we propose an extension that re-incorporates this discarded imaginary component. Our method leverages the full complex-valued representation to create a dual-component
attention score. We theoretically and empirically demonstrate that this approach enhances the modeling of long-context dependencies by preserving more positional information. Furthermore, evaluations on a suite of long-context
language modeling
benchmarks show that our method consistently improves performance over the standard RoPE, with the benefits becoming more significant as context length increases. The code is available at https://github.com/OpenMOSS/rope_pp.
Read the full abstract ▾
All authors Xiaoran Liu, Yuerong Song, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Zhaoxiang Liu, Shiguo Lian, Ziwei He, Xipeng Qiu
Computation and Language
LIME: Making LLM Data More Efficient with Linguistic Metadata Embeddings
Sebastian Sztwiertnia, Felix Friedrich, Kristian Kersting et al. (+2 more)
Pre-training decoder-only
language models relies on vast amounts of high-quality data, yet the availability of such data is increasingly reaching its limits. While metadata is commonly used to create and curate these
datasets, its potential as a direct training signal remains under-explored. We challenge this status quo and propose LIME (Linguistic Metadata
Embeddings), a method that enriches
token embeddings with metadata capturing syntax, semantics, and contextual properties. LIME substantially improves pre-training efficiency. Specifically, it adapts up to 56% faster to the training data distribution, while introducing only 0.01% additional parameters at negligible compute overhead. Beyond efficiency, LIME improves tokenization, leading to remarkably stronger
language modeling capabilities and generative task performance. These benefits persist across model scales (500M to 2B). In addition, we develop a variant with shifted metadata, LIME+1, that can guide
token generation. Given prior metadata for the next
token, LIME+1 improves
reasoning performance by up to 38% and arithmetic accuracy by up to 35%.
Read the full abstract ▾
All authors Sebastian Sztwiertnia, Felix Friedrich, Kristian Kersting, Patrick Schramowski, Björn Deiseroth
Computation and Language
TPA: Next Token Probability Attribution for Detecting Hallucinations in RAG
Pengqian Lu, Jie Lu, Anjin Liu et al. (+1 more)
Detecting
hallucinations in
Retrieval-Augmented Generation remains a challenge. Prior approaches attribute
hallucinations to a binary conflict between internal knowledge stored in FFNs and the retrieved context. However, this perspective is incomplete, failing to account for the impact of other components of the LLM, such as the user query, previously generated
tokens, the self
token, and the final LayerNorm adjustment. To comprehensively capture the impact of these components on
hallucination detection, we propose TPA which mathematically attributes each
token's probability to seven distinct sources: Query, RAG Context, Past
Token, Self
Token, FFN, Final LayerNorm, and Initial
Embedding. This attribution quantifies how each source contributes to the generation of the next
token. Specifically, we aggregate these attribution scores by Part-of-
Speech (POS) tags to quantify the contribution of each model component to the generation of specific linguistic categories within a response. By leveraging these patterns, such as detecting anomalies where Nouns rely heavily on LayerNorm, TPA effectively identifies hallucinated responses. Extensive experiments show that TPA achieves state-of-the-art performance.
Read the full abstract ▾
All authors Pengqian Lu, Jie Lu, Anjin Liu, Guangquan Zhang
Computation and Language
Enhancing Agentic RL with Progressive Reward Shaping and Value-based Sampling Policy Optimization
Jianghao Su, Xia Zeng, Luhui Liu et al. (+3 more)
Large
Language Models (LLMs) empowered with Tool-Integrated
Reasoning (TIR) can iteratively plan, call external tools, and integrate returned information to solve complex, long-horizon
reasoning tasks. Agentic Reinforcement Learning (Agentic RL) optimizes such models over full tool-interaction trajectories, but two key challenges hinder effectiveness: (1) Sparse, non-instructive rewards, such as binary 0-1 verifiable signals, provide limited guidance for intermediate steps and slow convergence; (2) Gradient degradation in Group Relative Policy Optimization (GRPO), where identical rewards within a rollout group yield zero advantage, which reducing sample efficiency. To address these challenges, we propose two complementary techniques: Progressive Reward Shaping (PRS) and Value-based Sampling Policy Optimization (VSPO). PRS is a curriculum-inspired reward design that introduces dense, stage-wise feedback - encouraging models to first master parseable and properly formatted tool calls, then optimize for factual correctness and answer quality. We instantiate PRS for short-form QA (with a length-aware BLEU to fairly score concise answers) and long-form QA (with LLM-as-a-Judge scoring to prevent reward hacking). VSPO is an enhanced GRPO variant that replaces zero advantages samples with
prompts selected by a task-value metric balancing difficulty and uncertainty, and applies value-smoothing clipping to stabilize gradient updates. Experiments on multiple short-form and long-form QA
benchmarks show that PRS consistently outperforms traditional binary rewards, and VSPO achieves superior stability, faster convergence, and higher final performance compared to SFT, PPO and GRPO baselines. Together, PRS and VSPO yield LLM-based TIR
agents that generalize better across domains.
Read the full abstract ▾
All authors Jianghao Su, Xia Zeng, Luhui Liu, Chao Luo, Ye Chen, Zhuoran Zhuang
Computation and Language · Cross-listed from cs.HC
Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novels
Yifei Huang, Tianyu Yan, Sitong Gong et al. (+5 more)
We present the Living Novel, an end-to-end system that transforms any literary work into an immersive, multi-character conversational experience. This system is designed to solve two fundamental challenges for LLM-driven characters. Firstly, generic LLMs suffer from persona drift, often failing to stay in character. Secondly,
agents often exhibit abilities that extend beyond the constraints of the story's world and logic, leading to both narrative incoherence (spoiler leakage) and robustness failures (frame-breaking). To address these challenges, we introduce a novel two-stage training pipeline. Our Deep Persona
Alignment (DPA) stage uses data-free reinforcement finetuning to instill deep character fidelity. Our Coherence and Robustness Enhancing (CRE) stage then employs a story-time-aware knowledge graph and a second
retrieval-grounded training pass to architecturally enforce these narrative constraints. We validate our system through a multi-phase evaluation using Jules Verne's Twenty Thousand Leagues Under the Sea. A lab study with a detailed ablation of system components is followed by a 5-day in-the-wild diary study. Our DPA pipeline helps our specialized model outperform GPT-4o on persona-specific metrics, and our CRE stage achieves near-perfect performance in coherence and robustness measures. Our study surfaces practical design guidelines for AI-driven narrative systems: we find that character-first self-training is foundational for believability, while explicit story-time constraints are crucial for sustaining coherent, interruption-resilient mobile-web experiences.
Read the full abstract ▾
All authors Yifei Huang, Tianyu Yan, Sitong Gong, Xiwei Gao, Caixin Kang, Ruicong Liu, Huchuan Lu, Bo Zheng
Computation and Language
Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning
Tong Wu, Yang Liu, Jun Bai et al. (+6 more)
We introduce Native Parallel Reasoner (NPR), a teacher-free framework that enables Large
Language Models (LLMs) to self-evolve genuine parallel
reasoning capabilities. NPR transforms the model from sequential emulation to native parallel cognition through three key innovations: 1) a self-distilled progressive training paradigm that transitions from “cold-start” format discovery to strict topological constraints without external supervision; 2) a novel Parallel-Aware Policy Optimization (PAPO)
algorithm that optimizes branching policies directly within the execution graph, allowing the model to learn adaptive decomposition via trial and error; and 3) a robust NPR Engine that refactors
memory management and flow control of SGLang to enable stable, large-scale parallel RL training. Across eight
reasoning benchmarks, NPR trained on Qwen3-4B achieves performance gains of up to 24.5% and inference speedups up to 4.6x. Unlike prior baselines that often fall back to autoregressive decoding, NPR demonstrates 100% genuine parallel execution, establishing a new standard for self-evolving, efficient, and scalable agentic
reasoning.
Read the full abstract ▾
All authors Tong Wu, Yang Liu, Jun Bai, Zixia Jia, Shuyi Zhang, Ziyong Lin, Yanting Wang, Song-Chun Zhu, Zilong Zheng