Computation and Language
Happiness is Sharing a Vocabulary: A Study of Transliteration Methods
Haeji Jung, Jinju Kim, Kyungjin Kim et al. (+2 more)
Transliteration has emerged as a promising means to bridge the gap between various
languages in
multilingual NLP, showing promising results especially for
languages using non-Latin scripts. We investigate the degree to which shared script, overlapping
token vocabularies, and shared phonology contribute to performance of
multilingual models. To this end, we conduct controlled experiments using three kinds of transliteration (romanization, phonemic transcription, and substitution ciphers) as well as orthography. We evaluate each model on three downstream tasks – named entity recognition (NER), part-of-
speech tagging (POS) and natural
language inference (NLI) – and find that romanization significantly outperforms other input types in 11 out of 12 evaluation settings, largely consistent with our hypothesis that it is the most effective approach. We further analyze how each factor contributed to the success, and suggest that having longer (subword)
tokens shared with pre-trained
languages leads to better utilization of the model.
Read the full abstract ▾
All authors Haeji Jung, Jinju Kim, Kyungjin Kim, Youjeong Roh, David R. Mortensen
Computation and Language · Cross-listed from cs.AI
DRIFT: Decompose, Retrieve, Illustrate, then Formalize Theorems
Meiru Zhang, Philipp Borchert, Milan Gritta et al. (+1 more)
Automating the formalization of mathematical statements for
theorem proving remains a major challenge for Large
Language Models (LLMs). LLMs struggle to identify and utilize the prerequisite mathematical knowledge and its corresponding formal representation in
languages like Lean. Current
retrieval-augmented autoformalization methods query external libraries using the informal statement directly, but overlook a fundamental limitation: informal statements lack direct mappings to mathematical
theorems and lemmata, nor do those
theorems translate trivially into the formal primitives of
languages like Lean. To address this, we introduce DRIFT, a novel framework that enables LLMs to decompose informal mathematical statements into smaller, more tractable "sub-components". This facilitates targeted
retrieval of premises from mathematical libraries such as Mathlib. Additionally, DRIFT retrieves illustrative
theorems to help models use premises more effectively in formalization tasks. We evaluate DRIFT across diverse
benchmarks (ProofNet, ConNF, and MiniF2F-test) and find that it consistently improves premise
retrieval, nearly doubling the F1 score compared to the DPR baseline on ProofNet. Notably, DRIFT demonstrates strong performance on the out-of-distribution ConNF
benchmark, with BEq+@10 improvements of 42.25% and 37.14% using GPT-4.1 and DeepSeek-V3.1, respectively. Our analysis shows that
retrieval effectiveness in mathematical autoformalization depends heavily on model-specific knowledge boundaries, highlighting the need for adaptive
retrieval strategies aligned with each model's capabilities.
Read the full abstract ▾
All authors Meiru Zhang, Philipp Borchert, Milan Gritta, Gerasimos Lampouras
Computation and Language
Is Implicit Knowledge Enough for LLMs? A RAG Approach for Tree-based Structures
Mihir Gupte, Paolo Giusto and Ramesh S
Large
Language Models (LLMs) are adept at generating responses based on information within their context. While this ability is useful for interacting with structured data like code files, another popular method,
Retrieval-Augmented Generation (RAG), retrieves relevant documents to augment the model's in-context learning. However, it is not well-explored how to best represent this retrieved knowledge for generating responses on structured data, particularly hierarchical structures like trees. In this work, we propose a novel bottom-up method to linearize knowledge from tree-like structures (like a GitHub repository) by generating implicit, aggregated summaries at each hierarchical level. This approach enables the knowledge to be stored in a knowledge base and used directly with RAG. We then compare our method to using RAG on raw, unstructured code, evaluating the accuracy and quality of the generated responses. Our results show that while response quality is comparable across both methods, our approach generates over 68% fewer documents in the retriever, a significant gain in efficiency. This finding suggests that leveraging implicit, linearized knowledge may be a highly effective and scalable strategy for handling complex, hierarchical data structures.
Read the full abstract ▾
Computation and Language
Toward Human-Centered Readability Evaluation
Bahar İlgen and Georges Hattab
Text simplification is essential for making public health information accessible to diverse populations, including those with limited health literacy. However, commonly used evaluation metrics in Natural
Language Processing (NLP), such as BLEU, FKGL, and SARI, mainly capture surface-level features and fail to account for human-centered qualities like clarity, trustworthiness, tone, cultural relevance, and actionability. This limitation is particularly critical in high-stakes health contexts, where communication must be not only simple but also usable, respectful, and trustworthy. To address this gap, we propose the Human-Centered Readability Score (HCRS), a five-dimensional evaluation framework grounded in Human-Computer Interaction (HCI) and health communication research. HCRS integrates automatic measures with structured human feedback to capture the relational and contextual aspects of readability. We outline the framework, discuss its integration into participatory evaluation workflows, and present a protocol for empirical validation. This work aims to advance the evaluation of health text simplification beyond surface metrics, enabling NLP systems that align more closely with diverse users' needs, expectations, and lived experiences.
Read the full abstract ▾
Computation and Language
Review of Inference-Time Scaling Strategies: Reasoning, Search and RAG
Zhichao Wang, Cheng Wan and Dong Nie
The performance gains of LLMs have historically been driven by scaling up model size and training data. However, the rapidly diminishing availability of high-quality training data is introducing a fundamental bottleneck, shifting the focus of research toward inference-time scaling. This paradigm uses additional computation at the time of deployment to substantially improve LLM performance on downstream tasks without costly model re-training. This review systematically surveys the diverse techniques contributing to this new era of inference-time scaling, organizing the rapidly evolving field into two comprehensive perspectives: Output-focused and Input-focused methods. Output-focused techniques encompass complex, multi-step generation strategies, including
reasoning (e.g., CoT, ToT, ReAct), various search and decoding methods (e.g., MCTS, beam search), training for long CoT (e.g., RLVR, GRPO), and model ensemble methods. Input-focused techniques are primarily categorized by few-shot and RAG, with RAG as the central focus. The RAG section is further detailed through a structured examination of query expansion, data,
retrieval and reranker, LLM generation methods, and multi-modal RAG.
Read the full abstract ▾
Computation and Language
HiligayNER: A Baseline Named Entity Recognition Model for Hiligaynon
James Ald Teves, Ray Daniel Cal, Josh Magdiel Villaluz et al. (+5 more)
The
language of Hiligaynon, spoken predominantly by the people of Panay Island, Negros Occidental, and Soccsksargen in the Philippines, remains underrepresented in
language processing research due to the absence of annotated corpora and baseline models. This study introduces HiligayNER, the first publicly available baseline model for the task of Named Entity Recognition (NER) in Hiligaynon. The
dataset used to build HiligayNER contains over 8,000 annotated sentences collected from publicly available news articles, social media posts, and literary texts. Two
Transformer-based models, mBERT and XLM-RoBERTa, were fine-tuned on this collected corpus to build versions of HiligayNER. Evaluation results show strong performance, with both models achieving over 80% in precision, recall, and F1-score across entity types. Furthermore, cross-lingual evaluation with Cebuano and Tagalog demonstrates promising transferability, suggesting the broader applicability of HiligayNER for
multilingual NLP in low-resource settings. This work aims to contribute to
language technology development for underrepresented Philippine
languages, specifically for Hiligaynon, and support future research in regional
language processing.
Read the full abstract ▾
All authors James Ald Teves, Ray Daniel Cal, Josh Magdiel Villaluz, Jean Malolos, Mico Magtira, Ramon Rodriguez, Mideth Abisado, Joseph Marvin Imperial
Computation and Language
Large Language Models for Full-Text Methods Assessment: A Case Study on Mediation Analysis
Wenqing Zhang, Trang Nguyen, Elizabeth A. Stuart et al. (+1 more)
Systematic reviews are crucial for synthesizing scientific evidence but remain labor-intensive, especially when extracting detailed methodological information. Large
language models (LLMs) offer potential for automating methodological assessments, promising to transform evidence synthesis. Here, using causal mediation analysis as a representative methodological domain, we benchmarked state-of-the-art LLMs against expert human reviewers across 180 full-text scientific articles. Model performance closely correlated with human judgments (accuracy correlation 0.71; F1 correlation 0.97), achieving near-human accuracy on straightforward, explicitly stated methodological criteria. However, accuracy sharply declined on complex, inference-intensive assessments, lagging expert reviewers by up to 15%. Errors commonly resulted from superficial linguistic cues – for instance, models frequently misinterpreted keywords like "longitudinal" or "sensitivity" as automatic evidence of rigorous methodological approache, leading to systematic misclassifications. Longer documents yielded lower model accuracy, whereas publication year showed no significant effect. Our findings highlight an important pattern for practitioners using LLMs for methods review and synthesis from full texts: current LLMs excel at identifying explicit methodological features but require human oversight for nuanced interpretations. Integrating automated information extraction with targeted expert review thus provides a promising approach to enhance efficiency and methodological rigor in evidence synthesis across diverse scientific fields.
Read the full abstract ▾
All authors Wenqing Zhang, Trang Nguyen, Elizabeth A. Stuart, Yiqun T. Chen
Computation and Language
Sarcasm Detection Using Deep Convolutional Neural Networks: A Modular Deep Learning Framework
Manas Zambre and Sarika Bobade
Sarcasm is a nuanced and often misinterpreted form of communication, especially in text, where tone and body
language are absent. This paper proposes a modular deep learning framework for sarcasm detection, leveraging Deep Convolutional Neural Networks (DCNNs) and contextual models such as BERT to analyze linguistic, emotional, and contextual cues. The system integrates sentiment analysis, contextual
embeddings, linguistic feature extraction, and
emotion detection through a multi-layer architecture. While the model is in the conceptual stage, it demonstrates feasibility for real-world applications such as chatbots and social media analysis.
Read the full abstract ▾
Computation and Language
RePro: Training Language Models to Faithfully Recycle the Web for Pretraining
Zichun Yu and Chenyan Xiong
High-quality data is a cornerstone of large
language model (LLM) pretraining, yet its growth has not kept pace with the needs of frontier models. In this paper, we introduce RePro, a novel web recycling method that trains a relatively small LM with reinforcement learning to generate effective and faithful rephrasings of pretraining data. Specifically, we design one quality reward and three faithfulness rewards, optimizing the LM rephraser to convert organic data into high-quality rephrasings while maintaining its core semantics and structure. In our experiment, we train a rephraser as small as 1B parameters to recycle 72B
tokens sampled from DCLM-RefinedWeb. Pretraining results on 400M, 1.4B, and 2.8B models demonstrate that RePro delivers 3.7%-14.5% relative accuracy gains over organic-only baseline on 22 downstream tasks, doubling the performance gains achieved by the state-of-the-art web recycling method that
prompts a 70B rephraser. Experiments with different amounts of recycled data highlight that RePro improves organic data efficiency by 2-3x. Individual and distributional analyses validate that RePro preserves more critical information and faithfully reflects the characteristics of organic data compared to prompting-based methods. Together, these results show that RePro provides an efficient and controllable path to effectively recycle organic data for pretraining. Our code is available at https://github.com/cxcscmu/RePro.
Read the full abstract ▾
Computation and Language
Unlocking LLM Safeguards for Low-Resource Languages via Reasoning and Alignment with Minimal Training Data
Zhuowei Chen, Bowei Zhang, Nankai Lin et al. (+2 more)
Recent advances in LLMs have enhanced AI capabilities, but also increased the risk posed by malicious requests, highlighting the need for effective LLM safeguards to detect such queries. Existing approaches largely rely on classifier-based methods that lack interpretability and perform poorly on low-resource
languages. To address these limitations, we propose ConsistentGuard, a novel
reasoning-based
multilingual safeguard, which enhances explainability via
reasoning and boosts knowledge transfer between
languages through
alignment. With only 1,000 training samples, our method demonstrates superior performance on three
datasets across six
languages, outperforming larger models trained with significantly more data, and exhibits strong interpretability and generalization ability. We also contribute a
multilingual benchmark extension and release our codes to support future research.
Read the full abstract ▾
All authors Zhuowei Chen, Bowei Zhang, Nankai Lin, Tian Hou, Lianxi Wang
Computation and Language · Cross-listed from cs.AR
Bhasha-Rupantarika: Algorithm-Hardware Co-design approach for Multilingual Neural Machine Translation
Mukul Lokhande, Tanushree Dewangan, Mohd Sharik Mansoori et al. (+5 more)
This paper introduces Bhasha-Rupantarika, a light and efficient
multilingual translation system tailored through
algorithm-hardware codesign for resource-limited settings. The method investigates model deployment at sub-octet precision levels (FP8, INT8, INT4, and FP4), with experimental results indicating a 4.1x reduction in model size (FP4) and a 4.2x speedup in inference speed, which correlates with an increased throughput of 66
tokens/s (improvement by 4.8x). This underscores the importance of ultra-low precision quantization for real-time deployment in IoT devices using FPGA accelerators, achieving performance on par with expectations. Our evaluation covers bidirectional translation between Indian and international
languages, showcasing its adaptability in low-resource linguistic contexts. The FPGA deployment demonstrated a 1.96x reduction in LUTs and a 1.65x decrease in FFs, resulting in a 2.2x enhancement in throughput compared to OPU and a 4.6x enhancement compared to HPTA. Overall, the evaluation provides a viable solution based on quantisation-aware translation along with hardware efficiency suitable for deployable
multilingual AI systems. The entire codes [https://github.com/mukullokhande99/Bhasha-Rupantarika/] and
dataset for reproducibility are publicly available, facilitating rapid integration and further development by researchers.
Read the full abstract ▾
All authors Mukul Lokhande, Tanushree Dewangan, Mohd Sharik Mansoori, Tejas Chaudhari, Akarsh J., Damayanti Lokhande, Adam Teman, Santosh Kumar Vishvakarma
Computation and Language
BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
Tao Yu, Zhengbo Zhang, Zhiheng Lyu et al. (+8 more)
Efficiently solving real-world problems with LLMs increasingly hinges on their ability to interact with dynamic web environments and autonomously acquire external information. While recent research like Search-R1 and WebDancer demonstrates strong performance in solving web tasks, they heavily rely on additional tools to convert the interactive web environment into static text content. This is in contrast to human browsing behaviors, which involve diverse interactions with the browser, such as scrolling, clicking, and typing. In this paper, we propose BrowserAgent, a more interactive
agent that solves complex tasks through human-inspired browser actions. BrowserAgent operates directly on raw web pages via Playwright through a set of predefined browser actions. We adopt a two-stage training (Supervised Fine-Tuning (SFT) and Rejection Fine-Tuning (RFT)) to improve the model's generalization abilities. Despite using significantly less training data than Search-R1, BrowserAgent achieves more competitive results across different Open-QA tasks. Additionally, we introduce an explicit
memory mechanism to store key conclusions across steps, further enhancing the model's
reasoning capabilities for long-horizon tasks. Notably, BrowserAgent-7B can achieve around 20% improvement over Search-R1 on multi-hop QA tasks like HotpotQA, 2Wiki, and Bamboogle. These results indicate that BrowserAgent can serve as a more advanced framework for more interactive and scalable web
agents.
Read the full abstract ▾
All authors Tao Yu, Zhengbo Zhang, Zhiheng Lyu, Junhao Gong, Hongzhu Yi, Xinming Wang, Yuxuan Zhou, Jiabing Yang, Ping Nie, Yan Huang, Wenhu Chen
Computation and Language
AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation
Omid Reza Heidari, Siobhan Reid and Yassine Yaakoubi
LLMs have advanced text-to-SQL generation, yet monolithic architectures struggle with complex
reasoning and schema diversity. We propose AGENTIQL, an
agent-inspired multi-expert framework that combines a
reasoning agent for question decomposition, a coding
agent for sub-query generation, and a refinement step for column selection. An adaptive router further balances efficiency and accuracy by selecting between our modular pipeline and a baseline parser. Several steps in the pipeline can be executed in parallel, making the framework scalable to larger workloads. Evaluated on the Spider
benchmark, AGENTIQL improves execution accuracy and interpretability and achieves up to 86.07% EX with 14B models using the Planner&Executor merging strategy. The attained performance is contingent upon the efficacy of the routing mechanism, thereby narrowing the gap to GPT-4-based SOTA (89.65% EX) while using much smaller open-source LLMs. Beyond accuracy, AGENTIQL enhances transparency by exposing intermediate
reasoning steps, offering a robust, scalable, and interpretable approach to semantic parsing.
Read the full abstract ▾
Computation and Language
You're Not Gonna Believe This: A Computational Analysis of Factual Appeals and Sourcing in Partisan News
Guy Mor-Lan, Tamir Sheafer and Shaul R. Shenhav
While media bias is widely studied, the epistemic strategies behind factual reporting remain computationally underexplored. This paper analyzes these strategies through a large-scale comparison of CNN and Fox News. To isolate reporting style from topic selection, we employ an article matching strategy to compare reports on the same events and apply the FactAppeal framework to a corpus of over 470K articles covering two highly politicized periods: the COVID-19 pandemic and the Israel-Hamas war. We find that CNN's reporting contains more factual statements and is more likely to ground them in external sources. The outlets also exhibit sharply divergent sourcing patterns: CNN builds credibility by citing Experts} and Expert Documents, constructing an appeal to formal authority, whereas Fox News favors News Reports and direct quotations. This work quantifies how partisan outlets use systematically different epistemic strategies to construct reality, adding a new dimension to the study of media bias.
Read the full abstract ▾
Computation and Language
FactAppeal: Identifying Epistemic Factual Appeals in News Media
Guy Mor-Lan, Tamir Sheafer and Shaul R. Shenhav
How is a factual claim made credible? We propose the novel task of Epistemic Appeal Identification, which identifies whether and how factual statements have been anchored by external sources or evidence. To advance research on this task, we present FactAppeal, a manually annotated
dataset of 3,226 English-
language news sentences. Unlike prior resources that focus solely on claim detection and verification, FactAppeal identifies the nuanced epistemic structures and evidentiary basis underlying these claims and used to support them. FactAppeal contains span-level annotations which identify factual statements and mentions of sources on which they rely. Moreover, the annotations include fine-grained characteristics of factual appeals such as the type of source (e.g. Active Participant, Witness, Expert, Direct Evidence), whether it is mentioned by name, mentions of the source's role and epistemic credentials, attribution to the source via direct or indirect quotation, and other features. We model the task with a range of encoder models and generative decoder models in the 2B-9B parameter range. Our best performing model, based on Gemma 2 9B, achieves a macro-F1 score of 0.73.
Read the full abstract ▾
Computation and Language
Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
Bowei He, Lihao Yin, Huiling Zhen et al. (+5 more)
Post-training compression has been a widely employed approach to scale down large
language model (LLM) and facilitate efficient inference. In various proposed compression methods, including
pruning and quantization, calibration data plays a vital role by informing the weight importance and activation dynamic ranges. However, how calibration data impacts the LLM capability after compression is less explored. Few of the existing works, though recognizing the significance of this study, only investigate the
language modeling or commonsense
reasoning performance degradation from limited angles, like the data sources or sample amounts. More systematic research is still needed to examine the impacts on different LLM capabilities in terms of compositional properties and domain correspondence of calibration data. In this work, we aim at bridging this gap and further analyze underlying influencing mechanisms from the activation pattern perspective. Especially, we explore the calibration data's impacts on high-level complex
reasoning capabilities, like math problem solving and code generation. Delving into the underlying mechanism, we find that the representativeness and diversity in activation space more fundamentally determine the quality of calibration data. Finally, we propose a calibration data curation framework based on such observations and analysis, enhancing the performance of existing post-training compression methods on preserving critical LLM capabilities. Our code is provided in \href{https://github.com/BokwaiHo/COLA.git}{Link}.
Read the full abstract ▾
All authors Bowei He, Lihao Yin, Huiling Zhen, Shuqi Liu, Han Wu, Xiaokun Zhang, Mingxuan Yuan, Chen Ma
Computation and Language
Dynamic Topic Evolution with Temporal Decay and Attention in Large Language Models
Di Wu and Shuaidong Pan
This paper proposes a modeling framework for dynamic topic
evolution based on temporal large
language models. The method first uses a large
language model to obtain contextual
embeddings of text and then introduces a temporal decay function and an
attention mechanism. These components allow the model to adjust the importance of semantic units according to time intervals and capture topic variations across different periods. The temporal representations are then mapped into a latent topic space, where a state transition
matrix is applied to describe the dynamic
evolution of topics. A joint optimization objective constrains both semantic modeling and temporal consistency, ensuring diversity and smoothness in topic generation. The design emphasizes the unified modeling of semantic representation and temporal
evolution, which improves topic coherence and diversity while enhancing stability and interpretability over time. Experiments on real-world corpora show that the framework effectively captures the generation, expansion, and decline of topics and outperforms existing models across multiple metrics. Overall, the proposed method provides a systematic solution for understanding dynamic semantic patterns in large-scale text, enriches the research paradigm of topic modeling, and supports complex text analysis tasks in multiple domains.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
A Layered Intuition – Method Model with Scope Extension for LLM Reasoning
Hong Su
Existing studies have introduced method-based
reasoning and scope extension as approaches to enhance Large
Language Model (LLM) performance beyond direct
matrix mappings. Building on these foundations, this paper summarizes and integrates these ideas into a unified Intuition-Method Layered Model with Scope Extension, designed to address indirected (unseen) issues more systematically. In this framework, intuition-based thinking provides rapid first-reaction answers, while method-based thinking decouples questions and solutions into transferable
reasoning units. Scope extension is then applied to broaden applicability, including vertical (cause analysis), horizontal (parallel and generalized issues), and for the first time, temporal and spatial extensions, which expand
reasoning across time and contextual dimensions. These extensions are organized into systematic knowledge trees that interconnect into a knowledge network, thereby increasing adaptability. To quantitatively evaluate this process, we propose the
entropy of method extension, which measures the independence and diversity of extensions as an indicator of the system's capacity to solve unseen questions. By logically connecting existing approaches with new extensions and introducing an
entropy-based evaluation framework, this work advances toward a more robust and extensible
reasoning paradigm for LLMs in real-world problem-solving.
Read the full abstract ▾
Computation and Language
BitMar: Low-Bit Multimodal Fusion with Episodic Memory for Edge Devices
Euhid Aman, Esteban Carlin, Hsing-Kuo Pao et al. (+3 more)
Cross-
attention transformers and other
multimodal vision-
language models excel at grounding and generation; however, their extensive, full-precision backbones make it challenging to deploy them on edge devices.
Memory-augmented architectures enhance the utilization of past context; however, most works rarely pair them with aggressive edge-oriented quantization. We introduce BitMar, a quantized
multimodal transformer that proposes an external human-like episodic
memory for effective image-text generation on hardware with limited resources. BitMar utilizes 1.58-bit encoders, one for text (BitNet-style) and one for
vision (DiNOv2-based), to create compact
embeddings that are combined and used to query a fixed-size key-value episodic
memory. During vector
retrieval, the BitNet decoder applies per-layer conditioning, which increases the contextual relevance of generated content. The decoder also employs
attention sinks with a sliding-window mechanism to process long or streaming inputs under tight
memory budgets. The combination of per-layer conditioning and sliding-window
attention achieves a strong quality-speed trade-off, delivering competitive captioning and
multimodal understanding at low latency with a small model footprint. These characteristics make BitMar well-suited for edge deployment.
Read the full abstract ▾
All authors Euhid Aman, Esteban Carlin, Hsing-Kuo Pao, Giovanni Beltrame, Ghaluh Indah Permata Sari, Yie-Tarng Chen
Computation and Language
Detecting Hallucinations in Authentic LLM-Human Interactions
Yujie Ren, Niklas Gruhlke and Anne Lauscher
As large
language models (LLMs) are increasingly applied in sensitive domains such as medicine and law,
hallucination detection has become a critical task. Although numerous
benchmarks have been proposed to advance research in this area, most of them are artificially constructed–either through deliberate
hallucination induction or simulated interactions–rather than derived from genuine LLM-human dialogues. Consequently, these
benchmarks fail to fully capture the characteristics of
hallucinations that occur in real-world usage. To address this limitation, we introduce AuthenHallu, the first
hallucination detection
benchmark built entirely from authentic LLM-human interactions. For AuthenHallu, we select and annotate samples from genuine LLM-human dialogues, thereby providing a faithful reflection of how LLMs hallucinate in everyday user interactions. Statistical analysis shows that
hallucinations occur in 31.4% of the query-response pairs in our
benchmark, and this proportion increases dramatically to 60.0% in challenging domains such as Math & Number Problems. Furthermore, we explore the potential of using vanilla LLMs themselves as
hallucination detectors and find that, despite some promise, their current performance remains insufficient in real-world scenarios. The data and code are publicly available at https://github.com/TAI-HAMBURG/AuthenHallu.
Read the full abstract ▾
Computation and Language
Merlin's Whisper: Enabling Efficient Reasoning in Large Language Models via Black-box Persuasive Prompting
Heming Xia, Cunxiao Du, Rui Li et al. (+3 more)
Large
reasoning models (LRMs) have demonstrated remarkable proficiency in tackling complex tasks through step-by-step thinking. However, this lengthy
reasoning process incurs substantial computational and latency overheads, hindering the practical deployment of LRMs. This work presents a new approach to mitigating overthinking in LRMs via black-box persuasive prompting. By treating LRMs as black-box communicators, we investigate how to persuade them to generate concise responses without compromising accuracy. We introduce Whisper, an iterative refinement framework that generates high-quality persuasive
prompts from diverse perspectives. Experiments across multiple
benchmarks demonstrate that Whisper consistently reduces
token usage while preserving performance. Notably, Whisper achieves a 3x reduction in average response length on simple GSM8K questions for the Qwen3 model series and delivers an average ~40%
token reduction across all
benchmarks. For closed-source APIs, Whisper reduces
token usage on MATH-500 by 46% for Claude-3.7 and 50% for Gemini-2.5. Further analysis reveals the broad applicability of Whisper across data domains, model scales, and families, underscoring the potential of black-box persuasive prompting as a practical strategy for enhancing LRM efficiency.
Read the full abstract ▾
All authors Heming Xia, Cunxiao Du, Rui Li, Chak Tou Leong, Yongqi Li, Wenjie Li
Computation and Language
VOLTAGE: A Versatile Contrastive Learning based OCR Methodology for ultra low-resource scripts through Auto Glyph Feature Extraction
Prawaal Sharma, Poonam Goyal, Vidisha Sharma et al. (+1 more)
UNESCO has classified 2500 out of 7000
languages spoken worldwide as endangered. Attrition of a
language leads to loss of traditional wisdom, folk literature, and the essence of the community that uses it. It is therefore imperative to bring digital inclusion to these
languages and avoid its extinction. Low resource
languages are at a greater risk of extinction. Lack of unsupervised Optical Character Recognition(OCR) methodologies for low resource
languages is one of the reasons impeding their digital inclusion. We propose VOLTAGE - a contrastive learning based OCR methodology, leveraging auto-glyph feature recommendation for cluster-based labelling. We augment the labelled data for diversity and volume using image transformations and Generative
Adversarial Networks. Voltage has been designed using Takri - a family of scripts used in 16th to 20th century in the Himalayan regions of India. We present results for Takri along with other Indic scripts (both low and high resource) to substantiate the universal behavior of the methodology. An accuracy of 95% for machine printed and 87% for handwritten samples on Takri script has been achieved. We conduct baseline and ablation studies along with building downstream use cases for Takri, demonstrating the usefulness of our work.
Read the full abstract ▾
All authors Prawaal Sharma, Poonam Goyal, Vidisha Sharma, Navneet Goyal
Computation and Language
UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
Guangxin He, Shen Nie, Fengqi Zhu et al. (+6 more)
Diffusion LLMs have attracted growing interest, with plenty of recent work emphasizing their great potential in various downstream tasks; yet the long-context behavior of
diffusion LLMs remains largely uncharted. We present a case study of post-training techniques for extending the context window of
diffusion LLMs (i.e., LLaDA) without retraining from scratch. We show that a simple modification to the standard Rotary Positional
Embeddings (RoPE) extension effectively accommodates the probabilistic modeling inherent in the
diffusion process, enabling stable scaling to longer context ranges. We further compare masking strategies used during post-training and analyze their impact on optimization stability and long-range recall. Instantiating these insights, we introduce UltraLLaDA, a
diffusion LLM with a 128K-
token context window that, in our empirical evaluation on long-context tasks, significantly outperforms training-free baselines. Our experimental results highlight the special positional extension as a key lever for scaling
diffusion LLMs to extended contexts and offer practical guidance for practitioners seeking 128K-scale context via efficient post-training.
Read the full abstract ▾
All authors Guangxin He, Shen Nie, Fengqi Zhu, Yuankang Zhao, Tianyi Bai, Ran Yan, Jie Fu, Chongxuan Li, Binhang Yuan
Computation and Language
Assessing Large Language Models for Structured Medical Order Extraction
A H M Rezaul Karim and Ozlem Uzuner
Medical order extraction is essential for structuring actionable clinical information, supporting decision-making, and enabling downstream applications such as documentation and workflow automation. Orders may be embedded in diverse sources, including electronic health records, discharge summaries, and multi-turn doctor-patient dialogues, and can span categories such as medications, laboratory tests, imaging studies, and follow-up actions. The MEDIQA-OE 2025 shared task focuses on extracting structured medical orders from extended conversational transcripts, requiring the identification of order type, description, reason, and provenance. We present the MasonNLP submission, which ranked 5th among 17 participating teams with 105 total submissions. Our approach uses a general-purpose, instruction-tuned LLaMA-4 17B model without domain-specific fine-tuning, guided by a single in-context example. This few-shot configuration achieved an average F1 score of 37.76, with notable improvements in reason and provenance accuracy. These results demonstrate that large, non-domain-specific LLMs, when paired with effective
prompt engineering, can serve as strong, scalable baselines for specialized clinical NLP tasks.
Read the full abstract ▾
Computation and Language
When or What? Understanding Consumer Engagement on Digital Platforms
Jingyi Wu and Junying Liang
Understanding what drives popularity is critical in today's digital service economy, where content creators compete for consumer
attention. Prior studies have primarily emphasized the role of content features, yet creators often misjudge what audiences actually value. This study applies Latent Dirichlet Allocation (LDA) modeling to a large corpus of TED Talks, treating the platform as a case of digital service provision in which creators (speakers) and consumers (audiences) interact. By comparing the thematic supply of creators with the demand expressed in audience engagement, we identify persistent mismatches between producer offerings and consumer preferences. Our longitudinal analysis further reveals that temporal dynamics exert a stronger influence on consumer engagement than thematic content, suggesting that when content is delivered may matter more than what is delivered. These findings challenge the dominant assumption that content features are the primary drivers of popularity and highlight the importance of timing and contextual factors in shaping consumer responses. The results provide new insights into consumer
attention dynamics on digital platforms and carry practical implications for marketers, platform managers, and content creators seeking to optimize audience engagement strategies.
Read the full abstract ▾
Computation and Language
FML-bench: Benchmarking Machine Learning Agents for Scientific Research
Qiran Zou, Hou Hei Lam, Wenhao Zhao et al. (+7 more)
Large
language models (LLMs) have sparked growing interest in machine learning research
agents that can autonomously propose ideas and conduct experiments. However, existing
benchmarks predominantly adopt an engineering-oriented perspective: they emphasize application-oriented tasks and evaluate primarily on final performance and computational cost, overlooking
agents' research processes and limiting assessment of their capabilities in scientific research settings. To more comprehensively evaluate
agents in scientific research settings, we introduce FML-bench, a
benchmark comprising 8 diverse and fundamental ML research tasks, and further propose complementary metrics, notably Exploration Diversity, which quantifies the variance of proposals across iterations and reveals how exploration patterns influence research outcomes. We evaluate state-of-the-art research
agents on FML-bench, showing that
agents employing broad exploration strategies exhibit higher exploration diversity and achieve superior performance, and that exploration diversity positively correlates with performance improvements across multiple tasks. We hope these findings and our
benchmark inform future
agent design and support the community in further investigating
agent behavior. Our
benchmark is available at https://github.com/qrzou/FML-bench.
Read the full abstract ▾
All authors Qiran Zou, Hou Hei Lam, Wenhao Zhao, Yiming Tang, Tingting Chen, Samson Yu, Tianyi Zhang, Chang Liu, Xiangyang Ji, Dianbo Liu
Computation and Language
NIM: Neuro-symbolic Ideographic Metalanguage for Inclusive Communication
Prawaal Sharma, Poonam Goyal, Navneet Goyal et al. (+1 more)
Digital communication has become the cornerstone of modern interaction, enabling rapid, accessible, and interactive exchanges. However, individuals with lower academic literacy often face significant barriers, exacerbating the "digital divide". In this work, we introduce a novel, universal ideographic metalanguage designed as an innovative communication framework that transcends academic, linguistic, and cultural boundaries. Our approach leverages principles of Neuro-symbolic AI, combining neural-based large
language models (LLMs) enriched with world knowledge and symbolic knowledge heuristics grounded in the linguistic theory of Natural Semantic Metalanguage (NSM). This enables the semantic decomposition of complex ideas into simpler, atomic concepts. Adopting a human-centric, collaborative methodology, we engaged over 200 semi-literate participants in defining the problem, selecting ideographs, and validating the system. With over 80% semantic comprehensibility, an accessible learning curve, and universal adaptability, our system effectively serves underprivileged populations with limited formal education.
Read the full abstract ▾
All authors Prawaal Sharma, Poonam Goyal, Navneet Goyal, Vidisha Sharma
Computation and Language
Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
Shaobo Wang, Cong Wang, Wenjie Fu et al. (+11 more)
As the demand for comprehensive evaluations of diverse model capabilities steadily increases,
benchmark suites have correspondingly grown significantly in scale. Despite notable advances in redundancy reduction and subset-level performance prediction, a systematic framework that effectively integrates these methods to ensure both prediction accuracy and ranking consistency is still largely elusive. In this paper, we first perform a sample-level analysis of
benchmark redundancy and identify several highly similar samples that can be eliminated. Besides, we frame
benchmark compression as an optimization problem with the aim of score reconstruction. Building on these, we then propose EssenceBench, a coarse-to-fine framework utilizing an iterative Genetic
Algorithm (GA), which takes the advantages of fitness-based subset search and attribution-based sample search. Compared to previous methods, our approach yields superior compression results with lower reconstruction error and markedly higher efficiency. In particular, on the HellaSwag
benchmark (10K samples), our method preserves the ranking of all models shifting within 5% using 25x fewer samples, and achieves 95% ranking preservation shifting within 5% using only 200x fewer samples.
Read the full abstract ▾
All authors Shaobo Wang, Cong Wang, Wenjie Fu, Yue Min, Mingquan Feng, Isabel Guan, Xuming Hu, Conghui He, Cunxiang Wang, Kexin Yang, Xingzhang Ren, Fei Huang, Dayiheng Liu, Linfeng Zhang
Computation and Language
End-to-end Speech Recognition with similar length speech and text
Peng Fan, Wenping Wang and Fei Deng
The mismatch of
speech length and text length poses a challenge in automatic
speech recognition (ASR). In previous research, various approaches have been employed to align text with
speech, including the utilization of Connectionist Temporal Classification (CTC). In earlier work, a key frame mechanism (KFDS) was introduced, utilizing intermediate CTC outputs to guide downsampling and preserve keyframes, but traditional methods (CTC) failed to align
speech and text appropriately when downsampling
speech to a text-similar length. In this paper, we focus on
speech recognition in those cases where the length of
speech aligns closely with that of the corresponding text. To address this issue, we introduce two methods for
alignment: a) Time Independence Loss (TIL) and b) Aligned Cross
Entropy (AXE) Loss, which is based on edit distance. To enhance the information on keyframes, we incorporate frame fusion by applying weights and summing the keyframe with its context 2 frames. Experimental results on AISHELL-1 and AISHELL-2
dataset subsets show that the proposed methods outperform the previous work and achieve a reduction of at least 86% in the number of frames.
Read the full abstract ▾
Computation and Language
Steering Over-refusals Towards Safety in Retrieval Augmented Generation
Utsav Maskey, Mark Dras and Usman Naseem
Safety
alignment in large
language models (LLMs) induces over-refusals – where LLMs decline benign requests due to aggressive safety filters. We analyze this phenomenon in
retrieval-augmented generation (RAG), where both the query intent and retrieved context properties influence refusal behavior. We construct RagRefuse, a domain-stratified
benchmark spanning medical, chemical, and open domains, pairing benign and harmful queries with controlled context contamination patterns and sizes. Our analysis shows that context arrangement / contamination, domain of query and context, and harmful-text density trigger refusals even on benign queries, with effects depending on model-specific
alignment choices. To mitigate over-refusals, we introduce SafeRAG-Steering, a model-centric
embedding intervention that steers the
embedding regions towards the confirmed safe, non-refusing output regions at inference time. This reduces over-refusals in contaminated RAG pipelines while preserving legitimate refusals.
Read the full abstract ▾