Computation and Language
KD4MT: A Survey of Knowledge Distillation for Machine Translation
Ona de Gibert, Joseph Attieh, Timothee Mickus et al. (+2 more)
Knowledge
Distillation (KD) as a research area has gained a lot of traction in recent years as a compression tool to address challenges related to ever-larger models in NLP. Remarkably, Machine Translation (MT) offers a much more nuanced take on this narrative: in MT, KD also functions as a general-purpose knowledge transfer mechanism that shapes supervision and translation quality as well as efficiency. This survey synthesizes KD for MT (KD4MT) across 105 papers (through October 1, 2025). We begin by introducing both MT and KD for non-experts, followed by an overview of the standard KD approaches relevant to MT applications. Subsequently, we categorize advances in the KD4MT literature based on (i) their methodological contributions and (ii) their practical applications. Our qualitative and quantitative analyses identify common trends in the field and highlight key research gaps as well as the absence of unified evaluation practice for KD methods in MT. We further provide practical guidelines for selecting a KD method in concrete settings and highlight potential risks associated with the application of KD to MT such as increased
hallucination and bias amplification. Finally, we discuss the role of LLMs in re-shaping the KD4MT field. To support further research, we complement our survey with a publicly available database summarizing the main characteristics of the surveyed KD methods and a glossary of key terms.
Read the full abstract ▾
All authors Ona de Gibert, Joseph Attieh, Timothee Mickus, Yves Scherrer, Jörg Tiedemann
Computation and Language
EduResearchBench: A Hierarchical Atomic Task Decomposition Benchmark for Full-Lifecycle Educational Research
Houping Yue, Zixiang Di, Mei Jiang et al. (+5 more)
While Large
Language Models (LLMs) are reshaping the paradigm of AI for Social Science (AI4SS), rigorously evaluating their capabilities in scholarly writing remains a major challenge. Existing
benchmarks largely emphasize single-shot, monolithic generation and thus lack the fine-grained assessments required to reflect complex academic research workflows. To fill this gap, we introduce EduResearchBench, the first comprehensive evaluation platform dedicated to educational academic writing. EduResearchBench is built upon our Hierarchical Atomic Task Decomposition (HATD) framework, which decomposes an end-to-end research workflow into six specialized research modules (e.g., Quantitative Analysis, Qualitative Research, and Policy Research) spanning 24 fine-grained atomic tasks. This taxonomy enables an automated evaluation pipeline that mitigates a key limitation of holistic scoring, where aggregate scores often obscure specific capability bottlenecks, and instead provides fine-grained, diagnostic feedback on concrete deficiencies. Moreover, recognizing the high cognitive load inherent in scholarly writing, we propose a curriculum learning strategy that progressively builds competence from foundational skills to complex methodological
reasoning and argumentation. Leveraging 55K raw academic samples, we curate 11K high-quality instruction pairs to train EduWrite, a specialized educational scholarly writing model. Experiments show that EduWrite (30B) substantially outperforms larger general-purpose models (72B) on multiple core metrics, demonstrating that in vertical domains, data quality density and hierarchically staged training curricula are more decisive than parameter scale.
Read the full abstract ▾
All authors Houping Yue, Zixiang Di, Mei Jiang, Bingdong Li, Hao Hao, Yu Song, Bo Jiang, Aimin Zhou
Computation and Language · Cross-listed from cs.LG
CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Models
Shuozhe Li, Jincheng Cao, Bodun Hu et al. (+3 more)
Reinforcement finetuning (RFT) has emerged as a powerful paradigm for unlocking
reasoning capabilities in large
language models. However, we identify a critical trade-off: while unconstrained RFT achieves strong
reasoning performance, it severely compromises model trustworthiness by amplifying
hallucination and worsening calibration; conversely, RKL-constrained RFT preserves trustworthiness but limits
reasoning gains due to its unbounded penalty on exploratory deviations. To resolve this tension, we introduce CARE-RFT (Confidence-Anchored Regularized Reinforcement Finetuning), a novel method that replaces standard reverse KL regularization with a skew reverse KL divergence. CARE-RFT provides a confidence-sensitive penalty: it is bounded for confident, consistently rewarded explorations to enable
reasoning, while unbounded elsewhere to preserve calibration. Extensive experiments across multiple model scales and RFT
algorithms show that CARE-RFT achieves a superior balance, matching the
reasoning performance of unconstrained RFT while recovering the trustworthiness and calibration of the base model. Our work establishes that careful, confidence-aware regularization is key to building both capable and trustworthy
reasoning models.
Read the full abstract ▾
All authors Shuozhe Li, Jincheng Cao, Bodun Hu, Aryan Mokhtari, Leqi Liu, Amy Zhang
Computation and Language · Cross-listed from cs.IR
SPARC-RAG: Adaptive Sequential-Parallel Scaling with Context Management for Retrieval-Augmented Generation
Yuxin Yang, Gangda Deng, Ömer Faruk Akgül et al. (+6 more)
Retrieval-Augmented Generation (RAG) grounds large
language model outputs in external evidence, but remains challenged on multi-hop question answering that requires long
reasoning. Recent works scale RAG at inference time along two complementary dimensions: sequential depth for iterative refinement and parallel width for coverage expansion. However, naive scaling causes context contamination and scaling inefficiency, leading to diminishing or negative returns despite increased computation. To address these limitations, we propose SPARC-RAG, a multi-
agent framework that coordinates sequential and parallel inference-time scaling under a unified context management mechanism. SPARC-RAG employs specialized
agents that maintain a shared global context and provide explicit control over the scaling process. It generates targeted, complementary sub-queries for each branch to enable diverse parallel exploration, and explicitly regulates exiting decisions based on answer correctness and evidence grounding. To optimize scaling behavior, we further introduce a lightweight fine-tuning method with process-level verifiable preferences, which improves the efficiency of sequential scaling and effectiveness of parallel scaling. Across single- and multi-hop QA
benchmarks, SPARC-RAG consistently outperforms previous RAG baselines, yielding an average +6.2 F1 improvement under lower inference cost.
Read the full abstract ▾
All authors Yuxin Yang, Gangda Deng, Ömer Faruk Akgül, Nima Chitsazan, Yash Govilkar, Akasha Tigalappanavara, Shi-Xiong Zhang, Sambit Sahu, Viktor Prasanna
Computation and Language · Cross-listed from cs.LG
ThinkTank-ME: A Multi-Expert Framework for Middle East Event Forecasting
Haoxuan Li, He Chang, Yunshan Ma et al. (+4 more)
Event
forecasting is inherently influenced by multifaceted considerations, including international relations, regional historical dynamics, and cultural contexts. However, existing LLM-based approaches employ single-model architectures that generate predictions along a singular explicit trajectory, constraining their ability to capture diverse geopolitical nuances across complex regional contexts. To address this limitation, we introduce ThinkTank-ME, a novel Think Tank framework for Middle East event
forecasting that emulates collaborative expert analysis in real-world strategic decision-making. To facilitate expert specialization and rigorous evaluation, we construct POLECAT-FOR-ME, a Middle East-focused event
forecasting benchmark. Experimental results demonstrate the superiority of multi-expert collaboration in handling complex temporal geopolitical
forecasting tasks. The code is available at https://github.com/LuminosityX/ThinkTank-ME.
Read the full abstract ▾
All authors Haoxuan Li, He Chang, Yunshan Ma, Yi Bin, Yang Yang, See-Kiong Ng, Tat-Seng Chua
Computation and Language · Cross-listed from cs.DB
Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs
Wei Zhou, Jun Zhou, Haoyu Wang et al. (+16 more)
Data preparation aims to denoise raw
datasets, uncover cross-
dataset relationships, and extract valuable insights from them, which is essential for a wide range of data-centric applications. Driven by (i) rising demands for application-ready data (e.g., for analytics, visualization, decision-making), (ii) increasingly powerful LLM techniques, and (iii) the emergence of infrastructures that facilitate flexible
agent construction (e.g., using Databricks Unity Catalog), LLM-enhanced methods are rapidly becoming a transformative and potentially dominant paradigm for data preparation. By investigating hundreds of recent literature works, this paper presents a systematic review of this evolving landscape, focusing on the use of LLM techniques to prepare data for diverse downstream tasks. First, we characterize the fundamental paradigm shift, from rule-based, model-specific pipelines to
prompt-driven, context-aware, and agentic preparation workflows. Next, we introduce a task-centric taxonomy that organizes the field into three major tasks: data cleaning (e.g., standardization, error processing, imputation), data integration (e.g., entity matching, schema matching), and data enrichment (e.g., data annotation, profiling). For each task, we survey representative techniques, and highlight their respective strengths (e.g., improved generalization, semantic understanding) and limitations (e.g., the prohibitive cost of scaling LLMs, persistent
hallucinations even in advanced
agents, the mismatch between advanced methods and weak evaluation). Moreover, we analyze commonly used
datasets and evaluation metrics (the empirical part). Finally, we discuss open research challenges and outline a forward-looking roadmap that emphasizes scalable LLM-data systems, principled designs for reliable agentic workflows, and robust evaluation protocols.
Read the full abstract ▾
All authors Wei Zhou, Jun Zhou, Haoyu Wang, Zhenghao Li, Qikang He, Shaokun Han, Guoliang Li, Xuanhe Zhou, Yeye He, Chunwei Liu, Zirui Tang, Bin Wang, Shen Tang, Kai Zuo, Yuyu Luo, Zhenzhe Zheng, Conghui He, Jingren Zhou, Fan Wu
Computation and Language
Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humans
Suhong Moon, Minwoo Kang, Joseph Suh et al. (+2 more)
Humans act via a nuanced process that depends both on rational deliberation and also on identity and contextual factors. In this work, we study how large
language models (LLMs) can simulate human action in the context of social dilemma games. While prior work has focused on "steering" (weak binding) of chat models to simulate personas, we analyze here how deep binding of base models with extended backstories leads to more faithful replication of identity-based behaviors. Our study has these findings: simulation fidelity vs human studies is improved by conditioning base LMs with rich context of narrative identities and checking consistency using instruction-tuned models. We show that LLMs can also model contextual factors such as time (year that a study was performed), question framing, and participant pool effects. LLMs, therefore, allow us to explore the details that affect human studies but which are often omitted from experiment descriptions, and which hamper accurate replication.
Read the full abstract ▾
All authors Suhong Moon, Minwoo Kang, Joseph Suh, Mustafa Safdari, John Canny
Computation and Language
Regional Bias in Large Language Models
M P V S Gopinadh, Kappara Lakshmi Sindhu, Soma Sekhar Pandu Ranga Raju P et al. (+1 more)
This study investigates regional bias in large
language models (LLMs), an emerging concern in AI
fairness and global representation. We evaluate ten prominent LLMs: GPT-3.5, GPT-4o, Gemini 1.5 Flash, Gemini 1.0 Pro, Claude 3 Opus, Claude 3.5 Sonnet, Llama 3, Gemma 7B, Mistral 7B, and Vicuna-13B using a
dataset of 100 carefully designed
prompts that probe forced-choice decisions between regions under contextually neutral scenarios. We introduce FAZE, a
prompt-based evaluation framework that measures regional bias on a 10-point scale, where higher scores indicate a stronger tendency to favor specific regions. Experimental results reveal substantial variation in bias levels across models, with GPT-3.5 exhibiting the highest bias score (9.5) and Claude 3.5 Sonnet scoring the lowest (2.5). These findings indicate that regional bias can meaningfully undermine the reliability,
fairness, and inclusivity of LLM outputs in real-world, cross-cultural applications. This work contributes to AI
fairness research by highlighting the importance of inclusive evaluation frameworks and systematic approaches for identifying and mitigating geographic biases in
language models.
Read the full abstract ▾
All authors M P V S Gopinadh, Kappara Lakshmi Sindhu, Soma Sekhar Pandu Ranga Raju P, Yesaswini Swarna
Computation and Language · Cross-listed from cs.CV
Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Highlights in Videos
Aditya K Surikuchi, Raquel Fernández and Sandro Pezzelle
Foundation models are used for many real-world applications involving
language understanding and generation from temporally-ordered
multimodal events. In this work, we study the ability of models to identify the most important sub-events in a
video, which is a fundamental prerequisite for summarizing,
reasoning over, or narrating
multimodal events. Specifically, we focus on football games and evaluate models on their ability to distinguish between important and non-important sub-events in a game. To this end, we construct a new
dataset by leveraging publicly available football highlight reels containing the most important moments of a game according to professional experts, without any additional annotation costs. Using our
dataset, we compare several state-of-the-art
multimodal models and show that they fail to reliably capture contextual importance. Analyses of models beyond standard evaluation metrics reveal their tendency to rely on a single dominant modality and their ineffectiveness in synthesizing necessary information from multiple sources. Our findings underline the importance of modular architectures that dynamically adapt to
multimodal heterogeneity and the need for complementary training procedures that maximize cross-modal synergy.
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spotting
Oguzhan Buyuksolak, Alican Gok and Osman Erman Okman
We introduce an efficient few-shot keyword spotting model for edge devices, EdgeSpot, that pairs an optimized version of a BC-ResNet-based acoustic backbone with a trainable Per-Channel Energy Normalization frontend and lightweight temporal self-
attention. Knowledge
distillation is utilized during training by employing a self-supervised teacher model, optimized with Sub-center ArcFace loss. This study demonstrates that the EdgeSpot model consistently provides better accuracy at a fixed false-alarm rate (FAR) than strong BC-ResNet baselines. The largest variant, EdgeSpot-4, improves the 10-shot accuracy at 1% FAR from 73.7% to 82.0%, which requires only 29.4M MACs with 128k parameters.
Read the full abstract ▾
Computation and Language
Machine-Assisted Grading of Nationwide School-Leaving Essay Exams with LLMs and Statistical NLP
Andres Karjus, Kais Allkivi, Silvia Maine et al. (+3 more)
Large
language models (LLMs) enable rapid and consistent automated evaluation of open-ended exam responses, including dimensions of content and argumentation that have traditionally required human judgment. This is particularly important in cases where a large amount of exams need to be graded in a limited time frame, such as nation-wide graduation exams in various countries. Here, we examine the applicability of automated scoring on two large
datasets of trial exam essays of two full national cohorts from Estonia. We operationalize the official curriculum-based rubric and compare LLM and statistical natural
language processing (NLP) based assessments with human panel scores. The results show that automated scoring can achieve performance comparable to that of human raters and tends to fall within the human scoring range. We also evaluate bias,
prompt injection risks, and LLMs as essay writers. These findings demonstrate that a principled, rubric-driven, human-in-the-loop scoring pipeline is viable for high-stakes writing assessment, particularly relevant for digitally advanced societies like Estonia, which is about to adapt a fully electronic examination system. Furthermore, the system produces fine-grained subscore profiles that can be used to generate systematic, personalized feedback for instruction and exam preparation. The study provides evidence that LLM-assisted assessment can be implemented at a national scale, even in a small-
language context, while maintaining human oversight and compliance with emerging educational and regulatory standards.
Read the full abstract ▾
All authors Andres Karjus, Kais Allkivi, Silvia Maine, Katarin Leppik, Krister Kruusmaa, Merilin Aruvee
Computation and Language
Teaching and Evaluating LLMs to Reason About Polymer Design Related Tasks
Dikshya Mohanty, Mohammad Saqib Hasan, Syed Mostofa Monsur et al. (+3 more)
Research in AI4Science has shown promise in many science applications, including
polymer design. However, current LLMs are ineffective in this problem space because: (i) most models lack
polymer-specific knowledge, and (ii) existing aligned models have limited coverage of knowledge and capabilities relevant to
polymer design. Addressing this, we introduce PolyBench, a large-scale training and test
benchmark dataset of more than 125K
polymer design-related tasks, leveraging a knowledge base of more than 13 million data points obtained from experimental and synthetic data sources to ensure broad coverage of
polymers and their properties. For effective
alignment using PolyBench, we introduce a knowledge-augmented
reasoning distillation method that augments this
dataset with structured CoT. Furthermore, tasks in PolyBench are organized from simple to complex analytical
reasoning problems, enabling generalization tests and diagnostic probes across the problem space. Experiments show that small- and mid- sized
language models (SLMs) with 7B to 32BB parameters, trained on PolyBench, outperform similar-sized models and remain competitive with closed-source frontier LLMs on PolyBench's test
dataset, while demonstrating performance gains on external
polymer benchmarks.
Dataset and associated code available at https://github.com/StonyBrookNLP/PolyBench.
Read the full abstract ▾
All authors Dikshya Mohanty, Mohammad Saqib Hasan, Syed Mostofa Monsur, Size Zheng, Benjamin Hsiao, Niranjan Balasubramanian
Computation and Language
A Longitudinal, Multinational, and Multilingual Corpus of News Coverage of the Russo-Ukrainian War
Dikshya Mohanty, Taisiia Sabadyn, Jelwin Rodrigues et al. (+3 more)
We present DNIPRO, a corpus of 246K news articles from the Russo-Ukrainian war (Feb 2022 – Aug 2024) spanning eleven outlets across five nation-states (Russia, Ukraine, U.S., U.K., China) and three
languages. The corpus features comprehensive metadata and human-evaluated annotations for stance, sentiment, and topical framing, enabling systematic analysis of competing geopolitical narratives. It is uniquely suited for empirical studies of narrative divergence, media framing, and information warfare. Our exploratory analyses reveal how media outlets construct incompatible realities through divergent attribution and topical selection without direct refutation of opposing narratives. DNIPRO empowers empirical research on narrative
evolution, cross-lingual information flow, and computational detection of implicit contradictions in fragmented information ecosystems.
Read the full abstract ▾
All authors Dikshya Mohanty, Taisiia Sabadyn, Jelwin Rodrigues, Chenlu Wang, Abhishek Kalugade, Ritwik Banerjee
Computation and Language
Generating Literature-Driven Scientific Theories at Scale
Peter Jansen, Peter Clark, Doug Downey et al. (+1 more)
Contemporary automated scientific discovery has focused on
agents for generating scientific experiments, while systems that perform higher-level scientific activities such as theory building remain underexplored. In this work, we formulate the problem of synthesizing theories consisting of qualitative and quantitative laws from large corpora of scientific literature. We study theory generation at scale, using 13.7k source papers to synthesize 2.9k theories, examining how generation using literature-grounding versus parametric knowledge, and accuracy-focused versus novelty-focused generation objectives change theory properties. Our experiments show that, compared to using parametric LLM
memory for generation, our literature-supported method creates theories that are significantly better at both matching existing evidence and at predicting future results from 4.6k subsequently-written papers
Read the full abstract ▾
All authors Peter Jansen, Peter Clark, Doug Downey, Daniel S. Weld
Computation and Language
Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification
Branislav Pecher, Jan Cegin, Robert Belanec et al. (+3 more)
Large
Language Models (LLMs) have demonstrated remarkable
multilingual capabilities, making them promising tools in both high- and low-resource
languages. One particularly valuable use case is generating synthetic samples that can be used to train smaller models in low-resource scenarios where human-labelled data is scarce. In this work, we investigate whether these synthetic data generation capabilities can serve as a form of
distillation, producing smaller models that perform on par with or even better than massive LLMs across
languages and tasks. To this end, we use a state-of-the-art
multilingual LLM to generate synthetic
datasets covering 11
languages and 4 classification tasks. These
datasets are then used to train smaller models via fine-tuning or instruction tuning, or as synthetic in-context examples for compact LLMs. Our experiments show that even small amounts of synthetic data enable smaller models to outperform the large generator itself, particularly in low-resource
languages. Overall, the results suggest that LLMs are best utilised as generators (teachers) rather than classifiers, producing data that empowers smaller and more efficient
multilingual models.
Read the full abstract ▾
All authors Branislav Pecher, Jan Cegin, Robert Belanec, Ivan Srba, Jakub Simko, Maria Bielikova
Computation and Language
GameTalk: Training LLMs for Strategic Conversation
Victor Conchello Vendrell, Max Ruiz Luyten and Mihaela van der Schaar
Strategic decision-making in multi-
agent settings is a key challenge for large
language models (LLMs), particularly when coordination and negotiation must unfold over extended conversations. While recent work has explored the use of LLMs in isolated decision tasks, little
attention has been given to optimizing long-term objectives through dialogue. We introduce GameTalk, a framework for training LLMs to make strategic decisions via multi-turn interactions. Unlike prior work that focuses on single-turn objectives or static action prediction, we train LLMs to optimize a global objective across full conversations. We achieve this by adapting fine-tuning methods like GRPO, DPO, and
STaR to incorporate reward signals that depend on the entire interaction. We evaluate this approach on a suite of increasingly complex games, designed to stress different aspects of
reasoning, coordination, and opponent modeling. Our results show that GameTalk significantly outperforms untrained models, especially under reward shaping, with DPO consistently yielding the strongest gains. These findings position conversational fine-tuning as a promising path for LLMs to reason, negotiate, and act in interactive environments.
Read the full abstract ▾
Computation and Language
Computer Environments Elicit General Agentic Intelligence in LLMs
Daixuan Cheng, Shaohan Huang, Yuxian Gu et al. (+6 more)
Agentic intelligence in large
language models (LLMs) requires not only model intrinsic capabilities but also interactions with external environments. Equipping LLMs with computers now represents a prevailing trend. However, the computer environment's intrinsic value has not been systematically investigated, particularly its potential to elicit general capabilities. Here we introduce LLM-in-Sandbox, which virtualizes the computer as a code sandbox with only basic functionalities, and demonstrate that this minimal setting elicits computer-based meta-capabilities for general task solving: external resource access, file management, and code execution. Without additional training, strong models achieve substantial gains (up to 15.5%) across mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following, while reducing
token consumption by up to 8 times. Furthermore, we develop LLM-in-Sandbox-RL to train models exclusively on non-agentic data within the sandbox, empowering weaker models to harness the environment and internalize these interactions. Our results demonstrate that computer environments elicit general intelligence, yield efficiency gains, and can be harnessed through training, serving as a promising foundation for generalist
agents.
Read the full abstract ▾
All authors Daixuan Cheng, Shaohan Huang, Yuxian Gu, Huatong Song, Guoxin Chen, Li Dong, Wayne Xin Zhao, Ji-Rong Wen, Furu Wei
Computation and Language
Automatic Classification of Arabic Literature into Historical Eras
Zainab Alhathloul and Irfan Ahmad
The Arabic
language has undergone notable transformations over time, including the emergence of new vocabulary, the obsolescence of others, and shifts in word usage. This
evolution is evident in the distinction between the classical and modern Arabic eras. Although historians and linguists have partitioned Arabic literature into multiple eras, relatively little research has explored the automatic classification of Arabic texts by time period, particularly beyond the domain of poetry. This paper addresses this gap by employing neural networks and deep learning techniques to automatically classify Arabic texts into distinct eras and periods. The proposed models are evaluated using two
datasets derived from two publicly available corpora, covering texts from the pre-Islamic to the modern era. The study examines class setups ranging from binary to 15-class classification and considers both predefined historical eras and custom periodizations. Results range from F1-scores of 0.83 and 0.79 on the binary-era classification task using the OpenITI and APCD
datasets, respectively, to 0.20 on the 15-era classification task using OpenITI and 0.18 on the 12-era classification task using APCD.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
LLM Prompt Evaluation for Educational Applications
Langdon Holmes, Adam Coscia, Scott Crossley et al. (+2 more)
As large
language models (LLMs) become increasingly common in educational applications, there is a growing need for evidence-based methods to design and evaluate LLM
prompts that produce personalized and pedagogically aligned out-puts. This study presents a generalizable, systematic approach for evaluating
prompts, demonstrated through an analysis of LLM-generated follow-up questions in a structured dialogue activity. Six
prompt templates were designed and tested. The templates incorporated established
prompt engineering patterns, with each
prompt emphasizing distinct pedagogical strategies. The
prompt templates were compared through a tournament-style evaluation framework that can be adapted for other educational applications. The tournament employed the Glicko2 rating system with eight judges evaluating question pairs across three dimensions: format, dialogue support, and appropriateness for learners. Data was sourced from 120 authentic user interactions across three distinct educational deployments. Results showed that a single
prompt related to strategic reading out-performed other templates with win probabilities ranging from 81% to 100% in pairwise comparisons. This
prompt combined persona and context manager pat-terns and was designed to support metacognitive learning strategies such as self-directed learning. The methodology showcases how educational technology re- searchers can systematically evaluate and improve
prompt designs, moving beyond ad-hoc
prompt engineering toward evidence-based
prompt development for educational applications.
Read the full abstract ▾
All authors Langdon Holmes, Adam Coscia, Scott Crossley, Joon Suh Choi, Wesley Morris
Computation and Language
Improving Training Efficiency and Reducing Maintenance Costs via Language Specific Model Merging
Alphaeus Dmonte, Vidhi Gupta, Daniel J Perry et al. (+1 more)
Fine-tuning a task-specific
multilingual large
language model (LLM) involves training the model on a
multilingual dataset with examples in all the required
languages. Updating one or more supported
languages with additional data or adding support for a new
language involves retraining the model, which can be computationally inefficient and creates a severe maintenance bottleneck. Recent research on merging
multilingual multitask models has shown promise in terms of improved quality, but its computational and maintenance efficiency remains unstudied. In this work, we provide the first focused analysis of this merging strategy from an efficiency perspective, evaluating it across three independent tasks. We demonstrate significant efficiency gains while maintaining parity in terms of quality: this merging approach reduces the initial training time by up to 50%. We also demonstrate that updating an individual
language and re-merging as part of model maintenance reduces training costs by more than 60%, compared to re-training the full
multilingual model. We show this on both public and proprietary industry
datasets confirming that the approach works well for industrial use cases in addition to academic settings already studied in previous work.
Read the full abstract ▾
All authors Alphaeus Dmonte, Vidhi Gupta, Daniel J Perry, Mark Arehart
Computation and Language · Cross-listed from cs.CV
Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing
Tingyu Song, Yanzhao Zhang, Mingxin Li et al. (+6 more)
Composed Image
Retrieval (CIR) is a pivotal and complex task in
multimodal understanding. Current CIR
benchmarks typically feature limited query categories and fail to capture the diverse requirements of real-world scenarios. To bridge this evaluation gap, we leverage image editing to achieve precise control over modification types and content, enabling a pipeline for synthesizing queries across a broad spectrum of categories. Using this pipeline, we construct EDIR, a novel fine-grained CIR
benchmark. EDIR encompasses 5,000 high-quality queries structured across five main categories and fifteen subcategories. Our comprehensive evaluation of 13
multimodal embedding models reveals a significant capability gap; even state-of-the-art models (e.g., RzenEmbed and GME) struggle to perform consistently across all subcategories, highlighting the rigorous nature of our
benchmark. Through comparative analysis, we further uncover inherent limitations in existing
benchmarks, such as modality biases and insufficient categorical coverage. Furthermore, an in-domain training experiment demonstrates the feasibility of our
benchmark. This experiment clarifies the task challenges by distinguishing between categories that are solvable with targeted data and those that expose intrinsic limitations of current model architectures.
Read the full abstract ▾
All authors Tingyu Song, Yanzhao Zhang, Mingxin Li, Zhuoning Guo, Dingkun Long, Pengjun Xie, Siyue Zhang, Yilun Zhao, Shu Wu
Computation and Language
synthocr-gen: A synthetic ocr dataset generator for low-resource languages- breaking the data barrier
Haq Nawaz Malik, Kh Mohmad Shafi and Tanveer Ahmad Reshi
Optical Character Recognition (OCR) for low-resource
languages remains a significant challenge due to the scarcity of large-scale annotated training
datasets.
Languages such as Kashmiri, with approximately 7 million speakers and a complex Perso-Arabic script featuring unique diacritical marks, currently lack support in major OCR systems including Tesseract, TrOCR, and PaddleOCR. Manual
dataset creation for such
languages is prohibitively expensive, time-consuming, and error-prone, often requiring word by word transcription of printed or handwritten text. We present SynthOCR-Gen, an open-source synthetic OCR
dataset generator specifically designed for low-resource
languages. Our tool addresses the fundamental bottleneck in OCR development by transforming digital Unicode text corpora into ready-to-use training
datasets. The system implements a comprehensive pipeline encompassing text
segmentation (character, word, n-gram, sentence, and line levels), Unicode normalization with script purity enforcement, multi-font rendering with configurable distribution, and 25+ data augmentation techniques simulating real-world document degradations including rotation, blur, noise, and scanner artifacts. We demonstrate the efficacy of our approach by generating a 600,000-sample word-segmented Kashmiri OCR
dataset, which we release publicly on HuggingFace. This work provides a practical pathway for bringing low-resource
languages into the era of
vision-
language AI models, and the tool is openly available for researchers and practitioners working with underserved writing systems worldwide.
Read the full abstract ▾
Computation and Language
Incremental Multilingual Text2Cypher with Adapter Combination
Makbule Gulcin Ozsoy
Large
Language Models enable users to access database using natural
language interfaces using tools like Text2SQL, Text2SPARQL, and Text2Cypher, which translate user questions into structured database queries. While these systems improve database accessibility, most research focuses on English with limited
multilingual support. This work investigates a scalable
multilingual Text2Cypher, aiming to support new
languages without re-running full fine-tuning, avoiding manual hyper-parameter tuning, and maintaining performance close to joint
multilingual fine-tuning. We train
language-specific LoRA adapters for English, Spanish, and Turkish and combined them via uniform linear merging or learned fusion MLP with dynamic gating. Experimental results show that the fusion MLP recovers around 75% of the accuracy gains from joint
multilingual fine-tuning while requiring only a smaller subset of the data, outperforming linear merging across all three
languages. This approach enables incremental
language expansion to new
languages by requiring only one LoRA adapter and a lightweight MLP retraining. Learned adapter fusion offers a practical alternative to expensive joint fine-tuning, balancing performance, data efficiency, and scalability for
multilingual Text2Cypher task.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Controlling Long-Horizon Behavior in Language Model Agents with Explicit State Dynamics
Sukesh Subaharan
Large
language model (LLM)
agents often exhibit abrupt shifts in tone and persona during extended interaction, reflecting the absence of explicit temporal structure governing
agent-level state. While prior work emphasizes turn-local sentiment or static
emotion classification, the role of explicit affective dynamics in shaping long-horizon
agent behavior remains underexplored. This work investigates whether imposing dynamical structure on an external affective state can induce temporal coherence and controlled recovery in multi-turn dialogue. We introduce an
agent-level affective subsystem that maintains a continuous Valence-Arousal-Dominance (VAD) state external to the
language model and governed by first- and second-order update rules. Instantaneous affective signals are extracted using a fixed, memoryless
estimator and integrated over time via exponential smoothing or momentum-based dynamics. The resulting affective state is injected back into generation without modifying model parameters. Using a fixed 25-turn dialogue protocol, we compare stateless, first-order, and second-order affective dynamics. Stateless
agents fail to exhibit coherent trajectories or recovery, while state persistence enables delayed responses and reliable recovery. Second-order dynamics introduce affective inertia and hysteresis that increase with momentum, revealing a trade-off between stability and responsiveness.
Read the full abstract ▾
Computation and Language
Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction
Tony Cristofano
Refusal behavior in aligned LLMs is often viewed as model-specific, yet we hypothesize it stems from a universal, low-dimensional semantic
circuit shared across models. To test this, we introduce Trajectory Replay via Concept-Basis Reconstruction, a framework that transfers refusal interventions from donor to target models, spanning diverse architectures (e.g., Dense to MoE) and training regimes, without using target-side refusal supervision. By aligning layers via concept fingerprints and reconstructing refusal directions using a shared “recipe” of concept atoms, we map the donor's ablation trajectory into the target's semantic space. To preserve capabilities, we introduce a weight-SVD stability guard that projects interventions away from high-variance weight subspaces to prevent collateral damage. Our evaluation across 8 model pairs confirms that these transferred recipes consistently attenuate refusal while maintaining performance, providing strong evidence for the semantic universality of safety
alignment.
Read the full abstract ▾
Computation and Language
Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain
Özgür Uğur, Mahmut Göksu, Mahmut Çimen et al. (+6 more)
This paper presents Mecellem models, a framework for developing specialized
language models for the Turkish legal domain through domain adaptation strategies. We make two contributions: (1)Encoder Model Pre-trained from Scratch: ModernBERT-based bidirectional encoders pre-trained on a Turkish-dominant corpus of 112.7 billion
tokens. We implement a checkpoint selection strategy that evaluates downstream
retrieval performance throughout training, revealing that optimal checkpoints achieve best
retrieval scores before pre-training loss reaches its minimum. Our encoder models achieve top-3 rankings on the Turkish
retrieval leaderboard, with smaller models (155M parameters) achieving comparable performance to larger reference models (307M-567M parameters). Our approach achieves 92.36% production efficiency compared to state-of-the-art models (embeddinggemma-300m: 100.00%, BAAI/bge-m3: 99.54%, newmindai/bge-m3-stsb: 94.38%), ranking fourth overall despite requiring less computational resources. SOTA models rely on multi-stage, computationally intensive training pipelines, making our single-stage pre-training followed by efficient post-training approach a cost-effective alternative; (2)Decoder Model with Continual Pre-training (CPT): Qwen3-1.7B and Qwen3-4B models adapted to Turkish legal domain through controlled curriculum learning. Four-phase CPT with optimal sample ratios enables gradual transition from general
language knowledge to specialized legal terminology and long-context
reasoning. This approach achieves 36.2% perplexity reduction on Turkish legal text, demonstrating domain adaptation gains.
Read the full abstract ▾
All authors Özgür Uğur, Mahmut Göksu, Mahmut Çimen, Musa Y{\i}lmaz, Esra Şavirdi, Alp Talha Demir, Rumeysa Güllüce, İclal Çetin, Ömer Can Sağbaş
Computation and Language
Transfer Learning from ImageNet for MEG-Based Decoding of Imagined Speech
Soufiane Jhilal, Stéphanie Martin and Anne-Lise Giraud
Non-invasive decoding of imagined
speech remains challenging due to weak, distributed signals and limited labeled data. Our paper introduces an image-based approach that transforms magnetoencephalography (MEG) signals into time-frequency representations compatible with pretrained
vision models. MEG data from 21 participants performing imagined
speech tasks were projected into three spatial scalogram mixtures via a learnable sensor-space convolution, producing compact image-like inputs for ImageNet-pretrained
vision architectures. These models outperformed classical and non-pretrained models, achieving up to 90.4% balanced accuracy for imagery vs. silence, 81.0% vs. silent reading, and 60.6% for vowel decoding. Cross-subject evaluation confirmed that pretrained models capture shared neural representations, and temporal analyses localized discriminative information to imagery-locked intervals. These findings show that pretrained
vision models applied to image-based MEG representations can effectively capture the structure of imagined
speech in non-invasive neural signals.
Read the full abstract ▾
Computation and Language
Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model
Chenghao Fan, Wen Heng, Bo Li et al. (+6 more)
Diffusion-based
language models (DLLMs) offer non-sequential, block-wise generation and richer data reuse compared to autoregressive (AR) models, but existing code DLLMs still lag behind strong AR baselines under comparable budgets. We revisit this setting in a controlled study and introduce Stable-DiffCoder, a block
diffusion code model that reuses the Seed-Coder architecture, data, and training pipeline. To enable efficient knowledge learning and stable training, we incorporate a block
diffusion continual pretraining (CPT) stage enhanced by a tailored warmup and block-wise clipped noise schedule. Under the same data and architecture, Stable-DiffCoder overall outperforms its AR counterpart on a broad suite of code
benchmarks. Moreover, relying only on the CPT and supervised fine-tuning stages, Stable-DiffCoder achieves stronger performance than a wide range of \~8B ARs and DLLMs, demonstrating that
diffusion-based training can improve code modeling quality beyond AR training alone. Moreover,
diffusion-based any-order modeling improves structured code modeling for editing and
reasoning, and through data augmentation, benefits low-resource coding
languages.
Read the full abstract ▾
All authors Chenghao Fan, Wen Heng, Bo Li, Sichen Liu, Yuxuan Song, Jing Su, Xiaoye Qu, Kai Shen, Wei Wei
Computation and Language · Cross-listed from cs.SE
Evaluating and Achieving Controllable Code Completion in Code LLM
Jiajun Zhang, Zeyu Cui, Lei Zhang et al. (+7 more)
Code completion has become a central task, gaining significant
attention with the rise of large
language model (LLM)-based tools in software engineering. Although recent advances have greatly improved LLMs' code completion abilities, evaluation methods have not advanced equally. Most current
benchmarks focus solely on functional correctness of code completions based on given context, overlooking models' ability to follow user instructions during completion-a common scenario in LLM-assisted programming. To address this limitation, we present the first instruction-guided code completion
benchmark, Controllable Code Completion
Benchmark (C3-Bench), comprising 2,195 carefully designed completion tasks. Through comprehensive evaluation of over 40 mainstream LLMs across C3-Bench and conventional
benchmarks, we reveal substantial gaps in instruction-following capabilities between open-source and advanced proprietary models during code completion tasks. Moreover, we develop a straightforward data synthesis pipeline that leverages Qwen2.5-Coder to generate high-quality instruction-completion pairs for supervised fine-tuning (SFT). The resulting model, Qwen2.5-Coder-C3, achieves state-of-the-art performance on C3-Bench. Our findings provide valuable insights for enhancing LLMs' code completion and instruction-following capabilities, establishing new directions for future research in code LLMs. To facilitate reproducibility and foster further research in code LLMs, we open-source all code,
datasets, and models.
Read the full abstract ▾
All authors Jiajun Zhang, Zeyu Cui, Lei Zhang, Jian Yang, Jiaxi Yang, Qiang Liu, Zilei Wang, Binyuan Hui, Liang Wang, Junyang Lin