Computation and Language
Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
Amir Mohammad Akhlaghi, Amirhossein Shabani, Mostafa Abdolmaleki et al. (+1 more)
The democratization of AI is currently hindered by the immense computational costs required to train Large
Language Models (LLMs) for low-resource
languages. This paper presents Persian-Phi, a 3.8B parameter model that challenges the assumption that robust
multilingual capabilities require massive model sizes or
multilingual baselines. We demonstrate how Microsoft Phi-3 Mini – originally a monolingual English model – can be effectively adapted to Persian through a novel, resource-efficient curriculum learning pipeline. Our approach employs a unique "warm-up" stage using bilingual narratives (Tiny Stories) to align
embeddings prior to heavy training, followed by continual pretraining and instruction tuning via Parameter-Efficient Fine-Tuning (PEFT). Despite its compact size, Persian-Phi achieves competitive results on Open Persian LLM Leaderboard in HuggingFace. Our findings provide a validated, scalable framework for extending the reach of state-of-the-art LLMs to underrepresented
languages with minimal hardware resources. The Persian-Phi model is publicly available at https://huggingface.co/amirakhlaghiqqq/PersianPhi.
Read the full abstract ▾
All authors Amir Mohammad Akhlaghi, Amirhossein Shabani, Mostafa Abdolmaleki, Saeed Reza Kheradpisheh
Computation and Language
Training Language Models to Use Prolog as a Tool
Niklas Mellgren, Peter Schneider-Kamp and Lukas Galke Poech
Language models frequently produce plausible yet incorrect
reasoning traces that are difficult to verify. We investigate fine-tuning models to use Prolog as an external symbolic
reasoning tool, training Qwen2.5-3B-Instruct with Group Relative Policy Optimization (GRPO) on a cleaned version of GSM8K (which we release as gsm8k-prolog-prover). We systematically vary
prompt structure, reward composition (execution, syntax, semantics, structure), and inference protocol (single-try, multiple-try, and two agentic modes). Our reinforcement learning approach outperforms supervised fine-tuning on GSM8K, and the resulting 3B model achieves zero-shot performance on MMLU-STEM and MMLU-Pro competitive with 7B few-shot baselines. Most importantly, we identify an accuracy–auditability trade-off: configurations tuned for correctness alone learn to delegate
reasoning to natural
language and use Prolog only for the final computation, while configurations rewarded for symbolic structure produce fully auditable programs at a cost in accuracy. We interpret this trade-off as a form of reward hacking and discuss its implications for deploying neurosymbolic systems in safety-critical domains. The source code for our experiments is available under https://github.com/aisilab/Prolog-as-a-Tool
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
Yezi Liu, Hanning Chen, Wenjun Huang et al. (+2 more)
Large
language models (LLMs) possess vast knowledge acquired from extensive training corpora, but they often cannot remove specific pieces of information when needed, which makes it hard to handle
privacy, bias mitigation, and knowledge correction. Traditional model unlearning approaches require computationally expensive fine-tuning or direct weight editing, making them impractical for real-world deployment. In this work, we introduce LoRA-based Unlearning with Negative Examples (LUNE), a lightweight framework that performs negative-only unlearning by updating only low-rank adapters while freezing the backbone, thereby localizing edits and avoiding disruptive global changes. Leveraging Low-Rank Adaptation (LoRA), LUNE targets intermediate representations to suppress (or replace) requested knowledge with an order-of-magnitude lower compute and
memory than full fine-tuning or direct weight editing. Extensive experiments on multiple factual unlearning tasks show that LUNE: (I) achieves effectiveness comparable to full fine-tuning and
memory-editing methods, and (II) reduces computational cost by about an order of magnitude.
Read the full abstract ▾
All authors Yezi Liu, Hanning Chen, Wenjun Huang, Yang Ni, Mohsen Imani
Computation and Language · Cross-listed from cs.LG
Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
Yezi Liu, Hanning Chen, Wenjun Huang et al. (+2 more)
Unlearning in large foundation models (e.g., LLMs) is essential for enabling dynamic knowledge updates, enforcing data deletion rights, and correcting model behavior. However, existing unlearning methods often require full-model fine-tuning or access to the original training data, which limits their scalability and practicality. In this work, we introduce Recover-to-Forget (R2F), a novel framework for efficient unlearning in LLMs based on reconstructing full-model gradient directions from low-rank LoRA adapter updates. Rather than performing backpropagation through the full model, we compute gradients with respect to LoRA parameters using multiple paraphrased
prompts and train a gradient decoder to approximate the corresponding full-model gradients. To ensure applicability to larger or black-box models, the decoder is trained on a proxy model and transferred to target models. We provide a theoretical analysis of cross-model generalization and demonstrate that our method achieves effective unlearning while preserving general model performance. Experimental results demonstrate that R2F offers a scalable and lightweight alternative for unlearning in pretrained LLMs without requiring full retraining or access to internal parameters.
Read the full abstract ▾
All authors Yezi Liu, Hanning Chen, Wenjun Huang, Yang Ni, Mohsen Imani
Computation and Language
Multilingual corpora for the study of new concepts in the social sciences and humanities:
Revekka Kyriakoglou and Anna Pappa
This article presents a hybrid methodology for building a
multilingual corpus designed to support the study of emerging concepts in the humanities and social sciences (HSS), illustrated here through the case of “non-technological innovation”. The corpus relies on two complementary sources: (1) textual content automatically extracted from company websites, cleaned for French and English, and (2) annual reports collected and automatically filtered according to documentary criteria (year, format, duplication). The processing pipeline includes automatic
language detection, filtering of non-relevant content, extraction of relevant segments, and enrichment with structural metadata. From this initial corpus, a derived
dataset in English is created for machine learning purposes. For each occurrence of a term from the expert lexicon, a contextual block of five sentences is extracted (two preceding and two following the sentence containing the term). Each occurrence is annotated with the thematic category associated with the term, enabling the construction of data suitable for supervised classification tasks. This approach results in a reproducible and extensible resource, suitable both for analyzing lexical variability around emerging concepts and for generating
datasets dedicated to natural
language processing applications.
Read the full abstract ▾
Computation and Language
Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models
Tomoki Doi, Masaru Isonuma and Hitomi Yanaka
Large
language models have the potential to generate explanations for their own predictions in a variety of styles based on user instructions. Recent research has examined whether these self-explanations faithfully reflect the models' actual behavior and has found that they often lack faithfulness. However, the question of how to improve faithfulness remains underexplored. Moreover, because different explanation styles have superficially distinct characteristics, it is unclear whether improvements observed in one style also arise when using other styles. This study analyzes the effects of training for faithful self-explanations and the extent to which these effects generalize, using three classification tasks and three explanation styles. We construct one-word constrained explanations that are likely to be faithful using a feature attribution method, and use these pseudo-faithful self-explanations for continual learning on instruction-tuned models. Our experiments demonstrate that training can improve self-explanation faithfulness across all classification tasks and explanation styles, and that these improvements also show signs of generalization to the multi-word settings and to unseen tasks. Furthermore, we find consistent cross-style generalization among three styles, suggesting that training may contribute to a broader improvement in faithful self-explanation ability.
Read the full abstract ▾
Computation and Language
Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data
Srihari Bandarupalli, Bhavana Akkiraju, Charan Devarakonda et al. (+2 more)
Automatic
speech recognition for low-resource
languages remains fundamentally constrained by the scarcity of labeled data and computational resources required by state-of-the-art models. We present a systematic investigation into cross-lingual continuous pretraining for low-resource
languages, using Perso-Arabic
languages (Persian, Arabic, and Urdu) as our primary case study. Our approach demonstrates that strategic utilization of unlabeled
speech data can effectively bridge the resource gap without sacrificing recognition accuracy. We construct a 3,000-hour
multilingual corpus through a scalable unlabeled data collection pipeline and employ targeted continual pretraining combined with morphologically-aware tokenization to develop a 300M parameter model that achieves performance comparable to systems 5 times larger. Our model outperforms Whisper Large v3 (1.5B parameters) on Persian and achieves competitive results on Arabic and Urdu despite using significantly fewer parameters and substantially less labeled data. These findings challenge the prevailing assumption that ASR quality scales primarily with model size, revealing instead that data relevance and strategic pretraining are more critical factors for low-resource scenarios. This work provides a practical pathway toward inclusive
speech technology, enabling effective ASR for underrepresented
languages without dependence on massive computational infrastructure or proprietary
datasets.
Read the full abstract ▾
All authors Srihari Bandarupalli, Bhavana Akkiraju, Charan Devarakonda, Vamsiraghusimha Narsinga, Anil Kumar Vuppala
Computation and Language
TeluguST-46: A Benchmark Corpus and Comprehensive Evaluation for Telugu-English Speech Translation
Bhavana Akkiraju, Srihari Bandarupalli, Swathi Sambangi et al. (+3 more)
Despite Telugu being spoken by over 80 million people,
speech translation research for this morphologically rich
language remains severely underexplored. We address this gap by developing a high-quality Telugu–English
speech translation
benchmark from 46 hours of manually verified CSTD corpus data (30h/8h/8h train/dev/test split). Our systematic comparison of cascaded versus end-to-end architectures shows that while IndicWhisper + IndicMT achieves the highest performance due to extensive Telugu-specific training data, finetuned SeamlessM4T models demonstrate remarkable competitiveness despite using significantly less Telugu-specific training data. This finding suggests that with careful hyperparameter tuning and sufficient parallel data (potentially less than 100 hours), end-to-end systems can achieve performance comparable to cascaded approaches in low-resource settings. Our metric reliability study evaluating BLEU, METEOR, ChrF++, ROUGE-L, TER, and BERTScore against human judgments reveals that traditional metrics provide better quality discrimination than BERTScore for Telugu–English translation. The work delivers three key contributions: a reproducible Telugu–English
benchmark, empirical evidence of competitive end-to-end performance potential in low-resource scenarios, and practical guidance for automatic evaluation in morphologically complex
language pairs.
Read the full abstract ▾
All authors Bhavana Akkiraju, Srihari Bandarupalli, Swathi Sambangi, Vasavi Ravuri, R Vijaya Saraswathi, Anil Kumar Vuppala
Computation and Language
Ensembling LLM-Induced Decision Trees for Explainable and Robust Error Detection
Mengqi Wang, Jianwei Wang, Qing Liu et al. (+5 more)
Error detection (ED), which aims to identify incorrect or inconsistent
cell values in tabular data, is important for ensuring data quality. Recent state-of-the-art ED methods leverage the pre-trained knowledge and semantic capability of large
language models (LLMs) to directly label whether a
cell is erroneous. However, this LLM-as-a-labeler pipeline produces predictions through an implicit black-box process with limited traceability and explicit justification, and relies on
stochastic single-pass inference, resulting in inconsistent and insufficiently robust detections across contexts. To address these limitations, we propose an LLM-as-an-inducer framework that uses an LLM to induce a decision tree for ED, termed TreeED, and ensembles multiple such trees for consensus detection, termed ForestED. Based on
prompts derived from data context, decision tree specifications, and output requirements, TreeED queries the LLM to induce a decision tree skeleton whose root-to-leaf paths specify the stepwise procedure for evaluating a sample. Each tree contains three types of nodes: (1) rule nodes that perform simple validation checks, such as format or range constraints; (2) Graph Neural Network (GNN) nodes that capture complex patterns, such as functional dependencies; and (3) leaf nodes that output the final decision as error or clean. ForestED employs uncertainty sampling to obtain multiple informative row subsets and constructs a decision tree for each subset using TreeED. It then applies an Expectation-Maximization-based
algorithm to jointly estimate tree reliability and optimize the consensus ED prediction. Experiments demonstrate that our methods are accurate, explainable, and robust, achieving an average F1-score improvement of 16.1% over the best baseline.
Read the full abstract ▾
All authors Mengqi Wang, Jianwei Wang, Qing Liu, Xiwei Xu, Zhenchang Xing, Liming Zhu, Michael Bain, Wenjie Zhang
Computation and Language · Cross-listed from cs.LG
Pay Less Attention to Function Words for Free Robustness of Vision-Language Models
Qiwei Tian, Chenhao Lin, Zhengyu Zhao et al. (+1 more)
To address the trade-off between robustness and performance for robust VLM, we observe that function words could incur vulnerability of VLMs against cross-modal
adversarial attacks, and propose Function-word De-
Attention (FDA) accordingly to mitigate the impact of function words. Similar to differential amplifiers, our FDA calculates the original and the function-word cross-
attention within
attention heads, and differentially subtracts the latter from the former for more aligned and robust VLMs. Comprehensive experiments include 2 SOTA baselines under 6 different attacks on 2 downstream tasks, 3
datasets, and 3 models. Overall, our FDA yields an average 18/13/53% ASR drop with only 0.2/0.3/0.6% performance drops on the 3 tested models on
retrieval, and a 90% ASR drop with a 0.3% performance gain on visual grounding. We demonstrate the scalability, generalization, and zero-shot performance of FDA experimentally, as well as in-depth ablation studies and analysis. Code is available at https://github.com/michaeltian108/FDA.
Read the full abstract ▾
All authors Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Chao Shen
Computation and Language
NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models
Feng Liang, Weixin Zeng, Runhao Zhao et al. (+1 more)
Large
Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural
language processing tasks. However, temporal
reasoning, particularly under complex temporal constraints, remains a major challenge. To this end, existing approaches have explored symbolic methods, which encode temporal structure explicitly, and reflective mechanisms, which revise
reasoning errors through multi-step inference. Nonetheless, symbolic approaches often underutilize the
reasoning capabilities of LLMs, while reflective methods typically lack structured temporal representations, which can result in inconsistent or hallucinated
reasoning. As a result, even when the correct temporal context is available, LLMs may still misinterpret or misapply time-related information, leading to incomplete or inaccurate answers. To address these limitations, in this work, we propose Neuro-Symbolic Temporal
Reasoning (NeSTR), a novel framework that integrates structured symbolic representations with hybrid reflective
reasoning to enhance the temporal sensitivity of LLM inference. NeSTR preserves explicit temporal relations through symbolic encoding, enforces logical consistency via verification, and corrects flawed inferences using abductive reflection. Extensive experiments on diverse temporal question answering
benchmarks demonstrate that NeSTR achieves superior zero-shot performance and consistently improves temporal
reasoning without any fine-tuning, showcasing the advantage of neuro-symbolic integration in enhancing temporal understanding in large
language models.
Read the full abstract ▾
All authors Feng Liang, Weixin Zeng, Runhao Zhao, Xiang Zhao
Computation and Language · Cross-listed from cs.CV
Generating Storytelling Images with Rich Chains-of-Reasoning
Xiujie Song, Qi Jia, Shota Watanabe et al. (+4 more)
A single image can convey a compelling story through logically connected visual clues, forming Chains-of-
Reasoning (CoRs). We define these semantically rich images as Storytelling Images. By conveying multi-layered information that inspires active interpretation, these images enable a wide range of applications, such as illustration and cognitive screening. Despite their potential, such images are scarce and complex to create. To address this, we introduce the Storytelling Image Generation task and propose StorytellingPainter, a two-stage pipeline combining the
reasoning of Large
Language Models (LLMs) with Text-to-Image (T2I) synthesis. We also develop a dedicated evaluation framework assessing semantic complexity, diversity, and text-image
alignment. Furthermore, given the critical role of story generation in the task, we introduce lightweight Mini-Storytellers to bridge the performance gap between small-scale and proprietary LLMs. Experimental results demonstrate the feasibility of our approaches.
Read the full abstract ▾
All authors Xiujie Song, Qi Jia, Shota Watanabe, Xiaoyi Pang, Ruijie Chen, Mengyue Wu, Kenny Q. Zhu
Computation and Language
Multilingual Agent-Based World Modeling for Social Science
Xuan Zhang, Wenxuan Zhang, Anxu Wang et al. (+2 more)
Multi-
agent role-playing has recently shown promise for studying social behavior with
language agents, but existing simulations are mostly monolingual without cross-lingual interaction, an essential property of real societies. We introduce MAWM, the first
Multilingual Agent-based World Modeling framework that supports multi-turn
multilingual interactions among generative
agents with diverse sociolinguistic profiles. MAWM enables two modes of analysis: (i) global public opinion modeling, which tracks how attitudes toward open-domain survey questions evolve across
languages and cultures, and (ii) media influence and information
diffusion, via autonomous news
agents that dynamically generate content and shape user behavior. To ground the simulation in realistic population distributions, we construct the MAPS
benchmark, which combines survey questions and demographic personas drawn from global population distributions. Experiments on simulation
alignment, along with social science case studies on cultural assimilation and normative
diffusion, show that native-
language simulation better reflects real survey data than English-only simulation, and that the
agent society in MAWM reproduces established sociocultural phenomena and highlights the value of
multilingual simulation as an alternative interpretable tool for computational social science.
Read the full abstract ▾
All authors Xuan Zhang, Wenxuan Zhang, Anxu Wang, See-Kiong Ng, Yang Deng
Computation and Language · Cross-listed from cs.AI
Enhancing knowledge tracing robustness for new question cold start in Intelligent Tutoring Systems
Wonbeen Lee, Channyoung Lee, Junho Sohn et al. (+1 more)
Intelligent Tutoring Systems (ITS) provide personalized learning paths by diagnosing learners' proficiency. Knowledge Tracing (KT) models play a central role in this diagnosis by estimating learners' evolving knowledge states. However, in real-world ITS services, diagnostic reliability may decrease when newly introduced questions have no prior interaction history. This study aims to empirically identify the key features that support KT model robustness under the question cold start situation. To this end, we designed Practical Integrated Cross-consistent Knowledge Tracing (PICKT), which integrates multiple types of features, and examined which feature contributes to robustness under the question cold start situation. Among these features, we further analyzed difficulty, texts, and relational information derived from the knowledge map, which have been identified as important factors in prior KT research, to examine how each contributes to robustness under the question cold start situation. The results showed that difficulty feature was particularly informative for highly challenging questions, where the likelihood of a correct response rate was remarkably low. Meanwhile, the fused texts and knowledge map features supported robustness by enabling the model to estimate the representation of unseen questions by leveraging semantically and structurally similar questions observed during training. These findings suggest that maintaining KT robustness in ITS applications requires prioritization of feature annotation aligned with the characteristics of educational services.
Read the full abstract ▾
All authors Wonbeen Lee, Channyoung Lee, Junho Sohn, Hansam Cho
Computation and Language · Cross-listed from cs.CV
Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Models
Fenghua Weng, Chaochao Lu, Xia Hu et al. (+2 more)
As
multimodal reasoning improves the overall capabilities of Large
Vision Language Models (LVLMs), recent studies have begun to explore safety-oriented
reasoning, aiming to enhance safety awareness by analyzing potential safety risks during the
reasoning process before generating the final response. Although such approaches improve safety awareness and interpretability, this single-pass think-then-answer paradigm remains vulnerable to contextual or visual jailbreak attacks. This reveals a critical flaw: single-pass
reasoning may overlook explicit harmful content in its own output. Our key insight is to exploit this wasted signal through reflection, which can effectively leverage the malicious content revealed in the first-pass
reasoning to enable genuine self-correction and prevent unsafe generations. Motivated by this, we propose Think-Reflect-Revise (TRR), a three-stage training framework designed to enhance the safety
alignment of LVLMs through policy-guided self-reflection. We first build a Reflective Safety
Reasoning (ReSafe)
dataset with 5,000 examples that follow a think-reflect-revise process. We then fine-tune the target model using the ReSafe
dataset to initialize reflective behavior, and finally reinforce policy-guided reflection through reinforcement learning. Experimental results show that TRR substantially improves the safety performance of LVLMs across both safety-awareness
benchmarks and jailbreak attack evaluations, increasing the overall safe response rate from 42.8% to 87.7% on Qwen2.5-VL-7B, while preserving stable performance on general
benchmarks such as MMMU and MMStar. The project page is available at https://think-reflect-revise.github.io/.
Read the full abstract ▾
All authors Fenghua Weng, Chaochao Lu, Xia Hu, Wenqi Shao, Wenjie Wang
Computation and Language
GUMBridge: a Corpus for Varieties of Bridging Anaphora
Lauren Levine and Amir Zeldes
Bridging is an anaphoric phenomenon where the referent of an entity in a discourse is dependent on a previous, non-identical entity for interpretation, such as in "There is 'a house'. 'The door' is red," where the door is specifically understood to be the door of the aforementioned house. While there are several existing resources in English for bridging anaphora, most are small, provide limited coverage of the phenomenon, and/or provide limited genre coverage. In this paper, we introduce GUMBridge, a new resource for bridging, which includes 16 diverse genres of English, providing both broad coverage for the phenomenon and granular annotations for the subtype categorization of bridging varieties. We also present an evaluation of annotation quality and report on baseline performance using open and closed source contemporary LLMs on three tasks underlying our data, showing that bridging resolution and subtype classification remain difficult NLP tasks in the age of LLMs.
Read the full abstract ▾
Computation and Language
DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning
Nithin Sivakumaran, Justin Chih-Yao Chen, David Wan et al. (+4 more)
Specialized visual tools can augment large
language models or
vision language models with expert knowledge (e.g., grounding, spatial
reasoning, medical knowledge, etc.), but knowing which tools to call (and when to call them) can be challenging. We introduce DART, a multi-
agent framework that uses disagreements between multiple debating visual
agents to identify useful visual tools (e.g., object detection, OCR, spatial
reasoning, etc.) that can resolve inter-
agent disagreement. These tools allow for fruitful multi-
agent discussion by introducing new information, and by providing tool-aligned agreement scores that highlight
agents in agreement with expert tools, thereby facilitating discussion. We utilize an aggregator
agent to select the best answer by providing the
agent outputs and tool information. We test DART on four diverse
benchmarks and show that our approach improves over multi-
agent debate as well as over single
agent tool-calling frameworks, beating the next-strongest baseline (multi-
agent debate with a judge model) by 3.4% and 2.4% on A-OKVQA and MMMU respectively. We also find that DART adapts well to new tools in applied domains, with a 1.3% improvement on the M3D medical
dataset over other strong tool-calling, single
agent, and multi-
agent baselines. Additionally, we measure text overlap across rounds to highlight the rich discussion in DART compared to existing multi-
agent methods. Finally, we study the tool call distribution, finding that diverse tools are reliably used to help resolve disagreement.
Read the full abstract ▾
All authors Nithin Sivakumaran, Justin Chih-Yao Chen, David Wan, Yue Zhang, Jaehong Yoon, Elias Stengel-Eskin, Mohit Bansal
Computation and Language · Cross-listed from cs.AI
A Neural Affinity Framework for Abstract Reasoning: Diagnosing the Compositional Gap in Transformer Architectures via Procedural Task Taxonomy
Miguel Ingram and Arthur Joseph Merritt III
Responding to Hodel et al.'s (2024) call for a formal definition of task relatedness in re-arc, we present the first 9-category taxonomy of all 400 tasks, validated at 97.5% accuracy via rule-based code analysis. We prove the taxonomy's visual coherence by training a CNN on raw grid pixels (95.24% accuracy on S3, 36.25% overall, 3.3x chance), then apply the taxonomy diagnostically to the original ARC-AGI-2 test set. Our curriculum analysis reveals 35.3% of tasks exhibit low neural affinity for
Transformers–a distributional bias mirroring ARC-AGI-2. To probe this misalignment, we fine-tuned a 1.7M-parameter
Transformer across 302 tasks, revealing a profound Compositional Gap: 210 of 302 tasks (69.5%) achieve >80%
cell accuracy (local patterns) but <10% grid accuracy (global synthesis). This provides direct evidence for a Neural Affinity Ceiling Effect, where performance is bounded by architectural suitability, not curriculum. Applying our framework to Li et al.'s independent ViTARC study (400 specialists, 1M examples each) confirms its predictive power: Very Low affinity tasks achieve 51.9% versus 77.7% for High affinity (p<0.001), with a task at 0% despite massive data. The taxonomy enables precise diagnosis: low-affinity tasks (A2) hit hard ceilings, while high-affinity tasks (C1) reach 99.8%. These findings indicate that progress requires hybrid architectures with affinity-aligned modules. We release our validated taxonomy,
Read the full abstract ▾
Computation and Language
Leveraging KV Similarity for Online Structured Pruning in LLMs
Jungmin Lee, Gwangeun Byeon, Yulhwa Kim et al. (+1 more)
Pruning has emerged as a promising direction for accelerating large
language model (LLM) inference, yet existing approaches often suffer from instability because they rely on offline calibration data that may not generalize across inputs. In this work, we introduce
Token Filtering, a lightweight online structured
pruning technique that makes
pruning decisions directly during inference without any calibration data. The key idea is to measure
token redundancy via joint key-value similarity and skip redundant
attention computations, thereby reducing inference cost while preserving critical information. To further enhance stability, we design a variance-aware fusion strategy that adaptively weights key and value similarity across heads, ensuring that informative
tokens are retained even under high
pruning ratios. This design introduces no additional
memory overhead and provides a more reliable criterion for
token importance. Extensive experiments on LLaMA-2 (7B/13B), LLaMA-3 (8B), and Mistral (7B) demonstrate that
Token Filtering consistently outperforms prior structured
pruning methods, preserving accuracy on commonsense
reasoning benchmarks and maintaining strong performance on challenging tasks such as MMLU, even with 50%
pruning.
Read the full abstract ▾
All authors Jungmin Lee, Gwangeun Byeon, Yulhwa Kim, Seokin Hong
Computation and Language
Do Large Language Models Truly Understand Cross-cultural Differences?
Shiwei Guo, Sihang Jiang, Qianxi He et al. (+6 more)
In recent years, large
language models (LLMs) have demonstrated strong performance on
multilingual tasks. Given its wide range of applications, cross-cultural understanding capability is a crucial competency. However, existing
benchmarks for evaluating whether LLMs genuinely possess this capability suffer from three key limitations: a lack of contextual scenarios, insufficient cross-cultural concept mapping, and limited deep cultural
reasoning capabilities. To address these gaps, we propose SAGE, a scenario-based
benchmark built via cross-cultural core concept
alignment and generative task design, to evaluate LLMs' cross-cultural understanding and
reasoning. Grounded in cultural theory, we categorize cross-cultural capabilities into nine dimensions. Using this framework, we curated 210 core concepts and constructed 4530 test items across 15 specific real-world scenarios, organized under four broader categories of cross-cultural situations, following established item design principles. The SAGE
dataset supports continuous expansion, and experiments confirm its transferability to other
languages. It reveals model weaknesses across both dimensions and scenarios, exposing systematic limitations in cross-cultural
reasoning. While progress has been made, LLMs are still some distance away from reaching a truly nuanced cross-cultural understanding. In compliance with the anonymity policy, we include data and code in the supplement materials. In future versions, we will make them publicly available online.
Read the full abstract ▾
All authors Shiwei Guo, Sihang Jiang, Qianxi He, Yanghua Xiao, Jiaqing Liang, Bi Yude, Minggui He, Shimin Tao, Li Zhang
Computation and Language
SETUP: Sentence-level English-To-Uniform Meaning Representation Parser
Emma Markle, Javier Gutierrez Bach and Shira Wein
Uniform Meaning Representation (UMR) is a novel graph-based semantic representation which captures the core meaning of a text, with flexibility incorporated into the annotation schema such that the breadth of the world's
languages can be annotated (including low-resource
languages). While UMR shows promise in enabling
language documentation, improving low-resource
language technologies, and adding interpretability, the downstream applications of UMR can only be fully explored when text-to-UMR parsers enable the automatic large-scale production of accurate UMR graphs at test time. Prior work on text-to-UMR parsing is limited to date. In this paper, we introduce two methods for English text-to-UMR parsing, one of which fine-tunes existing parsers for Abstract Meaning Representation and the other, which leverages a converter from Universal Dependencies, using prior work as a baseline. Our best-performing model, which we call SETUP, achieves an AnCast score of 84 and a SMATCH++ score of 91, indicating substantial gains towards automatic UMR parsing.
Read the full abstract ▾
Computation and Language
Replicating TEMPEST at Scale: Multi-Turn Adversarial Attacks Against Trillion-Parameter Frontier Models
Richard Young
Despite substantial investment in safety
alignment, the vulnerability of large
language models to sophisticated multi-turn
adversarial attacks remains poorly characterized, and whether model scale or inference mode affects robustness is unknown. This study employed the TEMPEST multi-turn attack framework to evaluate ten frontier models from eight vendors across 1,000 harmful behaviors, generating over 97,000 API queries across
adversarial conversations with automated evaluation by independent safety classifiers. Results demonstrated a spectrum of vulnerability: six models achieved 96% to 100% attack success rate (ASR), while four showed meaningful resistance, with ASR ranging from 42% to 78%; enabling extended
reasoning on identical architecture reduced ASR from 97% to 42%. These findings indicate that safety
alignment quality varies substantially across vendors, that model scale does not predict
adversarial robustness, and that thinking mode provides a deployable safety enhancement. Collectively, this work establishes that current
alignment techniques remain fundamentally vulnerable to adaptive multi-turn attacks regardless of model scale, while identifying deliberative inference as a promising defense direction.
Read the full abstract ▾