Computation and Language · Cross-listed from cs.AI
CUSE: Fine-Grained Multimodal Unlearning with Counterfactual Guidance and On-Policy Self-Distillation
Chenlu Ding, Jiancan Wu, Yanchen Luo et al. (+4 more)
Multimodal large
language models (MLLMs) can reproduce sensitive, copyrighted, or unsafe information from their training data. Effective unlearning must remove target information while preserving non-target content, even within the same response. Existing methods typically apply uniform forgetting objectives to complete responses and rely on fixed, off-policy supervision, resulting in coarse
token control and misalignment with the model's current behavior. We propose CUSE, a Counterfactual-guided Unlearning framework via on-policy Self-
distillation for
multimodal large
language models. It first constructs counterfactual inputs by removing target-related evidence and uses the resulting changes in
token probabilities to identify which
tokens require stronger forgetting. It then distills
token-level forgetting and retention targets on responses sampled from the current policy, with a frozen copy of the original model serving as the teacher model augmented with the corresponding forgetting- and retention-oriented privileged contexts. This design jointly determines what to forget and what to preserve. On MLLMU-Bench, CUSE reduces forget-set ROUGE-L by 33.6% while achieving 1.2% higher retain-set ROUGE-L than the strongest baseline.
Read the full abstract ▾
All authors Chenlu Ding, Jiancan Wu, Yanchen Luo, Tianyu Zhang, Yancheng Yuan, Xiang Wang, Xiangnan He
Computation and Language
The Constitution Speaks Nheengatu: An Open MT System and Reproducible Corpus Pipeline for the Amazonian Língua Geral
Fabrício Carraro, Rodolfo Joel Zevallos, Rodrigo Gonçalves de Souza et al. (+2 more)
Nheengatu (yrl), the Amazonian Língua Geral, has 6–10 thousand speakers and is co-official in São Gabriel da Cachoeira (Brazil), yet remains poorly served by evaluated Portuguese–Nheengatu MT resources. We present the largest parallel corpus assembled for the
language (14.5k Portuguese–Nheengatu pairs), anchored by the 2023 official translation of the Brazilian Federal Constitution; the first MT evaluation set built from transcribed spontaneous
speech; and, to our knowledge, the first open-weight MT models dedicated to bidirectional Portuguese–Nheengatu translation. Three findings organize the paper. First, translation into Nheengatu on held-out constitutional articles plateaus on development data regardless of training length or model size, and much of the remaining error sits in conventional legal terms that the written sources do not attest consistently; a terminology success rate over a frozen lexicon shows that glossary mechanisms at training time and lexical constraints at decoding each raise term realization, the latter at a cost in repetitive output that chrF++ barely registers. Second, a corpus expansion that adds transcribed conversation raises
speech chrF++ by 24–27 points while leaving written domains unchanged, a register gap that written-only
benchmarks cannot see. Third, a larger NLLB model helps translation into Portuguese but not into Nheengatu on legal or
speech text. For Portuguese-to-Nheengatu translation, our final systems reach 31.7–32.8 chrF++ on article-disjoint constitutional text, 64.1–64.5 on the near-domain set (62.1 after removing items with target-side content overlap), and 45.2–47.5 on
speech.
Read the full abstract ▾
All authors Fabrício Carraro, Rodolfo Joel Zevallos, Rodrigo Gonçalves de Souza, Caio Henrique Faustino da Silva, John E. Ortega
Computation and Language
Amharic-English Cross-Language Question Answering Using Statistical Machine Translation, Named Entity Recognition, and Semantic Indexing
Emebet Bekele and Fekade Getahun
The predominance of English-
language Web content creates an information-access barrier for speakers of low-resource
languages such as Amharic. This paper presents the design, imple-mentation, and evaluation of a bidirectional Amharic-English cross-
language question answering (CLQA) system that allows a user to formulate a natural-
language question in either Amharic or English and retrieve an answer from documents written in the other
language. The system integrates a statistical machine translation (SMT) component for query translation, a maximum-
entropy named entity recognition (NER) component for Amharic text processing, semantic-based indexing, question analysis, passage
retrieval, and answer extraction. A domain-diverse Amharic-English parallel corpus was used to train the SMT component, while monolingual cor-pora supported
language modeling. Documents were normalized, tokenized, stemmed, indexed, and semantically enriched before
retrieval. The original evaluation used precision and recall as the principal effectiveness measures. For Amharic-to-English
retrieval, the system achieved 72% precision and 79% recall; for English-to-Amharic
retrieval, it achieved 65% precision and 70% recall. These findings show that a modular translation-and-
retrieval architecture can provide useful cross-
language access even under low-resource conditions. This revised article preserves the original experimental claims while situating them in the context of subsequent multilingual
transformer and Amharic QA research.
Read the full abstract ▾
Computation and Language
Vela: User Behavior as Supervision Sparks LLM Intelligence
Yang Xiao, Yusong Sun, Haoyi Wu et al. (+17 more)
Large
language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommendation and advertising systems. These applications demand the user dimension of LLM intelligence: inferring a person's preferences, intent, and likely future actions from weeks to years of behavior data. Yet current models are trained on static text or synthetic data, neither of which captures how user behavior evolves over time. Real-world behavior records, already available at scale on content platforms and in personal-assistant
agents, offer a natural supervisory signal. We propose user behavior as supervision and introduce Vela, a framework that converts real user behavior data into trainable tasks and trajectories. From the anonymized behavior records of 100,000 users, Vela generates 100,000 tasks across 91 real-world task templates; difficulty-based filtering and trajectory acceptance criteria retain approximately 75,000 tasks with selected training trajectories. Fine-tuning Qwen3.6-35B-A3B on these trajectories improves performance over the base model by up to 11.5 points on user-related tasks and by an average of 4 points on general
benchmarks. Training on more user behavior data further improves user-related performance, and reinforcement learning after fine-tuning outperforms reinforcement learning from the base model by up to 7.7 points. These gains suggest that real-world user behavior is a scalable source of supervision for both user-centered capabilities and the broader intelligence of LLMs.
Read the full abstract ▾
All authors Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Weiqiang Hang, Luqing Wang, Zejian Wang, Zirui Tu, Jingsheng Mao, Shaobo Hu, Yang Luo, Jiale Guo, Jin Zhang, Yudan Liu, Zhaokai Luo, Ren Huan, Yao Hu, Chengyue Jiang, Wenjie Li, Chuan Mu
Computation and Language · Cross-listed from cs.AI
How to Build a Benchmark That Actually Measures Taste
Xiaofan Wu, Xi Zeng, Miaoxia Chen et al. (+8 more)
Expert-aligned "taste"
benchmarks should reward craft judgment, not the ability to game the test. We introduce TasteBench, a pairwise good-vs-bad
benchmark across visual, brand, book, and tech items under a frozen rubric with expert-only gold (panel n>=3; percent agreement >=0.60 primary gate; weak engagement labels train/analysis-only), together with a control protocol that separates craft judgment from test-gaming. Applying the protocol to a Round-1 pilot (primary_human_expert; three biological human experts; n=250 pairs) surfaces and quantifies three validity threats: two construction artifacts that inflate the trivial-baseline layer, and a difficulty ceiling that inflates every score. (1) Position: the preferred item is slot A with probability 0.956, so an always_item_a policy scores 0.956 (fair chance 0.5). (2) Lexical leakage: a filename
token cue scores 0.912. (3) Difficulty ceiling: craft domains show near-ceiling human agreement (book/brand/tech 1.0), and even under A/B randomization (eval_assignment_seed7; gold_side_A ~0.47) all five VLM cards reach 1.0 on craft, with only visual separating (~0.84-0.91; a post-hoc hard subset, n=107, leaves craft at 1.0). The protocol (expert-only gold, a weak-label firewall, side randomization at construction, filename scrubbing, and hard filters) makes each artifact measurable and, for the two construction artifacts, correctable; we release a reference eval harness and a side-randomization tool, and show the tool removes the position skew (0.916 to 0.495/0.505). The result is a calibrated view of what the
benchmark does and does not yet measure: position and leakage are fixable, whereas the craft-domain ceiling is a pair-difficulty limit that Round-2 must address. We do not evaluate engagement, era/circle conditioning, or calibration here, and we claim no general taste SOTA.
Read the full abstract ▾
All authors Xiaofan Wu, Xi Zeng, Miaoxia Chen, Peishan Chen, Shuyan Li, Chen Wang, Xiaohan Wang, Wanling Xu, Jiyun Yao, Hanyong Zhong, Jiahao Zhu
Computation and Language
Lingdan-V2: Large language models for clinically grounded reasoning in traditional Chinese medicine
Rui Hua, Yu Wei, Ziri Jin et al. (+15 more)
Large
language models have advanced medical
reasoning, but clinically grounded
reasoning in traditional Chinese medicine (TCM) remains challenging because prescription decisions require integrating domain knowledge, syndrome differentiation, treatment principles, herbal prescription composition, and dosage determination. We develop a TCM
language model family comprising models with 4 billion, 8 billion, and 14 billion parameters. The artificial intelligence contribution is a staged clinical
alignment framework integrating large scale continued pretraining on TCM and clinical records, complementary
reasoning supervision, and prescription-centered reinforcement learning (RL). We further formulate herbal prescription generation as a structured RL problem with prescription-level feedback, enabling direct optimization of clinical decision outputs beyond knowledge reproduction and generic instruction following. The engineering application is TCM clinical decision support, particularly herbal prescription recommendation. We evaluate the models on a public
benchmark containing 25,620 expert-verified items across 13 TCM knowledge and clinical
reasoning tasks and on a private retrospective
dataset of 4,348 spleen and stomach disease prescription cases. The 14 billion parameter reinforcement aligned model achieves a prescription generation F1 of 37.0 on the public
benchmark, compared with 32.3 for DeepSeek-R1 with 671 billion total parameters, and a herb-level F1 of 52.8 on the clinical
dataset with improved dosage agreement. Reinforcement post-training consistently improves prescription recommendation across all scales, while physician-source analysis shows that expert heterogeneity affects reward stability. These findings demonstrate a scalable pathway from domain grounding and
reasoning supervision to clinically oriented prescription decision support.
Read the full abstract ▾
All authors Rui Hua, Yu Wei, Ziri Jin, Zhuo Sun, Kai Chang, Jianan Xia, Zixin Shu, Xiaodong Li, Dongmei Jia, Fei Dong, Runshun Zhang, Jian Yu, Hao Xu, Haibin Yu, Jiansheng Li, Baoyan Liu, Wenjia Wang, Xuezhong Zhou
Computation and Language · Cross-listed from cs.CR
Towards Merging-conditional Backdoor Unalignment against Large Language Models
Yukun Chen, Yidou Chen, Yiming Li et al. (+4 more)
Model merging has become widely adopted for enhancing the task-specific performance of Large
Language Models (LLMs) without additional training. While the specialized model from third-party platforms may undergo safety audits (e.g., backdoor detection), we reveal a novel and critical threat: One-sided Merging-Conditional Backdoor (OMCB). OMCB remains dormant and undetectable within one standalone model but activates upon merging, triggering safety unalignment. We first theoretically analyze the existence and
geometric location of OMCB, revealing that the parameter shift crossing safety decision boundaries is the main cause of backdoor activation. Building upon this insight, we propose SMOKE, a meta-learning-based attack framework. SMOKE implants OMCB via a bi-level optimization strategy: the inner loop synthesizes surrogate meta-vectors through random low-rank initialization and finite-step maximization, while the outer loop optimizes the target model by simulating the merging process. Furthermore, preliminary analysis shows that the injected OMCB has bounded generalization loss under unknown merging configurations. Extensive experiments across diverse model architectures and merging techniques validate the effectiveness of SMOKE and its resistance to existing backdoor detections, highlighting a significant yet previously overlooked vulnerability in the model merging paradigm.
Read the full abstract ▾
All authors Yukun Chen, Yidou Chen, Yiming Li, Xinyu Zhang, Baosong Yang, Hongwei Yao, Zhan Qin
Computation and Language · Cross-listed from cs.SD
VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS
Wenbing Yang, Qihang Lu, Zihan Sun et al. (+4 more)
Instruction-following text-to-
speech (TTS) typically leaves voice design implicit, as conventional speaker
embeddings encode reference audio rather than text-driven targets. We propose VoTSpeech, which decouples voice design from
speech generation via an explicit intermediate voice latent. Built on a shared
language model backbone, VoTSpeech first predicts a continuous voice latent from text instructions using a flow-based module. This latent serves as a shared condition for both the
language model and the Acoustic DiT, guiding autoregressive sequence modeling and acoustic latent generation, respectively, before an audio VAE decoder converts the acoustic latents into waveforms. To capture speaker identity and complementary acoustic nuances, the voice latent is jointly supervised by speaker
embeddings and temporally pooled audio VAE features. With task-specific adaptation, VoTSpeech performs strongly on InstructTTSEval-ZH, demonstrating competitive capabilities against strong open-source models while maintaining high naturalness. These results confirm that explicit voice latent design and its conditioning interface effectively empower text-driven voice design for controllable
speech synthesis. Code, model, and audio samples are available at https://github.com/ywbn/VoTSpeech .
Read the full abstract ▾
All authors Wenbing Yang, Qihang Lu, Zihan Sun, Peilei Jia, Yingming Gao, Ya Li, Jun Gao
Computation and Language
PVFH-Bench: Benchmarking Premise-Induced Visual Fixation Hallucinations in Vision-Language Models
Xinhao Wang, Junbo Cui, Zhenghao Liu et al. (+4 more)
Large
Vision-
Language Models (LVLMs) have achieved remarkable progress across a wide range of
multimodal tasks. However, object
hallucination remains a persistent challenge, undermining model reliability in applications such as visual question answering, embodied
agents, and human-AI decision support systems. In this paper, inspired by cognitive science studies on perceptual bias, we investigate a new failure mode termed Premise-Induced Visual Fixation
Hallucination (PVFH). Specifically, PVFH refers to a premise-induced
reasoning error in which models become anchored to visually confusable evidence, accept a false textual premise, and subsequently generate responses about an absent yet visually plausible object. To systematically study this phenomenon, we introduce PVFH-Bench, a
benchmark that automatically constructs visually confusable but absent targets and pairs the core
hallucination setting with diagnostic subsets to trace how models move from object recognition to Premise-Induced Visual Fixation
Hallucination. Experimental results on 10 state-of-the-art LVLMs demonstrate that all evaluated models are highly vulnerable to Premise-Induced Visual Fixation
Hallucination. Moreover, we find that explicit chain-of-thought
reasoning can further exacerbate this failure mode by encouraging models to overinterpret weak visual cues as supporting evidence for hallucinated objects.
Read the full abstract ▾
All authors Xinhao Wang, Junbo Cui, Zhenghao Liu, Yuan Yao, Xiaohua Li, Yu Gu, Ge Yu
Computation and Language · Cross-listed from cs.IR
The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval
Jianbo Chen
Jev's documented interface evaluates typed questions independently against shared evidence. We audit request independence in seven open Jev-style checkpoints on 276 constructed three-passage states from 92 TREC
retrieval queries. Paired interventions separately change requested questions, displayed evidence, and candidate eligibility. Request expansion induces strict binary flips in 9.1%/8.0% of Nimble's judgments and changes 21.4%/18.9% of selection-or-abstention actions in TREC DL2019/2020 under a threshold of 0.5 fixed before inference. Correct-action rates change by 0.0/+8.2 percentage points, respectively, with 95% query-bootstrap intervals of [-6.0,6.0] and [3.8,12.6]. Field masking traces all 70 request-set flips to the schema fields serialized into each scoring
prompt. In Von, fixing a shared separator's position eliminates two eligibility-induced preference flips in selected diagnostic states. Laya-typed's default CUDA path produces five passage swaps without strict binary flips, and both yearly action-change intervals include zero; in an FP32 rerun, its probability movement vanishes at the model's four-decimal reporting resolution.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning
Haodong Chen, Shengtao Wen, Dong Liang et al. (+2 more)
Current evaluations of
multimodal unlearning typically focus on whether a model stops outputting target information. This surface-level criterion, however, masks two critical failure modes. First, knowledge suppression techniques often induce confabulation, leading the model to hallucinate facts about the unlearned subject. Second, refusal training merely alters observable behavior without erasing the underlying knowledge; such models might decline open queries but still identify correct answers under multiple-choice probing. To overcome these limitations, we propose Forget-Aware Self-
Distillation (FASD). This method is designed to achieve honest ignorance, a state where the model genuinely loses the target knowledge and learns to acknowledge its absence. FASD separates these goals into two complementary channels. The behavior channel distills appropriate refusals from an ignorance-prompted copy of the model itself. Crucially, target
tokens are masked from the teacher distribution to avoid accidentally reteaching the forgotten concepts. Simultaneously, the knowledge channel directly suppresses ground-truth answer spans independently of the model's generated responses. Experiments demonstrate that FASD successfully unlearns target knowledge, prevents information leakage in open-ended tasks, and preserves performance on retained data. The resulting model not only forgets the specified information but accurately recognizes the boundaries of its own knowledge. Code and data will be released upon publication.
Read the full abstract ▾
All authors Haodong Chen, Shengtao Wen, Dong Liang, Lingbing Guo, Xiang Chen
Computation and Language · Cross-listed from cs.IR
Fine-Tuning Strategy, Not Just Data: A Statistically Rigorous Audit of Gender and Stereotypical Bias in Embedding Models for HR Recommendation Systems
Ekta Yadav
Embedding models underpin modern job-candidate matching: resumes, job descriptions, and queries are encoded as dense vectors, and recommendations come from nearest-neighbor
retrieval over that space. Any social bias in the
embeddings therefore feeds directly into HR recommendations. We audit six multilingual sentence-
embedding models (LaBSE- and E5-family, both public and custom fine-tuned) using Bias-in-Bios (occupational gender bias) and StereoSet (stereotypes across profession, race, gender, and religion). We treat statistical validity as a first-class concern: we identify a non-independence error in a naive significance test, where pooling non-independent pairwise similarities inflated a trivial 0.0046 mean difference to $p \approx 0$, and replace it with a profession-level paired Wilcoxon signed-rank test, individual-resampling bootstrap confidence intervals, and permutation tests for StereoSet. Under the corrected methodology, knowledge-
distillation (KD) fine-tuning of E5-Base significantly increases measured bias on Bias-in-Bios ($\Delta = -0.177$, $p < 10^{-8}$; more negative means more bias), although this measure is sensitive to global
embedding geometry. The KD result is corroborated by pro-stereotype patterns in the StereoSet profession category (uncorrected $p < 0.05$). Binary-classification fine-tuning of the same base model shows no statistically distinguishable change ($p = 0.218$). Fine-tuning strategy is thus an actionable design lever for practitioners building HR recommenders. We also report which
benchmark categories are underpowered, and note that two domain-adapted models could not be included in significance testing due to a software-access constraint, which limits our conclusions.
Read the full abstract ▾
Computation and Language
Uniform Information Density-Based Preference Optimization
Yongyuan Li and Yang Xu
To enable large
language models to generate outputs that align with human preferences, optimization
algorithms based on the comparison between chosen and rejected output pairs have been developed and been proven effective. Some recent endeavors have utilized certain linguistically-derived assumptions about the preference pairs to guide the design of more precise optimization directions. However, human preferences are complex and hard to measure by simplistic criteria, and it is important to incorporate broader and more sophisticated insights about preferred vs. not-preferred
language. In this study, we adopt a well-established psycholinguistic theory, Uniform Information Density, to develop a novel preference optimization
algorithm, UIDPO. The basic assumption is that humans prefer to produce
language in which information is uniformly distributed, because uniformity induces less cognitive processing effort. To implement this idea into the
algorithm, we derive a compound metric for information uniformity by combining the global standard deviation and the local second-order difference, and integrate it to the objective of preference optimization. We compare UIDPO with SimPO, a previous state-of-the-art method, obtaining comparable and sometimes better performance on AlpacaEval2, ArenaHard, and MT-Bench. Additional linguistic analysis on complement clauses also shows that UIDPO can better reflect the tendencies of keeping information uniform in natural
language than baselines. Our results show that the subtle human preference over the information rate in
language is a helpful source for achieving better human
alignment for LLMs. The code repository is published here: https://github.com/CLCS-SUSTech/UIDPO.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Yilan Chen, Zian Su, Manish Prajapat et al. (+4 more)
Reinforcement learning with verifiable rewards improves the
reasoning capabilities of large
language models, but remains computationally expensive and sample inefficient. On-policy
distillation (OPD) provides dense
token-level supervision from a stronger teacher, yet standard OPD distills every
token regardless of whether the teacher's preference actually improves task reward. Existing
token-selection methods rely on proxies such as uncertainty or teacher–student divergence, which need not identify reward-improving supervision. We introduce Reward-Aligned
Distillation (RAD), a bilevel framework that weights
distillation signals based on their predicted effect on the student's post-update reward. RAD differentiates through a temporary optimizer step to score each
token's supervision, then updates the
token weights by projected ascent from the initial
token weights. The resulting weights can strengthen, suppress, or reverse individual
distillation-gradient contributions. At a matched training-response budget, RAD improves mean accuracy on six competition-math
benchmarks over the strongest baselines by 6.5 percentage points for Qwen3-1.7B (non-thinking) and 2.5 points for Qwen3-4B-Instruct-2507. RAD-Simple, a simplified variant, achieves comparable math accuracy at the same training-response budget. On LiveCodeBench-v6, RAD reaches 54.69% and 58.67% pass@1 with Qwen3-1.7B (non-thinking) and OLMo3-7B-Instruct, respectively, exceeding the strongest baselines by 8.7 and 6.4 points. These results support reward-guided weighting of teacher supervision across model families and tasks.
Read the full abstract ▾
All authors Yilan Chen, Zian Su, Manish Prajapat, Dingmin Wang, Rein Houthooft, Rui Song, Vijay Lingam
Computation and Language
LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding
Zeyu Shan, Yongxin Shi, Shihang Wu et al. (+4 more)
Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large
Language Models (LLMs) and Large
Vision-
Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Document understanding remain largely unexplored. To bridge this gap, we introduce LongHisDoc, the first
benchmark tailored for long historical document understanding. LongHisDoc comprises 1,012 expert-annotated QA pairs derived from 101 PDF documents, with an average of 85.1 pages and 42,776.4
tokens per document. It spans 10 categories and features four QA types: single-page, cross-page, multi-hop, and unanswerable. Answering these questions requires integrating
multimodal evidence from text, layout, and visual elements. Our evaluation reveals that the top-performing LVLM and LLM achieve only 62.9% and 69.9% accuracy, respectively, highlighting a substantial performance gap. Furthermore, we assess various modal retrievers and the end-to-end RAG performance to provide references for
Retrieval-Augmented Generation (RAG) systems. We believe that LongHisDoc serves as a comprehensive
benchmark to catalyze future research in full-book historical document understanding. The
benchmark and evaluation code are available at https://huggingface.co/
datasets/qweq12433454/LongHisDoc and https://github.com/SCUT-DLVCLab/LongHisDoc.
Read the full abstract ▾
All authors Zeyu Shan, Yongxin Shi, Shihang Wu, Peirong Zhang, Yuanyuan Fang, Zhaopeng Wang, Lianwen Jin
Computation and Language · Cross-listed from cs.LG
Query-Adaptive Neuron Steering for LLMs
Yanxi Pan, Jingwen Wu, Yan Ma et al. (+8 more)
Effectively eliciting the capabilities of large
language models (LLMs) at inference time remains an important challenge. Existing studies address this challenge through context engineering, which shapes user queries, or fixed steering, which directly intervenes in model computation. However, predefined contexts and fixed intervention strategies may not adequately accommodate the diverse knowledge and
reasoning requirements of individual queries. This raises a natural question: Can internal steering in LLMs be made query-adaptive? In this work, we investigate whether query-induced internal activations can guide adaptive steering in LLMs. Our pilot studies reveal subject- and layer-dependent
neuron activation patterns and show that amplifying subject-relevant
neurons can improve model performance. Based on these findings, we propose Q-NeuS, a query-adaptive
neuron steering framework that dynamically determines which
neurons to modulate and by how much for each input. Specifically, Q-NeuS constructs a compact subject-guided candidate
neuron space and employs a lightweight Query-Adaptive Gate to generate
neuron-wise modulation coefficients. To learn effective intervention strategies, we introduce Neu-GRPO, which adapts group-relative optimization to train the Query-Adaptive Gate using the relative performance of multiple
neuron intervention configurations. The base LLM remains frozen throughout training and inference, and no task or subject labels are required at inference time. Extensive experiments across multiple LLMs and
benchmarks demonstrate that Q-NeuS consistently improves model performance and exhibits broad applicability across model scales and strong cross-
benchmark transferability.
Read the full abstract ▾
All authors Yanxi Pan, Jingwen Wu, Yan Ma, Xiang Chen, Xinyi Zeng, Xuetong Wang, Yunyan Zhang, Yue Zhang, Yu Tian, Jianfeng Pan, Hongyi Zhou
Computation and Language
AphaVoice: Text-to-Speech Model for Aphasia Patient Simulation
Al Varrel Putra Kusuma, Fernando Bello and Joshua Brown
While
language therapy is the primary treatment for post-stroke aphasia, limited clinical resources often restrict patient access to proper care. Addressing this gap requires reliable training simulations for medical professionals. However, current AI efforts focus almost exclusively on
speech processing, leaving
speech generation largely unexplored. This study introduces AphaVoice, a novel text-to-
speech (TTS) framework explicitly engineered to synthesize highly realistic aphasic
speech. Evaluated against state-of-the-art TTS systems, AphaVoice demonstrates a superior ability to capture the clinical feature of dysfluency, outperforming existing models in temporal metrics while consistently maintaining speaker identity. By leveraging latent space shifting, the model also supports multi-speaker generation without sacrificing phonetic realism. Ultimately, AphaVoice provides a scalable, high-fidelity tool for medical training and clinical simulation related to aphasia.
Read the full abstract ▾
Computation and Language
Scale, Drivers, and Limits of Representational Alignment Across Large Language Models
Jialin Wu and Meenakshi Khosla
Comparing large
language models can reveal shared representational structure and systematic differences between them. However, aggregate similarity measures can obscure how agreement varies across
geometric scales and which linguistic properties support it. We introduce two complementary methods to resolve where models agree and what supports that agreement. Standard representational similarity analysis (RSA) summarizes similarity across all stimulus pairs, potentially obscuring differences in
alignment between nearby and distant inputs. Neighborhood-Weighted RSA (NW-RSA) addresses this limitation by weighting stimulus pairs according to their proximity in representational space, placing
alignment from local neighborhoods to global geometry on a common scale. Counterfactual RSA (CF-RSA) tests which linguistic properties support
alignment by comparing intact representations in one model with representations under controlled linguistic perturbations in another. Across diverse
language models, we find strong agreement in coarse representational geometry but substantial divergence in local neighborhoods.
Alignment is observed both across broadly sampled natural text and within linguistic equivalence classes that preserve particular aspects of meaning or structure. Counterfactual analyses reveal selective sensitivity: changes to logical scope reduce
alignment substantially more than the tested changes to hierarchical syntax, event roles, or discourse structure.
Alignment also decreases when sentence structure or lexical meaning is disrupted in word lists and jabberwocky sentences, suggesting that the regularities of natural
language support stronger convergence. These findings refine claims of representational universality by showing that strong overall similarity need not imply agreement in how models represent finer linguistic distinctions.
Read the full abstract ▾
Computation and Language
EdgeLLM-Nano: Auditing Polar-Guided Key-Cache Compression and Consistency-Anchored Pruning on a 33.6 MB On-Device Language Model
S M Mahbub Murshed, Hayden Bright, Rezaul Karim et al. (+1 more)
Deploying
language models (LMs) on the edge presents unique
memory, latency, and power challenges absent in high-performance computing. KV-cache compression emerges as a promising approach to address these challenges. However, the effectiveness of current KV-cache compression methods has not been thoroughly examined on edge devices. Moreover, we show that existing evaluation methods do not always involve the KV-cache under test. We propose EdgeLLM-Nano, a measurement study of KV-cache compression using a nano-scale decoder-only LM (
transformer) testbed that has 8.39M parameters, i.e., 33.6 MB, to enable end-to-end examination of each backend, seed, and power sample. Our results give a negative verdict for a polar-guided key-cache quantizer (round-to-nearest (RTN) control causes an order of magnitude less damage), as well as a consistency-anchored progressive weight pruner (one-shot magnitude-based
pruning performs as well as, if not better). We also introduce Long-Context Decode Fidelity (LCDF) evaluation, a
token-by-
token examination of teacher-forced log-losses using the KV-cache under test. Unlike perplexity that gives the same 5.461 score to all methods, LCDF distinguishes different KV-cache compression backends thanks to its paired statistics across seeds and effect sizes. Our companion Raspberry Pi 5 measurement shows all compression backends we tested to be less energy-efficient per
token compared to uncompressed FP16 inference. Except for KIVI that is close to lossless in terms of LCDF, our findings indicate KV-cache compression can be improved. LCDF offers a simple and effective way of shedding light on KV-cache compression methods, whose negative impacts have been overlooked by perplexity.
Read the full abstract ▾
All authors S M Mahbub Murshed, Hayden Bright, Rezaul Karim, Ali Shiri Sichani
Computation and Language
Evaluating Bias in Chinese Large Language Models across Tasks and Registers: Task Operationalization Effects and Irony Classification Risks
Mo Zhou
In Chinese content governance, large
language models are increasingly used to identify toxic and discriminatory expressions, yet both task operationalization and ironic expression can alter bias measurement readings. We conducted a preregistered multi-task behavioral audit of multiple Chinese large
language models. In the content classification experiment, the core panel's classification error rate rose from 10.9% in the formal register to 23.5% in the yinyang (an echoic-irony register) version, a risk difference of +12.7 percentage points. (95% CI [+9.7, +15.7]). Inter-rater agreement was lower in the yinyang register (majority-vote kappa = 0.299), so the increase contains a component of label uncertainty propagation, but the direction of increase was consistent under both adjudicated and majority-vote reference standards. Explicit irony-aware
prompts reduced the intervention panel's error rate by 9.6 percentage points. We operationalized the 29 shared proposition anchors into three task forms: social consensus estimation (Form A), high-stakes decision (Form B), and controlled association rating (Form C); independent manipulation checks found that all 58 A-C comparison units received unanimous agreement, whereas the low A-B convergence was confirmed by calibrated majority adjudication (87.8% supporting the minority judgment); on the irony-probe materials, the interactions of B and C relative to A were +0.577 and +1.121, respectively. Thus, when evaluating large
language models for Chinese content governance, the effect of task operationalization on bias readings is an empirical question that requires data to answer, and safety evaluation should incorporate the ironic register as a test condition.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
Re-executing Coding-Agent Evaluations: Unchanged Decisions and Unresolved Policy Risk
Tianyu Ding, Yaocheng Tong, Muzhi Yang et al. (+3 more)
Executable tests grade large
language model (LLM) coding-
agent patches for leaderboards, rewards and self-improvement. Two preregistered re-execution studies returned negative results. SWE-bench Pro v1's offline option rejected none of 59 accepted Go patches whose grading configuration changed. Their exploratory, unweighted upper confidence limit is 4.95% under candidate independence, leaving a 3% exposed-patch risk unresolved; the preregistered
estimator uses inverse sampling-probability weights. An extension was inconclusive. In a selected stratum, a reconstructed module cache built from the reference solution's dependencies rejected accepted alternatives from four models that passed with their own dependencies. Re-executing 1,929 selected
agent-version/task evaluations from the Darwin Gödel Machine's archive preserved best-
agent selection and system-variant ranking, with search history and scores of evaluations not rerun fixed. Exploratory host, image and adapted-grader comparisons also preserved them. Seven of 23 disagreements with archived verdicts did not recur on a second host; rebuilt images changed 16 three-run majority verdicts (0.83%) under four-CPU execution. We provide an evidence-to-claim summary, lessons for grader audits and an anonymous replication package. These results concern accepted patches and fixed-history decisions; general policy safety and adaptive-learning robustness require broader evaluations.
Read the full abstract ▾
All authors Tianyu Ding, Yaocheng Tong, Muzhi Yang, Fei Jiang, Lei Yang, Boya Yu
Computation and Language
Evaluating Semantic Relation Knowledge through Bidirectional Relation Prediction from Gloss Pairs
Zhihan Cao, Adam Nohejl and Hitomi Yanaka
Semantic relations organise word senses and are fundamental to lexical semantics. In order to evaluate semantic relation knowledge of large
language models (LLMs), we propose a challenging task in which an LLM predicts the relation between two glosses (definitions of senses) presented in two different orders. The predictions for the two orders are evaluated for consistency with the
symmetry or asymmetry of semantic relations. Our task is designed for both monolingual and cross-lingual evaluation, allowing us to examine whether relation knowledge remains accessible when the same senses are verbalised through glosses in different
languages. We construct a
dataset of 3,069 sense pairs, with each sense verbalised in English, Greek, Romanian, Turkish, and Japanese. Experiments with 17 LLMs in zero-shot and five-shot settings reveal generally poor performance. In each setting, only eight LLMs achieve a macro-averaged F1 score significantly above a random baseline, and only one (zero-shot) and two (five-shot) significantly outperform a neural baseline. In particular, LLMs often fail to generate bidirectionally consistent predictions for hypernymy and to make correct predictions for holonymy in either direction. Our results suggest limitations in current LLMs' semantic relation knowledge, both in identifying relations and in respecting asymmetry.
Read the full abstract ▾
Computation and Language
Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?
Duaa Nakshbandi, Zhiyu An, Yiping Li et al. (+2 more)
Large
Language Models (LLMs) are increasingly being used to summarize and verify claims in critical sources that inform consequential decisions. Such claims are full of plural definites. If three of five patients improved after taking drug A, the claim the patients improved after drug A is not true, since two did not, yet its negation, the patients didn't improve after drug A, is not true either, since three did. In formal semantics, this truth-value gap is known as the Homogeneity Gap: both the plural definite and its negation are neither true nor false when the property holds for only a fraction of the plural referent. A model that cannot represent this gap turns mixed evidence into a clean verdict, an error that can be costly in high-stakes use. We introduce HomogeneityNLI, a diagnostic
dataset of 1,000 minimal pair items crossing determiner, polarity, and group composition across ten logical conditions. Evaluating seven open-weight models, we uncover a pervasive Bivalence Bias: models assign plural definites their homogeneous truth conditions. In the zero-shot setting, no model recognizes the gap for both polarities in more than 15% of scenarios, and negated definites are judged False in 84% of cases. Explicit linguistic rules, chain-of-thought, and enumeration
prompts produce unstable, model-specific effects that often entrench the bias. Our findings indicate that current open-weight LLMs acquire the truth conditions of plural predication but not its trivalent logic, and cannot yet be trusted to summarize or verify claims whose evidence is mixed.
Read the full abstract ▾
All authors Duaa Nakshbandi, Zhiyu An, Yiping Li, Sachin Giri, Wan Du
Computation and Language · Cross-listed from cs.CR
MemForm: Benchmarking Memory Formation Poisoning in LLM Agents
Haowen Pan, Pengyang Shao, Yunshan Ma et al. (+4 more)
Long-term
memory enables LLM
agents to learn from past interactions but exposes
memory formation to
adversarial influence. Existing studies primarily organize
memory poisoning by attack channels, corruption patterns, or lifecycle stages, leaving formation interfaces insufficiently characterized. We formulate
Memory Formation Poisoning as
adversarial manipulation of the interfaces through which persistent
agent memory is produced and committed, and introduce MemForm, a controlled diagnostic
benchmark covering five interfaces: Content, Evidence, Scope, Selection, and Path Poisoning. We introduce Selection and Path Poisoning as controlled intervention categories that isolate retention and consolidation order effects, respectively. Each category pairs clean and poisoned conditions with interface-specific controls. Evaluations across six LLMs and two response formats reveal broad susceptibility at all five interfaces under the evaluated
memory protocols and implementations. Crucially, authentic experiences with successful outcomes do not guarantee reliable behavior after
memory formation: changes to contextual support, retention, or consolidation order can induce undesirable downstream behavior. In particular, consolidation order can alter behavior even with identical retained experiences, depending on the
memory formation procedure. Additional analyses show that interface applicability depends on
memory architecture and that Path Poisoning susceptibility depends on the updater. The evaluated defenses provide incomplete mitigation that varies by interface. These findings motivate protecting both what is committed to persistent
memory and how
memory is formed from experiences. Code and data are available at https://github.com/opanhw/memform-code.
Read the full abstract ▾
All authors Haowen Pan, Pengyang Shao, Yunshan Ma, Yanzheng Jin, Naixin Zhai, Xiang Wang, Xun Yang
Computation and Language
Scaling Laws for Multimodal Data Mixtures
Aditi Khandelwal, Ayush K. Tarun, Yixuan Xu et al. (+5 more)
Frontier AI systems are increasingly natively
multimodal, jointly pretrained on multiple modalities, such as text,
vision, and audio. However, the scientific understanding of the
multimodal data mixtures used to pretrain these systems — how much of each modality to mix, and at what potential interference cost to the others — is largely absent or gatekept. In this work, we derive data mixing scaling laws for three modalities: text,
vision, and audio.
Multimodal Large
Language Models (MLLMs) increasingly adopt Mixture-of-Experts (MoE) architectures, as MoEs scale efficiently and partially limit cross-modal interference. We therefore use MoEs to conduct 268 experiments ranging from 457M to 8.3B parameters, on up to 150 billion
tokens. Our work leads to multiple actionable insights into
multimodal data mixing for training MLLMs. First, compute optima are modality-specific: audio modality is model-heavy, favoring scaling parameters over
tokens, while text and
vision require balanced scaling of model size and data. Second, repeating
vision or audio data beyond 2x yields negligible benefits. Finally, cross-modal interactions are highly asymmetric:
vision strongly interferes with audio, whereas audio's interference with learning text and
vision is negligible. Overall, we provide a principled foundation for understanding the
multimodal data mixtures needed to train frontier MLLMs.
Read the full abstract ▾
All authors Aditi Khandelwal, Ayush K. Tarun, Yixuan Xu, Imanol Schlag, Golnoosh Farnadi, Siva Reddy, Luke Zettlemoyer, Antoine Bosselut