Computation and Language
HumanLM: Simulating Users with State Alignment Beats Response Imitation
Shirley Wu, Evelyn Choi, Arpandeep Khatua et al. (+7 more)
Large
Language Models (LLMs) are increasingly used to simulate how specific users respond to a given context, enabling more user-centric applications that rely on user feedback. However, existing user simulators mostly imitate surface-level patterns and
language styles, which fail to reflect the underlying states of real users (e.g., beliefs and
emotions). To address these limitations, we propose a novel training framework, HumanLM, which builds user simulators that accurately reflect real users. Our key insight is that, in addition to generating responses, the model should generate natural-
language latent states that align with ground-truth responses through reinforcement learning. These latent states correspond to a set of psychologically grounded state dimensions that drive how real users respond. HumanLM further synthesizes these aligned latent states into responses that accurately represent real users. For extensive evaluation, we develop Humanual, a comprehensive
benchmark for simulating real users based on public data. Humanual consists of six large-scale
datasets with 26k users and 216k responses in total, spanning diverse tasks such as generating user responses to daily life issues, political blogs, and chat sessions with LLM assistants. Across
datasets, HumanLM significantly outperforms alternative approaches, achieving an average relative improvement of 16.3% in
alignment scores from an LLM judge. In a real-time simulation study with 111 participants, HumanLM achieves the highest similarity to real user responses and competitive human-likeness scores.
Read the full abstract ▾
All authors Shirley Wu, Evelyn Choi, Arpandeep Khatua, Zhanghan Wang, Joy He-Yueya, Tharindu Cyril Weerasooriya, Wei Wei, Diyi Yang, Jure Leskovec, James Zou
Computation and Language
Developing an AI Assistant for Knowledge Management and Workforce Training in State DOTs
Divija Amaram, Lu Gao, Gowtham Reddy Gudla et al. (+1 more)
Effective knowledge management is critical for preserving institutional expertise and improving the efficiency of workforce training in state transportation agencies. Traditional approaches, such as static documentation, classroom-based instruction, and informal mentorship, often lead to fragmented knowledge transfer, inefficiencies, and the gradual loss of expertise as senior engineers retire. Moreover, given the enormous volume of technical manuals, guidelines, and research reports maintained by these agencies, it is increasingly challenging for engineers to locate relevant information quickly and accurately when solving field problems or preparing for training tasks. These limitations hinder timely decision-making and create steep learning curves for new personnel in maintenance and construction operations. To address these challenges, this paper proposes a
Retrieval-Augmented Generation (RAG) framework with a multi-
agent architecture to support knowledge management and decision making. The system integrates structured document
retrieval with real-time, context-aware response generation powered by a large
language model (LLM). Unlike conventional single-pass RAG systems, the proposed framework employs multiple specialized
agents for
retrieval, answer generation, evaluation, and query refinement, which enables iterative improvement and quality control. In addition, the system incorporates an open-weight
vision-
language model to convert technical figures into semantic textual representations, which allows figure-based knowledge to be indexed and retrieved alongside text. Retrieved text and figure-based context are then provided to an open-weight large
language model, which generates the final responses grounded in the retrieved evidence.
Read the full abstract ▾
All authors Divija Amaram, Lu Gao, Gowtham Reddy Gudla, Tejaswini Sanjay Katale
Computation and Language
From Exact Hits to Close Enough: Semantic Caching for LLM Embeddings
Dvir David Biton and Roy Friedman
The rapid adoption of large
language models (LLMs) has created demand for faster responses and lower costs. Semantic caching, reusing semantically similar requests via their
embeddings, addresses this need but breaks classic cache assumptions and raises new challenges. In this paper, we explore offline policies for semantic caching, proving that implementing an optimal offline policy is NP-hard, and propose several
polynomial-time heuristics. We also present online semantic aware cache policies that combine recency, frequency, and locality. Evaluations on diverse
datasets show that while frequency based policies are strong baselines, our novel variant improves semantic accuracy. Our findings reveal effective strategies for current systems and highlight substantial headroom for future innovation. All code is open source.
Read the full abstract ▾
Computation and Language
Benchmarking Legal RAG: The Promise and Limits of AI Statutory Surveys
Mohamed Afane, Emaan Hariri, Derek Ouyang et al. (+1 more)
Retrieval-augmented generation (RAG) offers significant potential for legal AI, yet systematic
benchmarks are sparse. Prior work introduced LaborBench to
benchmark RAG models based on ostensible ground truth from an exhaustive, multi-month, manual enumeration of all U.S. state unemployment insurance requirements by U.S. Department of Labor (DOL) attorneys. That prior work found poor performance of standard RAG (70% accuracy on Boolean tasks). Here, we assess three emerging tools not previously evaluated on LaborBench: the Statutory Research Assistant (STARA), a custom statutory research tool, and two commercial tools by Westlaw and LexisNexis marketing AI statutory survey capabilities. We make five main contributions. First, we show that STARA achieves substantial performance gains, boosting accuracy to 83%. Second, we show that commercial platforms fare poorly, with accuracy of 58% (Westlaw AI) and 64% (Lexis+ AI), even worse than standard RAG. Third, we conduct a comprehensive error analysis, comparing our outputs to those compiled by DOL attorneys, and document both
reasoning errors, such as confusion between related legal concepts and misinterpretation of statutory exceptions, and
retrieval failures, where relevant statutory provisions are not captured. Fourth, we discover that many apparent errors are actually significant omissions by DOL attorneys themselves, such that STARA's actual accuracy is 92%. Fifth, we chart the path forward for legal RAG through concrete design principles, offering actionable guidance for building AI systems capable of accurate multi-jurisdictional legal research.
Read the full abstract ▾
All authors Mohamed Afane, Emaan Hariri, Derek Ouyang, Daniel E. Ho
Computation and Language
How LLMs Cite and Why It Matters: A Cross-Model Audit of Reference Fabrication in AI-Assisted Academic Writing and Methods to Detect Phantom Citations
MZ Naser
Large
language models (LLMs) have been noted to fabricate scholarly citations, yet the scope of this behavior across providers, domains, and prompting conditions remains poorly quantified. We present one of the largest citation
hallucination audits to date, in which 10 commercially deployed LLMs were prompted across four academic domains, generating 69,557 citation instances verified against three scholarly databases (namely, CrossRef, OpenAlex, and Semantic Scholar). Our results show that the observed
hallucination rates span a fivefold range (between 11.4% and 56.8%) and are strongly shaped by model, domain, and
prompt framing. Our results also show that no model spontaneously generates citations when unprompted, which seems to establish
hallucination as
prompt-induced rather than intrinsic. We identify two practical filters: 1) multi-model consensus (with more than 3 LLMs citing the same work yields 95.6% accuracy, a 5.8-fold improvement), and 2) within-
prompt repetition (with more than 2 replications yields 88.9% accuracy). In addition, we present findings on generational model tracking, which reveal that improvements are not guaranteed when deploying newer LLMs, and on capacity scaling, which appears to reduce
hallucination within model families. Finally, a lightweight classifier trained solely on bibliographic string features is developed to classify hallucinated citations from verified citations, achieving AUC 0.876 in cross-validation and 0.834 in LOMO generalization (without querying any external database). This classifier offers a pre-screening tool deployable at inference time.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
Reverse-Engineering Model Editing on Language Models
Zhiyu Sun, Minrui Luo, Yu Wang et al. (+2 more)
Large
language models (LLMs) are pretrained on corpora containing trillions of
tokens and, therefore, inevitably memorize sensitive information. Locate-then-edit methods, as a mainstream paradigm of model editing, offer a promising solution by modifying model parameters without retraining. However, in this work, we reveal a critical vulnerability of this paradigm: the parameter updates inadvertently serve as a side channel, enabling attackers to recover the edited data. We propose a two-stage reverse-engineering attack named KSTER (KeySpaceReconsTruction-then-EntropyReduction) that leverages the low-rank structure of these updates. First, we theoretically show that the row space of the update
matrix encodes a “fingerprint" of the edited subjects, enabling accurate subject recovery via spectral analysis. Second, we introduce an
entropy-based
prompt recovery attack that reconstructs the semantic context of the edit. Extensive experiments on multiple LLMs demonstrate that our attacks can recover edited data with high success rates. Furthermore, we propose subspace camouflage, a defense strategy that obfuscates the update fingerprint with semantic decoys. This approach effectively mitigates reconstruction risks without compromising editing utility. Our code is available at https://github.com/reanatom/EditingAttack.
Read the full abstract ▾
All authors Zhiyu Sun, Minrui Luo, Yu Wang, Zhili Chen, Tianxing He
Computation and Language · Cross-listed from cs.LG
ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs
Yanlin Qi, Xinhang Chen, Huiqiang Jiang et al. (+3 more)
KV-cache
retrieval is essential for long-context LLM inference, yet existing methods struggle with distribution drift and high latency at scale. We introduce ParisKV, a drift-robust, GPU-native KV-cache
retrieval framework based on collision-based candidate selection, followed by a quantized inner-product reranking
estimator. For million-
token contexts, ParisKV supports CPU-offloaded KV caches via Unified Virtual Addressing (UVA), enabling on-demand top-$k$ fetching with minimal overhead. ParisKV matches or outperforms full
attention quality on long-input and long-generation
benchmarks. It achieves state-of-the-art long-context decoding efficiency: it matches or exceeds full
attention speed even at batch size 1 for long contexts, delivers up to 2.8$\times$ higher throughput within full
attention's runnable range, and scales to million-
token contexts where full
attention runs out of
memory. At million-
token scale, ParisKV reduces decode latency by 17$\times$ and 44$\times$ compared to MagicPIG and PQCache, respectively, two state-of-the-art KV-cache Top-$k$
retrieval baselines, code is available at https://github.com/amy-77/ParisKV/tree/main.
Read the full abstract ▾
All authors Yanlin Qi, Xinhang Chen, Huiqiang Jiang, Qitong Wang, Botao Peng, Themis Palpanas
Computation and Language · Cross-listed from cs.SE
On Sequence-to-Sequence Models for Automated Log Parsing
Adam Sorrenti and Andriy Miranskyy
Context: Log parsing is a critical standard operating procedure in software systems, enabling monitoring, anomaly detection, and failure diagnosis. However, automated log parsing remains challenging due to heterogeneous log formats, distribution shifts between training and deployment data, and the brittleness of rule-based approaches. Objectives: This study aims to systematically evaluate how sequence modelling architecture, representation choice, sequence length, and training data availability influence automated log parsing performance and computational cost. Methods: We conduct a controlled empirical study comparing four sequence modelling architectures:
Transformer, Mamba state-space, monodirectional LSTM, and bidirectional LSTM models. In total, 396 models are trained across multiple
dataset configurations and evaluated using relative Levenshtein edit distance with statistical significance testing. Results:
Transformer achieves the lowest mean relative edit distance (0.111), followed by Mamba (0.145), mono-LSTM (0.186), and bi-LSTM (0.265), where lower values are better. Mamba provides competitive accuracy with substantially lower computational cost. Character-level tokenization generally improves performance, sequence length has negligible practical impact on
Transformer accuracy, and both Mamba and
Transformer demonstrate stronger sample efficiency than recurrent models. Conclusion: Overall,
Transformers reduce parsing error by 23.4%, while Mamba is a strong alternative under data or compute constraints. These results also clarify the roles of representation choice, sequence length, and sample efficiency, providing practical guidance for researchers and practitioners.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
EventCast: Hybrid Demand Forecasting in E-Commerce with LLM-Based Event Knowledge
Congcong Hu, Yuang Shi, Fan Huang et al. (+4 more)
Demand
forecasting is a cornerstone of e-commerce operations, directly impacting inventory planning and fulfillment scheduling. However, existing
forecasting systems often fail during high-impact periods such as flash sales, holiday campaigns, and sudden policy interventions, where demand patterns shift abruptly and unpredictably. In this paper, we introduce EventCast, a modular
forecasting framework that integrates future event knowledge into time-series prediction. Unlike prior approaches that ignore future interventions or directly use large
language models (LLMs) for numerical
forecasting, EventCast leverages LLMs solely for event-driven
reasoning. Unstructured business data, which covers campaigns, holiday schedules, and seller incentives, from existing operational databases, is processed by an LLM that converts it into interpretable textual summaries leveraging world knowledge for cultural nuances and novel event combinations. These summaries are fused with historical demand features within a dual-tower architecture, enabling accurate, explainable, and scalable forecasts. Deployed on real-world e-commerce scenarios spanning 4 countries of 160 regions over 10 months, EventCast achieves up to 86.9% and 97.7% improvement on MAE and MSE compared to the variant without event knowledge, and reduces MAE by up to 57.0% and MSE by 83.3% versus the best industrial baseline during event-driven periods. EventCast has deployed into real-world industrial pipelines since March 2025, offering a practical solution for improving operational decision-making in dynamic e-commerce environments.
Read the full abstract ▾
All authors Congcong Hu, Yuang Shi, Fan Huang, Yang Xiang, Zhou Ye, Ming Jin, Shiyu Wang
Computation and Language
Blind to the Human Touch: Overlap Bias in LLM-Based Summary Evaluation
Jiangnan Fang, Cheng-Tse Liu, Hanieh Deilamsalehy et al. (+5 more)
Large
language model (LLM) judges have often been used alongside traditional,
algorithm-based metrics for tasks like summarization because they better capture semantic information, are better at
reasoning, and are more robust to paraphrasing. However, LLM judges show biases for length and order among others, and are vulnerable to various
adversarial input
prompts. While recent studies have looked into these biases, few have analyzed them at a more granular level in relation to a well-defined overlap metric. In this work we provide an LLM judge bias analysis as a function of overlap with human-written responses in the domain of summarization. We test 9 recent LLMs with parameter counts ranging from 1 billion to 12 billion, including variants of Gemma 3 and LLaMA 3. We find that LLM judges increasingly prefer summaries generated by other LLMs over those written by humans as the similarities (as measured by ROUGE and BLEU) between the judged summaries decrease, and this pattern extends to all but one model tested, and exists regardless of the models' own position biases. Additionally, we find that models struggle to judge even summaries with limited overlaps, suggesting that LLM-as-a-judge in the summary domain should rely on techniques beyond a simple comparison.
Read the full abstract ▾
All authors Jiangnan Fang, Cheng-Tse Liu, Hanieh Deilamsalehy, Nesreen K. Ahmed, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi
Computation and Language
Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization
Jaewook Lee, Alexander Scarlatos, Simon Woodhead et al. (+1 more)
With the emergence of large
language models (LLMs) as a powerful class of generative artificial intelligence (AI), their use in tutoring has become increasingly prominent. Prior works on LLM-based tutoring typically learn a single tutor policy and do not capture the diversity of tutoring styles. In real-world tutor-student interactions, pedagogical intent is realized through adaptive instructional strategies, with tutors varying the level of scaffolding, instructional directiveness, feedback, and affective support in response to learners' needs. These differences can all impact dialogue dynamics and student engagement. In this paper, we explore how tutor personas embedded in human tutor-student dialogues can be used to guide LLM behavior without relying on explicitly prompted instructions. We train a steering vector using preference optimization: an activation-space direction that guides model responses toward specific tutor personas. We find that this steering vector captures tutor-specific variation across dialogue contexts, improving semantic
alignment with ground-truth tutor utterances and increasing preference-based evaluations, while largely preserving lexical similarity. Analysis of the learned scaling coefficients further reveals interpretable structure across tutors, corresponding to consistent differences in tutoring behavior. These results demonstrate that activation steering offers an effective and interpretable way for controlling tutor-specific variation in LLMs using signals derived directly from human dialogue data.
Read the full abstract ▾
All authors Jaewook Lee, Alexander Scarlatos, Simon Woodhead, Andrew Lan
Computation and Language
SciClaimEval: Cross-modal Claim Verification in Scientific Papers
Xanh Ho, Yun-Ang Wu, Sunisth Kumar et al. (+4 more)
We present SciClaimEval, a new scientific
dataset for the claim verification task. Unlike existing resources, SciClaimEval features authentic claims, including refuted ones, directly extracted from published papers. To create refuted claims, we introduce a novel approach that modifies the supporting evidence (figures and tables), rather than altering the claims or relying on large
language models (LLMs) to fabricate contradictions. The
dataset provides cross-modal evidence with diverse representations: figures are available as images, while tables are provided in multiple formats, including images, LaTeX source, HTML, and JSON. SciClaimEval contains 1,664 annotated samples from 180 papers across three domains, machine learning, natural
language processing, and medicine, validated through expert annotation. We
benchmark 11
multimodal foundation models, both open-source and proprietary, across the
dataset. Results show that figure-based verification remains particularly challenging for all models, as a substantial performance gap remains between the best system and human baseline.
Read the full abstract ▾
All authors Xanh Ho, Yun-Ang Wu, Sunisth Kumar, Tian Cheng Xia, Florian Boudin, Andre Greiner-Petter, Akiko Aizawa
Computation and Language
Learning to Self-Verify Makes Language Models Better Reasoners
Yuxin Chen, Yu Wang, Yi Zhang et al. (+9 more)
Recent large
language models (LLMs) achieve strong performance in generating promising
reasoning paths for complex tasks. However, despite powerful generation ability, LLMs remain weak at verifying their own answers, revealing a persistent capability asymmetry between generation and self-verification. In this work, we conduct an in-depth investigation of this asymmetry throughout training
evolution and show that, even on the same task, improving generation does not lead to corresponding improvements in self-verification. Interestingly, we find that the reverse direction of this asymmetry behaves differently: learning to self-verify can effectively improve generation performance, achieving accuracy comparable to standard generation training while yielding more efficient and effective
reasoning traces. Building on this observation, we further explore integrating self-verification into generation training by formulating a multi-task reinforcement learning framework, where generation and self-verification are optimized as two independent but complementary objectives. Extensive experiments across
benchmarks and models demonstrate performance gains over generation-only training in both generation and verification capabilities.
Read the full abstract ▾
All authors Yuxin Chen, Yu Wang, Yi Zhang, Ziang Ye, Zhengzhou Cai, Yaorui Shi, Qi Gu, Hui Su, Xunliang Cai, Xiang Wang, An Zhang, Tat-Seng Chua
Computation and Language · Cross-listed from cs.CV
Wenjie Liu, Hao Wu, Xin Qiu et al. (+6 more)
Modern
multimodal large
language models (MLLMs) adopt a unified self-
attention design that processes visual and textual
tokens at every
Transformer layer, incurring substantial computational overhead. In this work, we revisit the necessity of such dense visual processing and show that projected visual
embeddings are already well-aligned with the
language space, while effective
vision-
language interaction occurs in only a small subset of layers. Based on these insights, we propose ViCA (
Vision-only Cross-
Attention), a minimal MLLM architecture in which visual
tokens bypass all self-
attention and feed-forward layers, interacting with text solely through sparse cross-
attention at selected layers. Extensive evaluations across three MLLM backbones, nine
multimodal benchmarks, and 26
pruning-based baselines show that ViCA preserves 98% of baseline accuracy while reducing visual-side computation to 4%, consistently achieving superior performance-efficiency trade-offs. Moreover, ViCA provides a regular, hardware-friendly inference pipeline that yields over 3.5x speedup in single-batch inference and over 10x speedup in multi-batch inference, reducing visual grounding to near-zero overhead compared with text-only LLMs. It is also orthogonal to
token pruning methods and can be seamlessly combined for further efficiency gains. Our code is available at https://github.com/EIT-NLP/ViCA.
Read the full abstract ▾
All authors Wenjie Liu, Hao Wu, Xin Qiu, Xudong Wang, Yingqi Fan, Yihan Zhang, Anhao Zhao, Yunpu Ma, Xiaoyu Shen
Computation and Language · Cross-listed from cs.AI
When Is Enough Not Enough? Illusory Completion in Search Agents
Dayoon Ko, Jihyuk Kim, Sohyeon Kim et al. (+5 more)
In agentic search, an LLM
agent searches the web, reads the pages it finds, and decides what to look for next before returning an answer. But can we trust an answer simply because the
agent returns it? Often not, and even a correct answer can be a lucky guess: on questions with several constraints, we find that
agents conclude the task is complete while a constraint remains unverified in up to 48% of their correct answers. We call this illusory completion. To see how it arises, we introduce the Epistemic Ledger, which tracks at every turn what the retrieved pages establish about each constraint and what the
agent claims. Across 13
agents, from 7B RL-trained models to frontier LLMs, training and scale raise accuracy but change the pattern of verification failures rather than eliminating them: constraints may be left unchecked, assumed without support, or retained despite refuting evidence. To measure what
agents lose without tracking their constraints, we show them each constraint's state, approximated by LiveLedger, a lightweight 4B tracker.
Agents then answer 4.4-16.1 points more questions correctly, suggesting that on their own, they may not track what they have verified and what remains.
Read the full abstract ▾
All authors Dayoon Ko, Jihyuk Kim, Sohyeon Kim, Haeju Park, Dahyun Lee, Gunhee Kim, Moontae Lee, Kyungjae Lee
Computation and Language · Cross-listed from q-bio.NC
Linguistic properties and model scale in brain encoding: from small to compressed language models
Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi et al. (+5 more)
Recent work has shown that scaling large
language models (LLMs) improves their
alignment with human
brain activity, yet it remains unclear what drives these gains and which representational properties are responsible. Although larger models often yield better task performance and
brain alignment, they are increasingly difficult to analyze mechanistically. This raises a fundamental question: what is the minimal model capacity required to capture
brain-relevant representations? To address this question, we systematically investigate how constraining model scale and numerical precision affects
brain alignment. We compare full-precision LLMs, small
language models (SLMs), and compressed variants (quantized and pruned) by predicting fMRI responses during naturalistic
language comprehension. Across model families up to 14B parameters, we find that 3B SLMs achieve
brain predictivity indistinguishable from larger LLMs, whereas 1B models degrade substantially, particularly in semantic
language regions.
Brain alignment is remarkably robust to compression: most quantization and
pruning methods preserve neural predictivity, with GPTQ as a consistent exception. Linguistic probing reveals a dissociation between task performance and
brain predictivity: compression degrades discourse, syntax, and morphology, yet
brain predictivity remains largely unchanged. Overall,
brain alignment saturates at modest model scales and is resilient to compression, challenging common assumptions about neural scaling and motivating compact models for
brain-aligned
language modeling.
Read the full abstract ▾
All authors Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju
Computation and Language
Improving Variable-Length Generation in Diffusion Language Models via Length Regularization
Zicong Cheng, Ruixuan Jia, Jia Li et al. (+3 more)
Diffusion Large
Language Models (DLLMs) are inherently ill-suited for variable-length generation, as their inference is defined on a fixed-length canvas and implicitly assumes a known target length. When the length is unknown, as in realistic completion and infilling, naively comparing confidence across mask lengths becomes systematically biased, leading to under-generation or redundant continuations. In this paper, we show that this failure arises from an intrinsic lengthinduced bias in generation confidence estimates, leaving existing DLLMs without a robust way to determine generation length and making variablelength inference unreliable. To address this issue, we propose LR-DLLM, a length-regularized inference framework for DLLMs that treats generation length as an explicit variable and achieves reliable length determination at inference time. It decouples semantic compatibility from lengthinduced uncertainty through an explicit length regularization that corrects biased confidence estimates. Based on this, LR-DLLM enables dynamic expansion or contraction of the generation span without modifying the underlying DLLM or its training procedure. Experiments show that LRDLLM achieves 51.3% Pass@1 on HumanEvalInfilling under fully unknown lengths (+13.4% vs. DreamOn) and 51.5% average Pass@1 on four-
language McEval (+14.3% vs. DreamOn).
Read the full abstract ▾
All authors Zicong Cheng, Ruixuan Jia, Jia Li, Guo-Wei Yang, Meng-Hao Guo, Shi-Min Hu
Computation and Language · Cross-listed from q-bio.NC
Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models
Yixuan Liu, Zhiyuan Ma, Likai Tang et al. (+5 more)
How large
language models (LLMs) align with the neural representation and computation of human
language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking
entropy,
curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a
geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced
entropy and
curvature, consistently better predicted human
language network activity. This
alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced
curvature remained a robust predictor of model-
brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven
geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing.
Read the full abstract ▾
All authors Yixuan Liu, Zhiyuan Ma, Likai Tang, Runmin Gan, Xinche Zhang, Jinhao Li, Chao Xie, Sen Song
Computation and Language · Cross-listed from cs.CR
MemPot: Defending Against Memory Extraction Attack with Optimized Honeypots
Yuhao Wang, Shengfang Zhai, Guanghao Jin et al. (+3 more)
Large
Language Model (LLM)-based
agents employ external and internal
memory systems to handle complex, goal-oriented tasks, yet this exposes them to severe extraction attacks, and effective defenses remain lacking. In this paper, we propose MemPot, the first theoretically verified defense framework against
memory extraction attacks by injecting optimized honeypots into the
memory. Through a two-stage optimization process, MemPot generates trap documents that maximize the
retrieval probability for attackers while remaining inconspicuous to benign users. We model the detection process as Wald's Sequential Probability Ratio Test (SPRT) and theoretically prove that MemPot achieves a lower average number of sampling rounds compared to optimal static detectors. Empirically, MemPot significantly outperforms state-of-the-art baselines, achieving a 50% improvement in detection AUROC and an 80% increase in True Positive Rate under low False Positive Rate constraints. Furthermore, our experiments confirm that MemPot incurs zero additional online inference latency and preserves the
agent's utility on standard tasks, verifying its superiority in safety, harmlessness, and efficiency.
Read the full abstract ▾
All authors Yuhao Wang, Shengfang Zhai, Guanghao Jin, Yinpeng Dong, Linyi Yang, Jiaheng Zhang
Computation and Language
Let's Simplify Step by Step: Guiding LLM Towards Multilingual Unsupervised Proficiency-Controlled Sentence Simplification
Jingshen Zhang, Xin Ying Qiu, Lifang Lu et al. (+4 more)
Large
language models demonstrate limited capability in proficiency-controlled sentence simplification, particularly when simplifying across large readability levels. We propose a framework that decomposes complex simplifications into manageable steps through dynamic path planning, semantic-aware exemplar selection, and chain-of-thought generation with conversation history for coherent
reasoning. Evaluation on five
languages across two
benchmarks shows our approach improves simplification effectiveness while reducing computational steps by 22-42%. Human evaluation confirms the fundamental trade-off between simplification effectiveness and meaning preservation. Notably, even human annotators struggle to agree on semantic preservation judgments, highlighting the inherent complexity of this task. Our work shows that while step-by-step simplification improves control, preserving semantic fidelity during extensive simplification remains an open challenge.
Read the full abstract ▾
All authors Jingshen Zhang, Xin Ying Qiu, Lifang Lu, Zhuhua Huang, Yutao Hu, Yuechang Wu, JunYu Lu
Computation and Language
From Native Memes to Global Moderation: Cross-Cultural Evaluation of Vision-Language Models for Hateful Meme Detection
Mo Wang, Kaixuan Ren, Pratik Jalan et al. (+5 more)
Cultural context profoundly shapes how people interpret online content, yet
vision-
language models (VLMs) remain predominantly trained through Western or English-centric lenses. This limits their
fairness and cross-cultural robustness in tasks like hateful meme detection. We introduce a systematic evaluation framework designed to diagnose and quantify the cross-cultural robustness of state-of-the-art VLMs across
multilingual meme
datasets, analyzing three axes: (i) learning strategy (zero-shot vs. one-shot), (ii) prompting
language (native vs. English), and (iii) translation effects on meaning and detection. Results show that the common “translate-then-detect” approach deteriorate performance, while culturally aligned interventions - native-
language prompting and one-shot learning - significantly enhance detection. Our findings reveal systematic convergence toward Western safety norms and provide actionable strategies to mitigate such bias, guiding the design of globally robust
multimodal moderation systems.
Read the full abstract ▾
All authors Mo Wang, Kaixuan Ren, Pratik Jalan, Ahmed Ashraf, Tuong Vy Vu, Rahul Seetharaman, Shah Nawaz, Usman Naseem
Computation and Language · Cross-listed from cs.LG
On the Importance of a Multi-Scale Calibration for Quantization
Seungwoo Son, Ingyu Seong, Junhan Kim et al. (+2 more)
Post-training quantization (PTQ) is a cornerstone for efficiently deploying large
language models (LLMs), where a small calibration set critically affects quantization performance. However, conventional practices rely on random sequences of fixed length, overlooking the variable-length nature of LLM inputs. Input length directly influences the activation distribution and, consequently, the weight importance captured by the Hessian, which in turn affects quantization outcomes. As a result, Hessian estimates derived from fixed-length calibration may fail to represent the true importance of weights across diverse input scenarios. We propose MaCa (Matryoshka Calibration), a simple yet effective method for length-aware Hessian construction. MaCa (i) incorporates multi-scale sequence length information into Hessian estimation and (ii) regularizes each sequence as an independent sample, yielding a more stable and fruitful Hessian for accurate quantization. Experiments on state-of-the-art LLMs (e.g., Qwen3, Gemma3, LLaMA3) demonstrate that MaCa consistently improves accuracy under low bit quantization, offering a lightweight enhancement compatible with existing PTQ frameworks. To the best of our knowledge, this is the first work to systematically highlight the role of multi-scale calibration in LLM quantization.
Read the full abstract ▾
All authors Seungwoo Son, Ingyu Seong, Junhan Kim, Hyemi Jang, Yongkweon Jeon
Computation and Language
SED-SFT: Selectively Encouraging Diversity in Supervised Fine-Tuning
Yijie Chen, Yijin Liu and Fandong Meng
Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has emerged as the standard post-training paradigm for large
language models (LLMs). However, the conventional SFT process, driven by Cross-
Entropy (CE) loss, often induces mode collapse, where models over-concentrate on specific response patterns. This lack of distributional diversity severely restricts the exploration efficiency required for subsequent RL. While recent studies have attempted to improve SFT by replacing the CE loss, aiming to preserve diversity or refine the update policy, they fail to adequately balance diversity and accuracy, thereby yielding suboptimal performance after RL. To address the mode collapse problem, we propose SED-SFT, which adaptively encourages diversity based on the
token exploration space. This framework introduces a selective
entropy regularization term with a selective masking mechanism into the optimization objective. Extensive experiments across eight mathematical
benchmarks demonstrate that SED-SFT significantly enhances generation diversity with a negligible computational overhead increase compared with CE loss, yielding average improvements of 2.06 and 1.20 points in subsequent RL performance over standard CE-based baselines on Llama-3.2-3B-Instruct and Qwen2.5-Math-7B-Instruct, respectively. The code is publicly available at https://github.com/pppa2019/SED-SFT
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
Pull Requests as a Training Signal for Repo-Level Code Editing
Qinglin Zhu, Tianyu Chen, Shuai Lu et al. (+8 more)
Repository-level code editing requires models to understand complex dependencies and execute precise multi-file modifications across a large codebase. While recent gains on SWE-bench rely heavily on complex
agent scaffolding, it remains unclear how much of this capability can be internalised via high-quality training signals. To address this, we propose Clean Pull Request (Clean-PR), a mid-training paradigm that leverages real-world GitHub pull requests as a training signal for repository-level editing. We introduce a scalable pipeline that converts noisy pull request diffs into Search/Replace edit blocks through reconstruction and validation, resulting in the largest publicly available corpus of 2 million pull requests spanning 12 programming
languages. Using this training signal, we perform a mid-training stage followed by an agentless-aligned supervised fine-tuning process with error-driven data augmentation. On SWE-bench, our model significantly outperforms the instruction-tuned baseline, achieving absolute improvements of 13.6% on SWE-bench Lite and 12.3% on SWE-bench Verified. These results demonstrate that repository-level code understanding and editing capabilities can be effectively internalised into model weights under a simplified, agentless protocol, without relying on heavy inference-time scaffolding.
Read the full abstract ▾
All authors Qinglin Zhu, Tianyu Chen, Shuai Lu, Lei Ji, Runcong Zhao, Murong Ma, Xiangxiang Dai, Yulan He, Lin Gui, Peng cheng, Yeyun Gong
Computation and Language
DLLM Agent: See Farther, Run Faster
Huiling Zhen, Weizhe Lin, Renxi Liu et al. (+15 more)
Diffusion large
language models (DLLMs) have emerged as an alternative to autoregressive (AR) decoding with appealing efficiency and modeling properties, yet their implications for agentic multi-step decision making remain underexplored. We ask a concrete question: when the generation paradigm is changed but the
agent framework and supervision are held fixed, do
diffusion backbones induce systematically different planning and tool-use behaviors, and do these differences translate into end-to-end efficiency gains? We study this in a controlled setting by instantiating DLLM and AR backbones within the same
agent workflow (DeepDiver) and performing matched
agent-oriented fine-tuning on the same trajectory data, yielding
diffusion-backed DLLM
Agents and directly comparable AR
agents. Across
benchmarks and case studies, we find that, at comparable accuracy, DLLM
Agents are on average over 30% faster end to end than AR
agents, with some cases exceeding 8x speedup. Conditioned on correct task completion, DLLM
Agents also require fewer interaction rounds and tool invocations, consistent with higher planner hit rates that converge earlier to a correct action path with less backtracking. We further identify two practical considerations for deploying
diffusion backbones in tool-using
agents. First, naive DLLM policies are more prone to structured tool-call failures, necessitating stronger tool-call-specific training to emit valid schemas and arguments. Second, for multi-turn inputs interleaving context and action spans,
diffusion-style span corruption requires aligned
attention masking to avoid spurious context-action information flow; without such
alignment, performance degrades. Finally, we analyze
attention dynamics across workflow stages and observe paradigm-specific coordination patterns, suggesting stronger global planning signals in
diffusion-backed
agents.
Read the full abstract ▾
All authors Huiling Zhen, Weizhe Lin, Renxi Liu, Kai Han, Yiming Li, Yuchuan Tian, Hanting Chen, Xiaoguang Li, Xiaosong Li, Chen Chen, Xianzhi Yu, Mingxuan Yuan, Youliang Yan, Peifeng Qin, Jun Wang, Yu Wang, Dacheng Tao, Yunhe Wang
Computation and Language
Measuring cross-language intelligibility between Romance languages with computational tools
Liviu P Dinu, Ana Sabina Uban, Bogdan Iordache et al. (+2 more)
We present an analysis of mutual intelligibility in related
languages applied for
languages in the Romance family. We introduce a novel computational metric for estimating intelligibility based on lexical similarity using surface and semantic similarity of related words, and use it to measure mutual intelligibility for the five main Romance
languages (French, Italian, Portuguese, Spanish, and Romanian), and compare results using both the orthographic and phonetic forms of words as well as different parallel corpora and vectorial models of word meaning representation. The obtained intelligibility scores confirm intuitions related to intelligibility asymmetry across
languages and significantly correlate with results of cloze tests in human experiments.
Read the full abstract ▾
All authors Liviu P Dinu, Ana Sabina Uban, Bogdan Iordache, Anca Dinu, Simona Georgescu
Computation and Language · Cross-listed from cs.LG
Sign-Based Optimizers Are Effective Under Heavy-Tailed Noise
Dingzhi Yu, Hongyi Tao, Yuanyu Wan et al. (+2 more)
While adaptive gradient methods are the workhorse of modern machine learning, sign-based optimization
algorithms such as Lion and
Muon have recently demonstrated superior empirical performance over AdamW in training large
language models (LLM). However, a theoretical understanding of why sign-based updates outperform variance-adapted methods remains elusive. In this paper, we aim to bridge the gap between theory and practice through the lens of heavy-tailed gradient noise, a phenomenon frequently observed in
language modeling tasks. Theoretically, we introduce a novel generalized heavy-tailed noise condition that captures the behavior of LLMs more accurately than standard finite variance assumptions. Under this noise model, we establish sharp convergence rates of SignSGD and Lion for generalized smooth function classes, matching or surpassing previous best-known bounds. Furthermore, we extend our analysis to
Muon and Muonlight, providing what is, to our knowledge, the first rigorous analysis of
matrix optimization under heavy-tailed stochasticity. These results offer a strong theoretical justification for the empirical superiority of sign-based optimizers, showcasing that they are naturally suited to handle the noisy gradients associated with heavy tails. Empirically, LLM pretraining experiments validate our theoretical insights and confirm that our proposed noise models are well-aligned with practice.
Read the full abstract ▾
All authors Dingzhi Yu, Hongyi Tao, Yuanyu Wan, Luo Luo, Lijun Zhang
Computation and Language · Cross-listed from cs.CR
Secure Code Generation via Online Reinforcement Learning with Vulnerability Reward Model
Tianyi Wu, Mingzhe Du, Yue Liu et al. (+4 more)
Large
language models (LLMs) are increasingly used in software development, yet their tendency to generate insecure code remains a major barrier to real-world deployment. Existing secure code
alignment methods often suffer from a functionality–security paradox, improving security at the cost of substantial utility degradation. We propose SecCoderX, an online reinforcement learning framework for functionality-preserving secure code generation. SecCoderX first bridges vulnerability detection and secure code generation by repurposing mature detection resources in two ways: (i) synthesizing diverse, reality-grounded vulnerability-inducing coding tasks for online RL rollouts, and (ii) training a
reasoning-based vulnerability reward model that provides scalable and reliable security supervision. Together, these components are unified in an online RL loop to align code LLMs to generate secure and functional code. Extensive experiments demonstrate that SecCoderX achieves state-of-the-art performance, improving Effective Safety Rate (ESR) by approximately 10% over unaligned models, whereas prior methods often degrade ESR by 14-54%. We release our code,
dataset and model checkpoints at https://github.com/AndrewWTY/SecCoderX.
Read the full abstract ▾
All authors Tianyi Wu, Mingzhe Du, Yue Liu, Chengran Yang, Terry Yue Zhuo, Jiaheng Zhang, See-Kiong Ng
Computation and Language · Cross-listed from cs.AI
Can LLMs Truly Embody Human Personality? Analyzing AI and Human Behavior Alignment in Dispute Resolution
Deuksin Kwon, Kaleen Shrestha, Bin Han et al. (+5 more)
Large
language models (LLMs) are increasingly used to simulate human behavior in social settings such as legal mediation, negotiation, and dispute resolution. However, it remains unclear whether these simulations reproduce the personality-behavior patterns observed in humans. Human personality, for instance, shapes how individuals navigate social interactions, including strategic choices and behaviors in emotionally charged interactions. This raises the question: Can LLMs, when prompted with personality traits, reproduce personality-driven differences in human conflict behavior? To explore this, we introduce an evaluation framework that enables direct comparison of human-human and LLM-LLM behaviors in dispute resolution dialogues with respect to Big Five Inventory (BFI) personality traits. This framework provides a set of interpretable metrics related to strategic behavior and conflict outcomes. We additionally contribute a novel
dataset creation methodology for LLM dispute resolution dialogues with matched scenarios and personality traits with respect to human conversations. Finally, we demonstrate the use of our evaluation framework with three contemporary closed-source LLMs and show significant divergences in how personality manifests in conflict across different LLMs compared to human data, challenging the assumption that personality-prompted
agents can serve as reliable behavioral proxies in socially impactful applications. Our work highlights the need for psychological grounding and validation in AI simulations before real-world use.
Read the full abstract ▾
All authors Deuksin Kwon, Kaleen Shrestha, Bin Han, Spencer Lin, James Hale, Jonathan Gratch, Maja Matarić, Gale M. Lucas