Computation and Language · Cross-listed from cs.CR
Token-Flow Firewall: Semantic Runtime Auditing for Persistent AI Agents
Puji Wang, Yingchen Zhang, Ruqing Zhang et al. (+2 more)
Persistent AI
agents extend large
language models (LLMs) beyond single-turn interaction into long-lived software systems. Unlike traditional chat assistants, unsafe content in these
agents can propagate through persistent state, reusable skills, and tool-mediated interactions, creating a substantially larger semantic attack surface. We observe that most security-critical interactions in such
agents are transmitted through natural-
language token flows, including
memory updates, tool arguments, retrieved files, and inter-component communications. This observation enables a new security formulation: unsafe behavior can be intercepted as risky semantic flows before reaching privileged runtime sinks. Based on this insight, we propose TokenWall, a runtime defense framework that acts as a semantic firewall over
agent token flows. TokenWall performs boundary-aware semantic
auditing over these flows, constructing structured source-sink audit records, applying lightweight local inspection before execution, and selectively escalating ambiguous high-risk cases to stronger arbitration modules. Unlike prior approaches that rely on sparse
auditing or remote large-model oversight, TokenWall enables full-coverage pre-execution mediation while reducing remote arbitration and latency. Experiments on CIK-Bench show that TokenWall reduces attack success rate to 12.5% while maintaining a 97.4% benign executable pass rate without human confirmation. TokenWall further introduces only 0.69 seconds of additional latency on benign cases, demonstrating that semantic runtime containment can achieve a practical security-utility trade-off for persistent AI
agents.
Read the full abstract ▾
All authors Puji Wang, Yingchen Zhang, Ruqing Zhang, Jiafeng Guo, Xueqi Cheng
Computation and Language · Cross-listed from cs.AI
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Lu Dai, Ziyang Rao, Yili Wang et al. (+3 more)
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream
reasoning tasks. We formalize this failure as the Knowing-Using Gap, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally with self-patching, an adaptation of activation patching that scans all layer pairs at every fine-tuning check-point. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-
circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. Experiments are done cross-domain for the robustness of this finding. Building on this diagnosis, we propose layer-wise representation self-
distillation (LRSD) that aligns knowledge representation from late storage layer to middle layer. LRSD keeps improving generalization after fine-tuning saturates and nearly doubles multi-hop chaining accuracy on Qwen, while leaving memorization intact.
Read the full abstract ▾
All authors Lu Dai, Ziyang Rao, Yili Wang, Hanqing Wang, Hao Liu, Hui Xiong
Computation and Language
Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition
Jing Jie Tan, Ban-Hoe Kwan, Danny Wee-Kiat Ng et al. (+6 more)
Personality recognition has traditionally been constrained by theory-dependent formulations, where models are trained to fit predefined psychological taxonomies rather than uncovering shared underlying behavioral structure. This limits generalization, as personality itself is better understood as theory-invariant, while existing annotations reflect only partial and sometimes inconsistent views of the same latent traits. In this work, we introduce JAM ((J)udge for (A)daptive (M)etric-
Alignment), a theory-agnostic framework that shifts learning from adapting to predefined personality theories toward discovering unified latent pseudo-facets that capture shared psychological structure. Rather than constraining the model to any personality taxonomy during training or inference, the framework learns generalizable psychological representations and can infer an individual's latent psychological profile directly from the textual samples, without requiring theory-specific labels. JAM achieves this through an
Attention-Pooled Graph Prototypical Network that learns structured representations via clustering in
embedding space, together with a Cross-Theory Harmonization (CTH) approach that integrates (i) Human-Guided Linkage and (ii) Machine-Induced Consensus to unify heterogeneous
datasets without relying on predefined labels. To further improve robustness and data quality, we incorporate an LLM-as-a-Judge mechanism operating in two configurations, (i) LLM-before-the-loop and (ii) LLM-in-the-loop which identifies ambiguous samples to guide adaptive metric learning. Experiments show that JAM improves cross-framework generalization and performance, establishing a strong step toward theory-agnostic personality inference and supporting low-resource personality theories. The related code repository, model weights, and artifacts are available at https://research.jingjietan.com/JAM
Read the full abstract ▾
All authors Jing Jie Tan, Ban-Hoe Kwan, Danny Wee-Kiat Ng, Yan-Chai Hum, Shih-Yu Lo, Po-An Chen, Noriyuki Kawarazaki, Kosuke Takano, Anissa Mokraoui
Computation and Language
Echoes Across Vietnam's Highlands, Delta, and Coast: A Multilingual Corpus for Cham, Khmer, and Tay-Nung
Anh Trac Duc Dinh, Khang Nhat Hoang Vo, Vinh Cong Doan et al. (+2 more)
Vietnam's ethnic minority
languages are almost absent from the field of Natural
Language Processing (NLP), and the challenge goes beyond data scarcity: Cham, Khmer, and Tay-Nung differ sharply in script, Vietnamese contact, and standardization, conditions under which standard
multilingual adaptation can learn the wrong signals. We introduce CKTN, the first corpus and
benchmark for these
languages (44,367 documents, 24M subword
tokens), spanning continued pretraining, category classification, and summary-document
retrieval. We show that existing
multilingual encoders severely fragment these
languages, and that common adaptation metrics can mislead: models may lower
language-modeling loss or excel at lexical-overlap
retrieval while still failing at semantic generalization across documents. We address this with a script-aware adaptation recipe - vocabulary augmentation combined with calibrated replaced-
token pretraining - that prevents the discriminator from exploiting trivial script mismatches. The result is an encoder with substantially less fragmentation and the strongest classification performance among evaluated models, exposing the limits of lexical-overlap
retrieval as an evaluation signal.
Read the full abstract ▾
All authors Anh Trac Duc Dinh, Khang Nhat Hoang Vo, Vinh Cong Doan, Tai Tien Ta, Khoa Duc Anh Lam
Computation and Language
Grounded Event Extraction from SEC 8-K Filings with a Fine-Grained Taxonomy
Rian Dolphin, Joe Dursun, Jarrett Blankenship et al. (+2 more)
Form 8-K filings are the primary channel through which U.S. public companies disclose material events, but the SEC item codes attached to them are coarse: a single item spans routine administrative changes and chief executive departures, and many of the most
market-moving disclosures fall into a catch-all item. Large
language models make fine-grained labelling feasible at corpus scale, but only if the labels can be traced to the source text and shown to be reliable. We present a two-stage system that tags 8-K disclosures against a three-tier taxonomy of 119 event types. The first stage constrains output to valid taxonomy entries and anchors every tag to a verbatim quote via fuzzy n-gram validation; the second re-grades each cited quote against the category definition to produce a quality score. Applying the system to 292,984 filings from 2022 to 2026 yields 601,088 grounded event tags, which we release. Over 5,125 stratified tags, an LLM judge finds precision rises monotonically with the quality score, from 12% to 96%, while unsupported tags fall from 8% to near zero. Ablation shows the score is calibrated only when assigned in a dedicated second pass. An event study on unsigned abnormal returns confirms, without any
language model, that the taxonomy separates economically distinct events sharing an item code.
Read the full abstract ▾
All authors Rian Dolphin, Joe Dursun, Jarrett Blankenship, Katie Adams, Quinton Pike
Computation and Language
TypeProbe: Recovering Type Representations from Hidden States of Pre-trained Code Models
Giuliano Gorgone and Fausto Carcassi
State-of-the-art code models achieve impressive performance, yet the extent to which they internally encode type information remains poorly understood. We probe the residual streams of pretrained code models for internal type representations using a parallel
dataset of Java and Python code examples. Our results show that cross-lingual type representations emerge even from untyped code. Moreover, we test whether hidden states linearly encode the result type implied by typed function application by training probes on one
language to infer argument and result types in the other. Finally, we find that this structure is partly robust to lexical perturbations and cross-
language syntactic variations. To the best of our knowledge, prior work on interpretability of code models has not directly targeted formal type semantics or cross-lingual type representations. We release our code and
datasets.
Read the full abstract ▾
Computation and Language
X-Alpha: An AI Quant Researcher with Memory-Driven Self-Evolution
Fengyuan Liu, Yuchen Fu, Yuqi Wang et al. (+3 more)
Human quantitative researchers mine alpha through an iterative process: they derive hypotheses from financial knowledge, implement and test them as factors, and use the resulting evidence to guide subsequent research. Yet automated alpha mining faces three key bottlenecks: noisy financial knowledge is not directly actionable, exploration can remain confined to similar mechanisms, and feedback may remain tied to individual mining runs. To address these bottlenecks, we introduce X-Alpha, an AI Quant Researcher that couples persistent research
memory with continuous hypothesis-to-code discovery. Report-to-
Memory Abstraction (RMA) uses a multi-layer filtering-and-abstraction process to convert financial reports into actionable, data-compatible research paths.
Memory-driven routing uses accumulated experience to redirect exploration and define a targeted search space for each cycle. Within each cycle, hypotheses are implemented as executable factors and refined through empirical validation and code-based
evolution. Empirical outcomes are consolidated into persistent research
memory to guide subsequent exploration, enabling
memory-driven self-
evolution across cycles. We evaluate X-Alpha on five equity indices spanning mainland China, the United States, and Hong Kong, with comprehensive experiments centered on CSI300. Experiments show strong alpha discovery performance against baselines, with improvements in factor quality and signal diversity and consistent generation-level gains across multiple
markets and prediction horizons.
Read the full abstract ▾
All authors Fengyuan Liu, Yuchen Fu, Yuqi Wang, Fengyuan Liu, Mukai Li, Qi Liu
Computation and Language · Cross-listed from cs.AI
Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment
Vinay Kumar Chaganti
High-volume structured extraction pays a large model's latency on every item, so distilling the task into a small on-device model is attractive: comparable output at a fraction of the time and cost. We measure what that
distillation actually delivers, per sub-task. Each news article is mapped to one JSON object with a short summary and five categorical labels. We distill an 8B
reasoning teacher (deepseek-r1:8b) into a 0.6B student (Qwen3-0.6B; QLoRA, three seeds), and add two teacher controls: a same-size non-
reasoning teacher and a larger managed pipeline. A blinded, reference-free, three-judge panel scores every arm against the full article, alongside two non-
distillation baselines, few-shot prompting and constrained decoding. The student runs at about 0.8 s per article against the teacher's 39 s, and recovers 58% of the base-to-teacher gap on summary quality, beating its primary baseline (constrained decoding) by +16.8 points and few-shot prompting by a secondary +4.9. A same-size non-
reasoning teacher trains a student no better than the untuned base, so the summary gain follows from the teacher's
reasoning nature rather than its scale. Capabilities then split by teacher: the
reasoning teacher transfers writing quality and the managed pipeline transfers label diversity, while a same-size instruction teacher's students stay more grounded on the 22 short, thin-source articles in the 93-item test set (74 versus 55 faithful), where the
reasoning-lineage student fabricates. That grounding difference is a consistent ordering rather than a significant aggregate effect, and the subgroup is small, so we report it as a direction. Because no single engine wins every field, the deliverable is a per-field routing map for on-device enrichment.
Read the full abstract ▾
Computation and Language
Best-of-$N$ TTS Evaluation is Confounded by ASR Family Alignment
Taehyung Yu and Seongjae Kang
Best-of-$N$ (BoN) inference improves content consistency in zero-shot text-to-
speech by selecting among multiple candidates with an automatic
speech recognition (ASR) verifier. We identify an evaluation confound: the apparent quality of a verifier depends strongly on the ASR family used for evaluation. On LibriSpeech-PC with F5-TTS, verifier rankings vary substantially across Whisper, wav2vec 2.0, and HuBERT evaluators, while same-family verifier and evaluator pairs recover considerably more oracle headroom than cross-family pairs despite highly similar representations. This pattern suggests identity- or lineage-level coupling rather than general representational similarity. To mitigate this bias, we propose two cross-family rank ensembles: rank averaging and conjunctive max-rank. Both improve mean word error rate across independent evaluators without degrading automatic similarity or quality metrics, and the best ensemble achieves a $12\%$ relative WER reduction over F5-TTS at $N=10$. These findings motivate cross-evaluator triangulation as a more reliable default for reporting BoN TTS performance.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
A Multi-Timescale Recursive Self-Improvement Engine for Open-Ended Persona Growth
Mengchen Li
Role-playing AI personas today do not grow: they hold a fixed character, so the relationship a user builds with them has nothing to accumulate on. We introduce AutoPersonas, a multi-timescale engine that applies recursive self-improvement (RSI) to persona growth: rather than improving its intelligence, the persona recursively revises the State, evidence, and life-environment that shape its own future. We identify self-locking as the runtime failure mode of this recursion: locally plausible events keep appearing while the generated life collapses toward familiar environments, weak relationships, suspended decisions, and stale life stages. We trace it to model-level convergence toward high-probability behavioral channels and system-level context
gravity from State,
memory, history, and environment summaries. A three-year compressed simulation exposed environment watermark shells, occurrence-hardening gaps, slow-change accumulation failures, recursive indecision, and weak relationship persistence. An eight-model 40-day stress test generated 1,600 events and found mean rolling 5-day action-category repetition of 95.2%-97.6%, with all models crossing 90% by day 11; semantic re-keeping found 79.0%-88.0% macro-theme repetition. The primary contribution is the definition and measurement of self-locking. We also report a mitigation as a black-box result, with internals withheld for commercial reasons: in a same-runtime 40-day A/B, our production divergence configuration reduced macro-theme repetition from 61.8% to 39.4% and nearly doubled cumulative theme count, and a juvenile-goblin fictional-world run reproduced this regime without hard real-world intrusions.
Read the full abstract ▾
Computation and Language
Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech
Hao Wu, RongQi Han, Zhen Wang et al. (+2 more)
This paper describes our self-designed system for Task 1 of the MLC-SLM 2026 Challenge for
multilingual two-speaker conversational
speech. The system combines a modular speaker diarization front end with a challenge-adapted Qwen3-ASR-1.7B recognizer. The diarization front end performs voice activity detection, subsegment generation, CAMPPlus speaker
embedding extraction, two-speaker spectral clustering, and RTTM-based audio
segmentation. The resulting speaker-attributed segments are grouped by
language or region and decoded by the adapted ASR model. For ASR adaptation, we first perform supervised full fine-tuning on the official training data, then apply LoRA fine-tuning with synthetic
speech generated by a three-pipeline TTS-based synthetic
speech augmentation framework, and finally refine the model using GRPO reinforcement learning with rewards based on WER/CER and penalties for
hallucination, repetition, and length deviation. On the official development set, the full system achieves an average tcpMER of 23.70, reducing the error rate by 6.83 absolute points relative to the released Qwen-ASR-1.7B performance. On the final evaluation set, the system achieves an average tcpMER of 17.97. Ablation results show that supervised fine-tuning provides the largest gain, while synthetic-
speech LoRA adaptation and reinforcement learning further improve robustness.
Read the full abstract ▾
All authors Hao Wu, RongQi Han, Zhen Wang, Wei Liang, Wei Xu
Computation and Language · Cross-listed from cs.AI
A First-Principles Theory of Slow Thinking and Active Perception
Hongkang Yang, Zhi-Qin John Xu, Feiyu Xiong et al. (+1 more)
As part of a series on first-principles modeling of cognitive functions, this paper attempts to provide a mathematical formulation of thinking and perception. It formally derives slow thinking or more generally, active perception, and encompasses the design, training and inference of slow thinking large
language models. Our starting point is the lifting and projection of probability distributions on the observable and latent spaces, with the objective of representing complex data distributions by simple function families such as neural networks. A theory called "active lifting" is proposed, based on the sampling of latent sequences and an intrinsic drive to reduce uncertainty with maximum rate. It derives a large design space, containing the slow thinking models in a subspace that we call the static theory. These models are positioned on the representation hierarchy and sampler hierarchy induced by the static theory, and can be upgraded by climbing the two hierarchies. Active lifting further derives an inference process with an internal time axis, and a training objective that resembles minimum-length coding as well as the invention of
languages. Thus, it characterizes the agency of perception, including the emergence of the slow thinking formats. Technical by-products of this theory include a three-stage pathway for improving slow thinking models, a unified approach to constructing encoders and generative models for all data modalities, a priori formation of human-like visual representations, and a possible solution to policy collapse.
Read the full abstract ▾
All authors Hongkang Yang, Zhi-Qin John Xu, Feiyu Xiong, Weinan E
Computation and Language
Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models
Aiwei Liu, Cheng Shi, Chuhan Wu et al. (+45 more)
Scaling Large
Language Models (LLMs) has been driven mainly by enlarging the
Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each
token while leaving the
Transformer backbone fixed. Depth-recurrent (looped)
Transformers pursue this goal but are hard to scale, because looped computation does not fit naturally with the pipeline parallelism used to train the largest models. We add computation along the sequence-length dimension, where the extra computation is simply a longer input and stays compatible with standard large-model training. We propose Hidden Decoding, a sequence-length scaling method applied during continued pretraining (CPT). It expands each
token into n streams with independent
embedding tables and keeps the intermediate streams' key-value cache as context, so each
token performs more internal computation without adding or widening
Transformer layers. To keep this affordable at scale, we introduce Stream-Factorized
Attention, in which most layers attend only within each stream and only a few layers mix across streams, reducing the
attention cost from quadratic to roughly linear in n. Experiments support two scaling results. At frontier scale, we train WeLM-HD4-80B and WeLM-HD4-617B at n=4 and improve their matched non-HD baselines, making Hidden Decoding the first demonstrated sequence-length scaling method at the 100B+ MoE scale. Across expansion factors, the gains grow as n increases, showing that sequence-length expansion is a practical fixed-backbone scaling path for frontier-scale LLMs.
Read the full abstract ▾
All authors Aiwei Liu, Cheng Shi, Chuhan Wu, Ci Lei, Di Lu, Donald He, Fan Zhang, Fanhao Kong, Feifei Zhang, Guan Wang, Haicheng Wang, Haoyu Liu, Houjin Yu, Jiachen Ding, Jiayi Feng, Jie Zhou, Jijun Chi, Jindi Shi, Jing Lei, Junjie Zhang, Laiyi Li, Le Tian, Linhao Zhang, Miao Fan, Sijun Zhang, Wei Jia, Weiwei Shi, Wenhan Li, Wentao Zhao, Wenteng Liang, Xiao Zhou, Xiaojin Zhou, Xihuai Wang, Xinyu Gao, Xuanliang Wang, Xuyang Ao, Yang Yu, Yangxiu You, Yinuo Zhao, Yufei Kuang, Yufei Wang, Yuan Liu, Yuan Liu, Yuwen Chen, Zhencong Tian, Zhongyin Zhao, Zilin Yu, Zitao Wang
Computation and Language
SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation
Wangyu Wu, Xiaojian Lin, Rong Fu et al. (+4 more)
Text-to-SQL is a fundamental task in natural
language processing that enables users to interact with structured databases using natural
language. While large
language models (LLMs) have demonstrated remarkable performance on this task, their substantial computational requirements hinder deployment in resource-constrained settings. In this paper, we introduce SQuaD-SQL (Small-Qualified and Distilled for SQL), a novel approach that empowers small
language models (SLMs) to approach the performance of LLMs on the Text-to-SQL task while significantly improving efficiency through knowledge
distillation and synthetic data generation. Our method comprises three key components: (1) LLM-based synthetic data generation, where structured knowledge is extracted from LLMs via carefully designed prompting strategies; (2) parameter-efficient fine-tuning, enabling full model training on a single consumer-grade GPU; and (3) domain-adaptive fine-tuning, where domain-specific synthetic data further enhances performance in targeted domains. Experiments on the WikiSQL
dataset demonstrate that SQuaD-SQL achieves an execution accuracy of 86.9% on the test set, approaching the performance of LLMs while offering faster inference and lower
memory usage. These results suggest that, with proper training strategies, SLMs can serve as practical and efficient alternatives for Text-to-SQL applications in resource-limited environments.
Read the full abstract ▾
All authors Wangyu Wu, Xiaojian Lin, Rong Fu, Zaiyang Yu, Xuhang Chen, Wenjun Yu, Zhenhong Chen
Computation and Language
LEXIC: Lightweight On-Device Decoding of Reading Comprehension from Eye Movements
Sumin Lee, Kyeonghun Kim, Subeen Lee et al. (+5 more)
Predicting comprehension from eye movements could support adaptive reading interfaces. We present LEXIC, a compact recurrent model that predicts response correctness from fixation sequences, word frequency, and character length. It has 41.6K parameters and requires no
language-model inference. Mean area under the receiver operating characteristic curve (AUROC) reaches 0.529 for Unseen Text and 0.554 for Unseen Reader on OneStop. Matched comparisons with AhnCNN show gains from both encoder redesign and lexical augmentation in these settings. Separate training and evaluation on SB-SAT also yield higher mean AUROC than AhnCNN. LEXIC occupies only 166 KB of model weights and requires 1.4 ms per trial on a single CPU core, supporting lightweight on-device inference.
Read the full abstract ▾
All authors Sumin Lee, Kyeonghun Kim, Subeen Lee, Jiwon Yang, Hyunsu Go, Eunseob Choi, Ken Ying-Kai Liao, Nam-Joon Kim
Computation and Language
ICDAR 2026 HIPE-OCRepair Competition on LLM-Assisted OCR Post-Correction for Historical Documents
Maud Ehrmann, Emanuela Boros, Juri Opitz et al. (+3 more)
We present the results of HIPE-OCRepair-2026, an ICDAR competition on LLM-assisted OCR post-correction of historical documents. OCR post-correction remains a long-standing challenge in digital heritage: large-scale collections of digitized documents are affected by legacy OCR errors, while re-digitization at scale remains impractical. Large
language models (LLMs) offers a major opportunity to revisit this challenge, yet their effectiveness across
languages, document types, and noise conditions - and their tendency to hallucinate - remains insufficiently understood. HIPE-OCRepair-2026 pursues two objectives: (i) to evaluate the capabilities of modern OCR post-correction systems, and (ii) to provide a reproducible evaluation framework anchored in the HIPE-OCRepair-2026
dataset, a harmonized
multilingual resource consolidating existing and newly curated historical
datasets. Participants were tasked with correcting noisy OCR transcripts from historical newspapers and printed works in English, French, and German (17th-20th century), working at the level of coherent transcription units (paragraphs or articles) without access to source images. The evaluation adopts a
retrieval-oriented rather than diplomatic scoring approach, reflecting the practical use case of search and access over digitized collections. Four teams submitted systems ranging from zero-shot prompting to continued pre-training and fine-tuning, offering insights into the merits of different adaptation strategies. Results show that modern LLM-assisted systems can significantly improve OCR quality, but performance varies across
datasets,
languages, and noise levels. Over-correction on low-noise inputs emerges as a recurring challenge, highlighting the importance of evaluation beyond character error reduction. The
dataset, scorer, and evaluation pipeline are publicly released to support future research.
Read the full abstract ▾
All authors Maud Ehrmann, Emanuela Boros, Juri Opitz, Andrianos Michail, Florian Wagner, Simon Clematide
Computation and Language
COALA: Robust Contextualized Speech-augmented Language Modeling for ASR via Contrastive Regularizer and Biasing Score Estimation
Jhih-Rong Guo, Bi-Cheng Yan, Tien-Hong Lo et al. (+1 more)
Contextual biasing seeks to integrate external knowledge into automatic
speech recognition (ASR) systems to accurately recognize domain-specific entities. In this paper, we propose COALA (Contextualized ASR Leveraging Biasing Scoring), a robust framework designed to enhance
speech-augmented
language models (SLMs) in complex multi-entity scenarios. Considering the inherent context-window limitations of SLMs, identifying relevant target entities from a large-scale biasing list is crucial for effective recognition. To this end, COALA maps SLM latent representations into a specialized discriminative space to quantify the matching intensity between audio segments and candidate entities. Furthermore, we address the training collapse in prior study when handling multi-target utterances-where multiple rare words co-occur. Experimental results on the LibriSpeech
benchmark demonstrate that COALA consistently achieves superior contextual biasing performance across various biasing list scales.
Read the full abstract ▾
All authors Jhih-Rong Guo, Bi-Cheng Yan, Tien-Hong Lo, Berlin Chen
Computation and Language · Cross-listed from cs.AI
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
Andrej Leban, Theo Saarinen and Yuekai Sun
Large
language models (LLMs) increasingly act as data-science
agents, combining abstract
reasoning with advanced tool use. Yet the relevant
benchmark landscape largely divides into symbolic causal
reasoning benchmarks without realistic data analysis or data analysis
benchmarks without causal data-generating structure or a range of causal data science tasks. Existing causal evaluation
datasets are often restricted to curated examples from existing sources, with diversity coming from limited templatized variations rather than from systematic generation of novel synthetic causal structures. We introduce CausalDS, a
benchmark for evaluating causal
reasoning in agentic data-science workflows. Each
benchmark instance is a scene consisting of a sampled structural causal model (SCM) with generated observational data and an accompanying synthetic natural-
language story grounded in a real domain. We ground the composition of the
benchmark components in empirical distributions obtained from real-world
datasets, thus retaining empirical structure while reducing the memorization risk through completely synthetic generation. From each scene, we then derive tasks spanning all three of Pearl's rungs. Most tasks include a data science coding component, where the model typically needs to use several tools to arrive at the final answer due to imperfect observations. Abstention is tested directly: for targets the graph does not identify, the graded answer is to abstain. The
benchmark thus jointly evaluates symbolic causal
reasoning, data science, uncertainty quantification, abstention, and tool use/coding. Because the generator is fully synthetic, the exam is not a fixed
dataset: composition, difficulty, and size are parameters, and fresh exams can be drawn at will.
Read the full abstract ▾
Computation and Language
MASTE: A Multi-Agent Pipeline for Zero-Shot Aspect Sentiment Triplet Extraction
Ao Hong, Lehang Wang, Zhirun Yue et al. (+3 more)
Aspect Sentiment Triplet Extraction (ASTE) requires jointly identifying (aspect, opinion, sentiment) triples from a given review sentence. While large
language models (LLMs) achieve strong zero-shot performance on many NLP
benchmarks, their effectiveness on ASTE remains limited, as single-pass generation forces the model to determine span boundaries, opinion grouping, and sentiment polarity in a single decoding step. Common remedies, such as few-shot in-context learning and chain-of-thought prompting, offer only marginal improvements and rely heavily on either in-domain demonstrations sampled from labeled training data or carefully engineered
reasoning prompts, neither of which is broadly available in zero-shot deployment. Inspired by the classical
agent paradigm, we propose MASTE, a multi-
agent pipeline for zero-shot Aspect Sentiment Triplet Extraction. MASTE decomposes ASTE into four sequential stages, where specialized
agents handle different compositional subtasks with explicit conditioning on prior outputs. This design enables entirely training-free zero-shot ASTE and generalizes across different backbones and
datasets. Extensive experiments on four ASTE
benchmarks show that MASTE substantially outperforms zero-shot and chain-of-thought LLM baselines under the same backbone, narrowing the gap to fully supervised methods without using any labeled triplets. Code is available at https://github.com/Hankerlove/MASTE.
Read the full abstract ▾
All authors Ao Hong, Lehang Wang, Zhirun Yue, Mingxin Wang, Zihan Wang, Houde Liu
Computation and Language
COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation
Yashal Shakti Kanungo, Gyanendra Das, Pooja A et al. (+1 more)
Online ads are essential to all businesses and ad headlines are one of their core creative component. Existing methods can generate headlines automatically and also optimize their click-through-rate (CTR) and quality. However, evolving ad formats and changing creative requirements make it difficult to generate optimized & customized headlines. We propose a novel method that uses prefix control
tokens along with BART fine-tuning. It yields the highest CTR and also allows users to control the length of generated headlines for use across different ad formats. The method is also flexible and can easily be adapted to other architectures, creative requirements and optimization criteria. Our experiments demonstrate a 25.82% increment in Rouge-L and a 5.82% increment in estimated CTR over previously published strong ad headline generation baseline.
Read the full abstract ▾
All authors Yashal Shakti Kanungo, Gyanendra Das, Pooja A, Sumit Negi
Computation and Language
Holographic Neural PCFG for Unsupervised Parsing
Ryosuke Yamaki, Daichi Mochihashi, Nobutaka Shimada et al. (+1 more)
Unsupervised constituency parsing aims to accurately induce latent tree structures from raw text alone. Recent neural parameterizations of PCFGs achieve strong performance in both supervised and unsupervised parsing, yet rely on high-capacity black-box networks for rule scoring – as exemplified by the Neural PCFG family – leaving rule probabilities without an interpretable mathematical form. In this paper, we propose
Holographic Neural PCFG (Hol-PCFG), which recasts PCFG rule scoring as algebraic relation modeling among grammar-symbol
embeddings. Hol-PCFG adapts
Holographic Embeddings (Nickel et al., 2016), which scores knowledge-graph triples via circular correlation, to the left-child, right-child, and lexical-emission relations over
torus-constrained
embeddings, giving every rule probability a closed form that carries the intrinsic structure of grammar rules by construction. Hol-PCFG achieves state-of-the-art parsing performance in six
languages while cutting rule-scoring parameters by 99.94% relative to the baseline model and training more stably. Additionally, we demonstrate that Hol-PCFG can parse Japanese directly from characters without any morphological
segmentation, retaining nearly the same morpheme-level performance.
Read the full abstract ▾
All authors Ryosuke Yamaki, Daichi Mochihashi, Nobutaka Shimada, Tadahiro Taniguchi
Computation and Language · Cross-listed from cs.LG
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Jiantong Jiang, Peiyu Yang, Rui Zhang et al. (+1 more)
Despite the rapid advancements of large
language models (LLMs), LLM serving systems remain
memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-throughput LLM inference serving. In this survey, we focus on system-aware KV infrastructure for serving LLMs (abbreviated as sKis). We revisit recent work from a system behavior perspective, organizing existing efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural). Furthermore, we analyze cross-behavior co-design affinity and behavior-objective links, highlighting future opportunities. Our work systematizes a rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.
Read the full abstract ▾
All authors Jiantong Jiang, Peiyu Yang, Rui Zhang, Feng Liu
Computation and Language
What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness
Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana et al. (+3 more)
Large
language models fine-tuned for
forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT)
reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-Air. We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the
prompt often changes the model's forecast while leaving the
reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the
reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation's influence. Finally, forced answering reveals that forecasts are largely fixed before
reasoning begins: a single pre-
reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated
tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating,
auditing, and triaging
language model forecasters and
reasoning models more broadly.
Read the full abstract ▾
All authors Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana, Christopher J. Earls, Srikar Varadaraj, Eric Ho
Computation and Language
Dhruv Agarwal, Anya Shukla, Tanya Goyal et al. (+1 more)
Large
language models (LLMs) are used worldwide, yet disproportionately reflect Western values, limiting their ability to represent diverse value systems. We introduce PLURAL, a large-scale, value-focused preference
dataset grounded in the Integrated Values Survey (IVS), a nationally representative survey spanning 92 countries. Using a two-stage generation pipeline, we transform survey responses into synthetic preference triplets that preserve normative value signals while producing realistic scenarios. We release an initial version of PLURAL containing ~500,000 preference triplets representing people in 20 diverse countries. We evaluate PLURAL in three ways: (i)
dataset-level validation showing that it preserves both cross-country value differences and within-country diversity from the original survey; (ii) automated evaluation showing that training on PLURAL improves
alignment with target countries' cultural profiles, reducing mean absolute error by up to 27.7% relative to strong baselines; and (iii) blind human evaluation with 176 evaluators in India, Brazil, and Japan, who judge PLURAL-aligned responses as more representative of their national values. Together, these results show that PLURAL contains learnable signal for value steering, offering a scalable resource for pluralistic
alignment.
Dataset: https://huggingface.co/
datasets/agdhruv/plural-
alignment
Read the full abstract ▾
All authors Dhruv Agarwal, Anya Shukla, Tanya Goyal, Aditya Vashistha
Computation and Language · Cross-listed from cs.AI
From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
Joongho Ahn and Moonsoo Kim
Enterprise large
language model (LLM) applications often begin as prototypes whose behavior is carried by
prompts and
retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-
agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-
language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer,
prompt instructions alone let recommendation-
language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.
Read the full abstract ▾
Computation and Language
Structured Pruning of Large Language Models via Power Transformation and Sign-Preserving Score Aggregation with Adaptive Feature Retention
Ryota Kobayashi, Tsubasa Hirakawa, Takayoshi Yamashita et al. (+4 more)
This paper proposes an improved structured
pruning method for large
language models (LLMs) that addresses key challenges in adapting Adaptive Feature Retention (AFR), an unstructured
pruning technique, to structured
pruning. When applying AFR to structured
pruning, three major problems arise: distribution mismatch between heterogeneous
pruning scores, loss of sign information indicating optimization direction consistency, and influence of outliers. To address these issues, we propose a unified approach combining power transformation for nonlinear distribution
alignment, sign-preserving score aggregation, and percentile-based outlier removal. Experiments on Llama-3-8B, Vicuna-v1.5-13B, and LLaVA-v1.5-13B demonstrate that our method maintains accuracy comparable to unstructured
pruning while achieving practical inference speedup through structured
pruning.
Read the full abstract ▾
All authors Ryota Kobayashi, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi, Yasunori Ishii, Tomoyuki Okuno, Kazuki Kozuka
Computation and Language
Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework
Riccardo Revalor, Jalees Rehman and Debjit Pal
Large-
Language Models (LLMs) can be prone to flawed and unfaithful
reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps. This raises three fundamental questions: How can we reliably quantify uncertainty in LLM
reasoning? Can semantic, structural, and causal awareness select more faithful
reasoning compared to naïve majority voting? and How robust is
reasoning topology under
adversarial conditions? To address these questions, we introduce GRAPHEVAL, a graph-based
reasoning framework that re-frames uncertainty quantification (UQ) as a holistic
reasoning fidelity problem. We propose a novel UQ metric, Graph
Reasoning Coherence Score (GRCS), that quantifies semantic-structural consensus of the
reasoning space and captures pathological mode collapse and confident
hallucinations. We find that GRCS is the only metric that is consistently negatively correlated with
reasoning faithfulness across both more capable and smaller models. Additionally, we introduce Graph Self-Consistency (GSC), a medoid-based decoding strategy that trades nominal accuracy for
reasoning fidelity, exposing the degree to which SC is inflated by unfaithful lucky guesses in smaller models, while preserving or improving accuracy in more capable ones. Finally, through
adversarial medoid ablation, we demonstrate that the GSC-selected path acts as a "load-bearing path" and forcing models away from it degrades
reasoning faithfulness and, in targeted cases, causes drops in accuracy.
Read the full abstract ▾
Computation and Language
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
Kalle Kujanpää, Ning Liu, Shahnawaz Alam et al. (+4 more)
Production LLM
agents often waste latency and reliability by regenerating code for the same procedural steps on every request. We replace this inference-time coding loop with an agentic tool-making pipeline that compiles repeated SOP steps into validated, versioned tools before deployment. The tool-maker grounds synthesis in the live environment as it collects execution traces, observes backend schemas and values, generates candidate tools, and repairs them against labeled cases. At runtime, the production
agent calls these tools directly and falls back to code generation only when needed. We deploy the approach in a Fulfillment Center alarm-triage system, where an
agent diagnoses alarms against a 44-node SOP over heterogeneous metric backends. In production, tool calls reduce p50 latency by 42%. On 1,500 historical alarms, they reduce end-to-end error rate by up to 53% by suppressing run-to-run variance in repeated steps. Because tools return compact structured verdicts, they also enable a simpler direct-call architecture, reducing p50 latency by a further 62% in a controlled ablation. Versioned tools also improve auditability and expose specification gaps and upstream data drift. Our results show that
agents that build and maintain their own tool libraries, a key element of self-evolving
agents, can make industrial LLM systems faster, more reliable, and easier to operate.
Read the full abstract ▾
All authors Kalle Kujanpää, Ning Liu, Shahnawaz Alam, Yeshwanth Reddy Sura, Tianyu Yang, Kristina Klinkner, Shervin Malmasi
Computation and Language
From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs
Yi Zhang and Julia Rayz
We introduce a Bloom-aligned framework for measuring educational control in Large
Language Models (LLMs): the ability to preserve a task's instructional intent while shifting its cognitive demand toward specified learning objectives. We apply this framework to programming tasks in computer science education to study the gap between solving tasks and adapting them for learners. Using revised Bloom's Taxonomy as an operational scale of cognitive demand, we evaluate two intervention settings: general difficulty control, where models are asked to make tasks harder or easier, and Bloom's control, where models are asked to target higher or lower Bloom's levels. We evaluate a matched Qwen3-Next model pair, comparing Qwen3-Next-80B-A3B-Instruct with Qwen3-Coder-Next across 2,520 tasks from three
benchmarks. The framework reveals a robust directional asymmetry: both models reliably increase cognitive demand, but struggle to lower it. We further characterize these outcomes with semantic-delta clustering and layer-wise Fisher's Discriminant Ratio probing. Within this controlled comparison, the general model shows clearer middle-layer separability for both general difficulty and Bloom-control contrasts, whereas the coder model shows weaker separability for general difficulty and a deeper peak for Bloom-control contrasts. These results show that strong execution performance does not automatically entail Bloom-aligned educational control.
Read the full abstract ▾