Computation and Language
BanglaDial-Abuse: A Corpus-Grounded Dataset for Regional Dialect Identification in Abusive Bangla Text
Hasin Almas Sifat
Regional linguistic variation remains an important challenge for Bangla natural
language processing, particularly in informal and non-standard text. This paper introduces BanglaDial-Abuse, a balanced Bengali-script
dataset developed for regional dialect identification in abusive and hostile Bangla text. The
dataset contains 1,000 sentences distributed equally across four linguistic varieties: Standard Bangla, Chattagram, Sylhet, and Barishal, with 250 samples per class. The resource was constructed using a corpus-grounded synthetic procedure incorporating regional variation in pronouns, possessive forms, verb morphology, negation, interrogative structures, postpositions, vocabulary, and Bengali-script spelling conventions while preserving the underlying hostile or abusive meaning. Descriptive analysis shows broadly comparable sentence-length distributions but partially distinct lexical spaces across the four classes. Pairwise Jaccard vocabulary similarity ranges from 0.37 to 0.56. The primary task is four-class regional dialect identification rather than binary abusive-text detection. The
dataset is publicly available through Zenodo under a Creative Commons Attribution 4.0 license. The current version is intended as a research and prototyping corpus rather than a native-speaker-validated gold-standard linguistic resource. Keywords: Bangla, Bengali, dialect identification, regional dialect, abusive
language, low-resource NLP, Chattagram, Sylhet, Barishal,
dataset
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Do Multilingual Encoders Produce Language-Consistent Semantic IDs?
Abhinav Bohra and Anuj Bohra
Semantic IDs (SIDs) compress item
embeddings into discrete code sequences used in generative
retrieval. We ask whether a multilingual encoder is sufficient for different-
language renderings of the same product to receive
language-consistent SIDs. Using Amazon ESCI listings rendered in English, Spanish, and Japanese, we test whether translations remain close to their English source, whether residual quantization is unusually sensitive to translation-induced movement, and whether multilingual or
language-balanced quantizer fitting improves SID agreement. Multilingual E5 places translations measurably apart: under an English-heavy fit, a Japanese translation preserves the first SID code of its English counterpart in only 7.7% of cases, compared with 89.0% for an English rewording. Distance-matched product-directed controls produce nearly the same full-SID mismatch as translation, providing no evidence that the quantizer selectively amplifies
language directions. Balancing the fitting mixture makes codebook use more uniform but further reduces cross-lingual prefix agreement: Spanish first-code consistency falls from 28.3% to 6.6%, while an English-only fit preserves it for 67.6% of Spanish translations. These results show that multilingual exposure and balanced codebook use alone do not guarantee
language-consistent SIDs.
Read the full abstract ▾
Computation and Language
Counting and Min-Cost Encoding for Tokenization in Large Language Models
Shuming Shi, Xiang Zhang, Hao Yu et al. (+7 more)
Mainstream large
language models rely on a tokenizer to encode text into a
token sequence. Different tokenizers may yield
token sequences of substantially different lengths for the same text. With a fixed model architecture, shorter
token sequences correspond to lower inference time. We propose a tokenizer training approach named Counting and Filtering (CNF) and a text encoding
algorithm called Min-Cost Encoding (MCE). MCE defines a cost function over a text segment, and determines the best segmentation by globally minimizing the overall segmentation cost. CNF builds a raw vocabulary by directly counting valid substrings, and then constructs the final vocabulary through a filtering step based on actual
token usage when segmenting the training corpus with MCE. The CNF-MCE conbination offers several advantages over BPE, including higher
token efficiency, greater scalability, and lower dependency. Across six text categories and two vocabulary-size groups, CNF-MCE consistently achieves better compression than the evaluated BPE tokenizers. With a 250K vocabulary, CNF-MCE increases compression rate by 26% and 30% on English web text over the o200k_base and qwen250k tokenizers. Experiments scaling the vocabulary to 1M entries on English web text demonstrate sustained improvements over BPE, with a
token efficiency improvement of over 60% and vocabulary utilization rising from 52.9% to 96.9%. The MCE
algorithm does not depend on a merge list (as in BPE) or
token probability (as in UnigramLM), making it applicable to a wide range of vocabularies, including those built from BPE, UnigramLM, CNF, and others.
Language models trained from scratch at the 1.8B and 8B scales achieve comparable average performance to models using the BPE tokenizers across 11
benchmarks. These results demonstrate that CNF-MCE can improve
token efficiency significantly while maintaining competitive downstream performance.
Read the full abstract ▾
All authors Shuming Shi, Xiang Zhang, Hao Yu, Wenbo Fei, Changjian Wang, Zhan Wang, Guoqing Pang, Guangye Yu, Quan Lu, Ning Jiang
Computation and Language
Madeleine: Learning Involuntary Recall for Conversational Memory from Simulated Lives
Zhiyun Shi
A long-term conversational assistant must recall the right
memory at the right moment, yet the
memory that matters most is often not similar to what the user says now. Current systems recover such associations by letting an LLM reason at write or read time, at a cost of hundreds to over a thousand LLM calls per
memory bank and up to several thousand context
tokens per query. We argue that association is a learnable relevance: the pointwise mutual information of
memories under how human lives unfold. We introduce Madeleine, which learns amortized association: offline, an LLM life simulator writes simulated lives, whose cue-trigger pairs teach a query encoder a residual association on top of frozen similarity; online, it calls no LLM and plugs into any vector
memory by replacing only the query encoder. On LoCoMo-Plus under the official protocol, Madeleine (I) reaches 66.6 when plugged into HyperMem, the highest among all systems evaluated under this protocol; (II) used alone, reaches the score of HyperMem as released (52.4 vs. 52.9) with zero LLM calls and about 1/21 of its answer context; and (III) lifts T-Mem by 26.2 points, significantly outperforms the same untrained backbone inside both systems, and leaves ordinary QA intact on the 4B backbone.
Read the full abstract ▾
Computation and Language
AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines
Sumin Lee, Sukmin Cho, Seungjae Lim et al. (+1 more)
Retrieval-based speculative decoding (SD) drafts
tokens by copying continuations from existing text, which suits coding
agents that repeatedly reproduce code, logs, and earlier attempts. Yet existing methods fall short in
agent pipelines: much of the reusable text is missing from their corpora or stored in a form that differs from what the
agent emits, and their draft lengths ignore that accept length varies across
agents and drifts over turns. We present AgSpec, a framework that supplies the corpus and draft-length policies that existing
retrieval engines lack in coding-
agent pipelines. AgSpec retrieves from session, workspace, and global corpora, retaining the ongoing session trajectory and indexing opened files in the
agent's emission format. It bounds each
agent's draft length with an offline-profiled cap and adapts the length online from verification feedback. On two repository-level multi-
agent coding
benchmarks, AgSpec outperforms five
retrieval-based drafters and EAGLE-3 in most evaluated settings, raising generation throughput over autoregressive decoding up to 4.37$\times$ at batch size 1 and 4.76$\times$ at batch size 16. AgSpec also remains effective on
benchmarks without a repository or a multi-
agent pipeline, showing that its gains generalize to coding
agents broadly.
Read the full abstract ▾
All authors Sumin Lee, Sukmin Cho, Seungjae Lim, Youngjin Kwon
Computation and Language
Precision over Scale: A Polish-Silesian Benchmark and a Translation System Outperforming Open-Source and Commercial Models
Grzegorz Kulik, Mikołaj Pokrywka, Adam Jatowt et al. (+1 more)
Dialectal machine translation remains challenging due to limited data and strong linguistic variation not captured by standard
benchmarks, which often assume standardized and well-edited text. We study Polish-Silesian MT using neural and rule-based systems, evaluating on SiLTT - a new Pol-Szl testset, alongside established BOUQuET and FLORES
benchmarks. Results show our rule-based system is consistently strongest on SiLTT and BOUQuET
datasets and that TranslateGemma fine-tuned on a curated
dataset improves over strong neural baselines but does not surpass the rule-based system in dialectal settings. We release SiLTT and our best neural model to support further research.
Read the full abstract ▾
All authors Grzegorz Kulik, Mikołaj Pokrywka, Adam Jatowt, Wojciech Kusa
Computation and Language
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
Yu Mao, Lei Yu, Zining Zhu et al. (+6 more)
We connect the spurious-reward paradox to a model's reachability and propose random-reward reinforcement learning (RL) as a useful tool for the probing enterprise, addressing a decade-long debate over what probing performance actually reveals about a model. There are two prevailing explanations for the surprising finding that even random rewards can improve the performance of large
language models (LLMs): one attributes the gains to particular mechanisms within RL training; the other to data contamination. Our results motivate a different view: spurious-reward RL can probe a model's reachability, or what further training can attain from its current state under specified constraints, beyond what is reflected in its current performance. Two OLMo checkpoints with the same accuracy on synthetic arithmetic (3.5%), for example, reach 8.5% and 55% in their best runs under the same correctness-rewarded RL. Examining OLMo checkpoints across pre-training and mid-training reveals three distinct regimes of training response: early on, RL produces little improvement even when correct answers are rewarded; later in pre-training, rewarding correct answers becomes effective while random rewards remain weak; and, upon entering mid-training, even random rewards can produce large gains. A similar ordering appears in a number-masked supervised fine-tuning (SFT) analysis of these checkpoints, suggesting that the pattern is not specific to a particular RL mechanism. Moreover, RL with random rewards offers a distinctive perspective on what training can make an LLM do, since its reward signal supplies no information about which answers are correct. By asking what training can attain without correctness feedback, it addresses the label-leakage side of a central problem in decodability-based probing: whether a successful probe reveals the model's capabilities or learns the task itself.
Read the full abstract ▾
All authors Yu Mao, Lei Yu, Zining Zhu, Yusheng Zheng, Haohang Li, Freda Shi, Yutong Yin, Zhaoran Wang, Jingcheng Niu
Computation and Language
JoinGR: Learning to Traverse Join Graphs for Table Retrieval
Sandipan De, Abhijit Chakraborty, Sambaran Bandyopadhyay et al. (+1 more)
Retrieving the right tables is a prerequisite for Text-to-SQL over realistic databases. Dense table retrievers rank schema elements independently, but this ignores a key source of evidence: some required tables are not mentioned in the question and become identifiable only through their join relationships to already relevant tables. We introduce JOINGR, a join-aware table
retrieval method that treats the database join graph as the
retrieval space. Columns are represented as graph nodes, while intra-table and foreign-key relationships are represented as typed edges. Given a question, JOINGR selects semantically similar anchor tables, traverses join edges with a query-conditioned scorer, and aggregates the resulting edge deposits into table scores. The scorer is a lightweight MLP on top of frozen query, node, and edge
embeddings, trained with a pairwise margin loss over gold tables. On BIRD and Spider
datasets, JOINGR is competitive with the strongest
retrieval baselines. On BEAVER, a challenging enterprise
benchmark with multi-hop table requirements, JOINGR substantially improves recall over dense
retrieval and re-ranking baselines. Cross-domain experiments show that the learned scorer transfers across
benchmarks, indicating that the method captures reusable joingraph traversal behavior.
Read the full abstract ▾
All authors Sandipan De, Abhijit Chakraborty, Sambaran Bandyopadhyay, Vivek Gupta
Computation and Language
Capturing In-Context Learning Dynamics with Task Operators
Guangzhi Xiong, Zhenghao He, Bohan Liu et al. (+3 more)
In-context learning (ICL) enables
language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fixed activation vectors extracted from specific layers or positions, but these input-independent interventions fail on complex tasks where the output depends on fine-grained interactions with the input. By analyzing the ICL forward pass, we show that each
attention head's output is an affine transformation of its context-masked counterpart, and that the parameters of this transformation are empirically stable across samples for a given task. Building on this, we introduce Task Operator (TO), which replays this transformation as an analytically derived update to the
attention output projection. Across lexical, algorithmic, and
reasoning tasks, TO achieves the best overall performance among prior methods and substantially narrows the gap between zero-shot inference and ICL. We further show that the extracted knowledge concentrates in a task-specific sparse
circuit across layers and positions, and that averaging operators from disjoint demonstration batches enables effective many-shot scaling without expanding the context window. Our code is available at https://github.com/gzxiong/task_operator.
Read the full abstract ▾
All authors Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang
Computation and Language
Sentence Specificity Scores for Collaborative Technical Documentation: A Domain-Transfer Study
Rocker D'Antonio, Thomas Benton Townsend and Dimitrios Michael Manias
Collaboration depends on shared context, and technical documentation is one way that context persists across people and AI teammates. Specificity, the amount and exactness of detail expressed in
language, shapes what information documentation captures and how precisely that information is communicated. This work audits sentence-specificity scoring artifacts on technical documentation and tests whether scores applied only after generation help choose among fixed LLM-generated revisions. Across Wikipedia and three technical-documentation corpora, the fixed general-domain predictor SpeciTeller and the pinned post-publication author-repository implementation of Ko et al.'s target-adapted predictor produce different corpus orders and same-sentence rank agreement from -0.066 to 0.510. Strict filtering and
token-length adjustment change these patterns without reconciling them. In the Gemma set, SpeciTeller ranking raises direction-valid selection from 71.7% to 83.3% (+11.7 points; 95% source-case bootstrap interval +1.7 to +21.7); in the GPT-OSS-120B set, SpeciTeller ranking raises direction-valid selection from 51.7% to 56.7% (+5.0 points; 95% source-case bootstrap interval -6.7 to +16.7), and every primary single-score GPT-OSS-120B interval includes zero. These findings tie score interpretation and decision value to the predictor and candidate set.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Empty Commitments: When Agents Promise What They Cannot Deliver
Jiaqi Tang, Bingyu Shen, Lan Wei et al. (+6 more)
A chatbot that says "I will remind you tomorrow" will not run again until the user writes. We call such a promise an empty commitment: a promise of action after the current turn that nothing in the
agent's tools or runtime can carry out. Unlike a broken promise, its emptiness is decided by the
agent's configuration at the moment of speaking, so it can be detected from a single turn, before deployment or at run time. We define empty commitments on top of commitment semantics, with three failure types, an anchoring condition for promises that a tool could make real, and an outcome taxonomy that separates these failures from honest deferrals and from over-refusal. We build a checker (a setup-blind detector, deterministic feasibility rules, and a response judge) validated against 400 human labels. On a controlled
benchmark of 293 follow-up requests across five setups that add one persistence affordance at a time, four open-weight models of 8-14B parameters fail on 45.9% of responses when no tool exists and nothing is stated. A frontier model fails on 4.4%, but it gets there by deferring and asking, not by using the tools it has: promises made without the enabling call remain in every model. Telling the model its runtime, the cheapest fix, cuts open-weight failures nearly in half where nothing is doable and changes nothing where a scheduler exists; a directive capability card removes most failures at the largest cost in over-refusal; running the checker in the loop and rewriting flagged replies removes more at a smaller cost. Code,
prompts, model outputs, and human labels are released.
Read the full abstract ▾
All authors Jiaqi Tang, Bingyu Shen, Lan Wei, Qing Lu, Bethel Ololade, Danny Galvis, Miles Q. Li, Bin Hu, Boyang Li
Computation and Language · Cross-listed from cs.AI
Beyond Final Accuracy: Auditing Communication in LLM Multi-Agent Systems
Shixuan Li, Wei Yang, Peiyu Zhang et al. (+3 more)
Multi-
agent communication aims to help
agents benefit from one another's information. Yet improvements in system performance leave a fundamental ambiguity: do they reflect effective communication, a favorable
agent architecture, or simply additional
reasoning? Because communication methods are commonly evaluated within the systems they were designed for, these factors are difficult to disentangle. Final accuracy further merges corrected errors and corrupted answers into a single outcome, obscuring how communication changes decisions. We introduce Independent–Communicate–Revise (ICR), a controlled framework that evaluates communication as answer revision following independent
reasoning. ICR fixes initial
reasoning trajectories, measures correction and preservation conditional on both
agents' initial correctness, and uses a no-message revision control to quantify gains beyond additional
reasoning. Across four
reasoning benchmarks, our audit of textual and latent communication reveals that similar aggregate accuracy can conceal substantially different revision behaviors. Compared with transmitting answers alone, full
reasoning increases correction while reducing preservation on all four
benchmarks, so richer messages amplify beneficial and harmful influence alike. Receiver-policy comparisons on MedQA and GPQA-D further show that a structured verification policy shifts every channel toward greater preservation and lower correction, while its effect on selectivity varies across channels and tasks. These findings challenge treating communication quality as an intrinsic property of a channel. ICR therefore recenters evaluation on selective revision, providing a unified framework for examining how message content and receiver policies jointly produce benefits and harms.
Read the full abstract ▾
All authors Shixuan Li, Wei Yang, Peiyu Zhang, Anzhe Cheng, Heng Ping, Paul Bogdan
Computation and Language
LawCompass: Navigating from Legal QA to Multi-Agent Deep Research with Grounded Evidence
Xiaoxia Cheng, Linnan Wang, Jiahao Ma et al. (+5 more)
Recent advances in Large
Language Models (LLMs) and
Retrieval-Augmented Generation (RAG) have significantly democratized access to legal information. Nevertheless, most existing legal assistants remain confined to multi-turn conversational QA, failing to support complex legal tasks that require systematic evidence
retrieval, multi-step
reasoning, and report-level synthesis. In this paper, we present LawCompass, an evidence-grounded legal assistant that navigates the transition from standard Legal QA to multi-
agent deep research. LawCompass provides three task-oriented functions: Legal QA, which delivers precise, evidence-backed answers to legal questions; Professional
Retrieval, which enables structured exploration of statutes and judicial cases via query rewriting; and Deep Research, which employs a multi-
agent workflow to decompose complex legal tasks and synthesize comprehensive research reports. Crucially, LawCompass maintains explicit citation links across all modules, empowering users to directly verify system outputs against original legal sources. Evaluation results demonstrate that LawCompass provides a practical and scalable paradigm for transforming conversational AI into trustworthy and evidence-grounded legal research assistance.
Read the full abstract ▾
All authors Xiaoxia Cheng, Linnan Wang, Jiahao Ma, Zhichuan Ye, Xuemei Zhou, Chuanyu Tong, Bo Jiang, Qing Zhu
Computation and Language
It Takes Workflows to Evolve Better Workflows
Xuehang Guo, Haoyu Wang, Haifeng Chen et al. (+3 more)
Tackling complex real-world tasks can exceed the capabilities of a single large
language model (LLM), motivating the use of multi-
agent workflows that coordinate specialized
agents to work together on these tasks. Recent methods train LLMs to construct better workflows from execution outcomes, but they optimize only the workflow generator, while the other
agents that build or execute each workflow remain fixed even though every outcome depends on all of them. However, extending training beyond the generator is challenging: the
agents are coupled, and a workflow's outcome is a single sparse score that cannot tell which
agent causes a failure. We propose FloWright, which leverages the workflow as a harness to optimize workflows. By introducing a hierarchical, structure-aware reward paradigm, FloWright enables one role to self-evolve and two or more roles to co-evolve, with no additional models, labels, or executions. Considering the limitation that workflows are commonly trained and evaluated on data that a single
agent can already handle, we further propose DataWright, an adaptive data hardening approach that converts existing
datasets into workflow-level tasks with increased difficulty. Across document, slide, chart, code, math, and finance tasks, small open models trained with FloWright achieve improved performance by up to $+7.41\%$, with co-evolving ($+5.03\%$) more roles gaining more than optimizing one of them alone ($+2.83\%$). Our project page: https://xhguo7.github.io/FloWright/.
Read the full abstract ▾
All authors Xuehang Guo, Haoyu Wang, Haifeng Chen, Yangyi Chen, Zhenhailong Wang, Qingyun Wang
Computation and Language · Cross-listed from cs.SE
Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents
Junyu Guo, Shangding Gu, Ming Jin et al. (+1 more)
Coding
agents can return plausible patches that omit required behavior. These failures are hard to review because long traces and confident summaries often hide what was missed. We ask when a nominally weaker reviewer can reliably decide whether a patch solves its issue. We study 411 execution-labeled traces from three
agents and 101 controlled cases. On 154 GPT-5.4 traces, structured but unchecked evidence raises both defect catch and over-rejection. We then provide official execution evidence as an upper-bound diagnostic. After choosing and freezing one of two formats per reviewer, five of six reviewers improve both rates on 122 held-out traces; two classify every trace correctly. Reviewer size is not a consistent predictor of quality. Because official tests are unavailable in deployment, we also evaluate a frozen cascade with patch-caused static errors and generated tests that first fail on the unpatched repository. On 121 scored held-out GPT-5.4 traces and 59 Gemini traces, its coverage is 0.89 and 0.86, risk is 0.33 and 0.26, catch is 0.76 and 0.80, and over-rejection is 0.66 and 0.67. Most false rejections occur when unresolved cases reach the reviewer. Official execution evidence shows the potential of weak review when decisive checks are available. Producing equally reliable checks without official tests remains the main bottleneck.
Read the full abstract ▾
All authors Junyu Guo, Shangding Gu, Ming Jin, Javad Lavaei
Computation and Language · Cross-listed from cs.AI
Pay for the Fault, Not the Flow: Label-Free In-Flow Multi-Agent Workflow Optimization
Xuehang Guo, Haoyu Wang, Shengyu Chen et al. (+4 more)
Large
language models (LLMs) increasingly construct multi-
agent workflows that decompose a complex task and assign specialist
agents from a pool. However, building such a workflow well remains challenging: how finely to divide the task, which
agent to trust with each subtask, and when to create a new specialist are all critical decisions a workflow constructor needs to settle up front. Thus, whether each subtask succeeds remains unknown until the workflow runs. Yet, improving a workflow is costly. Locating a fault usually requires a reference answer, a graded outcome, or a trained assessor, and the fix is applied to the whole workflow through re-execution, re-search, or retraining. We propose InFlowOp, which prices every decision in one label-free cost that weighs how well an
agent's competence meets what a subtask demands against how much that
agent takes to run. Before execution, InFlowOp bidirectionally determines the granularity of task decomposition and
agent assignment following from the cost rather than from a fixed template. During execution, InFlowOp corrects a fault with the cheapest move via the same cost that serves the workflow both as it is built and as it runs. Facing the workflow-level evaluation challenge, we introduce Braid, a
benchmark whose tasks require multi-
agent coordination beyond single-
agent capability. Across various domains and backbones, InFlowOp outperforms single
agent baselines by up to $+11.97\%$, achieving $+9.64\%$ with in-flow optimization. Our project page: https://xhguo7.github.io/InFlowOp/.
Read the full abstract ▾
All authors Xuehang Guo, Haoyu Wang, Shengyu Chen, Zach Chen, Wei Cheng, Qingyun Wang, Haifeng Chen
Computation and Language
Scaling and Distilling Text Embeddings for Better Diffusibility
Zekai Zhang, Yunjie Tian, Yanjin He et al. (+4 more)
Diffusion language models (DLMs) offer a promising alternative to autoregressive (AR)
language generation. Recent advances in continuous DLMs, which apply latent
diffusion to continuous text
embeddings, raise a practical question: which
embedding makes the best latent space, i.e., the most diffusible? To answer this, we search through different
embeddings and find that scaling the
embedding model to stronger ones within the same family (T5 to T5Gemma-1 to T5Gemma-2) greatly improves generative performance. But the raw T5Gemma-2
embeddings are still not optimal. They are so discriminative that even the
embeddings of plausible alternative words are separated, which makes the generation vulnerable to imperfect sampling. Consequently, continuous
diffusion often fails to reach any of them and ends up at an invalid
embedding instead. To address this, we distill T5Gemma-2 into a student encoder that learns the teacher's decoded probabilities as soft labels. Learning from such soft labels makes the student pull the alternative
embeddings closer while maintaining the encoding-decoding mechanism. The distilled
embeddings form a more connected and diffusible latent space, improving over the vanilla T5Gemma-2
embeddings. As a result, our medium-sized DLM achieves Gen. PPL 17.8 (against real-text PPL 15.4) at real-text
entropy on OpenWebText, outperforming GPT-2-M on Gen. PPL.
Read the full abstract ▾
All authors Zekai Zhang, Yunjie Tian, Yanjin He, Xiaoyan Zhang, Dongdi Zhao, Qing Qu, Di Fu
Computation and Language
Distilling Directional Verification
Jungseob Lee, Sugyeong Eo, Seongtae Hong et al. (+4 more)
Knowledge
distillation aims to transfer the factual knowledge of large
language models to smaller models for efficient deployment. Yet a teacher may recall a relation in one direction while failing to generate the answer in the reverse direction.
Distillation from its generated answers can therefore propagate this directional limitation to the student. The same teacher can nevertheless recognize such an answer by scoring the relation in the direction it knows. We introduce directional label
distillation, in which frozen teachers score candidate answers in that known direction and the best-scoring candidate becomes the student's training target. On facts about parents and their children, known-direction scoring yields more accurate labels than scoring the requested direction, even after tuned corrections for name priors. With prior-corrected scores, the better direction depends on the facts rather than the template, and reverses on mined facts whose notable entity is the parent rather than the child. With the evaluated children's forward facts withheld, students trained on known-direction labels improve open-ended accuracy on their trained queries by 13 to 15 points over students trained on prior-corrected reverse labels. After generated answers are matched to a fixed name list by lexical similarity, students reproduce nearly all selected labels. Their accuracy largely follows label quality. The label advantage holds on unscreened queries and when candidates are retrieved without inserting correct answers. Our findings show that directional verification mitigates the transfer of errors from teacher-generated answers to students by providing more accurate training targets. Code is available at https://github.com/js-lee-AI/directional-verification.
Read the full abstract ▾
All authors Jungseob Lee, Sugyeong Eo, Seongtae Hong, Seungyoon Lee, Chanjun Park, Jaehyung Seo, Heuiseok Lim
Computation and Language
The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization
Chao Li, Shigeng Wang and Anbang Yao
Post-training quantization (PTQ) methods typically use sequential quantization that partitions a pre-trained LLM into a series of units (e.g.,
transformer blocks), with one unit quantized at each stage. State-of-the-art PTQ methods are predominantly learning-based, optimizing auxiliary quantization parameters (e.g., scaling factors, rotation
matrices, clipping thresholds, and adapters) via gradient descent to minimize a reconstruction loss. A common practice is to use mean squared error (MSE) as the reconstruction loss function, yet its induced optimization behavior remains largely unexplored. In this work, we take a holistic view of sequential quantization and systematically investigate how optimization evolves from the first quantization stage to the last, aiming for a deep understanding of optimization in learning-based PTQ schemes. Through extensive empirical studies spanning representative learning-based PTQ methods, LLM families, model scales, architectures, quantization settings and various tasks, we consistently uncover Optimization Imbalance: reconstruction loss magnitudes vary dramatically across stages, accompanied by highly uneven gradient magnitudes and parameter updates under MSE. We term the cross-stage range of loss magnitudes the reconstruction loss scale, and reveal that MSE translates the unexpectedly large reconstruction loss scale into highly uneven gradient magnitudes, which in turn lead to uneven optimization strength across quantization stages. This finding suggests a general principle for improving learning-based PTQ: optimization strength across stages should be decoupled from the reconstruction loss scale. Theoretically, we show that root mean squared error (RMSE) variants defined at the sample, channel,
token, and element levels naturally realize this principle through implicit gradient normalization, outperforming MSE significantly as a drop-in replacement.
Read the full abstract ▾
Computation and Language
A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Models
Yingzhu Zhao, Vlad Pandelea, Han Yuan et al. (+4 more)
Large
language models (LLMs) have been increasingly used for financial document analysis, including earnings call transcripts (ECTs). Beyond generating standalone claims, users increasingly prefer grounded analyses that pair claims with verifiable citations from source documents to enable independent validation. However, evaluating such analytical claims typically requires extensive expert annotation, which is costly and difficult to scale, and real-world financial analysis commonly involves long context-question-answer triplets, further increasing task complexity. To address these challenges and
benchmark the current landscape of grounded analysis by LLMs, we propose a numeric evidence evaluation method that enables groundedness assessment without reliance on expert annotation. We also introduce an automated
dataset construction pipeline and construct ECTs-100 from the top 100 constituents of the S&P 500 to support
benchmark of both groundedness and correctness. In addition, we examine conscious incompetence, a practical failure mode in financial analysis in which LLMs must detect when available evidence is insufficient and refrain from producing unsupported
hallucinations. Empirical results show that LLMs perform well in groundedness but face notable limitations in correctness, with informational insufficiency presenting an additional challenge.
Read the full abstract ▾
All authors Yingzhu Zhao, Vlad Pandelea, Han Yuan, Bo Hu, Wuqiong Luo, Li Zhang, Zheng Ma
Computation and Language · Cross-listed from cs.IR
RPTune: Learned Context Curation for LLM Catalog Search
Chuxuan Hu, Hejie Cui, Norman Huang et al. (+3 more)
For small merchant businesses (SMBs) whose catalogs fit within a long-context LLM, full-catalog prompting offers a compelling alternative to multi-stage
retrieval designed primarily for large marketplaces with millions of items. However, fitting the full catalog into the context window does not ensure that the model can use it effectively, since LLMs do not exploit long contexts uniformly. We therefore study in-context catalog search through two complementary questions: (1) how to curate and present catalogs to the LLM, and (2) how to adapt the LLM for product selection on curated contexts. We propose RPTune, an end-to-end framework that couples learned catalog curation with LLM post-training using automatically generated, catalog-grounded supervision. An encoder-reorganizer curator orders and prunes products guided by downstream LLM feedback, while the resulting curated catalogs in turn improve the effectiveness of LLM post-training with a context-relative reward. We evaluate RPTune on 7 real merchants spanning distinct retail verticals, using 100 complex conversational queries per merchant. RPTune consistently improves search accuracy across both proprietary and open-weight LLMs, with context curation yielding gains of up to 31.4 percentage points and post-training adding a further 10.3 points on average.
Read the full abstract ▾
All authors Chuxuan Hu, Hejie Cui, Norman Huang, Shubham Kumar Bharti, Wang-Chiew Tan, Sercan Ö. Ar{\i}k
Computation and Language
Role-aware Heuristic Episodic Attention for Conversational LLMs
Wanyang Hong, Zhaoning Zhang, Yi Chen et al. (+5 more)
Large
language models often lose track of persistent instructions and relevant information as multi-turn conversations grow. We study this cumulative contextual decay through three related failure modes:
attention pollution, dilution, and drift. We propose REA (Role-aware Heuristic Episodic
Attention), a context-management framework that assigns different persistence and representation policies to instructions and episodic interactions. Instructional
Memory retains identified global constraints in a dedicated prefix. Episodic
Memory preserves user inputs and compresses model replies, while heuristic
retrieval selects raw text, compressed representations, or omission for each historical turn. On Long-MT-Bench+, REA improves the judge score from 6.32 to 7.36 on a 10-point scale, a 16.5% relative gain over the Vanilla baseline, and reduces average latency by 2.91$\times$. Additional evaluations show aggregate gains on three backbones spanning 1.7B-7B parameters and on Chinese and English role-playing tasks. These results support role-aware context management as a practical approach to maintaining conversational continuity and instruction adherence.
Read the full abstract ▾
All authors Wanyang Hong, Zhaoning Zhang, Yi Chen, Libo Zhang, Baihui Liu, Linbo Qiao, Zhiliang Tian, Dongsheng Li
Computation and Language
Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles
Alexei N. Skurikhin, Emily M. Taylor and Nathan A. DeBardeleben
As large
language models (LLMs) move from standalone assistants into agentic workflows, evaluation must extend beyond scalar leaderboard accuracy to account for operational reliability, cost, latency, and
token efficiency. We use an agentic ensemble of small
language models (SLMs) with an SLM-judge-mediated feedback loop as a case study for such beyond-leaderboard evaluation. On the 541-
prompt IFEval
benchmark, the best ensemble achieves 97.34% strict
prompt accuracy, exceeding the strongest standalone LLM baseline, gpt-5.4, by 5.81 percentage points while operating in a lower-cost regime. We then analyze the tokenomics and operational behavior behind this gain, including cost per sample,
token composition, useful-output goodput, feedback-loop recovery, latency decomposition, and performance across instruction categories and constraint counts. Our results show that agentic SLM ensembles can trade additional test-time
tokens and orchestration overhead for improved instruction-following fidelity, motivating multi-dimensional evaluation protocols for future agentic AI systems.
Read the full abstract ▾
Computation and Language
ReHoPER: Receding-Horizon Planning for Enhanced Reasoning
Saeed Ahmadnia and Cornelia Caragea
We propose ReHoPER, an inference-only, zero-shot method that improves large
language models'
reasoning by generating and answering intermediate questions along multiple paths before the final answer. It iteratively plans a horizon of candidate intermediate questions, selects one to answer, and replans from the updated history. ReHoPER is task-agnostic, using the same generic instructions across
datasets and models without labeled data or task-specific
prompt design. Across multiple
datasets, including iLLC, a new controlled
benchmark for compositional
reasoning, ReHoPER outperforms strong baselines, with the largest gains in the most compositional settings. Our implementation and the iLLC generator are publicly available to support future work.
Read the full abstract ▾
Computation and Language
Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations
Jungwon Park, Changin Choi, Jimyeong Kim et al. (+2 more)
As large
language models are increasingly deployed across diverse downstream tasks, efficient task adaptation has emerged as a central challenge. In response, a wide range of task adaptation methods have been proposed, spanning parameter-efficient fine-tuning, in-context learning, and
embedding-injection approaches. However, these lines of work have largely evolved within individual paradigms, leaving their cross-paradigm relationships and trade-offs underexplored, especially for recently emerging
embedding-based adaptations. This survey presents a unified framework that categorizes task adaptation methods by where and how task information is encoded: model weights, input
prompts, or injected task
embeddings. We provide a comprehensive taxonomy that integrates these paradigms, analyze their key strengths and limitations to explain how different adaptation paradigms have evolved, clarify relationships across paradigms, and highlight open problems for future research.
Read the full abstract ▾
All authors Jungwon Park, Changin Choi, Jimyeong Kim, Nojun Kwak, Wonjong Rhee
Computation and Language
The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
Yufa Zhou
Human
reasoning depends on how objects are related within propositions. How do relations organize the
language representations of contextual contents? We give an LLM a list of facts in its context (e.g., Alice eats an apple. Bob eats a pear.) and measure how its hidden state changes when the question switches from what Alice eats to what Bob eats. Averaged over many lists, this change is a steering vector, which we call the ordinal vector. It points to a fact by its order of mention, the order in which the facts were stated in the context. We find that LLMs represent the fact a question asks about by its order of mention, not by the name the question contains. We state this as the ordinal addressing hypothesis: each order of mention has a fact address in the model's state, shared by all contexts, and a question moves the state to the fact address of the fact it asks about, while the context supplies what that fact says. Across Qwen, Gemma, and Llama, fact addresses are (1) ordered by mention: query states are organized by the order of facts, not of names, even when one fact has multiple subjects; (2) steerable: added to a question about the first fact of a new list, the ordinal vector makes the model answer with the second fact of that list; (3) low-rank: they span a low-rank subspace in which the first-mentioned fact is the easiest to reach, surprisingly similar to human recall; and (4) emergent: they are shared in late-middle layers, hold from 1.5B to 32B parameters, and form early in pretraining.
Language models reach a stated fact by where it was mentioned, deepening our understanding of LLM
reasoning.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
Sungho Park, Wonjoong Kim, Jue Zhang et al. (+8 more)
Automated harness optimization can substantially improve LLM
agents by iteratively updating their
prompts, tool interfaces, and control logic from execution feedback. However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harness. We formulate this missing dimension of harness optimization as an automated curriculum learning problem and introduce ActiveSaddler. ActiveSaddler models the evolving curriculum as a non-stationary bandit with dynamically instantiated optimization targets. It abstracts recurring failures into reusable failure-pattern arms, estimates the potential learning progress from further targeting each pattern, and adaptively balances revisiting known weaknesses with exploring unseen scenarios for new ones. Optimization outcomes continually update both the set of discovered failure patterns and their priorities, allowing the curriculum to co-evolve with the harness. Experiments on GAIA2 and Terminal-Bench 2.0 show that ActiveSaddler consistently discovers stronger harnesses, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer using a scenario order fixed before optimization, respectively. Ablations further show that these gains depend on dynamically constructing optimization targets, estimating their evolving utility, and balancing continued optimization with new failure discovery. Together, these results establish automated curriculum learning as a new crucial optimization dimension for harness optimization.
Read the full abstract ▾
All authors Sungho Park, Wonjoong Kim, Jue Zhang, Wook-Shin Han, Pengfei Gao, Chanyoung Park, Yongqiang Yao, Rao Fu, Elsie Nallipogu, Qingwei Lin, Victor Rühle
Computation and Language
DeBERTa-ConPara: Attack-Aware and Deployment-Realistic Detection of AI-Generated Text
Mohamed Mady, Yupei Li, Johannes Reschke et al. (+1 more)
Robust detection of AI-generated text under deployment conditions is challenging: distribution shifts across domains and generators,
adversarial perturbations of the input surface, and the absence of target-domain labels for threshold calibration all degrade detectors that perform well in-domain. We present DeBERTa-ConPara, a deployment-oriented detector combining attack-aware Unicode preprocessing with a contextual
transformer encoder trained over HC3 Plus, M4, MAGE and RAID. Our central finding is that preprocessing acts in opposite directions depending on where it is applied: normalising the training corpus deduplicates it, collapsing 35.4% of RAID rows into copies of their clean siblings and deleting the
adversarial supervision, whereas normalising at inference is an effective defence. A factorial varying the two placements independently identifies raw training with normalised inference as the best configuration, reaching 99.61% AUROC, 99.01% TPR@5% FPR and 96.57% TPR@1% FPR on the official RAID hidden test, alongside 93.14% average balanced accuracy across HC3 Plus and MAGE under a fixed threshold. The gain is confined to two of twelve attack classes: homoglyph and zero-width-space insertion rise from 11.05% and 1.12% to 96.98%. The same signature reproduces in a zero-shot detector of different architecture, showing the effect belongs to the attacks rather than to our model. We additionally report two negative results: semantic-invariance augmentation through paraphrasing and supervised contrastive learning (ConPara) does not improve the best configuration, and the handcrafted feature-fusion branch is inert in distribution and harmful outside it.
Read the full abstract ▾
All authors Mohamed Mady, Yupei Li, Johannes Reschke, Björn W. Schuller
Computation and Language · Cross-listed from cs.RO
Kinematic MeanFlow: One-Step Action Generation Policy for Robotic Foundation Models
Jiawei Fan, Sifeng Wang, Yuqing Hou et al. (+1 more)
In this paper, we study how to achieve one-step action generation in Robotic Foundation Models (RFMs), aiming to overcome the high inference latency of multi-step flow matching. MeanFlow provides a promising framework for this goal, yet its direct application leads to performance collapse. We discover that this stems from two distinctive dynamics exhibited in the RFM velocity field: (1) the “local acceleration" exhibits stability early on, but surges sharply towards the end of the denoising process, and (2) the spread of its magnitudes across samples widens as denoising progresses. To address these issues, we introduce Kinematic MeanFlow (K-MF), a novel one-step action policy tailored for RFMs. Specifically, grounded in a kinematic identity, K-MF decouples the time derivative term in the MeanFlow formulation into two sub-interval terms separated by an intermediate point. This decoupled formulation enables the two terms to capture early-stage and late-stage denoising dynamics, respectively, while mitigating the error amplification across the process. As a result, our K-MF empowers RFMs to achieve one-step action generation in both training from scratch and fine-tuning paradigms across diverse tasks, while outperforming multi-step flow matching in most settings. In terms of inference efficiency, K-MF reduces action-head latency of GR00T-N1.6 by 67.5%~74.4% across L40 and Jetson Orin in eager and compiled modes, yielding end-to-end latency reductions of 30.3%~54.9%. Code will be available at https://github.com/IntelChina-AI/K-MF.
Read the full abstract ▾
All authors Jiawei Fan, Sifeng Wang, Yuqing Hou, Anbang Yao
Computation and Language
Child-Adapted Structured Phonological Representations for Interpretable Speech Sound Analysis
Abner Hernandez, Tomás Arias Vergara, Andreas Maier et al. (+1 more)
Structured phonological representations provide an interpretable alternative to generic
speech embeddings, but existing models are largely trained on adult
speech. We adapt PhonoQ-2.0 to child
speech using CHILDES-Aligned data and compare three
alignment-supervision conditions (Adult, Adult+Child, and Child-only) across two initialization strategies (Adult PhonoQ and scratch). Generalization is evaluated against manual child-
speech annotations. On 1,352 consonant targets from 58 typically developing children, child-
speech adaptation improves voicing recognition across all supervision conditions, from 0.922 macro-F1 for Adult PhonoQ to 0.972–0.987 after adaptation. Manner is more sensitive to
alignment supervision: Adult+Child MFA reaches 0.804 and 0.796, compared to approximately 0.70 under Adult MFA supervision. Place remains comparatively strong across systems (0.871–0.902), although per-class performance varies substantially. The velar-fronting contrast is preserved across all seven model variants. Longitudinal UltraPhonix analysis further reveals speaker-specific velar and post-alveolar changes that are largely preserved across models and broadly consistent with reported clinical progress.
Read the full abstract ▾
All authors Abner Hernandez, Tomás Arias Vergara, Andreas Maier, Paula Andrea Pérez-Toro