Computation and Language · Cross-listed from cs.LG
LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
Tianzhu Ye, Li Dong, Guanheng Chen et al. (+4 more)
Reinforcement learning (RL) on open-ended tasks compresses an LLM's rubric-based evaluation into a scalar reward, discarding rich textual feedback and conflating responses with distinct quality profiles. We propose Experiential Learning (EL), which repurposes the feedback model from an LLM-as-a-Judge into an LLM-as-a-Coach. The coach distills its assessment of each on-policy response into transferable experiential knowledge, which conditions a teacher model and is internalized by the policy through on-policy context
distillation. Compared with scalar rewards, this higher-bandwidth feedback channel provides dense supervision and preserves fine-grained preferences among high-quality responses. Across two policy families, with feedback from the policy itself or a proprietary model, EL consistently outperforms rubric-based RL on held-out and unseen open-ended tasks. Notably, EL generalizes better beyond the training distribution, and mitigates reward hacking. These findings establish experiential knowledge as a richer and more generalizable learning signal for post-training on non-verifiable tasks.
Read the full abstract ▾
All authors Tianzhu Ye, Li Dong, Guanheng Chen, He Zhu, Xun Wu, Shaohan Huang, Furu Wei
Computation and Language · Cross-listed from cs.IR
FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering
Jijun Chi, Zhenghan Tai, Hanwei Wu et al. (+21 more)
Financial question answering over U.S. Securities and Exchange Commission (SEC) filings requires retrieving and synthesizing heterogeneous evidence dispersed across long, standardized, and highly redundant disclosures. Existing
retrieval-augmented and multi-
agent systems typically derive
retrieval queries directly from the user's question and rank candidates by semantic similarity. Together, these choices create prior-corpus misalignment: a mismatch between model priors and the target filings' structure, terminology, and evidence standards. As a result, query generation misses corpus-specific evidence, while semantic reranking favors topically similar but evidentially invalid false-positive chunks. We propose FinSAgent, an evidence-grounded multi-
agent framework that reframes SEC filing QA as corpus-aligned
retrieval planning and corrects both ends with a single principle: inject corpus-side conditioning wherever model priors would otherwise dominate. FinSAgent combines (1) role-specialized
agents anchored to the mandated 10-K item structure, (2) database-aware query decomposition that conditions each
agent's sub-queries on a lightweight, summary-level view of the local corpus, and (3) multi-path
retrieval with a learned feature-gated reranker that separates evidential validity from semantic similarity. Across five offline financial QA
benchmarks, FinSAgent improves
retrieval coverage and answer correctness over strong single-
agent and multi-
agent baselines; in a three-arm randomized online experiment with 1,000 anonymous user ratings, it also receives higher scores than baselines.
Read the full abstract ▾
All authors Jijun Chi, Zhenghan Tai, Hanwei Wu, Tung Sum Thomas Kwok, Hailin He, Zixing Liao, Bohuai Xiao, Chaolong Jiang, Jianliang Lei, Jerry Huang, Peng Lu, Muzhi Li, Liheng Ma, Yihong Wu, Sicheng Lyu, Jingrui Tian, Yihan Li, Yanzhang Ma, Sizhe Guan, Dingtao Hu, Yufei Cui, Ling Zhou, Lei Ding, Xinyu Wang
Computation and Language
VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval
Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng et al. (+1 more)
Large
language models are increasingly used in practical systems, making efficient model selection important for reducing deployment cost. LLM routing has emerged as a practical solution for allocating each input query to an appropriate model under a desired cost-performance trade-off. Existing routing methods often estimate model suitability from the surface semantics or
embedding similarity of the input query. However, such methods may ignore the underlying difficulty of a query, leading to suboptimal routing decisions. To address the challenge, we propose VDAR-Router, a difficulty-aware
retrieval-based routing framework. For each input query, VDAR-Router first generates an explicit difficulty analysis. It then retrieves historical examples with similar difficulty profiles. Based on the retrieved records, it estimates candidate model suitability and selects the model using a reward function that considers both performance and cost. Experiments on three
datasets show that VDAR-Router consistently achieves better cost-performance trade-offs than existing baselines. These results demonstrate the effectiveness of difficulty-aware
retrieval for training-free LLM routing. Case studies further show that explicit query analysis helps retrieve more relevant examples and supports more reliable routing decisions.
Read the full abstract ▾
All authors Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng, An-Zi Yen
Computation and Language · Cross-listed from cs.AI
WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting
Zhaokai Wang, Tianlin Gui, Jiayuan Rao et al. (+3 more)
Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear prediction before the answer is available. We present WorldCupArena, a dynamic
benchmark for
language models and deep-research
agents. The 2026 FIFA World Cup is its first evaluation, and the same process can be reused for future leagues and cups. Before each match, a model either receives a common evidence package or searches for information itself. It predicts the result and score, likely players and events, match statistics, and the outcome of the competition. After the match, these predictions are compared with the recorded result. We report result accuracy, exact-score accuracy, and a scoreline score that gives some credit when a predicted score is close but not exact, together with scores for the other prediction tasks. Across systems, similar result accuracy can mask larger differences in detailed predictions. Four systems predicted champion Spain, and two of them also recovered the exact final pairing. Compared with betting-
market and human-fan baselines, the best system shows only small gains in result and exact-score accuracy, but a clearer gain in Scoreline. New schedules can be added as they begin, allowing the
benchmark to evaluate future models without using outcomes that are already known. Code, predictions and evaluation scripts will be publicly released.
Read the full abstract ▾
All authors Zhaokai Wang, Tianlin Gui, Jiayuan Rao, Shangzhe Di, Yihong Tang, Dingli Liang
Computation and Language
Modeling turn-taking with distant viewing: investigating silence thresholds in human and AI-generated discourse
Taylor Arnold, Nicolas Ballier and Artem Saloev
This study investigates silence gaps in two kinds of audiovisual material. We analysed thirty US situational comedies and fifty-one synthetic podcasts generated with Google NotebookLM. Gaps were compared across speaker gender, assigned from a fundamental-frequency threshold estimated in Praat, and across production settings.
Read the full abstract ▾
Computation and Language
Pancasila-Dilemmas: Evaluating Large Language Models on Indonesian Human Value Dilemmas Grounded in Pancasila
Supryadi, Irfan, Julianti et al. (+4 more)
The value
alignment of large
language models (LLMs) is crucial for ensuring responses align with human intention and value preferences. However, most evaluations of value
alignment focus on Western or universal values, while assessments grounded in the value systems of specific countries remain scarce. In this paper, we introduce Pancasila-Dilemmas, an evaluation
dataset of 1,834 questions derived from Indonesian news, classified by 5 values of Pancasila: Religion, Humanity, Unity, Democracy, and Social Justice. This
dataset reflects daily life in Indonesia, making it suitable for measuring the value
alignment of LLMs deployed for Indonesia. To ensure a more rigorous evaluation, we choose scenarios containing dilemmas. The
dataset is proofread by native speakers and answered by 5 diverse Indonesian citizens. We evaluate 50 closed- and open-source LLMs on our
dataset. Results reveal that all evaluated LLMs achieves less than 0.5 Probability Match Score (PMS) and 0.72 Max-Vote Agreement Score (MVAS). Compared by each values, LLMs mostly struggle in Religion and Unity dilemma cases. This highlights a significant gap in capturing Indonesian values. The
dataset is publicly available at https://github.com/tjunlp-lab/Pancasila-Dilemmas.
Read the full abstract ▾
All authors Supryadi, Irfan, Julianti, Darren Keanly Martin, Jayvin Fernando, Yuqi Ren, Deyi Xiong
Computation and Language
An Early Warning of Emerging Biosecurity Risks in Frontier LLMs
Zhida He, Xia Hu, Baichen Le et al. (+20 more)
Frontier large
language models (LLMs) are increasingly integrated into scientific workflows, yet their growing biological capabilities may outpace current safeguards. To assess the biological risks of frontier models, we develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework that couples model-level stress testing with wet-lab validation. Within this framework, Intern-BioBreaker generates targeted jailbreak
prompts to test whether aligned models can be induced to provide operational guidance for safety-sensitive biological tasks or produce sequence-level outputs with potentially harmful properties. Selected sequence outputs are then carried forward for DNA synthesis, host expression, and orthogonal
protein verification to assess whether model-generated designs can yield the intended biological products. Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models: (i) Intern-BioBreaker outperforms baseline attack models and reveals widespread bio-risk jailbreak vulnerabilities across both open-weight and proprietary frontier LLMs, with several targets reaching near-saturated or 100% task-level attack success rate (ASR); (ii) in sequence-level case studies, GPT-5.5 can be induced to generate modified viral candidate sequences with pathogenic potential; the corresponding translated
proteins may exhibit even stronger receptor-binding affinity and thus enhanced infection potential; and (iii) end-to-end verification shows that selected model-generated biological designs are not merely textual artifacts, but can be physically realized under controlled experimental settings. These findings underscore the need for stronger biological red-teaming, nucleic acid synthesis screening, and safety mechanisms that keep pace with model capabilities.
Read the full abstract ▾
All authors Zhida He, Xia Hu, Baichen Le, Chunxiao Li, Jiajia Li, Lijun Li, Chaochao Lu, Jing Shao, Youbang Sun, Hua Tang, Xiang Wang, Xiao Wang, Xiaoyu Wen, Tong Wu, Jia Xu, Peng Yu, Shu Yu, Jie Zhang, Qiaosheng Zhang, Yi Zhang, Xing-Ming Zhao, Tianhang Zheng, Ziyuan Zhou
Computation and Language · Cross-listed from cs.LG
L1 Augmented Attention as an Improved Vector Similarity Metric
Kurt Godden
Scaled dot product
attention conflates directional
alignment and vector magnitude, limiting its effectiveness as a similarity metric in
Transformer models. We introduce L1 augmented
attention, a simple and computationally parallelizable modification that subtracts a learned, head specific L1 distance between queries and keys from the dot product score. This hybrid similarity captures complementary
geometric information. Dot product rewards directional
alignment, while L1 penalizes coordinate deviations. To reduce the cost of L1 computation, we project queries and keys into low dimensional subspaces whose parameters specialize to preserve informative L1 structure. Evaluated on WikiText 2 using a compact
transformer, L1 augmented
attention achieves up to a 14.5% reduction in perplexity over the original
transformer baseline and outperforms an RBF L2 kernel. Analysis of norm variance and learned L1 weights reveals distinct
geometric roles across layers and strong head level specialization. These results demonstrate that enriching
attention with L1 geometry provides a principled and effective improvement to similarity computation in modern
language models, with practical benefits for both accuracy and parallel efficiency.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning for Parameter-Efficient Reasoning in Compact Models
Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov et al. (+3 more)
Large
language models achieve strong
reasoning performance, but often at prohibitive training cost - a challenge that is especially acute for compact models ($\leq 4 \, \mathrm{B}$ parameters) trained under limited budgets. We introduce MADA-RL, a post-training framework that specializes compact models into generator and critic roles and trains them with a debate-aware learning signal, fine-tuning only a small subset of parameters via LoRA adapters. Our central contribution is a counterfactual critic advantage: a dynamic, role-conditioned baseline that redefines the critic's advantage as its reward minus the generator ensemble's per-instance accuracy. This explicitly optimizes critics to improve over generator consensus rather than to merely reproduce a correct answer, yielding more targeted credit assignment than static mean-reward normalization. At deployment, the specialized
agents are composed in a lightweight multi-round protocol. Across five mathematical
reasoning benchmarks, MADA-RL raises the accuracy of the DeepSeek-R1-Distill-Qwen-1.5B model from $39.9 \, \%$ to $41.9 \, \%$ ($+2.0$ points, $p < 0.001$) using $16$ times fewer trainable parameters than fully fine-tuned baselines, placing it on the accuracy-trainable-parameter Pareto front. It approaches, but does not surpass, the strongest baselines (DeepScaleR, STILL-3), which are trained on substantially larger
datasets; we analyse this gap and the associated inference-time cost directly. A controlled study isolates the source of MADA-RL's gains: the counterfactual advantage produces the highest critic improvement rate of any model evaluated, indicating that trained critics learn to correct generator errors rather than to imitate them.
Read the full abstract ▾
All authors Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Zifeng Ding, Volker Tresp, Yunpu Ma
Computation and Language · Cross-listed from cs.CR
Self-State Attacks on Self-Hosted AI Agents: How Far Can OS Defenses Go?
Yimeng Chen, Nathanaël Denis, Roberto Di Pietro et al. (+1 more)
Self-hosted AI
agents maintain persistent
memory, instructions, and configuration that influence their future behavior. If an
agent is compromised, an attacker can exploit the
agent's legitimate write permissions to corrupt this self-state, making malicious and benign updates difficult to distinguish at the operating system (OS) level. We investigate how far existing OS mechanisms can prevent, detect, and recover from such self-state attacks. We formalize an attack space and evaluate representative OS defenses using four
agent workloads and a Linux telemetry pipeline. Our results show a consistent limitation across defense dimensions. File-level controls either leave alternative mutation paths open or, when complete over the tested operations, also block corresponding legitimate updates. Detectors flag a substantial part of legitimate activity, while more selective methods cover only part of the attack space. Finally, protected backups successfully restore corrupted state, but require a trusted recovery point and may incur rollback cost. Overall, our results show that the main limitation is not OS observability. Indeed, the OS can enforce, observe, attribute, and recover self-state changes. Yet, generic OS defenses lack the decision context needed to combine broad operation coverage with selective decisions. Effective protection therefore requires self-state-aware mechanisms that exploit additional context beyond generic file and syscall behavior.
Read the full abstract ▾
All authors Yimeng Chen, Nathanaël Denis, Roberto Di Pietro, Jürgen Schmidhuber
Computation and Language
What Transfers Under Source Shift? Definitions, Examples, and Fine-Tuning for Climate Disclosure Classification
Guosheng Li, Fenghui Ren, Bin Liu et al. (+5 more)
Climate disclosure classification is a fundamental task for analysing corporate
climate disclosures, yet such disclosures appear in many different sources – annual reports, press releases, and earnings calls – that differ in length, purpose, and writing style. Existing evaluations are mostly conducted within a single source, leaving open whether common LLM adaptation strategies remain effective under source shift. We reframe
climate disclosure classification as a cross-source adaptation problem and study three widely used adaptation strategies – definitions, examples, and fine-tuning – across eleven open- and closed-source LLMs, using two corpora that share the same label space but come from different sources. We find that all strategies bring positive cross-source gains on average, but the strongest in-source strategies are not the strongest cross-source ones: similarity-based
retrieval and LoRA fine-tuning gain most in-source but lose most of that advantage under source shift; randomly selected few-shot examples, a weaker in-source baseline, retain their advantage more reliably; definitions transfer most consistently, though only when their granularity matches the target text. Across these strategies, when the source changes, simpler is often safer.
Read the full abstract ▾
All authors Guosheng Li, Fenghui Ren, Bin Liu, Chuan Yu, Kaiying Ji, Lin Yue, Jun Shen, Sasa Qian
Computation and Language
DeLIVeR: Decomposed Learning for Information-grounded Veracity Recognition via Reinforced Knowledge Graph Exploration
Cong Hoan Nguyen, Thomas Hoang, Hieu Minh Duong et al. (+1 more)
Automated fact-checking remains a challenge for Large
Language Models (LLMs) due to "query brittleness" in traditional
retrieval systems. We propose DeLIVeR (Decomposed Learning for Information-grounded Veracity Recognition), a framework that treats evidence
retrieval as a reinforced strategic exploration task. DeLIVeR utilizes a Planner LLM to decompose complex claims into targeted question sets, which are used to traverse structured Knowledge Graphs (KGs) for high-precision evidence. We optimize the Planner's policy using Group Relative Policy Optimization (GRPO) with a reward system prioritizing structural diversity and verdict accuracy. Our evaluation on LIAR, FEVER, and PolitiFact shows that DeLIVeR significantly outperforms state-of-the-art baselines. Using Qwen2.5-7B, our framework achieved peak F1-scores of 83.73, 84.57, and 79.70 respectively, representing a 10-15% improvement over HippoRAG2. By shifting to a reinforced question-planning strategy, DeLIVeR effectively bridges multi-hop
reasoning gaps and provides an auditable, transparent path for verifiable misinformation detection.
Read the full abstract ▾
All authors Cong Hoan Nguyen, Thomas Hoang, Hieu Minh Duong, Long Nguyen
Computation and Language · Cross-listed from cs.DL
Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field
Tanay Aggarwal, Angelo Salatino, Francesco Osborne et al. (+1 more)
Knowledge Organization Systems like Ontologies and taxonomies are fundamental for structuring scientific knowledge, yet their manual curation presents a persistent bottleneck in knowledge management. While Large
Language Models (LLMs) offer a scalable mechanism for automated ontology generation, their capacity to classify complex, domain-specific semantics requires systematic evaluation. In this paper, we assess the performance of five small, open-source LLMs (up to 9 billion parameters) in identifying semantic relationships between biomedical concepts. To support this evaluation, we introduce MeSH-Rel-4K, a
dataset comprising 4K semantic relationships extracted from the Medical Subject Headings (MeSH). We analyse three adaptation strategies: standard prompting, Chain-of-Thought prompting, and fine-tuning. While parameter-constrained models traditionally struggle with the nuances of in-context logic, our results reveal that targeted fine-tuning increases the average F1-score by 34.1 percentage points. These results confirm that direct fine-tuning effectively exceeds the
reasoning bottlenecks of smaller LLMs, providing an accurate, automated methodology for the construction and
evolution of specialised biomedical ontologies.
Read the full abstract ▾
All authors Tanay Aggarwal, Angelo Salatino, Francesco Osborne, Enrico Motta
Computation and Language
Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI
Bogdan Raduta, Horia Velicu, Alexandru Preda et al. (+1 more)
Enterprises will not deploy AI
agents they cannot trust, and the most-cited reason for distrust is
hallucination: confident, fluent output that is simply not true. The common response is to wait for a model that does not hallucinate. We argue that this is the wrong target. Large
language models are, by construction, capable of generating unsupported text, and no amount of scale removes the possibility; a faithfulness judge bolted onto a raw model catches some errors but still ships others, and even well-curated
retrieval pipelines have been shown to fabricate citations. We reframe the goal: "zero
hallucination" is not a property a model possesses but a property a system enforces. We present
HALO (
Hallucination-Aware Layered Oversight), an assurance architecture which treats
hallucination as a containable failure mode rather than an eliminable one.
HALO composes six layers of defense: grounded generation over retrieved, approved content; constrained, deterministic execution that bounds where the model can err; multi-signal verification that scores every output for groundedness and
hallucination using both an LLM judge and evidence-based checks against the source text; calibrated abstention, so the system declines rather than guesses when grounding is insufficient; total traceability of every
retrieval, tool call, and generation; and continuous oversight that detects drift, alerts on threshold breaches, and closes the loop by regenerating and statistically validating improved
agents. We detail each layer, give particular
attention to evidence-based confidence (which verifies extractions against the source document rather than trusting the model's self-reported certainty), and illustrate the architecture on a regulated claims-extraction workload.
Read the full abstract ▾
All authors Bogdan Raduta, Horia Velicu, Alexandru Preda, Serban Chiricescu
Computation and Language · Cross-listed from cs.HC
AlphaOracle: Oracle bone script decipherment via human-workflow-inspired deep learning
Yuliang Liu, Haisu Guan, Pengjie Wang et al. (+11 more)
Approximately 3,000 of the 4,500 oracle bone script (OBS) characters remain undeciphered due to fragmentary inscriptions and sparse evidence. Current AI approaches fail to replicate expert workflows that integrate form analysis, contextual semantics, and philological
reasoning. We introduce AlphaOracle, a human-workflow-inspired framework that systematizes OBS decipherment using the largest digitized corpus to date. Its multi-stage pipeline comprises: (i) rubbing parsing; (ii) radical-based morphological analysis with diachronic modeling; (iii) contextual
retrieval with semantic
alignment; and (iv) philological validation against classical sources. Each stage generates explicit, confidence-weighted evidence chains, culminating in interpretable reports for scholarly verification. Across multiple test characters, AlphaOracle's readings strongly agreed with expert interpretations. In a study of 86 domain specialists, it reduced analysis time by 64% and 79% of participants rated it highly useful. Notably, AlphaOracle resolves the character "Lao" as a toponymic or clan designation, offering concrete revisions to Shang administrative and social interpretations. These results suggest that computational methods aligned with philological practice can facilitate OBS research and provide a conceptual reference for studies of other undeciphered scripts.
Read the full abstract ▾
All authors Yuliang Liu, Haisu Guan, Pengjie Wang, Xinyu Wang, Jinpeng Wan, Kaile Zhang, Handong Zheng, Xingchen Liu, Zhebin Kuang, Huanxin Yang, Bang Li, Yongge Liu, Lianwen Jin, Xiang Bai
Computation and Language · Cross-listed from cs.LG
Mobius Learning: Cyclic Depth Folding in Transformers
Tongtian Zhu
Transformer-based
language models organize computation along an ordered depth axis, where shallow and deep blocks often develop distinct representational roles. We challenge the conventional view that these roles must remain tied to a block's position in the ordered sequence. We introduce Mobius Learning, a training architecture based on cyclic depth folding, in which different data streams follow cyclically shifted block orders. The same block group is therefore applied early in the block sequence for some data streams and late for others, so it is optimized in both shallow and deep roles, a phenomenon we call depth-role superposition. Surprisingly, in four-worker experiments with a modded GPT-2 small (124M) model trained on 2.5B FineWeb
tokens using
Muon, Mobius Learning achieves lower validation loss than a fixed-order looped
Transformer at larger numbers of
Transformer block-sequence passes. This counterintuitive result shows that a block group need not remain confined to one fixed shallow or deep role within the block sequence and opens a new design space based on cyclic depth folding. Crucially, this structure makes Mobius Learning particularly well suited to
memory-constrained distributed training: raw training data remain local, while each worker stores one block group rather than the complete
Transformer block stack.
Read the full abstract ▾
Computation and Language
When a Name Is Not a Name: A Benchmark Dataset and Distilled Reasoning for Culturally Entangled Bangla Homographs in Low-Resource LLMs
Md. Asaduzzaman Shuvo
Many Bangla words are at once personal names and culturally loaded common nouns, "Maya" is both a girl's name and a word for affectionate compassion. Choosing the right reading demands cultural knowledge that is scarce in the pretraining data of modern
language models. We introduce Culturally Entangled Homograph (CEH) disambiguation and build a Bangla
benchmark of 1,516 expert-verified sentences (3,032 labelled occurrences) in which one word appears twice with two distinct readings, each labelled with a culturally grounded category and an explanation of the
reasoning behind it. Across open- and closed-source models, we find a systematic dominant-meaning bias: models default to the common-noun sense and overlook the name. A Bangla-specific model fails under every prompting regime we test, showing that
language-specific pretraining alone does not confer cultural grounding. We further show that contrastive chain-of-thought prompting can sharply reduce this bias without training, and that distilling cultural explanations teaches small (1-3B) models to reason toward the correct reading rather than memorise labels, cutting dominant-meaning bias from as high as 100% to under 5% and turning the failed Bangla-specific model into our strongest system.
Dataset and code are available at https://github.com/ashuvo25/BanglaCEH.
Read the full abstract ▾
Computation and Language
ESCUCHA: A Spanish Speech Benchmark for Heterogeneous Acoustic Conditions
Fernando López, Ana Ayala, Guillermo Segovia et al. (+4 more)
As large audio
language models (LALMs) advance, robust evaluation frameworks have become essential. In this context, Spanish
speech understanding under realistic acoustic conditions has received particularly little
attention. We introduce ESCUCHA, the first Spanish
speech understanding
benchmark designed to evaluate LALMs across heterogeneous acoustic conditions and
reasoning abilities. ESCUCHA comprises 1,000 human-curated questions paired with audio, totaling 162.9 hours sourced directly “from the wild” rather than drawn from existing
datasets, with durations ranging from a few seconds to over 80 minutes. The
benchmark emphasizes
reasoning, spanning 9 perceptual and 10
reasoning categories, and it captures linguistic diversity through multiple Spanish accents and non-normative
speech. ESCUCHA further includes multi-audio questions, spoken questions, and audio instructions, and it flags which questions support open-ended evaluation. Benchmarking several state-of-the-art
multimodal and
speech models reveals substantial performance gaps relative to trained humans.
Read the full abstract ▾
All authors Fernando López, Ana Ayala, Guillermo Segovia, Fernando Ibáñez, Ana Martínez, Pablo Gómez, Jordi Luque
Computation and Language
When to Use Extra Context: Evidence-Grounded Terminology Adaptation for Simultaneous Speech Translation
Zeyu Yang and Satoshi Nakamura
Extra context is valuable for simultaneous
speech translation of technical talks, but injecting the entire document context into every streaming segment is often too coarse. Through diagnostic experiments, we find that context gains mainly come from paper-specific terminology recovery rather than uniform semantic enhancement. We therefore propose EGTA, an Evidence-Grounded Terminology Adaptation framework that builds a document terminology
memory, selects compact candidate terms conditioned on the current streaming state, and adapts ASR/
speech-side and decoder-side decision spaces using only the selected terms. EGTA can be instantiated in cascaded, end-to-end, and generation-only SimulST settings without full-model fine-tuning. We evaluate EGTA on an ACL technical-talk SimulST evaluation suite consisting of MCIF-dev and ACL60/60-dev. On MCIF-dev, EGTA-RG improves BLEU by +1.05/+0.59, XCOMET-XL by +0.019/+0.006, named-entity recall by +79%/+73% relative, and acronym recall by +0.099/+0.171 on En$\rightarrow$Zh and En$\rightarrow$De. Across MCIF-dev latency settings, EGTA consistently improves XCOMET-XL, named-entity recall, and acronym recall. External validation on ACL60/60-dev further shows consistent terminology-recall gains without additional fine-tuning. Shuffled-
memory controls and activation audits provide evidence that the improvements are tied to paper-specific evidence
alignment rather than generic context prompting.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Reranking
HONOR Agentic Search Team: Zhengzong Chen, Lei Tang, Lijun Liu et al. (+26 more)
We present MagicSelector, a joint optimization framework integrating Counterfactual task decomposition, Progressive reranking, and Dynamic Top-K, designed to address the fundamental challenges of tool
retrieval in
agents. MagicSelector is a specialized framework capable of translating ambiguous user instructions into executable atomic subtasks and guiding high-precision tool
retrieval, effectively mitigating redundant noise and severe context distraction in out-of-domain (OOD) scenarios. We empower MagicSelector with these capabilities through three key contributions: (1) a preference-guided counterfactual task decomposition mechanism that utilizes a counterfactual reward to quantify the marginal causal gain of decomposition on
retrieval ranking, effectively imposing fine-grained structural supervision on logical coherence; (2) a progressive tool reranking method driven by self-
distillation hard negative mining, which optimizes both point-wise and list-wise relevance to enhance fine-grained discrimination among highly similar tools; and (3) a dual semantic boundary-aware dynamic Top-K strategy that adaptively monitors reranking score cliffs and inter-tool semantic shifts to dynamically truncate the candidate list, maximizing relevant tool recall while filtering long-tail noise. Evaluated on MTDTool, the first task decomposition
benchmark we constructed tailored for mobile multi-turn interactions with process-level annotations, MagicSelector yields promising performance. Extensive experiments demonstrate that MagicSelector significantly outperforms state-of-the-art methods in terms of tool
retrieval accuracy, OOD generalization capability, and overall
token efficiency, thereby demonstrating the effectiveness of our proposed framework.
Read the full abstract ▾
All authors HONOR Agentic Search Team: Zhengzong Chen, Lei Tang, Lijun Liu, Chuandi Jiang, Fan Yang, Keyun Chu, Chu Zhao, Shihao Liu, Minghang Li, Bo Liang, Can Wen, Hailong Wu, Jingnan Ju, Mian Liu, Nengbin Zhang, Peiqiang Wang, Penghe Nie, Qinhui Gu, Sijia Lv, Siqi Chen, Wei Zhang, Yang Xu, Yuhao Qian, Yuxiang Zhang, Zeng Cheng, Zhen Wang, Zuan Chen, Yuanyuan Zhao, Fei Huang
Computation and Language
Large Language Models for Citation Function Classification
Daniel Vodička, Jakub Šmíd, Pavel Král et al. (+1 more)
Citation function classification plays a crucial role in understanding the relationships between scientific publications and advancing bibliometric analysis. This study presents one of the first comprehensive evaluations of multiple state-of-the-art (SOTA) large
language models (LLMs) for citation function classification, achieving new SOTA results on the ACL-ARC
dataset. We systematically compare five models (Mistral 7B, Orca 2-7B, LLaMA 3.1-8B, Falcon 7B, and SciBERT) across zero-shot, few-shot, and fine-tuning approaches. Our fine-tuned Falcon 7B model achieves a 73.3% macro F1 score on ACL-ARC, representing a significant improvement over previous methods. Additionally, we introduce AC3, a novel
dataset featuring a seven-category annotation scheme that distinguishes between neutral acknowledgments and explicit evaluative stances (more opinion-oriented citations - criticizing, complimenting, contradicting). The
dataset is implemented across four context extraction variants to systematically evaluate the impact of contextual scope on classification performance. We also provide detailed analysis of model performance, experimental configurations, and limitations to guide future research in this domain. To our knowledge, this is one of the first studies dedicated to comprehensive model comparison for citation function classification, addressing a gap identified in recent surveys.
Read the full abstract ▾
All authors Daniel Vodička, Jakub Šmíd, Pavel Král, Christophe Cerisara
Computation and Language
C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Chuheng Du, Junyi Chen, Hanlin Tang et al. (+7 more)
Long-context inference is central to modern large
language model (LLM) applications such as
retrieval-augmented generation and multi-document
reasoning. To mitigate the growing inference cost, recent work has explored key-value (KV) cache reuse to reduce redundant prefill computation. However, existing reuse methods primarily focus on computation savings and overlook a critical bottleneck in long-context LLM serving: the cost of storing and accessing large KV caches. While KV compression appears to be a natural complement, naively combining compression with non-prefix KV reuse often leads to severe accuracy degradation. In this work, we propose C$^2$KV, a unified framework for non-prefix KV reuse that jointly optimizes KV extraction and inference-time concatenation. C$^2$KV learns a composable and compressed KV cache
manifold that is explicitly designed to be position-agnostic. Our approach introduces a lightweight sidecar Extractor with learnable compression
tokens and a structured
attention flow, enabling modular KV representations that can be flexibly reused and concatenated without modifying the frozen base model. We further employ a compression-concatenation co-training strategy to align extraction-time representations with their downstream reuse behavior. Extensive experiments across multiple long-context
benchmarks and model families demonstrate that C$^2$KV significantly reduces KV cache storage and transfer costs, achieving up to 17$\times$ inference speedup under long contexts, while preserving generation quality.
Read the full abstract ▾
All authors Chuheng Du, Junyi Chen, Hanlin Tang, Kan Liu, Tao Lan, Lin Qu, Chaoyue Niu, Shengzhong Liu, Guihai Chen, Fan Wu
Computation and Language
AEGIS: Awareness-Enhanced Guidance for Iterative Safeguard
Kyungwon Park, Sangmin Lee, Heejae Chon et al. (+1 more)
Span-level rationales are often assumed to improve controllability in text detoxification, but it remains unclear when such guidance helps and when it introduces trade-offs. We present Awareness-Enhanced Guidance for Iterative Safeguard (AEGIS) as an exploratory framework for studying span-guided
multilingual detoxification across English, Mandarin Chinese, and Korean. AEGIS combines span-level detector outputs with frozen generator backbones, allowing harmful spans, intensity labels, and target attributes to be provided as structured guidance during rewriting. Rather than claiming state-of-the-art detoxification performance, we analyze how span guidance affects the balance between toxicity reduction and meaning preservation across generator families, model scales, and
languages. Our results suggest that span-guided detoxification is conditionally useful: explicit rationales change the trade-off between toxicity reduction and meaning preservation, but their effects depend strongly on the generator backbone and the linguistic context. These findings highlight both the promise and the limitations of span-level control signals for
multilingual detoxification.
Read the full abstract ▾
All authors Kyungwon Park, Sangmin Lee, Heejae Chon, Hyungu Kang
Computation and Language
Tokenizing Crosslingual Homographs
Rotem Brillant and Yuval Pinter
Multilingual language models rely on shared subword vocabularies to represent multiple
languages within a limited number of
token units. While such sharing is often useful, it can also create cases in which identical surface forms are treated too uniformly across
languages, even when their meanings or usage differ. We investigate this limitation through cross-lingual homographs and false friends, and examine whether introducing
language information earlier in the tokenization process can improve their treatment. We propose a simple tokenizer-level intervention based on
language cues:
language-specific characters replacing initial characters of shared-vocabulary words, reducing common identity during vocabulary construction. In intrinsic analysis, we find through tokenizer-level statistics that BPE and UnigramLM often treat cross-lingual homographs in a largely
language agnostic way, whereas the context-sensitive SaGe tokenizer diverges more strongly; our intervention removes this gap. In downstream English-to-X machine translation, our cues yield modest improvements in several settings, especially under BPE, although the effect is not consistent across all
languages and evaluation sets. Overall, the findings suggest that adding lightweight
language information at the tokenizer level is a promising direction for further exploration.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.HC
It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation
Sadra Sabouri, Zeinabsadat Saghi, Jordan Lee Boyd-Graber et al. (+3 more)
Prior work on AI-assisted information evaluation has largely focused on what AI systems communicate, comparing explanation types and formats, with responses predominantly cast in directive rhetoric where the system delivers a verdict and the user passively accepts it. While debate-style interactions have recently shown promise in prompting critical evaluation over deference, the rhetorical patterns that structure AI responses and how they might induce reflection, uncertainty, or independent
reasoning remain largely unexamined. To address this, we investigated eight rhetorical patterns known to induce contemplation: Intentional Misleading, Interpretive Alternative, Scaffold Explanation, Triggering Distrust, Information Distortion, Alternative Framing, Socratic Questioning, and an Oracle baseline. Through a within-subject study with n=98 participants on a hint-on-demand fact verification task, we observed preliminary evidence that Scaffold Explanation were associated with the highest accuracy gains, and encouraging deeper reflection. Surprisingly, the
adversarial conditions also improved accuracy modestly. Participants preferred Alternative Framing most and Interpretive Alternative least, largely due to the latter's perceived time cost. We discuss the implications of designing conversational
agents with varied rhetorical styles and the trade-offs among user performance, satisfaction, and contemplation.
Read the full abstract ▾
All authors Sadra Sabouri, Zeinabsadat Saghi, Jordan Lee Boyd-Graber, Jonathan May, Jonathan K. Kummerfeld, Souti Chattopadhyay
Computation and Language · Cross-listed from cs.LG
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
Nikhil Ghosh, Tetiana Parshakova and Robert M. Gower
Low-rank adaptation (LoRA) makes finetuning large
language models cheaper by adding to each weight
matrix a trainable low-rank update parameterized as the product of two
matrices. These
matrices are usually trained with Adam, which treats them as a single flat vector of parameters and ignores both the
matrix and product structure of LoRA. Applying a
matrix-aware optimizer such as
Muon to each factor does not consistently improve over Adam, and neither do the product-aware
Muon variants proposed in concurrent works. To realize consistent gains, we introduce PoLoRA, a Preconditioned Orthogonalized LoRA optimizer built from three ingredients: a product-aware spectral update direction,
curvature preconditioning derived from controlling the per-sample loss change, and a magnitude rule that controls the sizes of both the factor and merged updates. We evaluate PoLoRA on instruction-tuning
datasets for code and math across models from 1B to 8B parameters, and find that it reaches the final held-out loss achieved by tuned Adam in 1.2-1.7 times fewer steps, while adding at most 3% per-step overhead. Compared to Adam, PoLoRA is also less sensitive to the learning rate, and its optimal learning rate is stable across ranks.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Is Progressive Disclosure All You Need for Long-Context Agents?
Yifeng He, Yinzhe Zhao, Jicheng Wang et al. (+1 more)
Long-document question answering usually forces a choice between loading the whole document into the context window and bolting on a separate retriever. Agentic AI suggests a broader option, giving the
agent the document path and letting it decide how and what to read.
Agent Skills, a standard for packaging expertise into folders an
agent loads on demand, supply a ready mechanism: progressive disclosure, which exposes only what a query needs, from a short description down to the specific passages. Practitioners rapidly adopted this pattern for book-length understanding tasks, but the evidence to support such choices has been anecdotal. We run the first controlled study of the pattern, comparing raw-document
navigation and several designs of
Agent Skills packs against a classical hybrid retriever across three
agent harnesses and three model families on InfiniteBench. On a single book, the gain depends on the harness, running large when the
agent navigates the raw document poorly but near zero when a strong
agent harness already divides and retrieves on its own. When scaling up to tasks that span many books, raw-document
navigation collapses while one-level progressive disclosure degrades more slowly and pulls ahead. A second, deeper routing level never helps and sometimes breaks accuracy outright, so one level is enough. Progressive disclosure buys context, not intelligence: it is redundant while a strong
agent can locate the right passages itself, and decisive once the corpus grows too large to navigate by reading.
Read the full abstract ▾
All authors Yifeng He, Yinzhe Zhao, Jicheng Wang, Hao Chen
Computation and Language · Cross-listed from cs.AR
D-NOVA: In-Storage Retrieval Accelerator via Dual-Bound 3D NAND-Optimized Similarity Search with Vector Adaptation
Chang Eun Song, Sumukh Pinge, Tianqi Zhang et al. (+3 more)
Retrieval-Augmented Generation (RAG) enhances the factual grounding of large
language model (LLM) inference by retrieving relevant information from external knowledge bases. However, its dense vector
retrieval introduces significant latency and energy overhead, becoming the primary performance bottleneck. Although recent in-storage accelerators aim to reduce data movement, they still rely on host or embedded processors outside the
memory, where nearly 70% of the total
retrieval time is spent. As a result, they cannot fully overcome the bandwidth limitations, leading to yet another
memory bottleneck. To tackle these limitations, we present D-NOVA, a hardware-software co-designed in-storage
retrieval accelerator. D-NOVA executes an inverted file (IVF)-based hierarchical
retrieval pipeline by deeply
embedding the search functionality directly into the NAND
memory array. This is achieved by incorporating a new distance metric, Dual-Bound Tight Similarity Sensing (DTS), which is specifically tailored for searching within the NAND string. In addition, we introduce a lightweight contrastive adapter that maps
embedding vectors into a DTS-friendly domain, recovering near-software recall while improving performance and energy efficiency. D-NOVA is up to 41.7x faster and 71x more energy-efficient than a CPU baseline, and achieves 12.13x higher throughput while being up to 1.26x more energy-efficient than state-of-the-art in-storage RAG accelerators, demonstrating the potential of fully in-storage vector search for scalable RAG acceleration.
Read the full abstract ▾
All authors Chang Eun Song, Sumukh Pinge, Tianqi Zhang, Sung Eun Kim, Tajana S. Rosing, Mingu Kang
Computation and Language · Cross-listed from cs.CR
Salience Induction against Multi-Hop RAG Agents: Threat and Defense
Xingfu Zhou, Pengfei Wang, Yuan Zhou et al. (+2 more)
Agentic
retrieval-augmented generation (RAG) systems increasingly retrieve external evidence and orchestrate tools for knowledge-intensive applications. In Multi-Hop question answering,
agents chain facts across documents. Existing defenses focus on content poisoning, which injects false facts, and
prompt injection, which embeds directives. We identify a third attack surface: the salience channel, through which fact position, emphasis, framing, and semantic proximity can redirect
reasoning even when all retrieved claims are true and no instructions are present. We formalize Salience Induction as truth-preserving edits that redirect Multi-Hop attribute binding while leaving the
retrieval trace semantically intact. We define six Salience-Editing operator classes and build an iterative proposer-verifier pipeline under factual and stealth constraints. We also introduce SalientWiki-MH, a decoy-annotated Multi-Hop
benchmark. Evaluations across five frontier model families (GPT, Claude, Gemini, DeepSeek, and Qwen) and three
agent architectures (ReAct, Reflexion, and tool-calling) show broad generalization. Under a 30% edit budget, Salience Induction achieves an 83.3% attack success rate; the strongest evaluated baseline defense leaves 75.7% post-defense ASR. Untargeted rewriting further reduces attacks only by degrading neutral task success. Our lightweight input-side defense, Salience Normalization, reduces attack success to 15.3% under standard attacks and 23.6% under an adaptive attack. These results show that truthfulness and instruction filtering alone are insufficient: robust agentic RAG also requires defenses against salience-relevance decoupling.
Read the full abstract ▾
All authors Xingfu Zhou, Pengfei Wang, Yuan Zhou, Wei Xie, Xu Zhou
Computation and Language
Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration
Jie Hu
Test-time collaboration, including self-consistency, best-of-N selection, critic models, and verifier pipelines, is often credited with broadly improving LLM
reasoning, yet its gains are uneven and sometimes negative. We ask when training-free collaboration should be expected to help. For a fixed candidate pool, we decompose a selector or verifier's net gain into measurable factors: recoverable mass, verification-signal coverage, conditional selection quality, and harm to already-correct outputs. This reframes collaboration as a candidate-selection problem rather than as an intrinsic property of a multi-
agent topology. Across LiveCodeBench, MATH Level-5 hard subjects, and GPQA-Diamond, gains are bounded first by the oracle gap and then by signal fidelity, which we measure directly as candidate-level agreement between verifier verdicts and official labels. On LiveCodeBench, a public-test verifier (MCC 0.825) gains +8.14 percentage points (pp) over a first-sample baseline; a generated-test verifier (MCC 0.248) improves by +2.70pp and is not statistically distinguishable from an LLM selector, but operates at near-zero harm versus the selector's 4.69% harm rate. On MATH, a symbolic answer-equivalence selector beats self-consistency by +4.67pp, while LLM selectors are negative. On GPQA-Diamond, recoverable mass is only 3.03% and 87.54% of candidate pools are answer-identical; a weaker model's pools shrink both further, suggesting that oracle gap is a joint property of task, model, and sampling configuration. Our framework yields a practical pre-deployment diagnostic: estimate the oracle gap, then measure coverage, signal fidelity, and harm before investing in collaboration.
Read the full abstract ▾