Computation and Language
Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models
Raúl Vázquez, Aman Sinha, Chuyuan Li et al. (+12 more)
In 2026, we held the fourth iteration of the SHROOM Shared Task series: SHROOM-
Visions (Shared-task on
Hallucinations and Related Observable Overgeneration Mistakes in
Vision language models), which is hosted at the UncertaiNLP Workshop co-located with EMNLP 2026. Following the success of the 2024 and 2025 tasks, this time we aim to tackle
hallucinations through a model-agnostic detection task focused on large
vision-
language models. Building on the recently introduced SHEEP
dataset, designed for long-term evaluation across model generations, the task invites participants to detect and classify fine-grained
hallucination spans in image-conditioned text generation (VQA, image captioning, etc.). The evaluation uses a five-class taxonomy of
hallucinations spanning four
languages: Chinese, English, French, and Italian. The shared task generated strong interest in the NLP community worldwide, with 27 teams contributing 600+ system submissions. The best systems achieve average scores of 0.58 in character-level correlation, 0.46 in label-conditioned correlation, and 0.51 in intersection-over-union (IoU) across four
languages, outperforming the baselines by 30-40 points.
Read the full abstract ▾
All authors Raúl Vázquez, Aman Sinha, Chuyuan Li, Artem Shelmanov, Artem Vazhentsev, Claudio Savelli, Eduardo Calò, Emilio Raimond, Stella Frank, Hengyu Luo, Flavio Giobergia, Vincent Segonne, Lorenzo Vaiani, Jörg Tiedemann, Timothee Mickus
Computation and Language
Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty
Tim Schopf, Tobias Schreieder and Akiko Aizawa
Automated novelty judgment can accelerate scientific discovery by enabling efficient evaluation, refinement, and comparison of research ideas. While large
language models are increasingly adopted for this task, we investigate a previously overlooked limitation in their judgment capabilities: despite generating
reasoning rationales that closely mirror those of human experts, their final novelty judgments often diverge substantially. We demonstrate that this miscalibration stems from a systematic bias towards judging ideas as "medium novel". To mitigate this, we propose Think-Probe-Respond (TPR), a lightweight approach that probes latent novelty judgments from hidden states during the
reasoning phase and uses the probed judgments to condition the final response. Across strong baselines, TPR improves novelty judgment performance by 22.30% and successfully mitigates the prevalent "medium novelty" bias.
Read the full abstract ▾
Computation and Language
Reconstructing the Right Episode: Evaluating Interleaved Conversational Memory Beyond Long Context
Zhexi Feng, Ruiyi Zhang, Yongbo Yang et al. (+1 more)
Conversations with chat assistants increasingly span many topics in a single long-running thread, challenging
memory systems. Existing long-context and
memory benchmarks often expose session or topic boundaries, or probe direct personal-
memory questions. These settings understate a harder assistant-
memory regime: a flat mixed-topic thread where the system must infer which earlier episode makes a later task decision valid. We introduce SCALE-QA, a constraint-grounded task QA
benchmark for flat unsegmented threads targeting episode integrity failure. The
dataset contains 3,000 audited questions across 10 domains, uses deterministic four-way multiple-choice grading, and includes a deterministic runtime builder; experiments use all 3,000 questions through 128k and a stratified 400-question diagnostic at 1M. SCALE-QA questions are ordinary task-oriented requests whose correct answer depends on causally related evidence introduced earlier in the conversation. We also propose Temporal-Semantic Interleaved
Memory Reconstruction (TSIM), which segments the turn stream into coherent episodes and indexes them through a hierarchical multi-view
memory stack with deterministic episode-level summary and cluster-routing views. Experiments show that SCALE-QA challenges strong RAG baselines and long-context LLMs alike; across three open-source and proprietary LLM backends, TSIM achieves the highest accuracy in every backend setting, gaining 5.6-17.6 accuracy points over the strongest corresponding baseline.
Read the full abstract ▾
All authors Zhexi Feng, Ruiyi Zhang, Yongbo Yang, Pengtao Xie
Computation and Language
Unmatched Does Not Mean False: Incomplete Reference Sets Can Reverse Calibration Rankings in Open-Ended Theory-of-Mind Tracking
Zhexi Feng, Wuxi Chen and Bingrui Zhang
Open-ended Theory-of-Mind (ToM) trackers emit valid beliefs absent from finite references. A finite-reference-plus-matcher pipeline marks unmatched outputs false, creating proxy labels that can reverse proper-score model selection on fixed outputs. Holding 259 beliefs and paired scores fixed, reference recoding lowers weighted prevalence from 0.783 to 0.295 and reverses strictly proper Brier risk: a frozen source-prior rule leads native confidence by 0.227 under reference labels and trails by 0.152 under blinded adjudication, in all six authored scenarios. A reference-only Platt recalibrator reverses further. An ICE-specific reversal appears in a released 301-question NQ-open DPR-BERT pipeline: its average-confidence baseline improves instance-level calibration error by 0.045 under exact match but worsens it by 0.074 under human correctness, with both intervals excluding zero. On independently authored OpenToM narratives, 90-96% of audited unmatched beliefs are literally true and the paired direction again reverses. An exact decomposition attributes the distortion to omitted truths, and a closed-form criterion correctly classifies comparisons from twelve released systems. Frozen-audit retrospective replay shows 50 attempted annotations recover ranking direction with probability at least 0.996. TriSource-Restore anchors full-frame reference labels and frozen automatic judgments to a probability-sampled human pilot, maintains at least nominal coverage, narrows intervals, and repairs confidence subject to a base-rate deployment gate.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
A Token-Level Analysis of Sampled-Token Reverse-KL On-Policy Distillation
Bing Shao, Jiazheng Zhang, Long Ma et al. (+13 more)
On-policy
distillation (OPD) supervises a student on its own trajectories with
token-level signals from a frozen teacher, yet how a sampled loss allocates updates across
tokens remains poorly understood. We analyze the gradient of the per-
token K2
estimator of reverse KL with respect to the student logits. The $\ell_1$ norm of this gradient factorizes into the absolute teacher–student log-probability gap and a student-side softmax factor that grows as the sampled
token becomes less likely under the student. In our math-
distillation runs, these per-
token norms are highly non-uniform: low-student-probability
tokens account for a disproportionate share of their sum and are also enriched in large teacher–student gaps. As a lightweight intervention suggested by this analysis, we study Surprise-aware Reweighting (SuRe), a detached, bounded weighting rule that further amplifies this existing allocation. Across two Qwen3 student scales, SuRe improves several math metrics over vanilla OPD and shows no clear degradation on the selected out-of-domain
benchmarks. Our primary contribution is therefore a gradient-level characterization of reverse-KL OPD trained with the K2
estimator, with SuRe as one empirical instantiation.
Read the full abstract ▾
All authors Bing Shao, Jiazheng Zhang, Long Ma, Yujiong Shen, Senjie Jin, Xin Guo, Yuming Yang, Mingxu Chai, Zhiheng Xi, Boyang Liu, Junlin Shang, Peixin Wang, Qi Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
Computation and Language
AutoVerifier: Residual-Guided Non-Parametric Optimization for Reference-Based Answer Verification
Zebei Zhao, Zhihao Shi and Minqi Shi
Reference-based verifiers are important for evaluating
reasoning models and providing accurate outcome rewards in reinforcement learning with verifiable rewards. To improve verification accuracy, prior work has explored rule-based, model-based, and tool-augmented verifiers for checking answer equivalence across diverse answer forms. However, the equivalence of answer forms such as $1+3.14$ and $1+\pi$ may depend on the question and scoring criterion. We frame such implicit assumptions as verifier inductive biases. To address this challenge, we propose AutoVerifier, a residual-guided non-parametric optimization method that learns these biases from recurring verifier errors. Specifically, AutoVerifier records these biases in rule cards and promotes them to code modules or
prompt guidance only after replay validation detects no direct regressions, keeping accepted updates auditable, editable, and reusable. Experiments on four verifier
benchmarks demonstrate that AutoVerifier outperforms state-of-the-art verifiers by a large margin.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editing
Haoyu Wang, Cheng Feng, Liuyang Bian et al. (+5 more)
Practical
video editing is not only pixel generation: an editor must turn a brief, a clip pool, music metadata, and hard constraints into an executable timeline. We study this decision layer as executable
video-editing planning and introduce RefineCut, which, unlike workflow systems that wrap a prompted frontier model, trains a compact open-weight planner for it. The planner edits a typed timeline through structured patches covering clip selection, trimming, ordering, transitions, and duration and music
alignment; a deterministic verifier applies each patch and checks it against an explicit constraint ledger. Because editing has no single ground-truth repair, we do not imitate teachers directly: RefineCut replays every multi-teacher branch through the verifier and keeps verifier-best repairs as supervision. A second stage, RefineCut-Evo, lets the student score its own repairs with the verifier and a task rubric and trains on high-margin preference pairs, so the final $8$B planner runs in a closed verifier loop with no teacher calls at inference. On RefineCut-Bench ($3{,}578$ tasks, $7{,}971$ captioned clips, $499$ music tracks, explicit ledgers), verifier-replayed
distillation lifts the planner from $0.620$ to $0.858$ on the protocol-specific
Video-Editing Score and RefineCut-Evo reaches $0.924$; the gain transfers to Llama-3.1-8B and GLM-4-9B, and in the same closed loop the $8$B planner matches or exceeds its frontier teachers. Code and RefineCut-Bench are publicly released; see the Data Availability statement.
Read the full abstract ▾
All authors Haoyu Wang, Cheng Feng, Liuyang Bian, Ruiyang Huang, Lei Wei, Yafei Wen, Xiaoxin Chen, Xiaoying Tang
Computation and Language
AWM: Answerable Working Memory for Long-Document VQA Agents
Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu et al. (+5 more)
Long-document visual question answering increasingly relies on VLM
agents that retrieve candidate pages, inspect page images, write findings to working
memory, and synthesize answers. Working
memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final-answer correctness and evidence-page access. This creates a
memory-quality blind spot: an
agent may reach the right page and answer correctly while leaving behind
memory too generic or incomplete to support answering once page context is removed. We introduce
memory-only answerability, a diagnostic that asks whether a reader can answer from the question and terminal working
memory alone. Building on this diagnostic, Answerable Working
Memory (AWM) treats terminal working
memory as an answerable evidence artifact, and AWM-GRPO incorporates this signal into the GRPO reward while preserving final-answer priority. Under GRPO, this reward assigns higher advantages to answer-correct trajectories whose terminal working
memory remains answerable. On MMLongBench-Doc, even when gold evidence pages are provided, 42.5% of correct answers still cannot be answered from terminal working
memory alone. AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on MMLongBench-Doc and LongDocURL and reduces the
memory-missing-correct rate by 2.7 points over answer-only GRPO.
Read the full abstract ▾
All authors Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, Rui Lu, Yuxiao Dong, Jie Tang, Evgeny Kharlamov
Computation and Language
From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation
Lukas Edman, Daryna Dementieva and Alexander Fraser
Large
language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate
speech detection, remains less clear. Prior studies comparing prompted LLMs with state-of-the-art encoder-based models (e.g., BERT variants (Roy et al., 2023; Dönmez et al., 2024)) have shown only marginal gains, suggesting that LLMs may not excel in hate
speech detection or mitigation. In this work, we revisit this question through the lens of instruction tuning. By thoroughly unifying 36 English hate
speech datasets spanning multiple labeling schemes, we fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate
speech mitigation. Our results demonstrate not only state-of-the-art performance on in-domain
benchmarks but also substantial improvements in cross-domain and cross-lingual generalization–areas where encoder-based specialist classifiers often struggle.
Read the full abstract ▾
Computation and Language
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
Guibin Zhang, Leo Lu, Fangzhou Xie et al. (+15 more)
Agent capability is not determined by the model alone. The
agent harness, encompassing
memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-
Agent, a harness intelligence model trained to synthesize task-adaptive
agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the
agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-
Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliable execution, and self-evolve by distilling performance signals from an expanding archive of prior harness configurations. Equipped with JIT-
Agent as a harness helper, DeepSeek-V4-Flash surpasses GPT-5.6 on DeepSearchQA (+9.1) and OdysseyBench (+4.3), while the already strong GLM-5.2 gains up to +20.2 points. Across controlled evaluations, JIT-
Agent-generated harnesses are performance-competitive with mature
agent runtimes such as OpenCode and Claude Code and consistently improve multi-scale model families of DeepSeek V4,
Mimo-V2.5, and Qwen3.6. To our knowledge, JIT-
Agent is the first model purpose-built for just-in-time harness generation, establishing harness intelligence as a trainable, transferable, and compounding dimension of
agent capability orthogonal to model scaling.
Read the full abstract ▾
All authors Guibin Zhang, Leo Lu, Fangzhou Xie, Kang Zhu, Junhao Wang, Zhifei Xie, Zhaochen Yu, Zihang Liu, Zhongxiang Sun, Qiankun Li, Yue Liao, Heng Chang, Xiaobin Hu, Qibing Ren, Wangchunshu Zhou, Chuanrui Hu, Yafeng Deng, Shuicheng Yan
Computation and Language
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
Lam So, Canhui Wu and Han Lin
Reasoning-oriented large
language models often achieve strong problem-solving performance by generating long chains of thought, but this behavior substantially increases inference cost and latency. In contrast, instruction-tuned models tend to answer more concisely, yet often lack comparable
reasoning ability. This accuracy-efficiency mismatch motivates a lightweight approach that combines the strengths of both models without full model retraining. In this paper, we propose GRIP (Granular Reward-guided Interpolation of Parameters), a reward-guided parameter interpolation framework for efficient
reasoning. Given a
reasoning model and an instruction model with identical architectures, GRIP assigns learnable interpolation ratios to individual modules and optimizes only these ratios while keeping both source models frozen. The interpolation ratios are trained with a reward signal that favors responses that are both correct and concise. Experiments show that GRIP achieves a better accuracy-efficiency trade-off than fixed or search-based merging baselines and further reveals module-wise fusion patterns associated with efficient
reasoning.
Read the full abstract ▾
Computation and Language
Cross-Dataset Stability of Expert-Informed Skill Prompting and Fine-Tuning for Chinese Metaphor Identification
Yufeng Wu and Meichun Liu
Metaphor-identification performance can change markedly across
datasets that differ in text distribution and annotation policy. We examine whether a fixed expert-informed procedure produces a more even cross-
dataset profile than task-specific parameter adaptation. Four prespecified conditions are compared for Chinese sentence-level metaphor identification: BERT fine-tuning (BERT-FT), QLoRA-based large
language model fine-tuning (LLM-FT), direct zero-shot LLM prompting (LLM-ZS), and zero-shot prompting with a frozen procedural Skill (Skill-ZS). The Skill operationalizes established criteria involving contextual meaning, basic meaning, contrast, and comparison. Evaluation covers CMRE Test and two external
datasets, CCIME and CMC. Fine-tuned scores are means over three seeds, whereas each zero-shot score comes from one deterministic configuration. Fine-tuning remains strongest on the native test set: BERT-FT reaches 91.76 Macro-F1. LLM-FT has the highest external mean (83.52), while Skill-ZS is close at 82.92 and has both the highest external floor (82.64) and the smallest observed range across all three
datasets (4.08 points). In the matched zero-shot comparison, adding the Skill reduces metaphorical predictions on every
dataset. This sharply lowers false positives on CCIME but increases false negatives on CMRE Test and CMC. The results position expert-informed Skill prompting as a complementary route to more even observed cross-
dataset performance, while fine-tuning retains its advantage in native-data accuracy. To our knowledge, this is the first study to compare an expert-informed procedural Skill with task-specific fine-tuning in the same cross-
dataset evaluation of Chinese sentence-level metaphor identification.
Read the full abstract ▾
Computation and Language
Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil et al. (+6 more)
Automatic
Speech Recognition (ASR) is typically evaluated using Word Error Rate (WER), which poorly reflects semantic similarity. While
embedding-based metrics correlate better with human judgments, the respective roles of encoder and decoder-based Large
Language Models (LLMs) remain underexplored. This paper presents a comparative study of both families for ASR evaluation. We analyze BERTScore and SemDist across different LLMs, layers, and pooling strategies, showing that both metrics can achieve strong correlation with human judgments when properly configured. For decoder models, we investigate generative LLMs in two settings: pairwise hypothesis selection via prompting and direct qualitative error classification. Our results show that encoder-based metrics remain highly competitive, while generative LLMs perform strongly in hypothesis comparison and improve the interpretability of ASR evaluation.
Read the full abstract ▾
All authors Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil, Sergio Burdisso, Petr Motlicek, Shiran Liu, Mickael Rouvier, Jane Wottawa, Richard Dufour
Computation and Language
Controllable Affective Generation via Latent Vector Steering
Xixian Yong, Siyuan Chang, Yingying Zhang et al. (+2 more)
Large
Language Models (LLMs) often produce emotionally flattened responses after
alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable affective generation via latent vector steering. EmoVec extracts
emotion-specific directions from paired neutral and
emotion-conditioned responses using contrastive activation addition, and further refines them through task-specific debiasing and principal subspace removal. During inference, these vectors are injected into the final residual stream with static or scenario-adaptive scaling, enabling continuous control over emotional intensity without updating model weights. Experiments across three LLMs and eight
emotions show that EmoVec consistently improves emotional salience while largely preserving semantic content, fluency, and coherence. Ablation studies and human evaluation further confirm the effectiveness of vector purification and adaptive scaling, establishing EmoVec as a practical inference-time method for affective control in deployed LLMs.
Read the full abstract ▾
All authors Xixian Yong, Siyuan Chang, Yingying Zhang, Xian Wu, Xiao Zhou
Computation and Language
EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports
Yu-Chien Tang, Yu-Hsiang Liu and An-Zi Yen
VLMs are increasingly positioned as daily assistants that perceive first-person environments, follow user dialogue, and decide how to help. Existing
egocentric benchmarks mainly evaluate visual understanding in isolation, leaving open whether models can arbitrate between visual evidence and user-provided
language when the two are helpful, irrelevant, or conflicting. We introduce EgoArgus, a human-annotated
dataset for evaluating
egocentric assistants on understanding and decision tasks in five dialogue-
video daily scenarios. Our results demonstrate that it is still challenging for current VLMs as reliable
egocentric assistants, which requires identifying which modality is trustworthy and deciding when intervention is warranted. Deeper analysis also shows that existing modality bias mitigation methods are quite restricted to enhance performance, providing insights to aid practioners into the deployment of current VLMs as daily assistants.
Read the full abstract ▾
Computation and Language
Virgil: Navigating Explainability for Transformer-based Language Models
Martino Ciaperoni, Sezer Kutluk, Benedetta Muscato et al. (+2 more)
Explainability for
transformer-based
language models is becoming crucial as these systems are deployed in high-stakes applications. As a result, the ecosystem of explainability tools is rapidly evolving, becoming richer, but also more fragmented and harder to navigate. To address this challenge, we present Virgil, an interactive system that lets practitioners and researchers, including non-experts, navigate explainability tools for
transformer language models. Supported by a curated knowledge base, the system enables users to discover and compare explainability tools within a unified interface.
Read the full abstract ▾
All authors Martino Ciaperoni, Sezer Kutluk, Benedetta Muscato, Marta Marchiori Manerba, Fosca Giannotti
Computation and Language · Cross-listed from cs.IR
When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory
Kazuki Nakayashiki
Provenance links keep the evidence behind an inherited belief reachable; an
agent with a verification budget must still choose which links to inspect. We study a consolidated
memory that states a decision constraint and whose source record has since been superseded by a record that withdraws it: provenance is immutable, the current record has changed, and the
memory is stale. In a controlled six-
memory scenario with a budget of two records, sixteen
language models rarely re-verified a constraint that read as settled: they inspected its provenance path in about one episode in five and, once the constraint had been superseded, produced stale-consistent decisions in 77.3%, 74.7% and 74.7% of episodes across a primary run, a replication and a held-out domain. Re-assigning one of the same two slots to the critical path removed most of them: +74.0, +72.7 and +61.3 points (positive in every model), +80.7 in a prospectively frozen interleaved replication with a repaired non-critical control, and +62.0 on a panel of 10 models from 9 organisations; a corrected re-run of the held-out scenario gave +73.3. The forced-critical policy uses experimenter knowledge of the critical path: it quantifies how much stale-decision risk the same budget can recover and is not a scheduler. Two further deposited experiments locate the failure and a remedy: in this store the constraint's path is selected in 17.0% of episodes at two slots and 88.7% at four of six (above uniform allocation), and at two slots a one-sentence, target-blind rule (prefer
memories that state a limit on a candidate direction) moved the
agent's own allocation onto the constraint's path and recovered the oracle contrast on decisions (+89.3 points) where that constraint limits the tempting action, while a content-free freshness cue did not materially redirect allocation and a content-matched control rule changed neither selection nor decisions.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference
Jiarui Hu, Zhiyuan Wen, Xiaoyun Liu et al. (+2 more)
Large
reasoning models often produce
reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes
reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its
entanglement with
reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general
reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-
reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public
benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces
reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter $\alpha$, enabling deployment-time adjustment to balance
token savings, accuracy, and generation stability.
Read the full abstract ▾
All authors Jiarui Hu, Zhiyuan Wen, Xiaoyun Liu, Jiaxing Shen, Yu Yang
Computation and Language
ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives
Jihao Zhu, Zhiwei Yang, Wenxiao Zhang et al. (+7 more)
Humanities and social science research requires close reading of long narrative materials such as novels, scripts, archives, and case reports, yet many users have limited access to costly proprietary long-context models. Compact, locally deployable
language models are a practical alternative, but directly feeding them an entire long context remains costly, hard to inspect, and prone to missing sparse evidence. We present ClueWeaver, an evidence-aware dual-
agent framework for long-narrative question answering with compact local models. A Finder identifies passages containing answer-critical clues through
retrieval-guided
segmentation, while an Interpreter derives the answer from the selected evidence, produces rationales with paragraph-ID citations, and applies an internal self-calibration pass for high-risk questions. Both
agents are optimized with reward-guided reinforcement learning: Finder rewards emphasize evidence retention and faithful paragraph-ID references, and Interpreter rewards emphasize correctness, grounding, and concise explanations. This decomposition makes evidence selection and
reasoning more inspectable than end-to-end prompting. Experiments across multiple long-context narrative question answering and claim verification settings show that ClueWeaver substantially improves local end-to-end
language models while providing evidence coverage and paragraph-referenced
reasoning traces. Code is available at https://github.com/Ameame1/ClueWeaver.
Read the full abstract ▾
All authors Jihao Zhu, Zhiwei Yang, Wenxiao Zhang, Junqian Zhao, Qi You, Fangqi Wang, Zheyuan Deng, Hanzhe Yang, Yu Liu, Jin B. Hong
Computation and Language
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Hongqiu Ni, Han Tian, Chi Zhang et al. (+2 more)
Prefix caching introduces a fundamental tradeoff in multi-
agent large
language model (LLM) serving: retaining a long system-
prompt key-value (KV) cache for an
agent accelerates future calls, yet it reduces the GPU
memory available for batching concurrent requests. In multi-stage workflows, existing schedulers tend to prioritize either immediate prefix locality or overall workflow progress. However, under a shared KV cache budget, optimizing either objective in isolation can prolong tasklevel job completion time (JCT) through downstream delays or frequent prefix replacement. To strike a balance, we here propose TOPAS, a Task-Oriented Prefix-Aware Scheduler that jointly decides which
agent prefixes to keep in the cache and which requests to schedule for execution. TOPAS scores candidate post-decision states by trading off the expected reduction in each task's longest remaining service path against the near-term benefit of downstream prefix reuse, accounting for the costs of prefix movement and preemption. A task-level aging mechanism is also incorporated to prevent starvation. We implement TOPAS within the SGLang framework and assess its performance on three synthetic DAGs and two MetaGPT software-development workflows. Compared with the best performing baseline for each workload and metric, TOPAS reduces the mean/p99 JCT by up to 39.8%/49.4% on the synthetic workloads, while lowering mean JCT by 9.8% on MetaGPT-SOP and mean/p99 JCT by 22.0%/26.6% on MetaGPT-TL.
Read the full abstract ▾
All authors Hongqiu Ni, Han Tian, Chi Zhang, Guopeng Li, Haisheng Tan
Computation and Language · Cross-listed from cs.IT
Serial Depth of Adaptive Parallel Sampling: Lower Bounds and Schedules for Masked Diffusion Models
Chuling Wen, Weijie Liang, Jian Lu et al. (+1 more)
Masked
diffusion models generate sequences in rounds, predicting several masked positions at once. Fewer rounds can accelerate decoding, but sampling positions independently can lose their dependence and bias the output. How few rounds suffice for a prescribed error budget? Beyond error bounds for individual decoding rules, we study sharp lower bounds on the number of rounds required for a given target distribution. We introduce serial depth: the minimum number of rounds, averaged over target sequences, needed to meet a forward Kullback-Leibler error budget. With exact conditional probabilities, we prove these bounds for all deterministic rules that may adapt to generated values and construct matching schedules for several distribution families, with depths ranging from $\log\log n$ to linear in sequence length $n$. To narrow the gap to trained models, we show that these lower bounds remain valid for learned predictors and give sufficient accuracy conditions on visited contexts for schedules to retain their error guarantees. On two pretrained models across five text domains, a schedule suggested by the theory achieves higher average log-likelihood on held-out text in five rounds than random selection in eight. A preregistered confirmation study certifies a relative improvement on independent dialogue data.
Read the full abstract ▾
All authors Chuling Wen, Weijie Liang, Jian Lu, Zhouchen Lin
Computation and Language · Cross-listed from cs.AI
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
Zhiyuan Li, Linyuan Gao, Xuechun Ding et al. (+3 more)
Reusable skill libraries allow large
language model (LLM)
agents to reuse procedural knowledge across tasks, but they also turn
memory access into a challenging
retrieval problem. Full-library prompting preserves coverage at high context cost, vector
retrieval returns compact neighborhoods but treats skills as independent text, and graph-based
retrieval can recover workflow context only when the edges that carry relevance are reliable. We propose CaSKG, a counterfactual-causal skill graph framework that calibrates procedural relations before
retrieval. CaSKG first builds a high-recall directed candidate graph from semantic, lexical, input/output, and structural evidence, with repair evidence and an optional LLM judge further refining candidate scores. It then applies direction-conditioned textual counterfactual probes that remove, substitute, and reorder skill pairs, aggregates the evidence with
Bayesian smoothing, and publishes a state-filtered weighted graph for task-conditioned expansion. The graph is constructed offline and used without changing the downstream
agent policy or task interface. Across six LLM backbones on ALFWorld ID-140 and ScienceWorld U211, CaSKG achieves the highest task score in all twelve combinations of model and
benchmark. Relative to Graph-of-Skills (GoS), it improves the six-model macro-average ScienceWorld score from 72.62 to 80.50 and ALFWorld success from 80.01% to 86.79%, while reducing mean environment steps on both
benchmarks. Qualitative and ablation analyses further show that calibrated edges help
retrieval preserve prerequisites, state-changing actions, verification routines, and final completion steps. These results position edge-confidence calibration as an effective route to compact and executable skill
retrieval at scale\footnote{Code is available at: https://github.com/ZhiyuanLi218/Caskg }.
Read the full abstract ▾
All authors Zhiyuan Li, Linyuan Gao, Xuechun Ding, Hongwei Chen, Yuan Wu, Yi Chang
Computation and Language · Cross-listed from cs.LG
A Storage-Retrieval Gap in Parametric Knowledge Graph Memory
Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov et al. (+1 more)
Graph
retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring
token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA
dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph,
embedding-based and weight-space geometry
retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($\rho = +0.329$) but not with functional retrievability. We quantify the byte and context-
token costs against graph
retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph
memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.
Read the full abstract ▾
All authors Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp
Computation and Language
ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains
Jinpu Jiang, Xuan Wu, Wenhao Song et al. (+6 more)
Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for Question Answering (QA) by integrating external information into Large
Language Models (LLMs). However, false, inaccurate, and misleading information in news and social media poses a serious challenge to real-world RAG systems, especially in multi-hop QA, where complex multi-step
reasoning can be misled by even a single deceptive misinformation segment in the retrieved documents. Existing approaches mainly rely on implicit
alignment or explicit regulation, but their limited ability to assess fine-grained information reliability makes them vulnerable to deceptive misinformation that is semantically relevant to the question yet factually incorrect, leading to erroneous answers. To address this limitation, we propose ReliableRAG, which, to the best of our knowledge, is the first reliability-driven framework that mitigates deceptive misinformation in multi-hop QA through fine-grained evaluation of individual triples. ReliableRAG first extracts information segments from source documents and represents them as structured triples. It then quantifies triple reliability by combining query-triple semantic relevance with triple credibility, retaining only the top-$K$ reliable and non-redundant triples. Based on these refined triples, ReliableRAG autoregressively constructs robust
reasoning chains to consolidate trustworthy evidence and filter deceptive misinformation, producing accurate answers faithful to reliable information. Experiments on three multi-hop QA
datasets show that ReliableRAG outperforms existing methods, substantially improving the factual reliability and robustness of RAG systems under deceptive misinformation injection.
Read the full abstract ▾
All authors Jinpu Jiang, Xuan Wu, Wenhao Song, Bo Yang, You Zhou, Hongwei Ge, Heow Pueh Lee, Yanchun Liang, Chunguo Wu
Computation and Language · Cross-listed from cs.AI
PonsRAG: A Pons-Inspired RAG Bridging Cognitive Islands for Coordinated Long Narrative Reasoning
Rongchen Zhao, Yu Chen, Juyuan Wang et al. (+4 more)
Long Narrative
Reasoning is an essential capability for processing and
reasoning over complex narratives. While
retrieval-augmented generation provides a promising framework, existing methods still face two critical challenges: cognitive islanding and cross-layer evidence disconnection. To address these issues, we propose PonsRAG, a coordinated RAG framework inspired by the biological pons. PonsRAG consists of two key components: Triple-Layer Indexing, which organizes documents into a connected knowledge structure to bridge cognitive islands, and Coordinated
Reasoning, which retrieves evidence across distinct layers and integrates cross-layer information into a unified context. We evaluate PonsRAG on four long-context narrative
benchmarks, and experimental results show that it outperforms the strongest baseline, achieving a 11.56% relative improvement in average accuracy on multi-choice tasks.
Read the full abstract ▾
All authors Rongchen Zhao, Yu Chen, Juyuan Wang, Zhouting Mo, Jianxing Yu, Wenqing Chen, Jingping Liu
Computation and Language
VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Text
Trieu Hai Nguyen and Van-Dung Hoang
Distinguishing AI-generated content from human-written text in Vietnamese remains difficult due to the limited availability of dedicated open-source tools. In this paper, we present VietAIDetector, an open-source software tool designed for practical Vietnamese AI-generated text detection. Operating on a zero-shot scoring engine derived from VietBinoculars~\cite{nguyen2025vietbinoculars}, the software requires no domain-specific retraining and accepts raw text, DOCX files, native PDF documents, and scanned PDFs via integrated optical character recognition. To handle long documents beyond conventional input lengths, the tool combines sliding-window chunking with document-level score aggregation. VietAIDetector offers an interactive Gradio web interface with configurable operating thresholds, including Youden's J statistic, the Closest Point approach, and a low false-positive rate criterion, with results exportable as downloadable PDF reports. Empirical validations confirm that this chunking pipeline mitigates threshold drift on multi-thousand-
token documents while maintaining practical runtime throughput under GPU deployment. The tool is publicly available at https://github.com/trieuntu/VietAIDetector.
Read the full abstract ▾
Computation and Language
MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
Jiaxin Yuan, Udari Madhushani Sehwag, Connor Martinez Lockhart et al. (+13 more)
Formal
theorem proving enables machine-verifiable evaluation of mathematical
reasoning, yet existing
benchmarks often emphasize aggregate proof accuracy, concentrate on a narrow range of mathematics, and provide limited evidence of robustness to equivalent reformulations. We introduce MathAdv, a diagnostic
benchmark spanning 13 domains across undergraduate- and graduate-level mathematics. Alongside Lean 4
theorem proving, MathAdv provides up to three auxiliary tasks: multiple-choice questions that probe mathematical knowledge, fill-in-the-blank problems that isolate informal
reasoning, and expert-crafted transformations that test robustness to problem presentation. Our evaluation of contemporary
theorem provers yields four findings: formalization remains a major bottleneck; performance varies substantially across mathematical domains; natural-
language guidance helps general-purpose LLMs but can hinder proof-specialized models; and mathematically equivalent reformulations expose substantial robustness limitations. Together, these results show how component-wise evaluation can reveal model capabilities and failure modes that aggregate
theorem-proving accuracy obscures. The
dataset and evaluation scripts are available at https://github.com/margotyjx/MathAdv.git.
Read the full abstract ▾
All authors Jiaxin Yuan, Udari Madhushani Sehwag, Connor Martinez Lockhart, Xiaoyu Liu, Shayan Shabihi, Jiaqi Wang, Chenghao Deng, Xiayimei Han, Vlassis Mastrantonis, Dmitrii Gudin, Shaopeng Zhu, Abdirisak Mohamed, Bilal Aytekin, Jiewen Lang, Zezheng Song, Furong Huang
Computation and Language
DCGC: Draft-Conditioned Global Correction for Complex Reasoning with Masked Diffusion Models
Minhae Oh, Nakyung Lee and Jungwoo Lee
Correcting flawed
reasoning traces remains a significant challenge for Large
Language Models (LLMs), whose autoregressive generation can propagate early mistakes into subsequent
reasoning. We introduce DCGC, a Masked
Diffusion Model (MDM) framework for global correction that uses an imperfect solution draft from an upstream solver as auxiliary context. DCGC combines task-specific Supervised Fine-Tuning (SFT) with a novel inference-time mechanism called Dynamic Dual-CFG. This mechanism separates problem-only and joint problem-draft branches and scales the draft-conditioned residual using a relative confidence gap. Across math, code, and knowledge
reasoning benchmarks, DCGC outperforms standard sampling and simpler CFG variants, with additional results suggesting transfer to different
diffusion backbones. In test-time setting where ground-truth failure labels are unavailable, DCGC improves full test set accuracy by correcting low-consensus upstream outputs, highlighting its utility as a verifier-free global correction module for difficult
reasoning instances.
Read the full abstract ▾
Computation and Language
OmniPhys: A Unified Multimodal Benchmark for Physics Understanding and Generation from Chinese Educational Corpora
Hao Chen, Yumin Lin, Nadila Yushanjiang et al. (+2 more)
Multimodal Large
Language Models (MLLMs) have demonstrated strong abilities in solving diverse visual and textual
reasoning tasks. However, their development in the physics domain is significantly hindered by the lack of a comprehensive
benchmark. To fill this gap, we introduce OmniPhys, a large-scale
benchmark for
multimodal physics understanding and
reasoning, covering middle school through university-level problems from Chinese Educational Corpora. OmniPhys consists of 15,246 questions and 19,850 images, accompanied by detailed annotations that support fine-grained analysis of
reasoning processes and knowledge usage. Beyond conventional evaluation, OmniPhys is a
benchmark that systematically evaluates
multimodal outputs in the physics domain, including models' ability to generate structured physics diagrams, which constitute a fundamental component of authentic physics problem solving. Extensive evaluations reveal critical gaps in the capabilities of current MLLMs, especially in complex
reasoning and visual generation. To address this, we release OmniPhys to serve as a foundational resource for advancing
multimodal intelligence in physics and scientific domains. Codes and data are available at https://github.com/ECNU-RAIL/OmniPhys-EMNLP2026.
Read the full abstract ▾
All authors Hao Chen, Yumin Lin, Nadila Yushanjiang, Xin Lin, Min Zhang
Computation and Language
Adaptive Triggering for Bias Correction in LLM Reasoning
Nayoung Kim, Mickey Mancenido and Huan Liu
Chain-of-thought prompting can expose and amplify demographic stereotypes within an LLM's intermediate
reasoning and create a failure mode that final-answer debiasing alone cannot address. Mitigating such bias during generation presents a fundamental timing problem: intervening too late allows biased
reasoning to propagate, while unnecessarily intervening can disrupt otherwise correct
reasoning. Existing approaches largely avoid this decision by either evaluating completed
reasoning chains post hoc or intervening at predetermined steps, leaving open when a developing
reasoning trajectory provides sufficient evidence to warrant correction. We formulate this decision as an online change-point detection problem. A per-step bias signal updates a CUSUM statistic and a targeted correction is injected only when accumulated evidence crosses a detector-specific threshold calibrated on held-out data. We instantiate the framework with a white-box signal derived from next-
token probabilities and a black-box signal obtained from an LLM judge, enabling deployment with both open-weight and hosted models. On gpt-4o-mini adaptive black-box triggering recovers most of the disambiguated-context accuracy lost under fixed-interval intervention while requiring substantially fewer interventions. That result holds even with an independent judge. Across six open-weight models, the white-box signal improves ambiguous-item accuracy on all six but reduces disambiguated-item accuracy on five because it cannot distinguish unsupported stereotype reliance from correct, stereotype-congruent evidence.
Read the full abstract ▾