Computation and Language · Cross-listed from cs.CV
Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans
Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani et al. (+2 more)
Human visual search is serial: the fovea must land on a candidate to confirm it, and those landings form a scanpath. Whether
multimodal large
language models (MLLMs), given the same foveated input, search as humans do bears on their use as models of human
vision and on
attention-
alignment scores. We compare three general-purpose MLLMs with human eye-movement scanpaths on goal-directed search (COCO-Search18), driving each model fixation by fixation through an identical, human-matched foveated view and assessing it along three axes: the decision of target presence, the efficiency of reaching the target, and the gaze process itself. The axes dissociate. On the decision and on target acquisition the models match or exceed humans, detecting present targets near ceiling and reaching them on the first saccade more often than people do. The gaze process is not human. Under the human-matched condition, all three share one signature: low-
entropy, large-amplitude, self-consistent scanpaths that agree with themselves far more closely than two humans agree with each other. That is consistent with a single-pass, non-serial architecture rather than a limit of acuity. Matched retinal input reproduces where humans look but not how the looking unfolds in time, and no degradation regime recovers human-like search at human-like success. The gap sits on a process axis that answer-
alignment and saliency metrics do not measure. Because they miss it, such metrics cannot certify human-like
vision, and zero-shot models suit outcome and spatial questions but not temporal, process-level ones.
Read the full abstract ▾
All authors Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Ulas Bagci, Alessandro Bruno
Computation and Language · Cross-listed from cs.HC
Computational KJ-Ho: An Analyst-Bias-Free Insight Extraction Framework from Large-Scale Qualitative Data Using Domain-Specialized LLMs
Kasumi Ban
The qualitative research methodologies that underpin consumer-insight generation - the KJ method, Grounded Theory, and Thematic Analysis - share a structural constraint: the cognitive processing capacity of the human analyst. Replication research further shows that conclusions vary substantially across analysts analyzing identical data (analyst bias). This paper proposes Computational KJ-Ho (the Kawakita Jiro method), a theoretical framework that computationally realizes the KJ method's epistemology - letting structure emerge from the data itself without imposing the analyst's preconceptions - an orientation we term "analyst-bias-free." The framework employs a domain-specialized LLM built through continued pre-training (CPT) on a marketing-research corpus and supervised fine-tuning (SFT) on expert-curated insight pairs, organized as a three-layer architecture: data structuring, insight extraction, and strategy generation. Two preliminary studies in the Japanese marketing context support the necessity of CPT-based domain specialization. The paper makes five contributions: (1) a theoretical integration of the KJ method, Grounded Theory, and Peircean abduction into a single epistemological commitment of data-driven explanation generation; (2) a three-layer architecture leveraging domain-specialized
embeddings for cross-interview analysis; (3) two novel evaluation metrics, InsightExtraction-F1 and MarketingQA; (4) explicit engagement with the WEIRD problem, centering a non-Western methodology; and (5) five practice-derived problem formulations from nearly three decades of marketing-research practice, translated into design requirements. The human analyst retains a supervisory role. This is a concept paper presented ahead of empirical validation.
Read the full abstract ▾
Computation and Language
D2-ScaleAgent: Dual-Dimensional Scaling for Long Document Understanding
Hao Zhang, Longrong Yang, Lunhao Duan et al. (+3 more)
Multi-modal
retrieval-augmented generation (RAG) is a key technique for visually rich long document understanding. Existing multi-modal RAG methods are progressively advancing toward multi-
agent systems: they first retrieve relevant pages based on a query, and then iteratively understand information within those pages. However, these methods typically rely on fixed workflows and lack the ability to dynamically scale computation at test time, often leading to insufficient evidence. To address this, we propose D2-ScaleAgent, an agentic framework that introduces a dual-dimensional scaling paradigm for
retrieval and
reasoning. The core of D2-ScaleAgent is a Verifier
agent-driven dynamic routing loop based on the intrinsic difficulty of the query, centered around a continuously updated evidence bank that serves as the
agent's dynamic working
memory: when
retrieval needs to be expanded, the
agent routes outward (
retrieval scaling), decomposing the query into attributes and performing parallel page
retrieval, followed by adaptive
pruning to ensure comprehensive evidence coverage. When fine-grained
reasoning is required, the
agent routes inward (
reasoning scaling), dynamically selecting sub-
agents with varying granularity and count to extract evidence from pages. Finally, D2-ScaleAgent achieves logical closure over the evidence chain. Extensive experiments demonstrate that D2-ScaleAgent is effective on long and visually rich document
benchmarks like MMLongBench-Doc, LongDocURL, etc.
Read the full abstract ▾
All authors Hao Zhang, Longrong Yang, Lunhao Duan, Ziyang Wang, Qing-Guo Chen, Shanshan Zhao
Computation and Language
Counting Documents Is Not Counting Text: Unit Bias in Web-PDF Corpus Statistics
Luca Foppiano
PDF corpora advertise their size in
tokens, but every rate they publish (coverage, OCR routing, re-fetch recovery,
language mix) is computed per document, and none decomposes its
token total. Because PDF length is extremely skewed, the two units can describe the same corpus very differently. We ask how the headline statistics of a web-PDF corpus change when each document is weighted by the text it contributes rather than counted once. We used CC-MAIN-2021-31-PDF-UNTRUNCATED (7.9M Common Crawl PDFs, 32.6B
tokens), the one public corpus that pairs the fragments Common Crawl stored with the re-fetched originals. Text mass is highly concentrated: 3.02% of text-bearing documents hold half the
tokens (Gini 0.807). The clearest consequence is Common Crawl's payload cap, which truncated 23.06% of these documents but 63.08% of their text. Reconstructing the truncated fragments and extracting both versions, two widely used text-layer parsers recover only 1.4% and 11.4% of that exposed text, so roughly 55-62% of the corpus's text is unrecoverable from the crawl by such pipelines; under the 5MiB cap adopted in March 2025, 30.19% of
tokens would still be exposed. We recommend that corpus statistics be reported in both units, documents and
tokens.
Read the full abstract ▾
Computation and Language
Mint-Agent: Introducing Finance-Native Agentic Foundation Models
Mint-Agent Team, Kun Wang, Gavin Zhang et al. (+7 more)
Financial
agents must do more than recall domain knowledge: they must be both reliable, executing precise operations over grounded evidence, and executive, sustaining long-horizon research whose conclusions remain auditable. We present Mint-
Agent, a family of finance-native agentic models designed around these two scales of financial intelligence. Mint-
Agent is built upon three pillars: data, harness, and
algorithm. Our data engine constructs clean, specialized tasks for atomic financial capabilities and long-horizon agentic execution from real-world financial sources. MintHarness enables stable interaction with open-ended environments and maintains auditable evidence trails across extended research trajectories. Our training recipe combines SFT, critical-step OPD, and RLVR to develop separate financial
reasoning and agentic execution experts, which are then unified through model merging and multi-teacher on-policy
distillation into compact, general-purpose financial
agents. This pipeline yields two flagship models, Mint-Cu (9B) and Mint-Ag (27B). Across professional financial
benchmarks, our models demonstrate two defining strengths: (1) Reliability: Mint-Ag achieves 98.33% on RFC-Bench, surpassing GPT-5.6-Sol and Claude-Opus-4.8 by 3.66 and 3.00 points; and (2) Executability: Mint-Cu reaches 69.86% on FinSearchComp T2, outperforming
Agents-A1-35B and Nex-N2-mini by 22.83 and 12.78 points, while Mint-Ag achieves 76.00% and 60.49% on FinanceAgentBench v1.1 and v2, respectively. These results establish a path toward trustworthy financial intelligence in which domain expertise, long-horizon execution, and auditable evidence are jointly engineered as a unified foundation for frontier agentic models.
Read the full abstract ▾
All authors Mint-Agent Team, Kun Wang, Gavin Zhang, Yaze Geng, Lei Tang, Yaoyang Yi, Zonghan Wu, Yifan Hu, Qingsong Wen, Yilei Shao
Computation and Language
Unadapted Multilingual ASR on a Garrusi Kurdish Evaluation Set: A Common-Reference Staged Normalization Analysis
Hiwa Asadpour
Evaluating
speech recognition for a Kurdish variety written in a Latin field orthography, using a model that outputs Arabic script, creates a measurement problem before a modelling one: direct scoring treats writing-system differences as recognition errors. Jointly normalizing reference and hypothesis avoids this, but also changes reference tokenization, mixing agreement gains with a change in the scoring denominator. I evaluate MMS-1B-all with the Central Kurdish (ckb) adapter, used as released without adaptation, on 1,722 Garrusi questionnaire segments from five speakers (9,763 reference word
tokens; 117.9 minutes). I use a common-reference design: the reference is folded once and fixed at 9,763
tokens, while only the hypothesis representation varies. The raw Arabic-script hypothesis scores 111.70% WER and 100.92% CER, with zero exact word matches. Latin transliteration gives 102.36% WER and 57.89% CER; folding it into the reference's reduced orthography gives 97.85% and 51.20%. Thus RAW-to-FOLDED reduces measured WER by 13.85 points and CER by 49.72 points; folding alone accounts for 4.51 and 6.69 points. Substantial error remains: 14.53% of reference
tokens are exact matches, edits are substitution-dominated, and per-segment WER is higher for shorter segments. A Southern Kurdish fine-tuned system (aranemini/southern-kurdish-asr), scored under the same design, performs worse on every speaker (1,703 segments), with 109.56% WER and 55.85% CER. However, 12,330 output characters fall outside the folding table, so these rates must be recomputed against the corrected fixed reference. The MMS output also contains 613 unconverted or unmapped characters, showing that part of the residual error reflects scoring-pipeline limits rather than recognition alone. I will release the fixed reference and segment-level results, subject to source-corpus sharing terms, to support independent checking.
Read the full abstract ▾
Computation and Language
HalluTracer: Pre-Decoding Truthfulness Prediction via Depth-Averaged Probe-Logit
Zhihao Guo, Zonghan Wu, Huan Huo et al. (+6 more)
Internal-state probes enable truthfulness prediction before a large
language model generates an answer. When detectors change both the layers they read and the rules used to combine them, the source of improved prediction becomes difficult to identify. We separate these choices and find that retaining more layers improves prediction even under fixed equal weighting. An exact Fisher-ratio decomposition explains why the additional benefit of linear reweighting is limited on these probe scores: information inlayer-wise differences largely overlaps with that captured by the depth mean. Estimating additional weights can then offset this small benefit when the data used to fit them are limited. These findings motivate our proposed method HalluTracer, which averages layer-wise probe logits to predict truthfulness before decoding. Across six models and four
benchmarks, including TruthfulQA, HalluTracer achieves the highest area under the receiver operating characteristic curve (AUROC) in 23 of 24 model–
benchmark pairs among the compared methods. The results support using evidence from across the network without requiring a correspondingly more flexible aggregation rule, clarifying the distinct roles of layer selection and weighting in pre-decoding detection.
Read the full abstract ▾
All authors Zhihao Guo, Zonghan Wu, Huan Huo, DaYong Ye, Junwei Zhang, Weiran Yao, Zhiwei Liu, Qingsong Wen, Yilei Shao
Computation and Language
Architecture-Dependent Causal Transfer of Activation States Across Large Language Models
Fernando Cardenas Piepereit
Direct communication between AI systems relies on natural
language as an intermediate layer, incurring encoding/decoding overhead,
token cost, and latency. We ask whether internal activation states can instead be transferred causally between different large
language model (LLM) architectures via a learned projection, evaluated at three levels: representational similarity, cross-model
retrieval from projected states, and end-to-end causal transfer via activation injection during generation. Using four architecturally diverse open-weight models (Qwen2-0.5B, Phi-3-mini, Mistral-7B, FLAN-T5-base), we find that representational
alignment in trained models exceeds a random-initialization null baseline and is best captured by a rank-based metric (mutual k-nearest-neighbour
alignment), more robust to activation-magnitude outliers than centered kernel
alignment (CKA) or Procrustes analysis. A learned projection network retrieves the correct target-model representation from a held-out set well above chance for the three causal decoder-only model pairs (45-50% top-1 accuracy vs. 5% chance) but at chance level for the encoder-based FLAN-T5. Injecting projected activations into a target model during generation produces a statistically significant, pre-registered causal effect on
retrieval-based output similarity for only one of the three decoder-only pairs (Qwen2-0.5B to Phi-3-mini: 23.3% vs. 0.0% under negative control, p=0.047, FDR-corrected); the two pairs targeting Mistral-7B show no such effect despite comparable representational
alignment at the hidden-state level. We interpret these results as evidence for causal transfer of the representational vehicle, not of meaning, and conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
Read the full abstract ▾
Computation and Language
IndicQE-APE: A Consolidated Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages
Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare et al. (+14 more)
Indic quality estimation (QE) and automatic post-editing (APE) data is spread across separate releases, so no single resource supports training and evaluation across tasks and
language pairs on one footing. We consolidate the WMT 2020-2024 shared-task lineage with an extended English-Malayalam resource into IndicQE-APE: $126{,}754$ instances over nine directional pairs, with up to four label types aligned on the same segment, a direct assessment, a human post-edit, word-level tags and an error explanation, and a test set stratified over four difficulty axes. We
benchmark six prompted LLMs and three COMET metrics on segment-level QE, and three systems on APE. Two of the axes are defined partly on direct assessment and select a compressed slice of it. Segments whose segment-level and
token-level signals disagree are ranked below equally scored segments of the same
language. Four-shot prompting costs every model at or below $3.4$B both correlation and output-format compliance. Unedited MT beats every APE system we run on three of the four pairs. The
benchmark (https://huggingface.co/
datasets/surrey-nlp/IndicQE-APE) and code (https://github.com/surrey-nlp/IndicQE-APE) are released.
Read the full abstract ▾
All authors Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare, Daria Sokova, Shenbin Qian, Girish Koushik, Tharindu Ranasinghe, Constantin Orăsan, Chrysoula Zerva, Ricardo Rei, Frédéric Blain, André F. T. Martins, Marco Turchi, Matteo Negri, Anoop Kunchukuttan, Mitesh M. Khapra, Pushpak Bhattacharyya
Computation and Language
Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning
Changhui Sun, Lanbo Liu, Hang Lei et al. (+11 more)
On-policy
distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories. This approach has achieved strong empirical gains and can often surpass conventional off-policy
distillation with substantially less data. However, standard
token-level OPD can provide only fragmented corrections along an erroneous student trajectory and cannot unfold a complete and correct repair path. Motivated by this limitation, we propose Step-Level On-Policy
Distillation (SOPD), which combines the long-horizon correction of supervised fine-tuning (SFT) with the on-policy advantage of OPD to provide step-level supervision over complete student-generated trajectories. We show that, at different limits of step length, SOPD reduces to SFT or approximates OPD. Compared with SFT, the teacher responses in SOPD are conditioned on student trajectories and therefore align more closely with student-visited states; compared with OPD, SOPD provides longer-horizon corrections rather than fragmented
token-level guidance. Across both
reasoning and
agent tasks, SOPD substantially outperforms conventional SFT and OPD. For example, on ALFWorld, SOPD improves the average success rate by 13.4 points over Vanilla OPD. We hope this work offers a new perspective for future research on
distillation methods.
Read the full abstract ▾
All authors Changhui Sun, Lanbo Liu, Hang Lei, Tong Ling, Jiahang Xie, Zhiyong Zheng, Yujia Wang, Hao Liu, Feng Xiao, Lu Liu, Yanlong Du, Zifeng Cheng, Ziwei Jiang, Qing Gu
Computation and Language · Cross-listed from cs.CV
Ao Shen, Yongheng Zhang, Yinghui Li et al. (+3 more)
Large
Multimodal Models (LMMs) for
video reasoning have long been hindered by the high computational cost of processing vast amounts of visual information. This dilemma motivates the transfer of the
reasoning capabilities of large models to smaller, more efficient ones. On-Policy
Distillation (OPD) offers a promising solution by matching output-
token distributions along student-generated trajectories. However,
video reasoning often depends on evidence accumulated across multiple frames. In this context, output-level supervision only captures information expressed through
token predictions and does not directly constrain the latent representations formed during
reasoning. To address this limitation, we propose Latent-OPD, which augments OPD with trajectory-level latent
distillation. Specifically, our method focuses on the position at the end of each trajectory, where hidden states effectively summarize the accumulated visual evidence and
reasoning context. Furthermore, we introduce a progressive teacher-lookahead strategy, which aligns middle-to-late student layers with increasingly deeper teacher layers. Experiments on six
video reasoning benchmarks show that Latent-OPD consistently outperforms output-only OPD. Notably, the improvements are particularly pronounced in scenarios with limited frames, long
videos, or tasks requiring complex evidence aggregation. These results establish Latent-OPD as a highly effective approach to frame-efficient
video reasoning.
Read the full abstract ▾
All authors Ao Shen, Yongheng Zhang, Yinghui Li, Manning Wang, Di Yin, Xing Sun
Computation and Language
FTA-Mem: Fact-Time-Affect Anchored Memory for Low-Density Long-Term Dialogue
Chang Liu, Junyi Zhao, Shuyi Zhang et al. (+4 more)
Long-term emotional-support
agents require
memory mechanisms for personalized understanding across sessions. However, emotional-support dialogue is often low-density: turns are incomplete, evidence is scattered, and user states evolve over time. Existing
memory methods usually rely on fixed units, such as turn-level notes or session summaries, which may lose details or introduce redundant noise. We propose FTA-Mem, a structured
memory framework for low-density long-term dialogue. FTA-Mem uses Boundary-preserving Window
Segmentation (BWS) to form coherent situation fragments, and constructs Fact-Time-Affect
Memory Units (FTA Units) that jointly encode factual content, temporal grounding, and affective context. Retrieved units are then synthesized into structured context for answer generation. Experiments on ES-MemEval and LoCoMo show that FTA-Mem improves overall long-term
memory question answering across
benchmarks with different information-density characteristics. On ES-MemEval, FTA-Mem achieves 0.3871 F1 and 0.6668 BERTScore. Further analysis shows that situation-level FTA construction better balances evidence preservation and construction cost than coarse session-level or overly fine-grained turn-pair construction, providing an effective granularity trade-off for long-term dialogue
memory.
Read the full abstract ▾
All authors Chang Liu, Junyi Zhao, Shuyi Zhang, Changsheng Ma, Yongfeng Tao, Minqiang Yang, Bin Hu
Computation and Language
Executable Code Knowledge: Code as a Native, Validation-Carrying Knowledge Representation for AI Coding Agents
Xueping Gao
AI coding
agents need more than relevant snippets: they need business semantics, validation evidence, relations, and assurance that their context is current. Existing systems usually infer or externalize this knowledge through
retrieval, summaries, graphs, rules, or reverse specifications. We investigate a complementary representation in which selected code units directly carry
agent-usable knowledge. We introduce Executable Code Knowledge (ECK) and define an Executable Code Knowledge Unit (ECKU) as a source-bound object combining stable identity, semantics, executable behavior, contracts, evidence, relations, provenance, validation state, and a query interface. Our Python prototype supports code-local authoring, manifest export, evidence execution, exact changed-line impact, freshness checking, and
agent-facing projections. Across three real Python repositories and 26 controlled patch tasks, direct ECK provides executable test coverage for 11/11 evidence-bearing tasks and exact selectors for 9/11; hiding declared evidence reduces exact recovery to 1/11 (paired exact McNemar p=0.0078). ECK-derived rules recover 11/11 exact selectors, showing that rules are effective delivery artifacts while ECK supplies source binding, validation state, impact, and freshness. Exact changed-line impact matches independently authored labels on all 26 patches (12 unit links; precision, recall, and F1 all 1.000). AST-bounded fingerprints classify 50 positive changes and 17 unrelated same-file controls correctly, whereas static rules snapshots detect none of the 50 stale cases. Model-backed patch-review and cross-layer studies measure projection fidelity rather than independent impact discovery. These results support a hybrid architecture:
retrieval for coverage, ECK for source and evidence governance, and projections for delivery.
Read the full abstract ▾
Computation and Language
Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?
Kristina Šekrst and Ana Kovačić
While various organizations now actively encourage LLM use in classrooms, we still lack rigorous, systematic evaluations of how well these models actually perform the fundamental tasks of
language pedagogy. This paper examines whether state-of-the-art LLMs can deliver the kind of corrective feedback and methodological explanations that
language learners need. The study tests multiple large
language models on their ability to identify, correct, and explain common learner mistakes in English, by systematically varying model parameters to investigate how these technical adjustments affect output quality, pedagogical clarity, and consistency, along with using
retrieval-augmented generation to query methodological data. The evaluation employs automated metrics (GLEU, BERTScore) but also human expert judgments to capture dimensions that purely computational measures miss: linguistic nuance, cultural sensitivity, and instructional appropriateness. While models demonstrate impressive surface-level correction abilities, their explanations often lack the terminological and domain knowledge that effective
language teaching requires, suggesting that current enthusiasm for AI-assisted
language learning may be outpacing our understanding of these systems' actual pedagogical competence.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.HC
PolyDebate: A Game-Orchestrated Multimodal System for Debate Skills Practice and Evaluation
Jianing Yin, Weng Pan Kuan, Xiaoyun Liu et al. (+4 more)
Debate is a structured form of persuasive communication that trains argument construction, rebuttal, oral delivery, and audience awareness. These skills are valued in education,
language learning, and professional communication. Recent AI debate systems and LLM-based judges have advanced argument generation and debate evaluation, but most remain text-centered and rarely support learners through a complete
multimodal practice experience. We introduce PolyDebate, a game-orchestrated
multimodal system for English debate practice and evaluation. PolyDebate guides learners through staged one-on-one (1v1) debates with an AI opponent, while skill cards, props, and coins make persuasive strategies explicit and turn practice into a game-like interaction. During each session, the system captures learner
speech and visual delivery evidence, generates context-aware opponent responses, and produces rubric-informed stage-level and overall feedback. PolyDebate is available as both an immersive Unity 3D game version and a web platform version that share the same workflow and evaluation services. Four studies covering AI opponent quality, evaluation coverage, AI judge feedback, and user perception show that PolyDebate brings debate interaction, gamified scaffolding,
multimodal assessment, and structured feedback together in a practical workflow for debate skills practice. The demonstration
video is available at https://youtu.be/mHwBG1_8Ebk.
Read the full abstract ▾
All authors Jianing Yin, Weng Pan Kuan, Xiaoyun Liu, Zhiyuan Wen, Yuxuan Li, Milos Stojmenovic, Jiannong Cao
Computation and Language
Domain-Agnostic Neural Topic Modeling with Contextual Token-Level Semantic Graph Representation
Seung-Won Seo, Won Ik Cho and Yongmin Yoo
Recent advances in neural topic models with pre-trained
language models (PLMs) have achieved strong performance by leveraging general-domain pre-training, yet their topic interpretability often degrades on specialized corpora. This limitation primarily stems from the geometry of the
embedding space, where domain-specific terms unseen during pre-training collapse into an indistinguishable region, and neither domain-specific re-training, word-level graph enrichment, nor parameter-efficient fine-tuning can restructure this space without inheriting the capacity ceiling of the underlying encoder. Our key insight is that a learnable graph layer operating on
token-level PLM
embeddings can acquire corpus-specific semantic structure that the frozen encoder lacks, because
token-level graphs preserve document-local context that word-level representations discard and joint optimization with the topic objective reshapes
embedding geometry directly from target-domain evidence. We instantiate this insight as DARTopic, a domain-agnostic framework that constructs
token-level semantic graphs from frozen PLM
embeddings and jointly trains a GNN encoder with topic inference. Across three
benchmarks spanning general, biomedical, and legal domains, DARTopic consistently outperforms strong baselines in topic coherence and document clus- tering without any encoder fine-tuning, while demonstrating robustness to PLM choice and favorable runtime efficiency over fine-tuning based alternatives.
Read the full abstract ▾
Computation and Language
STAIR: Semantic-Temporal Automaton for Interpretable Reasoning in Temporal Question Answering
Xinlong Dai, Jinchuan Zhang, Lei Gao et al. (+3 more)
By leveraging large-scale pretraining, LLMs can interpret diverse temporal expressions and question formulations without task-specific training. However, existing
prompt-based neuro-symbolic systems continue to rely on LLMs for both semantic interpretation and exact temporal inference. Consequently, discrete decisions regarding intervals, time anchors, and ordered states remain vulnerable to probabilistic errors and difficult to verify. We present STAIR, a Semantic-Temporal Automaton for Interpretable
Reasoning. STAIR separates semantic interpretation from precise temporal inference: an answer-free LLM adapter maps complex question formulations to normalized temporal intents, while a deterministic temporal automaton with finite control and guarded transitions executes the corresponding policies over canonicalized evidence. Following a rule-first design, STAIR resolves standard questions without invoking an LLM and applies semantic adaptation only when the rule path fails to produce an executable intent. This approach reduces free-form
reasoning, making temporal decisions verifiable and interpretable. Specifically, guarded execution supports precise point-time containment and before/after selection, while semantic adaptation handles non-exact intervals and time-anchored queries. Across the TimeQA-Easy, TimeQA-Hard, TempReason-L2, and TempReason-L3
datasets, STAIR consistently outperforms strong baselines in the TQA task using matched model settings, achieving average F1 improvements of 16.57% and 3.10% when utilizing the Qwen2.5-7B and GPT-4o-mini models, respectively. Furthermore, ablations and diagnostic analyses demonstrate that STAIR excels at handling both boundary-sensitive and order-sensitive queries, while its guarded execution and semantic adaptation ensure precise point-time
reasoning and inexact intervals, respectively.
Read the full abstract ▾
All authors Xinlong Dai, Jinchuan Zhang, Lei Gao, Xinzhe Hu, Yuefeng He, Hui Gao
Computation and Language · Cross-listed from cs.SD
Chen-An Li and Hung-yi Lee
Existing
speech retrieval systems rely on fixed similarity matching and cannot adapt to diverse user intents. We introduce INSPIRE, the first
benchmark for instruction-aware
speech retrieval, in which natural-
language instructions dynamically specify relevance criteria, including semantic content, speaker identity, speaking style, environmental sounds, and their combinations. We evaluate four
retrieval paradigms: large audio-
language models, cascaded pipelines, self-supervised
speech models, and contrastive audio-
language models. Our results reveal that no current method robustly handles all
retrieval intents. Text-based approaches perform relatively better at semantic
retrieval but struggle with paralinguistic attributes, while
speech-based models are moderately better at capturing acoustic properties but falter at following instructions. These findings highlight the need for unified architectures capable of instruction-aware
speech retrieval.
Read the full abstract ▾
Computation and Language
LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents
Xingjun Wang, Gongsheng Li, Qi Fan et al. (+3 more)
LLM
agents increasingly answer questions over dynamic raw-document collections, where files may change before preprocessing, and relevant evidence (spans, sections, pages, or tables) is query-dependent. Existing
retrieval-augmented approaches pre-materialize evidence via fixed chunking,
embeddings, or persistent indexes: effective for lookup, yet costly, stale-prone, and committed to a granularity before the query is known. We formulate in-context search as Budgeted Evidence Localization over a latent evidence space induced by dynamic raw documents and propose LENS (Latent Evidence Exploration and Search), an index-free framework. Instead of pre-materializing the evidence space, LENS maintains a query-conditioned belief over candidate units, iteratively selecting candidates via complementary lexical, local, and exploratory proposal policies, updating the belief via an LLM relevance oracle, and narrowing toward high-posterior regions under a controllable budget. Evidence is consolidated into compact, source-grounded regions of interest and compressed into self-organizing knowledge clusters reused across related queries. On a controlled 500-question evaluation with matched corpus snapshots, LENS reaches 62.4% exact match and 84.8% evidence recall vs. 65.2% exact match but 50.4% evidence recall for a ReAct-style baseline. Across scales, LENS gives the strongest supporting-fact localization and answer grounding. On a fixed 150-question fullwiki subset over the raw Wikipedia dump with zero indexing, LENS and ReAct are nearly tied in official answer quality (43.3% vs. 42.7% EM), with LENS grounding more answers in retrieved evidence (84.0% vs. 70.7%). A no-
retrieval Closed-Book reference highlights the contribution of model
memory. LENS is query-ready after corpus changes, needs no preprocessing or persistent index, and preserves source-grounded evidence localization throughout.
Read the full abstract ▾
All authors Xingjun Wang, Gongsheng Li, Qi Fan, Yunlin Mao, Luyan Su, Yingda Chen
Computation and Language
QUMem: Personalized Memory for Query-Conditioned User-State Inference in LLM Agents
Heng Wang, Yifei Li, Lingling Zhang et al. (+4 more)
Large
language model (LLM)
agents increasingly use external
memory systems to support personalization by drawing on long and evolving interaction histories, in which user preferences may be distributed across time, change with context, and conflict with earlier evidence. However, existing systems face three limitations: fixed-turn, fixed-
token, or session-based boundaries can mix unrelated dialogue or split an event from its causes, decisions, and outcomes; storing multiple pieces of user information from the same interaction as a single
memory binds together items that serve different functions and should be independently retrievable; and treating the current task as a single top-$k$
retrieval query can return fragments that are individually relevant but fail to jointly capture preference
evolution, temporal validity, and contextual applicability. We introduce QUMem, a structured
memory framework for query-conditioned user-state inference. QUMem first segments interaction histories into variable-length episodes according to semantic continuity, then decomposes each episode into independently retrievable factual, preference, and transferable insight
memories while preserving temporal positions and source evidence. At inference time, three sequential
agents identify task-specific information needs, plan multi-query
retrieval over the typed
memory stores, and jointly infer a temporally and contextually valid user state for downstream response generation. QUMem achieves state-of-the-art performance on both PersonaMem and KnowU-Bench, demonstrating the effectiveness of query-conditioned user-state inference for long-term personalization.
Read the full abstract ▾
All authors Heng Wang, Yifei Li, Lingling Zhang, Pengyu Li, Xinyu Che, Xinyu Zhang, Zesheng Yang
Computation and Language
HyperSkill: Self-Evolving LLM Agents via Hypergraph-Structured Skill Memory
Ruiyao Xu, Tiankai Yang and Wei-Chieh Huang
As agentic tasks grow in complexity, LLM
agents increasingly rely on experiential
memory to reuse procedural knowledge across tasks. Effective
memory design must jointly address what to store, how
memory is structured and retrieved, and how
memory evolves. Existing systems tackle each only partially: they store trajectories, insights, or workflows as isolated entries, discarding compositional relationships among subtasks and reusable skills; retrieve by flat
embedding similarity that ignores relational signals; and maintain
memory without leveraging its relational structure. We propose HyperSkill, a hypergraph-based
memory framework that jointly improves all three. HyperSkill represents
memory as a hypergraph with two node types, subtask steps and reusable skills, where each hyperedge links the subtasks and skills from a single trajectory. Dual-path
retrieval queries both subtask and trajectory levels, ranking skills by co-occurrence across retrieved trajectories. Periodic structure-informed maintenance prunes low-utility nodes and merges redundant skills via quality-weighted propagation. Across xBench,
GAIA, and WebWalkerQA with GPT-4o and Qwen3-30B-A3B, HyperSkill outperforms ten
memory baselines, yielding gains of up to +11.51 on
GAIA and +11.18 on WebWalkerQA.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
The Commercial Tax: Rent-vs-Own Blind Spots in Multi-Hop Retrieval Benchmarks
Luis M. Sanchez and Kosrow Dehnad
Enterprises connect
language models to their own data through
retrieval. The
benchmarks that rank multi-hop
retrieval systems leave out two facts a buyer needs before a published number can be used: whether the
retrieval backbone may be deployed commercially, and what it costs to build. On licensing: the field's dense-
retrieval anchor, NV-Embed-v2, is licensed cc-by-nc-4.0. Of the four leading MuSiQue systems we audit (HippoRAG-2, PropRAG, SAG, KET-RAG), three depend on it for their best numbers and none says so. On performance: we measure thirteen embedders from eight makers on one identical MuSiQue harness with bootstrap confidence intervals throughout. Until mid-2026 there was a real commercial tax: the best commercially-licensed embedder trailed the anchor by 2.31 Recall@5 points (95% CI [0.91, 3.71], p=0.001). NVIDIA's Nemotron-3-Embed-8B, released 2026-07-16, has closed it: +0.24 at Recall@5 (95% CI [-0.94, +1.43], p=0.69), -0.58 at Recall@10 (p=0.28). It matches the anchor, does not beat it, and is the only entrant that is commercially licensed, free to self-host, and indistinguishable from the anchor; every other entrant meeting the first two conditions sits 5.2 to 14.6 points below. The durable finding is the paid-versus-free divide: API embedders charge per
token on every re-index, self-hosted ones charge nothing. On cost: three of five audited systems (adding Microsoft's GraphRAG) do not disclose indexing cost, and the only published GraphRAG dollar figures span 11x inside one third-party paper (USD 2.30 vs USD 24.94 to index a 5.64 MB corpus once); extrapolated to 1 TB that undisclosed choice separates roughly USD 428K from $4.6M. Our cost model keeps one-time
embedding apart from recurring answering: at 1 TB,
embedding sits 7.5x-900x below graph construction, and a year of answering at 10,000 queries/day sits 350x or more below it.
Read the full abstract ▾
Computation and Language
Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrieval
Lihui Ding, Zihan Guo, Bingwei Lu et al. (+5 more)
Pseudo-query generation can alleviate the supervision bottleneck for
agent skill
retrieval, but existing document-level approaches typically leave the rich internal relations among capabilities, parameters, and usage examples implicit. As a result, generated queries may be topically relevant to a skill while lacking capability grounding and parameter consistency, raising the question of whether explicitly exploiting a skill document's internal structure can produce more effective
retrieval signals. We therefore propose Skill2Query, a framework that first parses a skill document into a Skill Knowledge Graph and then generates pseudo-queries through a three-stage process including style mimicking, query template generation, and parameter filling. The generated queries can be used for offline index augmentation, online query expansion, and retriever training. Four
benchmarks (TheoremQA, LogicBench, ToolQA, and CHAMP) are used to evaluate Skill2Query with large-scale skill candidate pools across multiple downstream applications, including skill
retrieval, retriever training, and end-to-end
agent execution. Using nearly 30K skills across diverse domains, we generate 700K category-diverse pseudo-queries. Skill2Query consistently improves sparse, dense, and skill-routing
retrieval, with an average Recall@1 gain of 6.70 percentage points across
retrieval settings. Skill2Query-generated training data also achieves the best Recall@1 and nDCG@1 among the evaluated generation baselines. Further evaluations with multiple LLM backends demonstrate that improved skill
retrieval translates into higher
agent task success rates. Code and resources are available at https://github.com/MatZaharia/Skill2Query.
Read the full abstract ▾
All authors Lihui Ding, Zihan Guo, Bingwei Lu, Chenyu Zhou, Yuanjian Zhou, Weinan Zhang, Jianghao Lin, Dongdong Ge
Computation and Language
CAPO: Constraint-Aware Prompt Optimization for LLM Agents
Victor Ye Dong, Reid Pryzant, Yi Liu et al. (+1 more)
Large
language models (LLMs) are increasingly deployed as
agents that rely on system
prompts to use tools and complete tasks. Such deployments impose distinct operational requirements, including appropriate tool use, concise
prompts and solution paths, and compliance with safety and formatting policies. For many practitioners, however, assembling domain-specific supervised data to post-train models to meet these requirements is infeasible. We introduce CAPO (Constraint-Aware
Prompt Optimization), a primal-dual method that combines pool-based rewrites with adaptive constraint weighting to optimize system
prompts under explicit operational constraints. Across agentic
benchmarks, CAPO more reliably reaches empirically feasible operating points while improving task performance. CAPO also generalizes beyond agentic settings, achieving strong results on assistant-style evaluations with output-format and safety/
privacy constraints. We further introduce DCAPO (Dynamically Trained CAPO), which trains a feedback- and dual-conditioned rewriter with pool-based GRPO while keeping the task
agent frozen. Across task
agents of different sizes, DCAPO produces a feasible
prompt in every evaluated domain and matches or improves the task accuracy achieved by the evaluated baselines. A surrogate analysis characterizes how finite-pool and discrete-rewrite errors enter the inexact primal-dual procedure.
Read the full abstract ▾
All authors Victor Ye Dong, Reid Pryzant, Yi Liu, Jian Jiao
Computation and Language
Agentic-DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech
Pengcheng Wang, Sheng Li, Jiyi Li et al. (+1 more)
Synthetic conversational
speech has become an important resource for developing and evaluating conversational
speech systems. However, existing dialogue synthesis pipelines typically generate dialogue content first and then insert interruptions, overlap, and backchannels using handcrafted markers or timing rules, making conversational timing prescribed rather than interaction-driven. We present Agentic-DuplexGen, a dialogue synthesis framework that explicitly decouples content, timing, and acoustics. An LLM first generates the dialogue script, and then two full-duplex conversational models perform the script while listening to each other in real time. This allows conversational timing to emerge naturally while preserving the scripted content. Finally, a high-fidelity text-to-
speech model re-renders the interaction without altering its timing. As a demonstration of the proposed framework, we construct a patient–clinician conversational
speech corpus with construction-time annotations, including word timestamps, speaker activity, overlap regions, and interaction events. Experimental results show that the proposed framework produces conversational dynamics closer to real dialogue than conventional stitching-based synthesis.
Read the full abstract ▾
All authors Pengcheng Wang, Sheng Li, Jiyi Li, Takahiro Shinozaki
Computation and Language · Cross-listed from cs.CR
Coverage Is Not Containment: A Fundamental Limit of Admission-Time Defenses Against Coordinated Poisoning of Vector Retrieval
Prashant Kumar Pathak and Tarun Kumar Sharma
Retrieval-augmented generation (RAG) answers a question by retrieving passages from a vector store and trusting them as context, so anyone who can add documents can try to steer the answer. A recent, appealing defense filters poisoning at ingestion, rejecting any document that behaves like a hub. We show it – and every ingestion-time filter – is defeated by a coordinated adversary that injects a handful of individually unremarkable documents which together surround one target query and seize its top-k (on BGE-large / BEIR, m=10 documents take 10/10; 9.9/10 on a live HNSW index). The attack is not theoretical. Realized as ordinary fluent text and run end-to-end through a BGE-large + HNSW + Qwen2.5-7B pipeline, it makes the generator emit the attacker's planted claim in 88% of targets, versus 0% without the injection. And no admission-time defense stops it: at ingestion an attack cone is geometrically identical to a legitimate niche upload, so – measuring this directly – the strongest trained classifier, given every feature and thousands of examples, separates the two no better than chance, catching 4.2% of attacks at a 1% false-positive rate. We prove this limit for the entire class of ingestion-time statistics (any decision from documents and reference queries alone), and it reproduces – and worsens – across two corpora and five encoders. The one signal that separates an attack from legitimate niche ingestion – a query's demand – is invisible before
retrieval, which is also the escape: a
retrieval-time detector that observes demand catches 100% of the attacks at the same 1% false-positive rate. Coverage of the query space by an admission gate is not containment of coordinated poisoning; robust defense must move past the front door, to demand.
Read the full abstract ▾
Computation and Language
$R^3$-Bench: LLMs Struggle with Resource-Rational Reasoning under Shared Budgets
Peisong Wang, Zhiwei Ma, Bowen Liu et al. (+6 more)
In cognitive science, resource rationality asks how an
agent should allocate limited computation to maximize expected value. Most
reasoning and
agent benchmarks use independent per-task budgets; existing shared-budget studies do not calibrate suite performance against the same model's demonstrated single-problem competence. We introduce $R^3$-Bench, which evaluates six-problem suites under shared budgets across mathematics, competitive programming, and abstract
reasoning in tool-free and agentic settings. Matched single-problem response curves define an offline empirical oracle over observed successes. Across 72 main-table
cells for six models, the oracle mean matches or exceeds the contest mean in all
cells and is strictly higher in 71. Under moderate tool-free pressure, equal-allocation replay also exceeds contest performance for four of six models. Trajectory diagnostics reveal limited strategy updating and pressure-dependent failure patterns. In a three-model diagnostic under strong agentic pressure, at least one fixed scheduler exceeds the contest mean in six of nine
cells, but no policy dominates across domains. These results expose a persistent gap between demonstrated competence and shared-budget realization.
Read the full abstract ▾
All authors Peisong Wang, Zhiwei Ma, Bowen Liu, Feixue Liu, Aochuan Chen, Chenyi Zi, Hongchuan Zeng, Yuhan Li, Jia Li
Computation and Language
ReRef-3D: A Benchmark for Spatial Referring Expression-Guided 3D Object Placement
Mary Lynn Martin, Yifei Zhang, Martha Palmer et al. (+1 more)
We introduce ReRef-3D, a
benchmark for
language-guided single-object placement in controlled 3D tabletop scenes. It contains 33,826 instructions across 998 CLEVR-derived scenes, spanning 16 placement types and direct, one-hop, and two-hop references. Given that an instruction defines a region of acceptable placements rather than one coordinate, predictions are evaluated by relation satisfaction and physical validity. Each instruction also includes a verified natural rephrasing. We evaluate LLaVA-3D, 3D-LLM, 3D-LLaVA, LL3DA, and PlaceIt3D, with PlaceIt3D evaluated under both point-based and region-based supervision. Region-supervised PlaceIt3D achieves the highest placement accuracy at about 80% and LLaVA-3D reaches about 70%. Performance varies by relation type, but nearest and aligned remain the most difficult. Rephrasing has little effect on performance overall.
Read the full abstract ▾
All authors Mary Lynn Martin, Yifei Zhang, Martha Palmer, Maria Leonor Pacheco
Computation and Language · Cross-listed from cs.AI
Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency
Parsa Mazaheri and Kasra Mazaheri
Automated checking pipelines increasingly place one
language model as the checker and another (or the same one) as the fixer. We ask whether that wiring changes what the checker reports. Measuring false alarms on human-verified-correct ProcessBench traces with the present task held byte-identical, we find that a completed audit -> repair episode already in the model's context lowers false alarms in 15 of 15 model x wording combinations, by 2.8 to 11.5 percentage points against a length-matched non-audit control, a 9 to 25% reduction relative to that control. The direction contradicts what the accumulated-message literature predicts: an episode whose audit reported an error lowers false alarms further still, at all five wordings on the model where that manipulation lands cleanly, though a negativity asymmetry predicts more flagging. Decomposing the episode finds repair content and audit verdict complementary: different components carry the effect on different model families. Signal-detection analysis locates the change in the threshold rather than in discrimination – the criterion moves in 15 of 15 combinations and survives correction in 13 while d' survives in none, though the d' test is half as sensitive by construction – and a hand audit of 50 false alarms finds 82% simply wrong, so at this operating point the shift need not be harmful. With
reasoning enabled the effect keeps its relative size on both models tested, and the threshold reading holds there too.
Read the full abstract ▾
Computation and Language
From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents
Zhengzhao Ma, Boxi Cao, Yaojie Lu et al. (+3 more)
Reliable uncertainty quantification (UQ) is essential for deploying large
language model (LLM)
agents in complex interactive environments. Existing UQ methods largely rely on local signals, such as
token probabilities, predictive
entropy, or per-step confidence, and therefore overlook the long-range dependencies through which errors accumulate across an execution trajectory. As a result, they may fail to identify
agent failures whose causes originate several
reasoning or interaction steps before the final answer. We propose RUPA (Relational Uncertainty Propagation for
Agents), a trajectory-level UQ framework for LLM
agents. RUPA represents an execution history as a directed trajectory graph in which
reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges. It then propagates uncertainty over this graph to capture how execution risk accumulates and transfers across interaction steps. The propagated signal is combined with trajectory-level behavioral features and goal-
alignment information to produce a confidence estimate for the full
agent trajectory. We evaluate RUPA on representative
agent benchmarks, including $\tau$-2, Terminal-Bench-2, and
GAIA, using 6 open-source LLMs spanning multiple model families. Experimental results show that RUPA consistently outperforms existing UQ methods by providing more accurate uncertainty estimates, enabling earlier failure detection, and improving uncertainty-guided
agent execution across diverse
agent tasks. These results demonstrate that explicitly modeling relational dependency is crucial to reliable UQ for long-horizon LLM
agents, providing a practical foundation for trustworthy
agent execution.
Read the full abstract ▾
All authors Zhengzhao Ma, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun