Computation and Language · Cross-listed from cs.IR
KGiRAG: An Iterative GraphRAG Approach for Responding Sensemaking Queries
Isabela Iacob, Melisa Marian and Gheorghe Cosmin Silaghi
Recent literature highlights the potential of graph-based approaches within large
language model (LLM)
retrieval-augmented generation (RAG) pipelines for answering queries of varying complexity, particularly those that fall outside the LLM's prior knowledge. However, LLMs are prone to
hallucination and often face technical limitations in handling contexts large enough to ground complex queries effectively. To address these challenges, we propose a novel iterative, feedback-driven GraphRAG architecture that leverages response quality assessment to iteratively refine outputs until a sound, well-grounded response is produced. Evaluating our approach with queries from the HotPotQA
dataset, we demonstrate that this iterative RAG strategy yields responses with higher semantic quality and improved relevance compared to a single-shot baseline.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation
Abdolamir Karbalaie, Fernando Seoane and Farhad Abtahi
Ambient AI "scribe" systems promise to reduce clinical documentation burden, but automatic
speech recognition (ASR) errors can remain unnoticed without careful review, and high-quality human reference transcripts are often unavailable for calibrating uncertainty. We investigate whether cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal to prioritize human verification in medical transcription workflows. Using 50 publicly available medical education audio clips (8 h 14 min), we transcribed each clip with eight ASR systems spanning commercial APIs and open-source engines. We aligned multi-model outputs, built consensus pseudo-references, and quantified
token-level agreement using a majority-strength metric; we further characterized disagreements by type (content vs. punctuation/formatting) and assessed per-model agreement via leave-one-model-out (jackknife) consensus scoring. Inter-model reliability was low (ICC[2,1] = 0.131), indicating heterogeneous failure modes across systems. Across 76,398 evaluated
token positions, 72.1% showed near-unanimous agreement (7-8 models), while 2.5% fell into high-risk bands (0-3 models), with high-risk mass varying from 0.7% to 11.4% across accent groups. Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass. These results suggest that cross-model disagreement provides a sparse, localizable signal that can surface potentially unreliable transcript spans without human-verified references, enabling targeted review; clinical accuracy of flagged regions remains to be established.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
AgenticGEO: A Self-Evolving Agentic System for Generative Engine Optimization
Jiaqi Yuan, Jialu Wang, Zihan Wang et al. (+3 more)
Generative search engines represent a transition from traditional ranking-based
retrieval to Large
Language Model (LLM)-based synthesis, transforming optimization goals from ranking prominence towards content inclusion. Generative Engine Optimization (GEO), specifically, aims to maximize visibility and attribution in black-box summarized outputs by strategically manipulating source content. However, existing methods rely on static heuristics, single-
prompt optimization, or engine preference rule
distillation that is prone to overfitting. They cannot flexibly adapt to diverse content or the changing behaviors of generative engines. Moreover, effectively optimizing these strategies requires an impractical amount of interaction feedback from the engines. To address these challenges, we propose AgenticGEO, a self-evolving agentic framework formulating optimization as a content-conditioned control problem, which enhances intrinsic content quality to robustly adapt to the unpredictable behaviors of black-box engines. Unlike fixed-strategy methods, AgenticGEO employs a MAP-Elites archive to evolve diverse, compositional strategies. To mitigate interaction costs, we introduce a Co-Evolving Critic, a lightweight surrogate that approximates engine feedback for content-specific strategy selection and refinement, efficiently guiding both evolutionary search and inference-time planning. Through extensive in-domain and cross-domain experiments on two representative engines, AgenticGEO achieves state-of-the-art performance and demonstrates robust transferability, outperforming 14 baselines across 3
datasets. Our code and model are available at: https://github.com/AIcling/agentic_geo.
Read the full abstract ▾
All authors Jiaqi Yuan, Jialu Wang, Zihan Wang, Qingyun Sun, Ruijie Wang, Jianxin Li
Computation and Language
Fast-Slow Thinking RM: Efficient Integration of Scalar and Generative Reward Models
Jiayun Wu, Peng Zhang, Yuanyuan Lu et al. (+3 more)
Reward models (RMs) are critical for aligning Large
Language Models via Reinforcement Learning from Human Feedback (RLHF). While Generative Reward Models (GRMs) achieve superior accuracy through chain-of-thought (CoT)
reasoning, they incur substantial computational costs. Conversely, Scalar Reward Models (SRMs) offer efficiency but suffer from limited performance and adaptability in complex scenarios. We introduce Fast-Slow Thinking Reward Models (F/S-RM), a hybrid RM architecture inspired by Dual Process Theory. It trains a single model to integrate two distinct reward paradigms: scalar-style first-
token pairwise judgment (fast thinking) and CoT-based judgment (slow thinking), regulated by a dual-confidence activation mechanism that determines when to activate slow thinking. Under hybrid inference, F/S-RM achieves state-of-the-art accuracy with an average score of 84.3 across
benchmarks, while reducing
token consumption by 22.5%.
Read the full abstract ▾
All authors Jiayun Wu, Peng Zhang, Yuanyuan Lu, Shan Qu, Ning Gu, Tun Lu
Computation and Language
CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language
Roy Uziel, Omer Belhasin, Itay Levy et al. (+4 more)
Masked
Diffusion Models (MDMs) provide an efficient non-causal alternative to autoregressive generation but often struggle with
token dependencies and semantic incoherence due to their reliance on discrete marginal distributions. We address these limitations by shifting the
diffusion process into a continuous sentence-level semantic space. We propose CRoCoDiL (Continuous and Robust Conditioned
Diffusion for
Language), a unified fine-tuning approach that jointly trains an encoder-demasker architecture, grounding the MDM demasking in continuous latent representations. This leads to the formation of a novel autoencoder in which decoding is obtained by an MDM
algorithm. Relying on the same framework, we introduce two unconditional text synthesis
algorithms: Continuous-Then-Discrete (ConThenDisc), a hybrid-
diffusion approach that first generates latent representations in continuous space and then decodes these to
tokens via an MDM, and Continuous-Within-Discrete (ConWithinDisc), a multi-
diffusion strategy that refines latent representations throughout the discrete sampling process. Experiments using LLaDA show that our methods achieve superior generation quality and more than 10x faster sampling speeds in an unconditional setting.
Read the full abstract ▾
All authors Roy Uziel, Omer Belhasin, Itay Levy, Akhiad Bercovich, Ran El-Yaniv, Ran Zilberstein, Michael Elad
Computation and Language
Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs
Hengwei Ye, Yuanting Guan, Yuxuan Ge et al. (+7 more)
Multimodal Large
Language Models (MLLMs) combine the linguistic strengths of LLMs with the ability to process
multimodal data, enbaling them to address a broader range of visual tasks. Because MLLMs aim at more general, human-like competence than
language-only models, we take inspiration from the Wechsler Intelligence Scales - an established battery for evaluating children by decomposing intelligence into interpretable, testable abilities. We introduce KidGym, a comprehensive 2D grid-based
benchmark for assessing five essential capabilities of MLLMs: Execution, Perception
Reasoning, Learning,
Memory and Planning. The
benchmark comprises 12 unique tasks, each targeting at least one core capability, specifically designed to guage MLLMs' adaptability and developmental potential, mirroring the stages of children's cognitive growth. Additionally, our tasks encompass diverse scenarios and objects with randomly generated layouts, ensuring a more accurate and robust evluation of MLLM capabilities. KidGym is designed to be fully user-customizable and extensible, allowing researchers to create new evaluation scenarios and adjust difficuly levels to accommodate the rapidly growing MLLM community. Through the evaluation of state-of-the-art MLLMs using KidGym, we identified significant insights into model capabilities and revealed several limitations of current models. We release our
benchmark at: https://bobo-ye.github.io/KidGym/.
Read the full abstract ▾
All authors Hengwei Ye, Yuanting Guan, Yuxuan Ge, Tianying Zhu, Zhenhan Guan, Yijia Zhong, Yijing Zhang, Han Zhang, Yingna Wu, Zheng Tian
Computation and Language
RedacBench: Can AI Erase Your Secrets?
Hyunjun Jeon, Kyuyoung Kim and Jinwoo Shin
Modern
language models can readily extract sensitive information from unstructured text, making redaction – the selective removal of such information – critical for data security. However, existing
benchmarks for redaction typically focus on predefined categories of data such as personally identifiable information (PII) or evaluate specific techniques like masking. To address this limitation, we introduce RedacBench, a comprehensive
benchmark for evaluating policy-conditioned redaction across domains and strategies. Constructed from 514 human-authored texts spanning individual, corporate, and government sources, paired with 187 security policies, RedacBench measures a model's ability to selectively remove policy-violating information while preserving the original semantics. We quantify performance using 8,053 annotated propositions that capture all inferable information in each text. This enables assessment of both security – the removal of sensitive propositions – and utility – the preservation of non-sensitive propositions. Experiments across multiple redaction strategies and state-of-the-art
language models show that while more advanced models can improve security, preserving utility remains a challenge. To facilitate future research, we release RedacBench along with a web-based playground for
dataset customization and evaluation. Available at https://hyunjunian.github.io/redaction-playground/.
Read the full abstract ▾
Computation and Language
Framing Effects in Independent-Agent Large Language Models: A Cross-Family Behavioral Analysis
Zice Wang and Zhenyu Zhang
In many real-world applications, large
language models (LLMs) operate as independent
agents without interaction, thereby limiting coordination. In this setting, we examine how
prompt framing influences decisions in a threshold voting task involving individual-group interest conflict. Two logically equivalent
prompts with different framings were tested across diverse LLM families under isolated trials. Results show that
prompt framing significantly influences choice distributions, often shifting preferences toward risk-averse options. Surface linguistic cues can even override logically equivalent formulations. This suggests that observed behavior reflects a tendency consistent with a preference for instrumental rather than cooperative rationality when success requires risk-bearing. The findings highlight framing effects as a significant bias source in non-interacting multi-
agent LLM deployments, informing
alignment and
prompt design.
Read the full abstract ▾
Computation and Language
URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models
Vinh Nguyen, Cuong Dang, Jiahao Zhang et al. (+6 more)
Retrieval-Augmented Generation (RAG) has emerged as a widely adopted approach for enhancing LLMs in scenarios that demand extensive factual knowledge. However, current RAG evaluations concentrate primarily on correctness, which may not fully capture the impact of
retrieval on LLM uncertainty and reliability. To bridge this gap, we introduce URAG, a comprehensive
benchmark designed to assess the uncertainty of RAG systems across various fields like healthcare, programming, science, math, and general text. By reformulating open-ended generation tasks into multiple-choice question answering, URAG allows for principled uncertainty quantification via conformal prediction. We apply the evaluation pipeline to 8 standard RAG methods, measuring their performance through both accuracy and prediction-set sizes based on LAC and APS metrics. Our analysis shows that (1) accuracy gains often coincide with reduced uncertainty, but this relationship breaks under
retrieval noise; (2) simple modular RAG methods tend to offer better accuracy-uncertainty trade-offs than more complex
reasoning pipelines; and (3) no single RAG approach is universally reliable across domains. We further show that (4)
retrieval depth, parametric knowledge dependence, and exposure to confidence cues can amplify confident errors and
hallucinations. Ultimately, URAG establishes a systematic
benchmark for analyzing and enhancing the trustworthiness of
retrieval-augmented systems. Our code is available on GitHub.
Read the full abstract ▾
All authors Vinh Nguyen, Cuong Dang, Jiahao Zhang, Hoa Tran, Minh Tran, Trinh Chau, Thai Le, Lu Cheng, Suhang Wang
Computation and Language
TriageSim: A Conversational Emergency Triage Simulation Framework from Structured Electronic Health Records
Dipankar Srirag, Quoc Dung Nguyen, Aditya Joshi et al. (+2 more)
Research in emergency triage is restricted to structured electronic health records (EHR) due to regulatory constraints on nurse-patient interactions. We introduce TriageSim, a simulation framework for generating persona-conditioned triage conversations from structured records. TriageSim enables multi-turn nurse-patient interactions with explicit control over disfluency and decision behaviour, producing a corpus of ~800 synthetic transcripts and corresponding audio. We use a combination of automated analysis for linguistic, behavioural and acoustic fidelity alongside manual evaluation for medical fidelity using a random subset of 50 conversations. The utility of the generated corpus is examined via conversational triage classification. We observe modest agreement for acuity levels across three modalities: generated synthetic text, ASR transcripts, and direct audio inputs. We provide the code for TriageSim at https://github.com/dipankarsrirag/triage-sim.git.
Read the full abstract ▾
All authors Dipankar Srirag, Quoc Dung Nguyen, Aditya Joshi, Padmanesan Narasimhan, Salil Kanhere
Computation and Language
GLoRIA: Gated Low-Rank Interpretable Adaptation for Dialectal ASR
Pouya Mehralian, Melissa Farasyn, Anne Breitbarth et al. (+2 more)
Automatic
Speech Recognition (ASR) in dialect-heavy settings remains challenging due to strong regional variation and limited labeled data. We propose GLoRIA, a parameter-efficient adaptation framework that leverages metadata (e.g., coordinates) to modulate low-rank updates in a pre-trained encoder. GLoRIA injects low-rank
matrices into each feed-forward layer, with a gating MLP determining the non-negative contribution of each LoRA rank-1 component based on location metadata. On the GCND corpus, GLoRIA outperforms geo-conditioned full fine-tuning, LoRA, and both dialect-specific and unified full fine-tuning, achieving state-of-the-art word error rates while updating under 10% of parameters. GLoRIA also generalizes well to unseen dialects, including in extrapolation scenarios, and enables interpretable adaptation patterns that can be visualized geospatially. These results show metadata-gated low-rank adaptation is an effective, interpretable, and efficient solution for dialectal ASR.
Read the full abstract ▾
All authors Pouya Mehralian, Melissa Farasyn, Anne Breitbarth, Anne-Sophie Ghyselen, Hugo Van hamme
Computation and Language · Cross-listed from cs.HC
A Directed Graph Model and Experimental Framework for Design and Study of Time-Dependent Text Visualisation
Songhai Fan, Simon Angus, Tim Dwyer et al. (+3 more)
Exponential growth in the quantity of digital news, social media, and other textual sources makes it difficult for humans to keep up with rapidly evolving narratives about world events. Various visualisation techniques have been touted to help people to understand such discourse by exposing relationships between texts (such as news articles) as topics and themes evolve over time. Arguably, the understandability of such visualisations hinges on the assumption that people will be able to easily interpret the relationships in such visual network structures. To test this assumption, we begin by defining an abstract model of time-dependent text visualisation based on directed graph structures. From this model we distill motifs that capture the set of possible ways that texts can be linked across changes in time. We also develop a controlled synthetic text generation methodology that leverages the power of modern LLMs to create fictional, yet structured sets of time-dependent texts that fit each of our patterns. Therefore, we create a clean user study environment (n=30) for participants to identify patterns that best represent a given set of synthetic articles. We find that it is a challenging task for the user to identify and recover the predefined motif. We analyse qualitative data to map an unexpectedly rich variety of user rationales when divergences from expected interpretation occur. A deeper analysis also points to unexpected complexities inherent in the formation of synthetic
datasets with LLMs that undermine the study control in some cases. Furthermore, analysis of individual decision-making in our study hints at a future where text discourse visualisation may need to dispense with a one-size-fits-all approach and, instead, should be more adaptable to the specific user who is exploring the visualisation in front of them.
Read the full abstract ▾
All authors Songhai Fan, Simon Angus, Tim Dwyer, Ying Yang, Sarah Goodwin, Helen Purchase
Computation and Language
RO-N3WS: Enhancing Generalization in Low-Resource ASR with Diverse Romanian Speech Benchmarks
Alexandra Diaconu, Mădălina Vînaga and Bogdan Alexe
We introduce RO-N3WS, a
benchmark Romanian
speech dataset designed to improve generalization in automatic
speech recognition (ASR), particularly in low-resource and out-of-distribution (OOD) conditions. RO-N3WS comprises over 126 hours of transcribed audio collected from broadcast news, literary audiobooks, film dialogue, children's stories, and conversational podcast
speech. This diversity enables robust training and fine-tuning across stylistically distinct domains. We evaluate several state-of-the-art ASR systems (Whisper, Wav2Vec 2.0) in both zero-shot and fine-tuned settings, and conduct controlled comparisons using synthetic data generated with expressive TTS models. Our results show that even limited fine-tuning on real
speech from RO-N3WS yields substantial WER improvements over zero-shot baselines. We will release all models, scripts, and data splits to support reproducible research in
multilingual ASR, domain adaptation, and lightweight deployment.
Read the full abstract ▾
Computation and Language
Detecting AI-Generated Essays in Writing Assessment: Responsible Use and Generalizability Across LLMs
Jiangang Hao
Writing is a foundational literacy skill that underpins effective communication, fosters critical thinking, facilitates learning across disciplines, and enables individuals to organize and articulate complex ideas. Consequently, writing assessment plays a vital role in evaluating
language proficiency, communicative effectiveness, and analytical
reasoning. The rapid advancement of large
language models (LLMs) has made it increasingly easy to generate coherent, high-quality essays, raising significant concerns about the authenticity of student-submitted work. This chapter first provides an overview of the current landscape of detectors for AI-generated and AI-assisted essays, along with guidelines for their responsible use. It then presents empirical analyses to evaluate how well detectors trained on essays from one LLM generalize to identifying essays produced by other LLMs, based on essays generated in response to public GRE writing
prompts. These findings provide guidance for developing and retraining detectors for practical applications.
Read the full abstract ▾
Computation and Language
Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects
Xiaoyu Luo, Wenrui Yu, Qiongxiu Li et al. (+1 more)
Autoregressive
language models (ARMs) have been shown to memorize and occasionally reproduce training data verbatim, raising concerns about
privacy and copyright liability.
Diffusion language models (DLMs) have recently emerged as a competitive alternative, yet their memorization behavior remains largely unexplored due to fundamental differences in generation dynamics. To address this gap, we present a systematic theoretical and empirical characterization of memorization in DLMs. We propose a generalized probabilistic extraction framework that unifies prefix-conditioned decoding and
diffusion-based generation under arbitrary masking patterns and
stochastic sampling trajectories.
Theorem 4.3 establishes a monotonic relationship between sampling resolution and memorization: increasing resolution strictly increases the probability of exact training data extraction, implying that autoregressive decoding corresponds to a limiting case of
diffusion-based generation by setting the sampling resolution maximal. Extensive experiments across model scales and sampling strategies validate our theoretical predictions. Under aligned prefix-conditioned evaluations, we further demonstrate that DLMs exhibit substantially lower memorization-based leakage of personally identifiable information (PII) compared to ARMs.
Read the full abstract ▾
All authors Xiaoyu Luo, Wenrui Yu, Qiongxiu Li, Johannes Bjerva
Computation and Language
Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training
Valentin Lacombe, Valentin Quesnel and Damien Sileo
Training on verifiable symbolic data is a promising way to expand the
reasoning frontier of
language models beyond what standard pre-training corpora provide. Yet existing procedural generators often rely on fixed puzzles or templates and do not deliver the distributional breadth needed at scale. We introduce
Reasoning Core, a scalable suite that procedurally generates verifiable symbolic
reasoning data across core formal domains: PDDL planning over randomized domains, first-order logic with equality, context-free grammar parsing and generation, causal
reasoning over random
Bayesian networks, and systems of equations. Each task is paired with an external solver for rigorous verification and admits continuous difficulty control for curriculum design. Examples can optionally include solver-derived
reasoning traces, enabling supervised training from the earliest pre-training stages, and the same interface provides verifiable reward functions for reinforcement learning. Our experiments show that mixing
Reasoning Core data into pre-training improves downstream
reasoning while preserving, or slightly improving,
language modeling quality. Zero-shot evaluations confirm these tasks challenge frontier models such as GPT-5. The code and data are publicly available under the MIT license.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Tool Verification for Test-Time Reinforcement Learning
Ruotong Liao, Nikolai Röhrich, Xiaohan Wang et al. (+4 more)
Test-time reinforcement learning (TTRL) has emerged as a promising paradigm for Recursive Self-Improving AI (RSI) by adapting Large
Reasoning Models (LRMs) on unlabeled test inputs, using self-consensus rewards derived from majority voting over sampled rollouts. However, majority voting can mistake popularity for correctness: a spurious yet high-frequency unverified consensus may become a biased reward signal, causing test-time RL to reinforce frequent but wrong answers and collapse into an incorrect mode. We address this false-popular failure mode with T$^3$RL (Tool-Verification for Test-Time Reinforcement Learning), a verification-aware test-time RL framework. T$^3$RL grounds pseudo-label construction in external tool evidence. Concretely, a verifier utilizes external tool evidence (e.g., from code execution) to upweight verified rollouts during a verification-aware voting, producing more reliable pseudo-labels for training. Across various math difficulties (MATH-500, AMC, and AIME 2024) and diverse backbone families, T$^3$RL significantly improves over TTRL, with better performance on harder problems. T$^3$RL is positioned as a verified online data synthesizer, highlighting the role of tool verification in reliable online adaptation. T$^3$RL is extensible to verifiable domains and provides training and inference efficiency.
Read the full abstract ▾
All authors Ruotong Liao, Nikolai Röhrich, Xiaohan Wang, Yuhui Zhang, Yasaman Samadzadeh, Volker Tresp, Serena Yeung-Levy
Computation and Language
Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale
Hao Li, Chunjiang Mu, Jianhao Chen et al. (+5 more)
The rapid proliferation of Claude
agent skills has raised the central question of how to effectively leverage, manage, and scale the
agent skill ecosystem. In this paper, we propose AgentSkillOS, the first principled framework for skill selection, orchestration, and ecosystem-level management. AgentSkillOS comprises two stages: (i) Manage Skills, which organizes skills into a capability tree via node-level recursive categorization for efficient discovery; and (ii) Solve Tasks, which retrieves, orchestrates, and executes multiple skills through DAG-based pipelines. To evaluate the
agent's ability to invoke skills, we construct a
benchmark of 30 artifact-rich tasks across five categories: data computation, document creation, motion
video, visual design, and web interaction. We assess the quality of task outputs using LLM-based pairwise evaluation, and the results are aggregated via a Bradley-Terry model to produce unified quality scores. Experiments across three skill ecosystem scales (200 to 200K skills) show that tree-based
retrieval effectively approximates oracle skill selection, and that DAG-based orchestration substantially outperforms native flat invocation even when given the identical skill set. Our findings confirm that structured composition is the key to unlocking skill potential. Our GitHub repository is available at:https://github.com/ynulihao/AgentSkillOS.
Read the full abstract ▾
All authors Hao Li, Chunjiang Mu, Jianhao Chen, Siyue Ren, Zhiyao Cui, Yiqun Zhang, Lei Bai, Shuyue Hu
Computation and Language · Cross-listed from cs.IR
Scaling Retrieval Augmented Generation with RAG Fusion: Lessons from an Industry Deployment
Luigi Medrano, Arush Verma and Mukul Chhabra
Retrieval-Augmented Generation (RAG) systems commonly adopt
retrieval fusion techniques such as multi-query
retrieval and reciprocal rank fusion (RRF) to increase document recall, under the assumption that higher recall leads to better answer quality. While these methods show consistent gains in isolated
retrieval benchmarks, their effectiveness under realistic production constraints remains underexplored. In this work, we evaluate
retrieval fusion in a production-style RAG pipeline operating over an enterprise knowledge base, with fixed
retrieval depth, re-ranking budgets, and latency constraints. Across multiple fusion configurations, we find that
retrieval fusion does increase raw recall, but these gains are largely neutralized after re-ranking and truncation. In our setting, fusion variants fail to outperform single-query baselines on KB-level Top-$k$ accuracy, with Hit@10 decreasing from $0.51$ to $0.48$ in several configurations. Moreover, fusion introduces additional latency overhead due to query rewriting and larger candidate sets, without corresponding improvements in downstream effectiveness. Our analysis suggests that recall-oriented fusion techniques exhibit diminishing returns once realistic re-ranking limits and context budgets are applied. We conclude that
retrieval-level improvements do not reliably translate into end-to-end gains in production RAG systems, and argue for evaluation frameworks that jointly consider
retrieval quality, system efficiency, and downstream impact.
Read the full abstract ▾
Computation and Language
CrimeNER: Curated Dataset and Methods for Research and Investigation in Language Intelligence
Miguel Lopez-Duran, Julian Fierrez, Aythami Morales et al. (+7 more)
The extraction of critical information from crime-related documents is a crucial task for law enforcement agencies. The extraction of this information can be interpreted as a Named-Entity Recognition (NER) task. However, there is a considerable lack of adequately annotated data on general real-world crime scenarios. To address this issue, we present CrimeNER, a case study of crime-related NER, and a general crime-related Named-Entity Recognition database (CrimeNER-db), consisting of more than 1.5K annotated documents extracted from public reports of terrorist attacks and the US Department of Justice's press notes. We define 4 coarse types of crime entity and 21 fine-grained entity types. We address the quality of the presented database with fully supervised experiments using flat and nested NER architectures and zero- and few-shot experiments using Large
Language Models (LLMs). The database is available on GitHub.
Read the full abstract ▾
All authors Miguel Lopez-Duran, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Gonzalo Mancera, Javier Irigoyen, Oscar Delgado, Francisco Jurado, Alvaro Ortigosa
Computation and Language
LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
Guanzheng Chen, Michael Qizhe Shieh and Lidong Bing
Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the
reasoning capabilities of Large
Language Models (LLMs) by optimizing them against factual outcomes. However, this paradigm falters in long-context scenarios, as its reliance on internal parametric knowledge is ill-suited for tasks requiring contextual grounding–the ability to find and reason over externally provided information. We identify a key reason for this failure: a reward based solely on the final answer is too sparse to effectively guide the model for identifying relevant evidence. We formally prove that the outcome-only reward leads to significant vanishing gradients for the context grounding process, rendering learning intractable. To overcome this bottleneck, we introduce LongRLVR to augment the sparse answer reward with a dense and verifiable context reward. This auxiliary signal directly incentivizes the model for selecting the correct grounding information, providing a robust learning gradient that solves the underlying optimization challenge. We validate our method on challenging long-context
benchmarks using Qwen and LLaMA models. LongRLVR consistently and significantly outperforms the standard RLVR across all models and
benchmarks, e.g., boosting a 14B model's scores on RULER-QA from 73.17 to 88.90 and on LongBench v2 from 39.8 to 46.5. Our work demonstrates that explicitly rewarding the grounding process is a critical and effective strategy for unlocking the full
reasoning potential of LLMs in long-context applications. Our code is available at https://github.com/real-absolute-AI/LongRLVR.
Read the full abstract ▾
Computation and Language
LLMs as Strategic Actors: Behavioral Alignment, Risk Calibration, and Argumentation Framing in Geopolitical Simulations
Veronika Solopova, Viktoria Skorik, Maksym Tereshchenko et al. (+2 more)
Large
language models (LLMs) are increasingly proposed as
agents in strategic decision environments, yet their behavior in structured geopolitical simulations remains under-researched. We evaluate six popular state-of-the-art LLMs alongside results from human results across four real-world crisis simulation scenarios, requiring models to select predefined actions and justify their decisions across multiple rounds. We compare models to humans in action
alignment, risk calibration through chosen actions' severity, and argumentative framing grounded in international relations theory. Results show that models approximate human decision patterns in base simulation rounds but diverge over time, displaying distinct behavioural profiles and strategy updates. LLM explanations for chosen actions across all models exhibit a strong normative-cooperative framing centered on stability, coordination, and risk mitigation, with limited
adversarial reasoning.
Read the full abstract ▾
All authors Veronika Solopova, Viktoria Skorik, Maksym Tereshchenko, Alina Haidun, Ostap Vykhopen
Computation and Language · Cross-listed from cs.LG
Recursive Models for Long-Horizon Reasoning
Chenxiao Yang, Nathan Srebro and Zhiyuan Li
Modern
language models reason within bounded context, an inherent constraint that poses a fundamental barrier to long-horizon
reasoning. We identify recursion as a core principle for overcoming this barrier, and propose recursive models as a minimal realization, where the model can recursively invoke itself to solve subtasks in isolated contexts. We prove that any computable problem admits a recursive decomposition of
reasoning in which each subtask requires only exponentially smaller active context than standard autoregressive models; this strictly surpasses any context management approach confined to a single sequence, such as summarization. We further generalize our framework to modern agentic systems with arbitrary context processing and control flows, and prove that recursive models can achieve optimal power within this broader class. Experimentally, we test two settings: fine-tuning a pretrained base model for recursive SAT solving, and training a small model from scratch on Go traces generated by exact game-tree search. Both show improved long-horizon accuracy with small active contexts.
Read the full abstract ▾
Computation and Language
Recursive Think-Answer Process for LLMs and VLMs
Byung-Kwan Lee, Youngchae Chee and Yong Man Ro
Think-Answer reasoners such as DeepSeek-R1 have made notable progress by leveraging interpretable internal
reasoning. However, despite the frequent presence of self-reflective cues like "Oops!", they remain vulnerable to output errors during single-pass inference. To address this limitation, we propose an efficient Recursive Think-Answer Process (R-TAP) that enables models to engage in iterative
reasoning cycles and generate more accurate answers, going beyond conventional single-pass approaches. Central to this approach is a confidence generator that evaluates the certainty of model responses and guides subsequent improvements. By incorporating two complementary rewards-Recursively Confidence Increase Reward and Final Answer Confidence Reward-we show that R-TAP-enhanced models consistently outperform conventional single-pass methods for both large
language models (LLMs) and
vision-
language models (VLMs). Moreover, by analyzing the frequency of "Oops"-like expressions in model responses, we find that R-TAP-applied models exhibit significantly fewer self-reflective patterns, resulting in more stable and faster inference-time
reasoning. We hope R-TAP pave the way evolving into efficient and elaborated methods to refine the
reasoning processes of future AI.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Efficient and High-Fidelity Omni Modality Retrieval
Chuong Huynh, Manh Luong and Abhinav Shrivastava
Multimodal retrieval is the task of aggregating information from queries across heterogeneous modalities to retrieve desired targets. State-of-the-art
multimodal retrieval models can understand complex queries, yet they are typically limited to two modalities: text and
vision. This limitation impedes the development of universal
retrieval systems capable of comprehending queries that combine more than two modalities. To advance toward this goal, we present OmniRet, the first
retrieval model capable of handling complex, composed queries spanning three key modalities: text,
vision, and audio. Our OmniRet model addresses two critical challenges for universal
retrieval: computational efficiency and representation fidelity. First, feeding massive
token sequences from modality-specific encoders to Large
Language Models (LLMs) is computationally inefficient. We therefore introduce an
attention-based resampling mechanism to generate compact, fixed-size representations from these sequences. Second, compressing rich omni-modal data into a single
embedding vector inevitably causes information loss and discards fine-grained details. We propose
Attention Sliced Wasserstein Pooling to preserve these fine-grained details, leading to improved omni-modal representations. OmniRet is trained on an aggregation of approximately 6 million query-target pairs spanning 30
datasets. We
benchmark our model on 13
retrieval tasks and a MMEBv2 subset. Our model demonstrates significant improvements on composed query, audio and
video retrieval tasks, while achieving on-par performance with state-of-the-art models on others. Furthermore, we curate a new Audio-Centric
Multimodal Benchmark (ACM). This new
benchmark introduces two critical, previously missing tasks-composed audio
retrieval and audio-visual
retrieval to more comprehensively evaluate a model's omni-modal
embedding capacity.
Read the full abstract ▾