Computation and Language · Cross-listed from cs.CV
UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding
Fei Tang, Bofan Chen, Zhengxi Lu et al. (+8 more)
GUI grounding, which localizes interface elements from screenshots given natural
language queries, remains challenging for small icons and dense layouts. Test-time zoom-in methods improve localization by cropping and re-running inference at higher resolution, but apply cropping uniformly across all instances with fixed crop sizes, ignoring whether the model is actually uncertain on each case. We propose UI-Zoomer, a training-free adaptive zoom-in framework that treats both the trigger and scale of zoom-in as a prediction uncertainty quantification problem. A confidence-aware gate fuses spatial consensus among
stochastic candidates with
token-level generation confidence to selectively trigger zoom-in only when localization is uncertain. When triggered, an uncertainty-driven crop sizing module decomposes prediction variance into inter-sample positional spread and intra-sample box extent, deriving a per-instance crop radius via the law of total variance. Extensive experiments on ScreenSpot-Pro, UI-
Vision, and ScreenSpot-v2 demonstrate consistent improvements over strong baselines across multiple model architectures, achieving gains of up to +13.4%, +10.3%, and +4.2% respectively, with no additional training required.
Read the full abstract ▾
All authors Fei Tang, Bofan Chen, Zhengxi Lu, Tongbo Chen, Songqin Nong, Tao Jiang, Wenhao Xu, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Computation and Language
Interpretable Stylistic Variation in Human and LLM Writing Across Genres, Models, and Decoding Strategies
Swati Rallapalli, Shannon Gallagher, Ronald Yurko et al. (+4 more)
Large
Language Models (LLMs) are now capable of generating highly fluent, human-like text. They enable many applications, but also raise concerns such as large scale spam, phishing, or academic misuse. While much work has focused on detecting LLM-generated text, only limited work has gone into understanding the stylistic differences between human-written and machine-generated text. In this work, we perform a large scale analysis of stylistic variation across human-written text and outputs from 11 LLMs spanning 8 different genres and 4 decoding strategies using Douglas Biber's set of lexicogrammatical and functional features. Our findings reveal insights that can guide intentional LLM usage. First, key linguistic differentiators of LLM-generated text seem robust to generation conditions (e.g.,
prompt settings to nudge them to generate human-like text, or availability of human-written text to continue the style); second, genre exerts a stronger influence on stylistic features than the source itself; third, chat variants of the models generally appear to be clustered together in stylistic space, and finally, model has a larger effect on the style than decoding strategy, with some exceptions. These results highlight the relative importance of model and genre over prompting and decoding strategies in shaping the stylistic behavior of machine-generated text.
Read the full abstract ▾
All authors Swati Rallapalli, Shannon Gallagher, Ronald Yurko, Tyler Brooks, Chuck Loughin, Michele Sezgin, Violet Turri
Computation and Language
From Weights to Activations: Is Steering the Next Frontier of Adaptation?
Simon Ostermann, Daniil Gurgurov, Tanja Baeumel et al. (+4 more)
Post-training adaptation of
language models is commonly achieved through parameter updates or input-based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as steering. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods. In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.
Read the full abstract ▾
All authors Simon Ostermann, Daniil Gurgurov, Tanja Baeumel, Michael A. Hedderich, Sebastian Lapuschkin, Wojciech Samek, Vera Schmitt
Computation and Language · Cross-listed from cs.LG
$\pi$-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data
Yaocheng Zhang, Yuanheng Zhu, Wenyue Chong et al. (+7 more)
Deep search
agents have emerged as a promising paradigm for addressing complex information-seeking tasks, but their training remains challenging due to sparse rewards, weak credit assignment, and limited labeled data. Self-play offers a scalable route to reduce data dependence, but conventional self-play optimizes students only through sparse outcome rewards, leading to low learning efficiency. In this work, we observe that self-play naturally produces a question construction path (QCP) during task generation, an intermediate artifact that captures the reverse solution process. This reveals a new source of privileged information: self-play can provide high-quality privileged information for the self-
distillation at low cost and at scale, without relying on human feedback or curated privileged information. Leveraging this insight, we propose Privileged Information Self-Play ($\pi$-Play), a novel multi-
agent self-
evolution framework combining self-play and self-
distillation. In $\pi$-Play, an examiner generates tasks together with QCPs, and a teacher employs QCP as privileged context to densely supervise a student via self-
distillation. This design transforms sparse-reward self-play into a dense-feedback co-
evolution. Extensive experiments show that data-free $\pi$-Play surpasses fully supervised search
agents and improves evolutionary efficiency by 2-3$\times$ over conventional self-play. Code is available at https://github.com/zhyaoch/pi-play.
Read the full abstract ▾
All authors Yaocheng Zhang, Yuanheng Zhu, Wenyue Chong, Songjun Tu, Qichao Zhang, Jiajun Chai, Xiaohan Wang, Wei Lin, Guojun Yin, Dongbin Zhao
Computation and Language
From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution
Pavel Chizhov, Egor Bogomolov and Ivan P. Yamshchikov
Efficiency and safety of Large
Language Models (LLMs), among other factors, rely on the quality of tokenization. A good tokenizer not only improves inference speed and
language understanding but also provides extra defense against jailbreak attacks and lowers the risk of
hallucinations. In this work, we investigate the efficiency of code tokenization, in particular from the perspective of data source diversity. We demonstrate that code tokenizers are prone to producing unused, and thus under-trained,
tokens due to the imbalance in repository and
language diversity in the training data, as well as the dominance of source-specific, repetitive
tokens that are often unusable in future inference. By modifying the BPE objective and introducing merge skipping, we implement different techniques under the name Source-Attributed BPE (SA-BPE) to regularize BPE training and minimize overfitting, thereby substantially reducing the number of under-trained
tokens while maintaining the same inference procedure as with regular BPE. This provides an effective tool suitable for production use.
Read the full abstract ▾
Computation and Language
Dual-Enhancement Product Bundling: Bridging Interactive Graph and Large Language Model
Zhe Huang, Peng Wang, Yan Zheng et al. (+2 more)
Product bundling boosts e-commerce revenue by recommending complementary item combinations. However, existing methods face two critical challenges: (1) collaborative filtering approaches struggle with cold-start items owing to dependency on historical interactions, and (2) LLMs lack inherent capability to model interactive graph directly. To bridge this gap, we propose a dual-enhancement method that integrates interactive graph learning and LLM-based semantic understanding for product bundling. Our method introduces a graph-to-text paradigm, which leverages a Dynamic Concept Binding Mechanism (DCBM) to translate graph structures into natural
language prompts. The DCBM plays a critical role in aligning domain-specific entities with LLM tokenization, enabling effective comprehension of combinatorial constraints. Experiments on three
benchmarks (POG, POG_dense, Steam) demonstrate 6.3%-26.5% improvements over state-of-the-art baselines.
Read the full abstract ▾
All authors Zhe Huang, Peng Wang, Yan Zheng, Sen Song, Longjun Cai
Computation and Language · Cross-listed from cs.LG
Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning
Zekai Lin, Chao Xue, Di Liang et al. (+8 more)
Supervised Fine-Tuning (SFT) of large
language models often suffers from task interference and catastrophic forgetting. Recent approaches alleviate this issue by isolating task-critical parameters during training. However, these methods represent a static solution to a dynamic problem, assuming that parameter importance remains fixed once identified. In this work, we empirically demonstrate that parameter importance exhibits temporal drift over the course of training. To address this, we propose Evolving Parameter Isolation (EPI), a fine-tuning framework that adapts isolation decisions based on online estimates of parameter importance. Instead of freezing a fixed subset of parameters, EPI periodically updates isolation masks using gradient-based signals, enabling the model to protect emerging task-critical parameters while releasing outdated ones to recover plasticity. Experiments on diverse multi-task
benchmarks demonstrate that EPI consistently reduces interference and forgetting compared to static isolation and standard fine-tuning, while improving overall generalization. Our analysis highlights the necessity of synchronizing isolation mechanisms with the evolving dynamics of learning diverse abilities.
Read the full abstract ▾
All authors Zekai Lin, Chao Xue, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng
Computation and Language · Cross-listed from cs.AI
Memory Transfer Learning: How Memories are Transferred Across Domains in Coding Agents
Kangsan Kim, Minki Kang, Taeil Kim et al. (+3 more)
Memory-based self-
evolution has emerged as a promising paradigm for coding
agents. However, existing approaches typically restrict
memory utilization to homogeneous task domains, failing to leverage the shared infrastructural foundations, such as runtime environments and programming
languages, that exist across diverse real-world coding problems. To address this limitation, we investigate
Memory Transfer Learning (MTL) by harnessing a unified
memory pool from heterogeneous domains. We evaluate performance across 6 coding
benchmarks using four
memory representations, ranging from concrete traces to abstract insights. Our experiments demonstrate that cross-domain
memory improves average performance by 3.7%, primarily by transferring meta-knowledge, such as validation routines, rather than task-specific code. Importantly, we find that abstraction dictates transferability; high-level insights generalize well, whereas low-level traces often induce negative transfer due to excessive specificity. Furthermore, we show that transfer effectiveness scales with the size of the
memory pool, and
memory can be transferred even between different models. Our work establishes empirical design principles for expanding
memory utilization beyond single-domain silos. Project page: https://memorytransfer.github.io/
Read the full abstract ▾
All authors Kangsan Kim, Minki Kang, Taeil Kim, Yanlai Yang, Mengye Ren, Sung Ju Hwang
Computation and Language
Davyd Naveriani, Albert Zeyer, Ralf Schlüter et al. (+1 more)
Diffusion language models have recently emerged as a leading alternative to standard
language models, due to their ability for bidirectional
attention and parallel text generation. In this work, we explore variants for their use in
speech recognition. Specifically, we introduce a comprehensive guide to incorporating masked
diffusion language models (MDLM) and uniform-state
diffusion models (USDMs) for rescoring ASR hypotheses. Additionally, we design a new joint-decoding method that combines CTC and USDM by integrating the framewise probability distributions derived from CTC with the labelwise probability distributions computed by USDM at each decoding step, thereby generating new candidates that combine strong
language knowledge from USDM and acoustic information from CTC. Our findings reveal that USDM, as well as MDLM, can significantly improve the accuracy of recognized text. We publish all our code and recipes.
Read the full abstract ▾
All authors Davyd Naveriani, Albert Zeyer, Ralf Schlüter, Hermann Ney
Computation and Language · Cross-listed from cs.AI
Derek Lilienthal, Manisha Mukherjee and Sameera Horawalavithana
Physical
reasoning over images requires integrating perception, domain knowledge, and multi-step symbolic inference, yet
Vision-
Language Models (VLMs) fall far short of human performance on physics
benchmarks. Reinforcement learning can improve
reasoning via reward signals, but it remains unclear whether these signals must come from outside the model or can be derived from the model itself. We compare external rewards, which rely on an outside verifier (rule-based format and answer-correctness checks, or an LLM-judge rubric), with an internal
attention-based reward computed from the model's own
attention over image regions, requiring no labels or judge. We find that external rewards drive most of the accuracy gains, while preserving more coherent
reasoning when rewarded for
reasoning tokens. The internal
attention-based reward, provides an annotation-free grounding signal that improves spatial-relation
reasoning and is complementary to external rewards: adding it to format and accuracy rewards, at no additional annotation cost, improves average accuracy on Molmo~2 while generalizing to out-of-distribution tasks. Our results suggest that internal
attention rewards can supplement external supervision and lead to improved physical
reasoning.
Read the full abstract ▾
Computation and Language
Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models
Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov et al. (+1 more)
Large
language models (LLMs) are prone to generating factually incorrect outputs. Recent work has applied conformal prediction to provide uncertainty estimates and statistical guarantees for the factuality of LLM generations. However, existing approaches are typically not
prompt-adaptive, limiting their ability to capture input-dependent variability. As a result, they may filter out too few items (leading to over-coverage) or too many (under-coverage) for a given task or
prompt. We propose an adaptive conformal prediction approach that extends conformal score transformation methods to LLMs, with applications to long-form generation and multiple-choice question answering. This enables
prompt-dependent calibration, retaining marginal coverage guarantees while improving conditional coverage. In addition, the approach naturally supports selective prediction, allowing unreliable claims or answer choices to be filtered out in downstream applications. We evaluate our approach on multiple white-box models across diverse domains and show that it significantly outperforms existing baselines in terms of conditional coverage.
Read the full abstract ▾
All authors Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov, Maxim Panov
Computation and Language
Leveraging LLM-GNN Integration for Open-World Question Answering over Knowledge Graphs
Hussein Abdallah, Ibrahim Abdelaziz, Panos Kalnis et al. (+1 more)
Open-world Question Answering (OW-QA) over knowledge graphs (KGs) aims to answer questions over incomplete or evolving KGs. Traditional KGQA assumes a closed world where answers must exist in the KG, limiting real-world applicability. In contrast, open-world QA requires inferring missing knowledge based on graph structure and context. Large
language models (LLMs) excel at
language understanding but lack structured
reasoning. Graph neural networks (GNNs) model graph
topology but struggle with semantic interpretation. Existing systems integrate LLMs with GNNs or graph retrievers. Some support open-world QA but rely on structural
embeddings without semantic grounding. Most assume observed paths or complete graphs, making them unreliable under missing links or multi-hop
reasoning. We present GLOW, a hybrid system that combines a pre-trained GNN and an LLM for open-world KGQA. The GNN predicts top-k candidate answers from the graph structure. These, along with relevant KG facts, are serialized into a structured
prompt (e.g., triples and candidates) to guide the LLM's
reasoning. This enables joint
reasoning over symbolic and semantic signals, without relying on
retrieval or fine-tuning. To evaluate generalization, we introduce GLOW-BENCH, a 1,000-question
benchmark over incomplete KGs across diverse domains. GLOW outperforms existing LLM-GNN systems on standard
benchmarks and GLOW-BENCH, achieving up to 53.3% and an average 38% improvement. GitHub code and data are available.
Read the full abstract ▾
All authors Hussein Abdallah, Ibrahim Abdelaziz, Panos Kalnis, Essam Mansour
Computation and Language
How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
Joel Niklaus, Atsuki Yamaguchi, Michal Štefánik et al. (+9 more)
Synthetic data is a standard component in training large
language models, yet systematic comparisons across design dimensions, including rephrasing strategy, generator model, and source data, remain absent. We conduct extensive controlled experiments, generating over one trillion
tokens, to identify critical factors in rephrasing web text into synthetic pretraining data. Our results reveal that structured output formats, such as tables, math problems, FAQs, and tutorials, consistently outperform both curated web baselines and prior synthetic methods. Notably, increasing the size of the generator model beyond 1B parameters provides no additional benefit. Our analysis also demonstrates that the selection of the original data used for mixing substantially influences performance. By applying our findings, we develop \textbf{FinePhrase}, a 486-billion-
token open
dataset of rephrased web text. We show that FinePhrase outperforms all existing synthetic data baselines while reducing generation costs by up to 30 times. We provide the
dataset, all
prompts, and the generation framework to the research community.
Read the full abstract ▾
All authors Joel Niklaus, Atsuki Yamaguchi, Michal Štefánik, Guilherme Penedo, Hynek Kydlíček, Elie Bakouch, Lewis Tunstall, Edward Emanuel Beeching, Thibaud Frere, Colin Raffel, Leandro von Werra, Thomas Wolf
Computation and Language
Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMs
Sasha Boguraev and Kyle Mahowald
We show how causal interventions in
Transformer models provide insights into English syntax by focusing on a long-standing challenge for syntactic theory: syntactic islands. Extraction from coordinated verb phrases is often degraded, yet acceptability varies gradiently with lexical content (e.g., "I know what he hates art and loves" vs. "I know what he looked down and saw"). We show that modern
Transformer language models replicate human judgments across this gradient. Using causal interventions that isolate functionally relevant subspaces in
Transformer blocks,
attention modules, and MLPs, we demonstrate that extraction from coordination islands engages the same filler-gap mechanisms as canonical wh-dependencies, but that these mechanisms are selectively blocked to varying degrees. By projecting a large corpus of unrelated text onto these causally identified subspaces, we derive a novel linguistic hypothesis: the conjunction "and" is represented differently in extractable versus non-extractable constructions, corresponding to expressions encoding relational dependencies versus purely conjunctive uses. These results illustrate how mechanistic interpretability can inform syntax, generating new hypotheses about linguistic representation and processing.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
CollabCoder: Plan-Code Co-Evolution via Collaborative Decision-Making for Efficient Code Generation
Duy Tung Doan, Quang Huy Phung, Dzung Nguyen et al. (+1 more)
Automated code generation remains a persistent challenge in software engineering, as conventional multi-
agent frameworks are often constrained by static planning, isolated execution, high computational overhead, and limited adaptability to complex tasks. This paper introduces CollabCoder, a novel Plan-Code Co-
Evolution framework that improves code generation through dynamic multi-
agent collaboration. The core idea is to design a collaborative decision-making process between the plan module and the code module to decide which module should be executed for the debugging process. Extensive experiments on widely used
benchmarks demonstrate that CollabCoder consistently improves code quality and robustness across tasks. Importantly, CollabCoder achieves performance comparable to or exceeding current state-of-the-art methods while reducing computational overhead, with efficiency gains becoming more pronounced as
benchmark difficulty increases. On the more challenging LiveCodeBench and xCodeEval
benchmarks, our approach improves performance by 11-20% over strong baselines while reducing the number of API calls by an average of 4-10 per execution.
Read the full abstract ▾
All authors Duy Tung Doan, Quang Huy Phung, Dzung Nguyen, Khac-Hoai Nam Bui
Computation and Language
Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection
Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein et al. (+1 more)
Annotating data remains a costly bottleneck for supervised NLP. Active learning (AL) reduces the number of human labels needed by selecting only the most informative instances, while instruction-tuned LLMs attack the same bottleneck from the other side, making labels cheap enough to annotate entire corpora. This raises two questions: can LLM labels replace human labels within the AL loop, and does AL remain necessary when entire corpora can be cheaply labeled? We investigate both by training supervised hate
speech classifiers on a new
dataset of 278K German political TikTok comments, comparing human and LLM annotation under matched conditions. LLM annotation at scale outperforms human-supervised classifiers at roughly one-tenth the cost, for both a closed-source (GPT-5.2) and an open-weight (Qwen3.5-122B-A10B) LLM, and the advantage is robust under soft-label evaluation. It hinges on the annotation interface: only a two-question decomposition mirroring the human annotation task unlocks it. AL provides no reliable advantage over random sampling in our prefiltered pool. Error structure depends on the LLM: only GPT-5.2 matches the human FP/FN balance, while other variants over-flag border-control and economic-competition discourse. Humans remain essential as evaluators; for training labels, the question shifts to which LLM, which interface, and what shape of pool.
Read the full abstract ▾
All authors Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein, Hinrich Schütze
Computation and Language
Beyond Static Personas: Situational Personality Steering for Large Language Models
Zesheng Wei, Mengxiang Li, Zilei Wang et al. (+1 more)
Personalized Large
Language Models (LLMs) facilitate more natural, human-like interactions in human-centric applications. However, existing personalization methods are constrained by limited controllability and high resource demands. Furthermore, their reliance on static personality modeling restricts adaptability across varying situations. To address these limitations, we first demonstrate the existence of situation-dependency and consistent situation-behavior patterns within LLM personalities through a multi-perspective analysis of persona
neurons. Building on these insights, we propose IRIS, a training-free,
neuron-based Identify-Retrieve-Steer framework for advanced situational personality steering. Our approach comprises situational persona
neuron identification, situation-aware
neuron retrieval, and similarity-weighted steering. We empirically validate our framework on PersonalityBench and our newly introduced SPBench, a comprehensive situational personality
benchmark. Experimental results show that our method surpasses best-performing baselines, demonstrating IRIS's generalization and robustness to complex, unseen situations and different models architecture.
Read the full abstract ▾
All authors Zesheng Wei, Mengxiang Li, Zilei Wang, Yang Deng
Computation and Language
Robust Reward Modeling for Large Language Models via Causal Decomposition
Yunsheng Lu, Zijiang Yang, Licheng Pan et al. (+1 more)
Reward models are central to aligning large
language models, yet they often overfit to spurious cues such as response length and overly agreeable tone. Most prior work weakens these cues directly by penalizing or controlling specific artifacts, but it does not explicitly encourage the model to ground preferences in the
prompt's intent. We learn a decoder that maps a candidate answer to the latent intent
embedding of the input. The reconstruction error is used as a signal to regularize the reward model training. We provide theoretical evidence that this signal emphasizes
prompt-dependent information while suppressing
prompt-independent shortcuts. Across math, helpfulness, and safety
benchmarks, the decoder selects shorter and less sycophantic candidates with 0.877 accuracy. Incorporating this signal into RM training in Gemma-2-2B-it and Gemma-2-9B-it increases RewardBench accuracy from 0.832 to 0.868. For Best-of-N selection, our framework increases length-controlled win rates while producing shorter outputs, and remains robust to lengthening and mild off-topic drift in controlled rewrite tests.
Read the full abstract ▾
All authors Yunsheng Lu, Zijiang Yang, Licheng Pan, Zhixuan Chu
Computation and Language
MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment
Zihao Liu, Hantao Zhou, Jiguo Li et al. (+5 more)
User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-
Evolution (IPSE), which gradually optimizes user profiles by comparing and
reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral
alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.
Read the full abstract ▾
All authors Zihao Liu, Hantao Zhou, Jiguo Li, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He, Peng Wang
Computation and Language
ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Execution
Shouzheng Huang, Meishan Zhang, Baotian Hu et al. (+1 more)
Large
Language Models (LLMs) enhance their problem-solving capability by utilizing external tools. However, in open-world scenarios with massive and evolving tool repositories, existing methods relying on static
embedding retrieval or parameter memorization of tools struggle to align user intent with tool semantics or generalize to unseen tools, respectively, leading to suboptimal accuracy of open-world tool
retrieval and execution. To address these, we present ToolOmni, a unified agentic framework that enables LLMs for open-world tool use by proactive
retrieval and grounded execution within a
reasoning loop. First, we construct a cold-start multi-turn interaction
dataset to instill foundational agentic capabilities via Supervised Fine-Tuning (SFT). Then, we introduce open-world tool learning based on a Decoupled Multi-Objective GRPO
algorithm, which simultaneously optimizes LLMs for both tool
retrieval accuracy and execution efficacy in online environments. Extensive experiments demonstrate that ToolOmni achieves state-of-the-art performance both in
retrieval and execution, surpassing strong baselines by a significant margin of +10.8% in end-to-end execution success rate, while exhibiting exceptional robustness and generalization capabilities.
Read the full abstract ▾
All authors Shouzheng Huang, Meishan Zhang, Baotian Hu, Min Zhang
Computation and Language
QuantileMark: A Message-Symmetric Multi-bit Watermark for LLMs
Junlin Zhu, Baizhou Huang and Xiaojun Wan
As large
language models become standard backends for content generation, practical provenance increasingly requires multi-bit watermarking. In provider-internal deployments, a key requirement is message
symmetry: the message itself should not systematically affect either text quality or verification outcomes. Vocabulary-partition watermarks can break message
symmetry in low-
entropy decoding: some messages are assigned most of the probability mass, while others are forced to use tail
tokens. This makes
embedding quality and message decoding accuracy message-dependent. We propose QuantileMark, a white-box multi-bit watermark that embeds messages within the continuous cumulative probability interval $[0, 1)$. At each step, QuantileMark partitions this interval into $M$ equal-mass bins and samples strictly from the bin assigned to the target symbol, ensuring a fixed $1/M$ probability budget regardless of context
entropy. For detection, the verifier reconstructs the same partition under teacher forcing, computes posteriors over latent bins, and aggregates evidence for verification. We prove message-unbiasedness, a property ensuring that the base distribution is recovered when averaging over messages. This provides a theoretical foundation for generation-side
symmetry, while the equal-mass design additionally promotes uniform evidence strength across messages on the detection side. Empirical results on C4 continuation and LFQA show improved multi-bit recovery and detection robustness over strong baselines, with negligible impact on generation quality. Our code is available at GitHub (https://github.com/zzzjunlin/QuantileMark).
Read the full abstract ▾
Computation and Language
From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Models
Wenxuan Li, Zhenfei Zhang, Mi Zhang et al. (+4 more)
Large
language models (LLMs) may memorize sensitive or copyrighted content, raising significant
privacy and legal concerns. While machine unlearning has emerged as a potential remedy, prevailing paradigms rely on user-provided forget sets, making unlearning requests difficult to audit and exposing systems to secondary leakage and malicious abuse. We propose MAGE, a
Memory-grAph Guided Erasure framework for user-minimized, corpus-free unlearning. Given only a lightweight user anchor that identifies a target entity, MAGE probes the target LLM to recover target-related memorization, organizes it into a weighted local
memory graph, and synthesizes scoped supervision for unlearning. MAGE is model-agnostic, can be plugged into standard unlearning methods, and requires no access to the original training corpus. Experiments on two
benchmarks, TOFU and RWKU, demonstrate that MAGE's self-generated supervision achieves effective unlearning performance comparable to supervision generated with external reference, while preserving overall utility. These results support a practical and auditable unlearning workflow driven by minimal anchors rather than user-supplied forget corpora.
Read the full abstract ▾
All authors Wenxuan Li, Zhenfei Zhang, Mi Zhang, Geng Hong, Mi Wen, Xiaoyu You, Min Yang
Computation and Language · Cross-listed from cs.CY
Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking
Alexander Nemecek, Osama Zafar, Yuqiao Xu et al. (+2 more)
Watermarking is becoming the default mechanism for AI content authentication, with governance policies and frameworks referencing it as infrastructure for content provenance. Yet across text, image, and audio modalities, watermark signal strength, detectability, and robustness depend on statistical properties of the content itself, properties that vary systematically across
languages, cultural visual traditions, and demographic groups. We examine how this content dependence creates modality-specific pathways to bias. Reviewing the major watermarking
benchmarks across modalities, we find that, with one exception, none report performance across
languages, cultural content types, or population groups. To address this, we propose three concrete evaluation dimensions for pluralistic watermark benchmarking: cross-lingual detection parity, culturally diverse content coverage, and demographic disaggregation of detection metrics. We argue that watermarking is part of the pluralistic
alignment pipeline and should be held to the same evaluation standards. We connect this to governance frameworks currently mandating watermarking deployment without requiring
fairness evaluation. Our position is that evaluation must precede deployment, and that the same bias
auditing requirements applied to AI models should extend to the verification layer.
Read the full abstract ▾
All authors Alexander Nemecek, Osama Zafar, Yuqiao Xu, Wenbiao Li, Erman Ayday
Computation and Language
MedRCube: A Multidimensional Framework for Fine-Grained and In-Depth Evaluation of MLLMs in Medical Imaging
Zhijie Bao, Fangke Chen, Licheng Bao et al. (+4 more)
The potential of
Multimodal Large
Language Models (MLLMs) in domain of medical imaging raise the demands of systematic and rigorous evaluation frameworks that are aligned with the real-world medical imaging practice. Existing practices that report single or coarse-grained metrics are lack the granularity required for specialized clinical support and fail to assess the reliability of
reasoning mechanisms. To address this, we propose a paradigm shift toward multidimensional, fine-grained and in-depth evaluation. Based on a two-stage systematic construction pipeline designed for this paradigm, we instantiate it with MedRCube. We
benchmark 33 MLLMs, Lingshu-32B achieve top-tier performance. Crucially, MedRCube exposes a series of pronounced insights inaccessible under prior evaluation settings. Furthermore, we introduce a credibility evaluation subset to quantify
reasoning credibility, uncover a highly significant positive association between shortcut behavior and diagnostic task performance, raising concerns for clinically trustworthy deployment. The resources of this work can be found at https://github.com/F1mc/MedRCube.
Read the full abstract ▾
All authors Zhijie Bao, Fangke Chen, Licheng Bao, Chenhui Zhang, Wei Chen, Jiajie Peng, Zhongyu Wei
Computation and Language
Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA
Yuanlei Zheng, Pei Fu, Hang Li et al. (+9 more)
Multi-page Document Visual Question Answering requires
reasoning over semantics, layouts, and visual elements in long, visually dense documents. Existing OCR-free methods face a trade-off between capacity and precision: end-to-end models scale poorly with document length, while visual
retrieval-based pipelines are brittle and passive. We propose Doc-$V^*$, an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation. Doc-$V^*$ begins with a thumbnail overview, then actively navigates via semantic
retrieval and targeted page fetching, and aggregates evidence in a structured working
memory for grounded
reasoning. Trained by imitation learning from expert trajectories and further optimized with Group Relative Policy Optimization, Doc-$V^*$ balances answer accuracy with evidence-seeking efficiency. Across five
benchmarks, Doc-$V^*$ outperforms open-source baselines and approaches proprietary models, improving out-of-domain performance by up to 47.9% over RAG baseline. Other results reveal effective evidence aggregation with selective
attention, not increased input pages.
Read the full abstract ▾
All authors Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai
Computation and Language · Cross-listed from cs.IR
Hybrid Retrieval for COVID-19 Literature: Comparing Rank Fusion and Projection Fusion with Diversity Reranking
Harishkumar Kishorkumar Prajapati
We present a hybrid
retrieval system for COVID-19 scientific literature, evaluated on the TREC-COVID
benchmark (171,332 papers, 50 expert queries). The system implements six
retrieval configurations spanning sparse (SPLADE), dense (BGE), rank-level fusion (RRF), and a projection-based vector fusion (B5) approach. RRF fusion achieves the best relevance (nDCG@10 = 0.828), outperforming dense-only by 6.1% and sparse-only by 14.9%. Our projection fusion variant reaches nDCG@10 = 0.678 on expert queries while being 33% faster (847 ms vs. 1271 ms) and producing 2.2x higher ILD@10 than RRF. Evaluation across 400 queries – including expert, machine-generated, and three paraphrase styles – shows that B5 delivers the largest relative gain on keyword-heavy reformulations (+8.8%), although RRF remains best in absolute nDCG@10. On expert queries, MMR reranking increases intra-list diversity by 23.8-24.5% at a 20.4-25.4% nDCG@10 cost. Both fusion pipelines evaluated for latency remain below the sub-2 s target across all query sets. The system is deployed as a Streamlit web application backed by Pinecone serverless indices.
Read the full abstract ▾
Computation and Language
On Cost-Effective LLM-as-a-Judge Improvement Techniques
Ryan Lail and Luke Markham
Using a
language model to score or rank candidate responses has become a scalable alternative to human evaluation in reinforcement learning from human feedback (RLHF) pipelines, benchmarking, and application layer evaluations. However, output reliability depends heavily on prompting and aggregation strategy. We present an empirical investigation of four drop-in techniques – ensemble scoring, task-specific criteria injection, calibration context, and adaptive model escalation – for improving LLM judge accuracy on RewardBench 2, with a unifying lens of noise control on the
stochastic judge: ensembling as Monte Carlo averaging over per-call noise, criteria injection as between-response discrimination sharpening, and per-response score variance as an uncertainty signal. Ensemble scoring and task-specific criteria injection (the latter virtually cost free) together reach up to 85.8% accuracy, +13.5pp over baseline. Calibration context and adaptive model escalation also improve over baseline but are dominated by criteria + ensembling on the cost-accuracy Pareto frontier. Small models benefit disproportionately from ensembling, making high-accuracy LLM judges accessible at low cost. We show that these techniques generalise across model providers, evaluating on both OpenAI GPT and Anthropic Claude families.
Read the full abstract ▾
Computation and Language
Learning the Cue or Learning the Word? Analyzing Generalization in Metaphor Detection for Verbs
Sinan Kurtyigit, Sabine Schulte im Walde and Alexander Fraser
Metaphor detection models achieve strong
benchmark performance, yet it remains unclear whether this reflects transferable generalization or lexical memorization. To address this, we analyze generalization in metaphor detection through RoBERTa, the shared backbone of many state-of-the-art systems, focusing on English verbs using the VU Amsterdam Metaphor Corpus. We introduce a controlled lexical hold-out setup where all instances of selected target lemmas are strictly excluded from fine-tuning, and compare predictions on these Held-out lemmas against Exposed lemmas (verbs seen during fine-tuning). While the model performs best on Exposed lemmas, it maintains robust performance on Held-out lemmas. Further analysis reveals that sentence context alone is sufficient to match full-model performance on Held-out lemmas, whereas static verb-level
embeddings are not. Together, these results suggest that generalization is primarily driven by "learning the cue" (transferable contextual patterns), while "learning the word" (verb-specific memorization) provides an additive boost when lexical exposure is available.
Read the full abstract ▾
Computation and Language
Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Models
Dhruv Sahnan, Subhabrata Dutta, Tanmoy Chakraborty et al. (+2 more)
Professional fact-checkers rely on domain knowledge and deep contextual understanding to verify claims. Large
language models (LLMs) and large
reasoning models (LRMs) lack such grounding and primarily reason from available evidence alone, creating a mismatch between expert-led and fully automated claim verification. To mitigate this gap, we posit human-AI collaboration as a more promising path forward, where expert feedback, grounded in real-world knowledge and domain expertise, guides the model's
reasoning. However, existing LRMs are hard to calibrate to natural
language feedback, particularly in a multi-turn interaction setup. We propose Co-FactChecker, a framework for human-AI collaborative claim verification. We introduce a new interaction paradigm that treats the model's thinking trace as a shared scratchpad. Co-FactChecker translates expert feedback into trace-edits that introduce targeted modifications to the trace, sidestepping the shortcomings of dialogue-based interaction. We provide theoretical results showing that trace-editing offers advantages over multi-turn dialogue, and our automatic evaluations demonstrate that Co-FactChecker outperforms existing autonomous and human-AI collaboration approaches. Human evaluations further show that Co-FactChecker is preferred over multi-turn dialogue, producing higher quality
reasoning and verdicts along with relatively easier to interpret and more useful thinking traces.
Read the full abstract ▾
All authors Dhruv Sahnan, Subhabrata Dutta, Tanmoy Chakraborty, Preslav Nakov, Iryna Gurevych
Computation and Language
Beyond Arrow's Impossibility: Fairness as an Emergent Property of Multi-Agent Collaboration
Sayan Kumar Chaki, Antoine Gourru and Julien Velcin
Fairness in
language models is typically studied as a property of a single, centrally optimized model. As large
language models become increasingly agentic, we propose that
fairness emerges through interaction and exchange. We study this via a controlled hospital triage framework in which two
agents negotiate over three structured debate rounds. One
agent is aligned to a specific ethical framework via
retrieval-augmented generation (RAG), while the other is either unaligned or adversarially prompted to favor demographic groups over clinical need. We find that
alignment systematically shapes negotiation strategies and allocation patterns, and that neither
agent's allocation is ethically adequate in isolation, yet their joint final allocation can satisfy
fairness criteria that neither would have reached alone. Aligned
agents partially moderate bias through contestation rather than override, acting as corrective patches that restore access for marginalized groups without fully converting a biased counterpart. We further observe that even explicitly aligned
agents exhibit intrinsic biases toward certain frameworks, consistent with known left-leaning tendencies in LLMs. We connect these limits to Arrow's Impossibility
Theorem: no aggregation mechanism can simultaneously satisfy all desiderata of collective rationality, and multi-
agent deliberation navigates rather than resolves this constraint. Our results reposition
fairness as an emergent, procedural property of decentralized
agent interaction, and the system rather than the individual
agent as the appropriate unit of evaluation.
Read the full abstract ▾