Computation and Language
SemPA: Improving Sentence Embeddings of Large Language Models through Semantic Preference Alignment
Ziyang Chen, Zhenxuan Huang, Yile Wang et al. (+3 more)
Traditional sentence
embedding methods employ
token-level contrastive learning on non-generative pre-trained models. Recently, there have emerged
embedding methods based on generative large
language models (LLMs). These methods either rely on fixed
prompt templates or involve modifications to the model architecture. The former lacks further optimization of the model and results in limited performance, while the latter alters the internal computational mechanisms of the model, thereby compromising its generative capabilities. We propose SemPA, a novel approach that boosts the sentence representations while preserving the generative ability of LLMs via semantic preference
alignment. We leverage sentence-level Direct Preference Optimization (DPO) to efficiently optimize LLMs on a paraphrase generation task, where the model learns to discriminate semantically equivalent sentences while preserving inherent generative capacity. Theoretically, we establish a formal connection between DPO and contrastive learning under the Plackett-Luce model framework. Empirically, experimental results on both semantic textual similarity tasks and various
benchmarks for LLMs show that SemPA achieves better semantic representations without sacrificing the inherent generation capability of LLMs.
Read the full abstract ▾
All authors Ziyang Chen, Zhenxuan Huang, Yile Wang, Weiqin Wang, Lu Yin, Hui Huang
Computation and Language
Compositional Steering of Large Language Models with Steering Tokens
Gorjan Radevski, Kiril Gashteovski, Giwon Hong et al. (+2 more)
Deploying LLMs in real-world applications requires controllable output that satisfies multiple desiderata at the same time. While existing work extensively addresses LLM steering for a single behavior, compositional steering – i.e., steering LLMs simultaneously towards multiple behaviors – remains an underexplored problem. In this work, we propose compositional steering
tokens for multi-behavior steering. We first embed individual behaviors, expressed as natural
language instructions, into dedicated
tokens via self-
distillation. Contrary to most prior work, which operates in the activation space, our behavior steers live in the space of input
tokens, enabling more effective zero-shot composition. We then train a dedicated composition
token on pairs of behaviors and show that it successfully captures the notion of composition: it generalizes well to unseen compositions, including those with unseen behaviors as well as those with an unseen number of behaviors. Our experiments across different LLM architectures show that steering
tokens lead to superior multi-behavior steering of verifiable constraints (e.g., length, format, structure,
language) compared to competing approaches (instructions, activation steering, and LoRA merging). Moreover, we show that steering
tokens complement natural
language instructions, with their combination resulting in further gains.
Read the full abstract ▾
All authors Gorjan Radevski, Kiril Gashteovski, Giwon Hong, Carolin Lawrence, Goran Glavaš
Computation and Language · Cross-listed from cs.AI
Reinforced Efficient Reasoning via Semantically Diverse Exploration
Ziqi Zhao, Zhaochun Ren, Jiahong Zou et al. (+9 more)
Reinforcement learning with verifiable rewards (RLVR) has proven effective in enhancing the
reasoning of large
language models (LLMs). Monte Carlo Tree Search (MCTS)-based extensions improve upon vanilla RLVR (e.g., GRPO) by providing tree-based
reasoning rollouts that enable fine-grained and segment-level credit assignment. However, existing methods still suffer from limited exploration diversity and inefficient
reasoning. To address the above challenges, we propose reinforced efficient
reasoning via semantically diverse explorations, i.e., ROSE, for LLMs. To encourage more diverse
reasoning exploration, our method incorporates a semantic-
entropy-based branching strategy and an $\varepsilon$-exploration mechanism. The former operates on already sampled
reasoning rollouts to capture semantic uncertainty and select branching points with high semantic divergence to generate new successive
reasoning paths, whereas the latter stochastically initiates
reasoning rollouts from the root, preventing the search process from becoming overly local. To improve efficiency, we design a length-aware segment-level advantage
estimator that rewards concise and correct
reasoning while penalizing unnecessarily long
reasoning chains. Extensive experiments on various mathematical
reasoning benchmarks with Qwen and Llama models validate the effectiveness and efficiency of ROSE. Codes are available at https://github.com/ZiqiZhao1/ROSE-rl.
Read the full abstract ▾
All authors Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin
Computation and Language · Cross-listed from cs.AI
Publishing FAIR and Machine-actionable Reviews in Materials Science: The Case for Symbolic Knowledge in Neuro-symbolic Artificial Intelligence
Jennifer D'Souza, Soren Auer, Eleni Poupaki et al. (+6 more)
Scientific reviews are central to knowledge integration in materials science, yet their key insights remain locked in narrative text and static PDF tables, limiting reuse by humans and machines alike. This article presents a case study in atomic layer deposition and etching (ALD/E) where we publish review tables as FAIR, machine-actionable comparisons in the Open Research Knowledge Graph (ORKG), turning them into structured, queryable knowledge. Building on this, we contrast symbolic querying over ORKG with large
language model-based querying, and argue that a curated symbolic layer should remain the backbone of reliable neurosymbolic AI in materials science, with LLMs serving as complementary, symbolically grounded interfaces rather than standalone sources of truth.
Read the full abstract ▾
All authors Jennifer D'Souza, Soren Auer, Eleni Poupaki, Alex Watkins, Anjana Devi, Riikka L. Puurunen, Bora Karasulu, Adrie Mackus, Erwin Kessels
Computation and Language
ArcAligner: Adaptive Recursive Aligner for Compressed Context Embeddings in RAG
Jianbo Li, Yi Jiang, Sendong Zhao et al. (+3 more)
Retrieval-Augmented Generation (RAG) helps LLMs stay accurate, but feeding long documents into a
prompt makes the model slow and expensive. This has motivated context compression, ranging from
token pruning and summarization to
embedding-based compression. While researchers have tried ”compressing” these documents into smaller summaries or mathematical
embeddings, there is a catch: the more you compress the data, the more the LLM struggles to understand it. To address this challenge, we propose ArcAligner (Adaptive recursive context *Aligner*), a lightweight module integrated into the
language model layers to help the model better utilize highly compressed context representations for downstream generation. It uses an adaptive ”gating” system that only adds extra processing power when the information is complex, keeping the system fast. Across knowledge-intensive QA
benchmarks, ArcAligner consistently beats compression baselines at comparable compression rates, especially on multi-hop and long-tail settings. The source code is publicly available.
Read the full abstract ▾
All authors Jianbo Li, Yi Jiang, Sendong Zhao, Bairui Hu, Haochun Wang, Bing Qin
Computation and Language
Hán Dān Xué Bù (Mimicry) or Qīng Chū Yú Lán (Mastery)? A Cognitive Perspective on Reasoning Distillation in Large Language Models
Yueqing Hu, Xinyang Peng, Shuting Peng et al. (+2 more)
Recent Large
Reasoning Models trained via reinforcement learning exhibit a "natural"
alignment with human cognitive costs. However, we show that the prevailing paradigm of
reasoning distillation – training student models to mimic these traces via Supervised Fine-Tuning (SFT) – fails to transmit this cognitive structure. Testing the "Hán Dān Xué Bù" (Superficial Mimicry) hypothesis across 14 models, we find that
distillation induces a "Functional
Alignment Collapse": while teacher models mirror human difficulty scaling ($\bar{r}=0.64$), distilled students significantly degrade this
alignment ($\bar{r}=0.34$), often underperforming their own pre-
distillation baselines ("Negative Transfer"). Our analysis suggests that SFT induces a "Cargo Cult" effect, where students ritualistically replicate the linguistic form of
reasoning (verbosity) without internalizing the teacher's dynamic resource allocation policy. Consequently,
reasoning distillation decouples computational cost from cognitive demand, revealing that human-like cognition is an emergent property of active reinforcement, not passive imitation.
Read the full abstract ▾
All authors Yueqing Hu, Xinyang Peng, Shuting Peng, Hanqi Wang, Tianhong Wang
Computation and Language
Can Large Language Models Resolve Semantic Discrepancy in Self-Destructive Subcultures? Evidence from Jirai Kei
Peng Wang, Xilin Tao, Siyi Yao et al. (+5 more)
Self-destructive behaviors are linked to complex psychological states and can be challenging to diagnose. These behaviors may be even harder to identify within subcultural groups due to their unique expressions. As large
language models (LLMs) being deployed across various fields, some researchers have begun exploring their application for detecting self-destructive behaviors. Motivated by this, we investigate self-destructive behavior detection within subcultures using current LLM-based methods. However, these methods have two main challenges: (1) Knowledge Lag: Subcultural slang evolves rapidly, faster than LLMs' training cycles; and (2) Semantic Misalignment: it is challenging to grasp the specific and nuanced expressions unique to subcultures. To address these issues, we propose Subcultural
Alignment Solver (SAS), a multi-
agent framework that incorporates automatic
retrieval and subculture
alignment, significantly boosting the performance of LLMs in detecting self-destructive behavior. Our experimental results show that SAS outperforms the current advanced multi-
agent framework OWL. Notably, it competes well with fine-tuned LLMs. We hope that SAS will advance the field of self-destructive behavior detection in subcultural contexts and serve as a valuable resource for future researchers.
Read the full abstract ▾
All authors Peng Wang, Xilin Tao, Siyi Yao, Jiageng Wu, Yuntao Zou, Zhuotao Tian, Libo Qin, Dagang Li
Computation and Language · Cross-listed from cs.LG
On the Hidden Objective Biases of Group-based Reinforcement Learning
Aleksandar Fontana, Marco Simoni, Giulio Rossolini et al. (+2 more)
Group-based reinforcement learning methods, like Group Relative Policy Optimization (GRPO), are widely used nowadays to post-train large
language models. Despite their empirical success, they exhibit structural mismatches between reward optimization and the underlying training objective. In this paper, we present a theoretical analysis of GRPO style methods by studying them within a unified surrogate formulation. This perspective reveals recurring properties that affect all the methods under analysis: (i) non-uniform group weighting induces systematic gradient biases on shared prefix
tokens; (ii) interactions with the AdamW optimizer make training dynamics largely insensitive to reward scaling; and (iii) optimizer momentum can push policy updates beyond the intended clipping region under repeated optimization steps. We believe that these findings highlight fundamental limitations of current approaches and provide principled guidance for the design of future formulations.
Read the full abstract ▾
All authors Aleksandar Fontana, Marco Simoni, Giulio Rossolini, Andrea Saracino, Paolo Mori
Computation and Language
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization
Xueyun Tian, Minghua Ma, Bingbing Xu et al. (+6 more)
Supervised fine-tuning (SFT) on chain-of-thought (CoT) trajectories demonstrations is a common approach for enabling
reasoning in large
language models. Standard practices typically only retain trajectories with correct final answers (positives) while ignoring the rest (negatives). We argue that this paradigm discards substantial supervision and exacerbates overfitting, limiting out-of-domain (OOD) generalization. Specifically, we surprisingly find that incorporating negative trajectories into SFT yields substantial OOD generalization gains over positive-only training, as these trajectories often retain valid intermediate
reasoning despite incorrect final answers. To understand this effect in depth, we systematically analyze data, training dynamics, and inference behavior, identifying 22 recurring patterns in negative chains that serve a dual role: they moderate loss descent to mitigate overfitting during training and boost policy
entropy by 35.67% during inference to facilitate exploration. Motivated by these observations, we further propose Gain-based LOss Weighting (GLOW), an adaptive, sample-aware scheme that exploits such distinctive training dynamics by rescaling per-sample loss based on inter-epoch progress. Empirically, GLOW efficiently leverages unfiltered trajectories, yielding a 5.51% OOD gain over positive-only SFT on Qwen2.5-7B and boosting MMLU from 72.82% to 76.47% as an RL initialization.
Read the full abstract ▾
All authors Xueyun Tian, Minghua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen
Computation and Language · Cross-listed from cs.AI
ConMax: Confidence-Maximizing Compression for Efficient Chain-of-Thought Reasoning
Minda Hu, Zexuan Qiu, Zenan Xu et al. (+3 more)
Recent breakthroughs in Large
Reasoning Models (LRMs) have demonstrated that extensive Chain-of-Thought (CoT) generation is critical for enabling intricate cognitive behaviors, such as self-verification and backtracking, to solve complex tasks. However, this capability often leads to “overthinking”, where models generate redundant
reasoning paths that inflate computational costs without improving accuracy. While Supervised Fine-Tuning (SFT) on
reasoning traces is a standard paradigm for the 'cold start' phase, applying existing compression techniques to these traces often compromises logical coherence or incurs prohibitive sampling costs. In this paper, we introduce ConMax (Confidence-Maximizing Compression), a novel reinforcement learning framework designed to automatically compress
reasoning traces while preserving essential
reasoning patterns. ConMax formulates compression as a reward-driven optimization problem, training a policy to prune redundancy by maximizing a weighted combination of answer confidence for predictive fidelity and thinking confidence for
reasoning validity through a frozen auxiliary LRM. Extensive experiments across five
reasoning datasets demonstrate that ConMax achieves a superior efficiency-performance trade-off. Specifically, it reduces inference length by 43% over strong baselines at the cost of a mere 0.7% dip in accuracy, proving its effectiveness in generating high-quality, efficient training data for LRMs.
Read the full abstract ▾
All authors Minda Hu, Zexuan Qiu, Zenan Xu, Kun Li, Bo Zhou, Irwin King
Computation and Language
Text as a Universal Interface for Transferable Personalization
Yuting Liu, Jian Guan, Jia-Nan Li et al. (+4 more)
We study the problem of personalization in large
language models (LLMs). Prior work predominantly represents user preferences as implicit, model-specific vectors or parameters, yielding opaque “black-box” profiles that are difficult to interpret and transfer across models and tasks. In contrast, we advocate natural
language as a universal, model- and task-agnostic interface for preference representation. The formulation leads to interpretable and reusable preference descriptions, while naturally supporting continual
evolution as new interactions are observed. To learn such representations, we introduce a two-stage training framework that combines supervised fine-tuning on high-quality synthesized data with reinforcement learning to optimize long-term utility and cross-task transferability. Based on this framework, we develop AlignXplore+, a universal preference
reasoning model that generates textual preference summaries. Experiments on nine
benchmarks show that our 8B model achieves state-of-the-art performanc – outperforming substantially larger open-source models – while exhibiting strong transferability across tasks, model families, and interaction formats.
Read the full abstract ▾
All authors Yuting Liu, Jian Guan, Jia-Nan Li, Wei Wu, Jiang-Ming Yang, Jianzhe Zhao, Guibing Guo
Computation and Language
A Unified Spoken Language Model with Injected Emotional-Attribution Thinking for Human-like Interaction
Qing Wang, Zehan Li, Yaodong Song et al. (+6 more)
This paper presents a unified spoken
language model for emotional intelligence, enhanced by a novel data construction strategy termed Injected Emotional-Attribution Thinking (IEAT). IEAT incorporates user emotional states and their underlying causes into the model's internal
reasoning process, enabling
emotion-aware
reasoning to be internalized rather than treated as explicit supervision. The model is trained with a two-stage progressive strategy. The first stage performs
speech-text
alignment and emotional attribute modeling via self-
distillation, while the second stage conducts end-to-end cross-modal joint optimization to ensure consistency between textual and spoken emotional expressions. Experiments on the Human-like Spoken Dialogue Systems Challenge (HumDial) Emotional Intelligence
benchmark demonstrate that the proposed approach achieves top-ranked performance across emotional trajectory modeling, emotional
reasoning, and empathetic response generation under both LLM-based and human evaluations.
Read the full abstract ▾
All authors Qing Wang, Zehan Li, Yaodong Song, Hongjie Chen, Jian Kang, Jie Lian, Jie Li, Yongxiang Li, Xuelong Li
Computation and Language
GenProve: Learning to Generate Text with Fine-Grained Provenance
Jingxuan Wei, Xingyue Wang, Yanghaoyu Liao et al. (+5 more)
Large
language models (LLM) often hallucinate, and while adding citations is a common solution, it is frequently insufficient for accountability as users struggle to verify how a cited source supports a generated claim. Existing methods are typically coarse-grained and fail to distinguish between direct quotes and complex
reasoning. In this paper, we introduce Generation-time Fine-grained Provenance, a task where models must generate fluent answers while simultaneously producing structured, sentence-level provenance triples. To enable this, we present ReFInE (Relation-aware Fine-grained Interpretability & Evidence), a
dataset featuring expert verified annotations that distinguish between Quotation, Compression, and Inference. Building on ReFInE, we propose GenProve, a framework that combines Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization (GRPO). By optimizing a composite reward for answer fidelity and provenance correctness, GenProve significantly outperforms 14 strong LLMs in joint evaluation. Crucially, our analysis uncovers a
reasoning gap where models excel at surface-level quotation but struggle significantly with inference-based provenance, suggesting that verifiable
reasoning remains a frontier challenge distinct from surface-level citation.
Read the full abstract ▾
All authors Jingxuan Wei, Xingyue Wang, Yanghaoyu Liao, Jie Dong, Yuchen Liu, Caijun Jia, Bihui Yu, Junnan Zhu
Computation and Language
Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences
Arkadiusz Modzelewski, Paweł Golik, Anna Kołos et al. (+1 more)
Large
Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmful purposes. This leads us to our central question: Is LLM-generated persuasion more difficult to automatically detect than human-written persuasion? To address this, we categorize controllable generation approaches for producing persuasive content with LLMs and introduce Persuaficial, a high-quality
multilingual benchmark covering six
languages: English, German, Polish, Italian, French and Russian. Using this
benchmark, we conduct extensive empirical evaluations comparing human-authored and LLM-generated persuasive texts. We find that although overtly persuasive LLM-generated texts can be easier to detect than human-written ones, subtle LLM-generated persuasion consistently degrades automatic detection performance. Beyond detection performance, we provide the first comprehensive linguistic analysis contrasting human and LLM-generated persuasive texts, offering insights that may guide the development of more interpretable and robust detection tools.
Read the full abstract ▾
All authors Arkadiusz Modzelewski, Paweł Golik, Anna Kołos, Giovanni Da San Martino
Computation and Language
V-FAT: Benchmarking Visual Fidelity Against Text-bias
Ziteng Wang, Yujie He, Guanliang Li et al. (+3 more)
Recent advancements in
Multimodal Large
Language Models (MLLMs) have demonstrated impressive performance on standard visual
reasoning benchmarks. However, there is growing concern that these models rely excessively on linguistic shortcuts rather than genuine visual grounding, a phenomenon we term Text Bias. In this paper, we investigate the fundamental tension between visual perception and linguistic priors. We decouple the sources of this bias into two dimensions: Internal Corpus Bias, stemming from statistical correlations in pretraining, and External Instruction Bias, arising from the
alignment-induced tendency toward sycophancy. To quantify this effect, we introduce V-FAT (Visual Fidelity Against Text-bias), a diagnostic
benchmark comprising 4,026 VQA instances across six semantic domains. V-FAT employs a Three-Level Evaluation Framework that systematically increases the conflict between visual evidence and textual information: (L1) internal bias from atypical images, (L2) external bias from misleading instructions, and (L3) synergistic bias where both coincide. We introduce the Visual Robustness Score (VRS), a metric designed to penalize "lucky" linguistic guesses and reward true visual fidelity. Our evaluation of 12 frontier MLLMs reveals that while models excel in existing
benchmarks, they experience significant visual collapse under high linguistic dominance.
Read the full abstract ▾
All authors Ziteng Wang, Yujie He, Guanliang Li, Siqi Yang, Jiaqi Xiong, Songxiang Liu
Computation and Language
Faithful Summarisation under Disagreement via Belief-Level Aggregation
Favour Yahdii Aghaebe, Tanefa Apekey, Elizabeth Williams et al. (+1 more)
Opinion and multi-document summarisation often involve genuinely conflicting viewpoints, yet many existing approaches, particularly LLM-based systems, implicitly smooth disagreement and over-represent majority opinions. This limits the faithfulness of generated summaries in opinion-heavy settings. We introduce a disagreement-aware synthesis pipeline that separates belief-level aggregation from
language generation. Documents are first represented as structured belief sets and aggregated using distance-based belief merging operators that explicitly model conflict. Large
language models are then used only to realise the aggregated beliefs as natural
language summaries. We evaluate the approach across multiple model families and scales, comparing it to methods that perform explicit aggregation during generation. Our results show that while sufficiently large models can match belief-level aggregation when aggregation is handled at generation time, this behaviour is not stable across architectures or capacities. In contrast, belief-level aggregation combined with simple prompting yields consistently strong disagreement-aware performance across models, while maintaining fluent and grounded summaries.
Read the full abstract ▾
All authors Favour Yahdii Aghaebe, Tanefa Apekey, Elizabeth Williams, Nafise Sadat Moosavi
Computation and Language
CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters
Ao Sun, Xiaoyu Wang, Zhe Tan et al. (+4 more)
As Large
Language Models (LLMs) serve a global audience,
alignment must transition from enforcing universal consensus to respecting cultural pluralism. We demonstrate that dense models, when forced to fit conflicting value distributions, suffer from Mean Collapse, converging to a generic average that fails to represent diverse groups. We attribute this to Cultural Sparsity, where gradient interference prevents dense parameters from spanning distinct cultural modes. To resolve this, we propose \textbf{CuMA} (Cultural Mixture of Adapters), a framework that frames
alignment as a conditional capacity separation problem. By incorporating demographic-aware routing, CuMA internalizes a Latent Cultural
Topology to explicitly disentangle conflicting gradients into specialized expert subspaces. Extensive evaluations on WorldValuesBench, Community
Alignment, and PRISM demonstrate that CuMA achieves state-of-the-art performance, significantly outperforming both dense baselines and semantic-only MoEs. Crucially, our analysis confirms that CuMA effectively mitigates mean collapse, preserving cultural diversity. Our code is available at https://github.com/Throll/CuMA.
Read the full abstract ▾
All authors Ao Sun, Xiaoyu Wang, Zhe Tan, Yu Li, Jiachen Zhu, Yuheng Jia, Shu Su
Computation and Language
Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agent
Mingyue Cheng, Daoyu Wang, Qi Liu et al. (+7 more)
Synthesizing informative commercial reports from massive and noisy web sources is critical for high-stakes business decisions. Although recent deep research
agents (DRAs) achieve notable progress, their reports remain limited in quality, reliability, and coverage. These mainly stem from ambiguous intents that cause search drift, retrieved web content that rapidly saturates the context window, and single-pass synthesis that limits report comprehensiveness. In this work, we propose Mind2Report, a cognitive deep research
agent that emulates commercial analysts to synthesize expert-level reports. Mind2Report first probes fine-grained commercial intent to establish a structured outline, then recursively explores web sources and distills validated evidence into research
memory to preserve context efficiency. Meanwhile, the research
memory and outline continuously co-evolve, refining the report structure to avoid rigid initial planning. Finally, Mind2Report iteratively synthesizes the report based on the evolving outline and accumulated evidence. Together, these designs enable reliable and context-efficient long-horizon commercial deep research. To rigorously evaluate commercial DRAs, we further construct QRC-Eval, comprising 200 real-world commercial tasks and a holistic evaluation framework covering report quality, reliability, and coverage. Extensive experiments demonstrate that Mind2Report consistently outperforms leading proprietary and open-source DRAs, while ablations verify the effectiveness of each module and further analyze the challenges they address. We expect this work to advance the development of commercial deep research
agents.
Read the full abstract ▾
All authors Mingyue Cheng, Daoyu Wang, Qi Liu, Shuo Yu, Xiaoyu Tao, Yuqian Wang, Chengzhong Chu, Yu Duan, Mingkang Long, Enhong Chen
Computation and Language · Cross-listed from cs.AI
Higher-Order Knowledge Representations for Agentic Scientific Reasoning
Isabella A. Stewart and Markus J. Buehler
Scientific inquiry requires systems-level
reasoning that integrates heterogeneous experimental data, cross-domain knowledge, and mechanistic evidence into coherent explanations. While Large
Language Models (LLMs) offer inferential capabilities, they often depend on
retrieval-augmented contexts that lack structural depth. Traditional Knowledge Graphs (KGs) attempt to bridge this gap, yet their pairwise constraints fail to capture the irreducible higher-order interactions that govern emergent physical behavior. To address this, we introduce a methodology for constructing hypergraph-based knowledge representations that faithfully encode multi-entity relationships. Applied to a corpus of ~1,100 manuscripts on biocomposite scaffolds, our framework constructs a global hypergraph of 161,172 nodes and 320,201 hyperedges, revealing a scale-free
topology (power law exponent ~1.23) organized around highly connected conceptual hubs. This representation prevents the combinatorial explosion typical of pairwise expansions and explicitly preserves the co-occurrence context of scientific formulations. We further demonstrate that equipping agentic systems with hypergraph traversal tools, specifically using node-intersection constraints, enables them to bridge semantically distant concepts. By exploiting these higher-order pathways, the system successfully generates grounded mechanistic hypotheses for novel composite materials, such as linking cerium
oxide to PCL scaffolds via chitosan intermediates. This work establishes a "teacherless" agentic
reasoning system where hypergraph
topology acts as a verifiable guardrail, accelerating scientific discovery by uncovering relationships obscured by traditional graph methods.
Read the full abstract ▾
Computation and Language
EvolSQL: Structure-Aware Evolution for Scalable Text-to-SQL Data Synthesis
Xuanguang Pan, Chongyang Tao, Jiayuan Bai et al. (+5 more)
Training effective Text-to-SQL models remains challenging due to the scarcity of high-quality, diverse, and structurally complex
datasets. Existing methods either rely on limited human-annotated corpora, or synthesize
datasets directly by simply prompting LLMs without explicit control over SQL structures, often resulting in limited structural diversity and complexity. To address this, we introduce EvolSQL, a structure-aware data synthesis framework that evolves SQL queries from seed data into richer and more semantically diverse forms. EvolSQL starts with an exploratory Query-SQL expansion to broaden question diversity and improve schema coverage, and then applies an adaptive directional
evolution strategy using six atomic transformation operators derived from the SQL Abstract Syntax Tree to progressively increase query complexity across relational, predicate, aggregation, and nesting dimensions. An execution-grounded SQL refinement module and schema-aware deduplication further ensure the creation of high-quality, structurally diverse mapping pairs. Experimental results show that a 7B model fine-tuned on our data outperforms one trained on the much larger SynSQL
dataset using only 1/18 of the data.
Read the full abstract ▾
All authors Xuanguang Pan, Chongyang Tao, Jiayuan Bai, Jianling Gao, Zhengwei Tao, Xiansheng Zhou, Gavin Cheung, Shuai Ma
Computation and Language
A Navigational Approach for Comprehensive RAG via Traversal over Proposition Graphs
Maxime Delmas, Lei Xu and André Freitas
Standard RAG pipelines based on chunking excel at simple factual
retrieval but fail on complex multi-hop queries due to a lack of structural connectivity. Conversely, initial strategies that interleave
retrieval with
reasoning often lack global corpus awareness, while Knowledge Graph (KG)-based RAG performs strongly on complex multi-hop tasks but suffers on fact-oriented single-hop queries. To bridge this gap, we propose a novel RAG framework: ToPG (Traversal over Proposition Graphs). ToPG models its knowledge base as a heterogeneous graph of propositions, entities, and passages, effectively combining the granular fact density of propositions with graph connectivity. We leverage this structure using iterative Suggestion-Selection cycles, where the Suggestion phase enables a query-aware traversal of the graph, and the Selection phase provides LLM feedback to prune irrelevant propositions and seed the next iteration. Evaluated on three distinct QA tasks (Simple, Complex, and Abstract QA), ToPG demonstrates strong performance across both accuracy- and quality-based metrics. Overall, ToPG shows that query-aware graph traversal combined with factual granularity is a critical component for efficient structured RAG systems. ToPG is available at https://github.com/idiap/ToPG.
Read the full abstract ▾
Computation and Language
MisSpans: Fine-Grained False Span Identification in Cross-Domain Fake News
Zhiwei Liu, Paul Thompson, Jiaqi Rong et al. (+5 more)
Online misinformation is increasingly pervasive, yet most existing
benchmarks and methods evaluate veracity at the level of whole claims or paragraphs using coarse binary labels, obscuring how true and false details often co-exist within single sentences. These simplifications also limit interpretability: global explanations cannot identify which specific segments are misleading or differentiate how a detail is false (e.g., distorted vs. fabricated). To address these gaps, we introduce MisSpans, the first multi-domain, human-annotated
benchmark for span-level misinformation detection and analysis, consisting of paired real and fake news stories. MisSpans defines three complementary tasks: MisSpansIdentity for pinpointing false spans within sentences, MisSpansType for categorising false spans by misinformation type, and MisSpansExplanation for providing rationales grounded in identified spans. Together, these tasks enable fine-grained localisation, nuanced characterisation beyond true/false and actionable explanations. Expert annotators were guided by standardised guidelines and consistency checks, leading to high inter-annotator agreement. We evaluate 15 representative LLMs, including
reasoning-enhanced and non-
reasoning variants, under zero-shot and one-shot settings. Results reveal the challenging nature of fine-grained misinformation identification and analysis, and highlight the need for a deeper understanding of how performance may be influenced by multiple interacting factors, including model size and
reasoning capabilities, along with domain-specific textual features. This project will be available at https://github.com/lzw108/MisSpans.
Read the full abstract ▾
All authors Zhiwei Liu, Paul Thompson, Jiaqi Rong, Baojie Qu, Runteng Guo, Min Peng, Qianqian Xie, Sophia Ananiadou
Computation and Language
Projected Autoregression: Autoregressive Language Generation in Continuous State Space
Oshri Naparstek
Standard autoregressive
language models generate text by repeatedly selecting a discrete next
token, coupling prediction with irreversible commitment at every step. We show that
token selection is not the only viable autoregressive interface. Projected Autoregression replaces
token selection with continuous prediction in
embedding space followed by discrete projection at commitment time. The model predicts next-
token vectors via regression and contrastive objectives, while discrete
tokens arise only by nearest-neighbor projection. An optional mutable suffix (“
liquid tail”) enables iterative refinement before commitment, but the central change is more basic: next-step prediction is continuous, and discrete
tokens are produced only as a downstream interface. Projected Autoregression establishes a concrete alternative to
token-selection autoregression:
language generation can be organized around continuous-state prediction with delayed discrete commitment. Refinement remains local to a short causal suffix within a left-to-right causal process, rather than a sequence-wide denoising process. This separation has two consequences. First, it induces a distinct generation regime: even with immediate projection ($K{=}1$), continuous prediction yields text structure and dynamics that differ from tested
token-space AR baselines, including a compute-matched best-of-16 reranking baseline. Second, it exposes a continuous control surface inside autoregressive generation: direction rate, history noise, delayed commitment, state-space guidance, and
embedding geometry act directly on the evolving generative state before
token commitment. Taken together, these results place repeated
token selection within a larger family of autoregressive interfaces and expose continuous state space as a broader algorithmic design space for
language generation.
Read the full abstract ▾
Computation and Language
RAAR: Retrieval Augmented Agentic Reasoning for Cross-Domain Misinformation Detection
Zhiwei Liu, Runteng Guo, Baojie Qu et al. (+4 more)
Cross-domain misinformation detection is challenging, as misinformation arises across domains with substantial differences in knowledge and discourse. Existing methods often rely on single-perspective cues and struggle to generalize to challenging or underrepresented domains, while
reasoning large
language models (LLMs), though effective on complex tasks, are limited to same-distribution data. To address these gaps, we introduce RAAR, the first
retrieval-augmented agentic
reasoning framework for cross-domain misinformation detection. To enable cross-domain transfer beyond same-distribution assumptions, RAAR retrieves multi-perspective source-domain evidence aligned with each target sample's semantics, sentiment, and writing style. To overcome single-perspective modeling and missing systematic
reasoning, RAAR constructs verifiable multi-step
reasoning paths through specialized multi-
agent collaboration, where perspective-specific
agents produce complementary analyses and a summary
agent integrates them under verifier guidance. RAAR further applies supervised fine-tuning and reinforcement learning to train a single multi-task verifier to enhance verification and
reasoning capabilities. Based on RAAR, we trained the RAAR-8b and RAAR-14b models. Evaluation on three cross-domain misinformation detection tasks shows that RAAR substantially enhances the capabilities of the base models and outperforms other cross-domain methods, advanced LLMs, and LLM-based adaptation approaches. The project will be released at https://github.com/lzw108/RAAR.
Read the full abstract ▾
All authors Zhiwei Liu, Runteng Guo, Baojie Qu, Yuechen Jiang, Min Peng, Qianqian Xie, Sophia Ananiadou
Computation and Language
When AI Settles Down: Late-Stage Stability as a Signature of AI-Generated Text Detection
Ke Sun, Guangsheng Bao, Han Cui et al. (+1 more)
Zero-shot detection methods for AI-generated text typically aggregate
token-level statistics across entire sequences, overlooking the temporal dynamics inherent to autoregressive generation. We analyze over 120k text samples and reveal Late-Stage
Volatility Decay: AI-generated text exhibits rapidly stabilizing log probability fluctuations as generation progresses, while human writing maintains higher variability throughout. This divergence peaks in the second half of sequences, where AI-generated text shows 24–32% lower
volatility. Based on this finding, we propose two simple features: Derivative Dispersion and Local
Volatility, which computed exclusively from late-stage statistics. Without perturbation sampling or additional model access, our method achieves state-of-the-art performance on EvoBench and MAGE
benchmarks and demonstrates strong complementarity with existing global methods.
Read the full abstract ▾
All authors Ke Sun, Guangsheng Bao, Han Cui, Yue Zhang
Computation and Language · Cross-listed from cs.AI
DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
Guanzhi Deng, Bo Li, Ronghao Chen et al. (+5 more)
Mixture-of-Experts (MoE) has become a prominent paradigm for scaling Large
Language Models (LLMs). Parameter-efficient fine-tuning methods, such as LoRA, are widely adopted to adapt pretrained MoE LLMs to downstream tasks. However, existing approaches typically assign identical LoRA ranks to all expert modules, ignoring the heterogeneous specialization of pretrained experts. This uniform allocation leads to a resource mismatch: task-relevant experts are under-provisioned, while less relevant ones receive redundant parameters. To address this, we propose DR-LoRA, a Dynamic Rank LoRA framework for fine-tuning pretrained MoE models. Specifically, DR-LoRA initializes all expert LoRA modules with a small active rank and uses an expert saliency score, which combines routing frequency and gradient-based rank importance, to identify which experts would benefit most from additional capacity. It then periodically expands the active ranks of the task-critical expert LoRA, progressively constructing a heterogeneous rank distribution tailored to the target task. Experiments on three MoE models across six tasks show that DR-LoRA consistently outperforms LoRA and other strong baselines, demonstrating that task-adaptive heterogeneous rank allocation is an effective strategy to improve active capacity utilization in MoE fine-tuning.
Read the full abstract ▾
All authors Guanzhi Deng, Bo Li, Ronghao Chen, Xiujin Liu, Zhuo Han, Huacan Wang, Lijie Wen, Linqi Song
Computation and Language · Cross-listed from cs.AI
Defense Against Indirect Prompt Injection via Tool Result Parsing
Qiang Yu, Xinran Cheng and Chuanyi Liu
As LLM
agents transition from digital assistants to physical controllers in autonomous systems and robotics, they face an escalating threat from indirect
prompt injection. By
embedding adversarial instructions into the results of tool calls, attackers can hijack the
agent's decision-making process to execute unauthorized actions. This vulnerability poses a significant risk as
agents gain more direct control over physical environments. Existing defense mechanisms against Indirect
Prompt Injection (IPI) generally fall into two categories. The first involves training dedicated detection models; however, this approach entails high computational overhead for both training and inference, and requires frequent updates to keep pace with evolving attack vectors. Alternatively,
prompt-based methods leverage the inherent capabilities of LLMs to detect or ignore malicious instructions via
prompt engineering. Despite their flexibility, most current
prompt-based defenses suffer from high Attack Success Rates (ASR), demonstrating limited robustness against sophisticated injection attacks. In this paper, we propose a novel method that provides LLMs with precise data via tool result parsing while effectively filtering out injected malicious code. Our approach achieves competitive Utility under Attack (UA) while maintaining the lowest Attack Success Rate (ASR) to date, significantly outperforming existing methods. Code is available at GitHub.
Read the full abstract ▾
Computation and Language
Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework
Junhyuk Choi, Jeongyoun Kwon, Heeju Kim et al. (+4 more)
Multi-
agent systems utilizing large
language models often assign authoritative roles to improve performance, yet the impact of authority bias on
agent interactions remains underexplored. We present the first systematic analysis of role-based authority bias in free-form multi-
agent evaluation using ChatEval. Applying French and Raven's power-based theory, we classify authoritative roles into legitimate, referent, and expert types and analyze their influence across 12-turn conversations. Experiments with GPT-4o and DeepSeek R1 reveal that Expert and Referent power roles exert stronger influence than Legitimate power roles. Crucially, authority bias emerges not through active conformity by general
agents, but through authoritative roles consistently maintaining their positions while general
agents demonstrate flexibility. Furthermore, authority influence requires clear position statements, as neutral responses fail to generate bias. These findings provide key insights for designing multi-
agent frameworks with asymmetric interaction patterns.
Read the full abstract ▾
All authors Junhyuk Choi, Jeongyoun Kwon, Heeju Kim, Haeun Cho, Hayeong Jung, Sehee Min, Bugeun Kim
Computation and Language
NC2C: Automated Convexification of Generic Non-Convex Optimization Problems
Xinyue Peng, Yanming Liu, Yihan Cang et al. (+4 more)
Non-convex optimization problems are pervasive across mathematical programming, engineering design, and scientific computing, often posing intractable challenges for traditional solvers due to their complex objective functions and constrained landscapes. To address the inefficiency of manual convexification and the over-reliance on expert knowledge, we propose NC2C, an LLM-based end-to-end automated framework designed to transform generic non-convex optimization problems into solvable convex forms using large
language models. NC2C leverages LLMs' mathematical
reasoning capabilities to autonomously detect non-convex components, select optimal convexification strategies, and generate rigorous convex equivalents. The framework integrates symbolic
reasoning, adaptive transformation techniques, and iterative validation, equipped with error correction loops and feasibility domain correction mechanisms to ensure the robustness and validity of transformed problems. Experimental results on a diverse
dataset of 100 generic non-convex problems demonstrate that NC2C achieves an 89.3% execution rate and a 76% success rate in producing feasible, high-quality convex transformations. This outperforms baseline methods by a significant margin, highlighting NC2C's ability to leverage LLMs for automated non-convex to convex transformation, reduce expert dependency, and enable efficient deployment of convex solvers for previously intractable optimization tasks.
Read the full abstract ▾
All authors Xinyue Peng, Yanming Liu, Yihan Cang, Yuwei Zhang, Xinyi Wang, Songhang Deng, Jiannan Cao
Computation and Language · Cross-listed from cs.CV
CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Models
Tobia Poppi, Burak Uzkent, Amanmeet Garg et al. (+7 more)
Video-
language models (VLMs) achieve strong
multimodal understanding but remain prone to
hallucinations, especially when
reasoning about actions and temporal order. Existing mitigation strategies, such as textual filtering or random
video perturbations, often fail to address the root cause: over-reliance on
language priors rather than fine-grained visual dynamics. We propose a scalable framework for counterfactual
video generation that synthesizes
videos differing only in actions or temporal structure while preserving scene context. Our pipeline combines
multimodal LLMs for action proposal and editing guidance with
diffusion-based image and
video models to generate semantic hard negatives at scale. Using this framework, we build CounterVid, a synthetic
dataset of ~26k preference pairs constructed from short counterfactual action clips and targeting both action recognition and controlled action-sequence ordering. We further introduce MixDPO, a unified Direct Preference Optimization approach that jointly leverages textual and visual preferences. Across Qwen2.5-VL and InternVL3 backbones, MixDPO substantially improves action recognition and temporal ordering, yields gains on most standard
video hallucination benchmarks, and largely preserves general
video understanding. Our source code, trained models, and
dataset are available at https://aimagelab.github.io/CounterVid.
Read the full abstract ▾
All authors Tobia Poppi, Burak Uzkent, Amanmeet Garg, Lucas Porto, Garin Kessler, Yezhou Yang, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara, Florian Schiffers