Computation and Language
DeepPrune: Parallel Scaling without Inter-trace Redundancy
Shangqing Tu, Yaxuan Li, Yushi Bai et al. (+2 more)
Parallel scaling has emerged as a powerful paradigm to enhance
reasoning capabilities in large
language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy – our analysis reveals that over 80% of parallel
reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic
pruning. Our method features a specialized judge model trained with out-of-distribution data (AIME 2022, AIME 2023, and MATH 500) using oversampling techniques to accurately predict answer equivalence from partial
reasoning traces, achieving 0.7072 AUROC on unseen
reasoning models. Combined with an online greedy clustering
algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging
benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple
reasoning models demonstrate that DeepPrune achieves remarkable
token reduction of 65.73%–88.50% compared to conventional consensus sampling, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel
reasoning, making high-performance
reasoning more efficient. Our code and data are here: https://deepprune.github.io/.
Read the full abstract ▾
All authors Shangqing Tu, Yaxuan Li, Yushi Bai, Lei Hou, Juanzi Li
Computation and Language · Cross-listed from cs.CV
The Visual Iconicity Challenge: Evaluating Vision-Language Models on Sign Language Form-Meaning Mapping
Onur Keleş, Asl{\i} Özyürek, Gerardo Ortega et al. (+2 more)
Iconicity, the resemblance between linguistic form and meaning, is pervasive in signed
languages, offering a natural testbed for visual grounding. For
vision-
language models (VLMs), the challenge is to recover such essential mappings from dynamic human motion rather than static context. We introduce the Visual Iconicity Challenge, a novel
video-based
benchmark that adapts psycholinguistic measures to evaluate VLMs on three tasks: (i) phonological sign-form prediction (e.g., handshape, location), (ii) transparency (inferring meaning from visual form), and (iii) graded iconicity ratings. We assess 13 state-of-the-art VLMs in zero- and few-shot settings on Sign
Language of the Netherlands and compare them to human baselines. On phonological form prediction, VLMs recover some handshape and location detail but remain below human performance; on transparency, they are far from human baselines; and only top models correlate moderately with human iconicity ratings. Interestingly, models with stronger phonological form prediction correlate better with human iconicity judgment, indicating shared sensitivity to visually grounded structure. Our findings validate these diagnostic tasks and motivate human-centric signals and
embodied learning methods for modelling iconicity and improving visual grounding in
multimodal models.
Read the full abstract ▾
All authors Onur Keleş, Asl{\i} Özyürek, Gerardo Ortega, Kadir Gökgöz, Esam Ghaleb
Computation and Language · Cross-listed from cs.AI
Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
Bianca-Mihaela Ganescu, Suchir Salhan, Andrew Caines et al. (+1 more)
Training
vision-
language models on cognitively-plausible amounts of data requires rethinking how models integrate
multimodal information. Within the constraints of the
Vision track for the BabyLM Challenge 2025, we propose a lightweight decoder-based architecture with (1)
token-wise dynamic gating for adaptive fusion of linguistic and visual cues, (2) feature modulation and channel
attention to maximise the utility of limited visual information and (3) auxiliary contrastive objectives for visual grounding. Evaluation on five
benchmarks (BLiMP, BLiMP Supplement, EWoK, Winoground and VQA) shows competitive or superior performance to
multimodal baselines. More notably, our dynamic gate discovers interpretable patterns without explicit supervision, favouring visual cues for content words and linguistic cues for function words. While we identify limitations in the Challenge constraints, such as the information bottleneck created by global image
embeddings and training instability from the
dataset split, our findings establish dynamic gating as a powerful tool for efficient
multimodal learning, offering both interpretability and performance even under severe constraints.
Read the full abstract ▾
All authors Bianca-Mihaela Ganescu, Suchir Salhan, Andrew Caines, Paula Buttery
Computation and Language
LeWiDi-2025 at NLPerspectives: Third Edition of the Learning with Disagreements Shared Task
Elisa Leonardelli, Silvia Casola, Siyao Peng et al. (+8 more)
Many researchers have reached the conclusion that AI models should be trained to be aware of the possibility of variation and disagreement in human judgments, and evaluated as per their ability to recognize such variation. The LEWIDI series of shared tasks on Learning With Disagreements was established to promote this approach to training and evaluating AI models, by making suitable
datasets more accessible and by developing evaluation methods. The third edition of the task builds on this goal by extending the LEWIDI
benchmark to four
datasets spanning paraphrase identification, irony detection, sarcasm detection, and natural
language inference, with labeling schemes that include not only categorical judgments as in previous editions, but ordinal judgments as well. Another novelty is that we adopt two complementary paradigms to evaluate disagreement-aware systems: the soft-label approach, in which models predict population-level distributions of judgments, and the perspectivist approach, in which models predict the interpretations of individual annotators. Crucially, we moved beyond standard metrics such as cross-
entropy, and tested new evaluation metrics for the two paradigms. The task attracted diverse participation, and the results provide insights into the strengths and limitations of methods to modeling variation. Together, these contributions strengthen LEWIDI as a framework and provide new resources,
benchmarks, and findings to support the development of disagreement-aware technologies.
Read the full abstract ▾
All authors Elisa Leonardelli, Silvia Casola, Siyao Peng, Giulia Rizzi, Valerio Basile, Elisabetta Fersini, Diego Frassinelli, Hyewon Jang, Maja Pavlovic, Barbara Plank, Massimo Poesio
Computation and Language
Shuang Chen, Yue Guo, Yimeng Ye et al. (+7 more)
Recent advances in
multimodal large
reasoning models (MLRMs) have substantially improved their ability to solve complex textual and visual tasks. However, these models tend to overthink on simple problems, producing unnecessarily lengthy
reasoning traces, while under-exploring on challenging ones, leading to missed solutions. To address this imbalance, we propose ARES, a unified open-source framework for adaptive
reasoning that dynamically allocates exploration effort based on task difficulty. Our approach is motivated by two key empirical findings: (i) while single-
token entropy is noisy, high window-
entropy (HWE)
tokens (
token-level
entropies averaged under a sliding window) can reliably capture
reasoning-critical moments; and (ii) reducing HWE usage benefits easy problems, while increasing it is essential for solving hard ones. Building on these insights, ARES introduces a two-stage training pipeline. In the Adaptive Cold-Start stage, we curate
multimodal and textual data paired with
reasoning traces of length proportional to problem difficulty, equipping the model with initial difficulty awareness. In the second stage, we develop Adaptive
Entropy Policy Optimization (AEPO), which uses HWE
tokens as exploration triggers to decide when to explore, and a hierarchical
entropy reward with dynamic KL control to decide how much to explore. Extensive experiments demonstrate that ARES achieves superior performance and
reasoning efficiency across diverse mathematical, logical, and
multimodal benchmarks, while closing the gap to leading commercial systems under significantly lower inference costs.
Read the full abstract ▾
All authors Shuang Chen, Yue Guo, Yimeng Ye, Shijue Huang, Wenbo Hu, Haoxi Li, Manyuan Zhang, Jiayu Chen, Song Guo, Nanyun Peng
Computation and Language · Cross-listed from cs.LG
xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
Cheng Qian, Zuxin Liu, Shirley Kokane et al. (+10 more)
Modern LLM deployments confront a widening cost-performance spectrum: premium models deliver strong
reasoning but are expensive, while lightweight models are economical yet brittle on complex tasks. Static escalation rules and keyword heuristics under-utilize this spectrum and fail to adapt across task types. We present xRouter, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models. The router is trained end-to-end with reinforcement learning using an explicit, cost-aware reward that encodes cost-performance trade-offs, eliminating the need for hand-engineered routing rules. Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting, as well as the deployment and evaluation pipelines. Across diverse
benchmarks, xRouter achieves strong cost-performance trade-offs (e.g., substantial cost reductions at comparable task completion rates), and provides empirical insights into what reliably helps learned routing and what does not, ranging from model trainability to the difficulty of eliciting sophisticated orchestration behaviors in small open models. We hope these findings and our open implementation will serve as a practical substrate for advancing learned, cost-aware LLM orchestration.
Read the full abstract ▾
All authors Cheng Qian, Zuxin Liu, Shirley Kokane, Akshara Prabhakar, Jielin Qiu, Haolin Chen, Zhiwei Liu, Heng Ji, Weiran Yao, Shelby Heinecke, Silvio Savarese, Caiming Xiong, Huan Wang
Computation and Language
Single layer tiny Co$^4$ outpaces GPT-2 and GPT-BERT
Noor Ul Zain, Mohsin Raza and Ahsan Adeel
We show that a tiny Co$^4$ machine(Adeel,2025) with a single layer, two heads, and 8M parameters, operating at an approximate cost of $O(N)$ (where $N$ is the number of input
tokens), outpaces the BabyLM Challenge baselines GPT-2 (124M, 12 layers, $O(N^2))$ and GPT-BERT (30M, 12 layers, $O(N^2))$ in just two epochs, while both are trained for ten. Co$^4$ achieves orders-of-magnitude greater training efficiency on 10M
tokens, demonstrating highly sample efficient pretraining. Using the BabyLM challenge evaluation pipeline across complex
benchmarks, Co$^4$ exhibits strong zero-shot and fine-tuning performance on SuperGLUE tasks. Specifically, Co$^4$ outperforms GPT-2 on 5 out of 7 zero-shot metrics and 6 out of 7 fine-tuning tasks, and GPT-BERT on 4 out of 7 metrics in both cases. These results suggest the need to rethink prevailing deep learning paradigms and associated scaling laws.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
Heming Zou, Yunliang Zang, Wutong Xu et al. (+2 more)
Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for foundation models, but it suffers from parameter interference, resulting in suboptimal performance. Although Mixture-of-Experts (MoE)-based LoRA variants show promise in mitigating intra-task correlations in single-task instruction tuning, they introduce additional router parameters and remain ineffective in multi-task model merging where inter-task interference arises. Inspired by the fly olfactory
circuit, we propose FlyLoRA, an implicit MoE-based LoRA variant that introduces: (1) rank-wise expert activation in the up-projection
matrix, and (2) an implicit router that unifies expert routing and down-projection, where a frozen sparse random projection
matrix replaces the traditional dense trainable version. This design resolves the trade-off between intra-task decorrelation and computational efficiency by eliminating the need for an explicit router, while inherently mitigating inter-task interference due to the orthogonality property of random
matrices. Extensive experiments across four domains – general knowledge understanding, scientific question answering, mathematical
reasoning, and code generation – demonstrate consistent performance improvements over existing methods. Beyond empirical gains, FlyLoRA highlights how biological structures can inspire innovations in AI technologies. Code is available at https://github.com/gfyddha/FlyLoRA.
Read the full abstract ▾
All authors Heming Zou, Yunliang Zang, Wutong Xu, Yao Zhu, Xiangyang Ji
Computation and Language
If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models
Jasmin Orth, Philipp Mondorf and Barbara Plank
Conditional acceptability refers to how plausible a conditional statement is perceived to be. It plays an important role in communication and
reasoning, as it influences how individuals interpret implications, assess arguments, and make decisions based on hypothetical scenarios. When humans evaluate how acceptable a conditional "If A, then B" is, their judgments are influenced by two main factors: the $\textit{conditional probability}$ of $B$ given $A$, and the $\textit{semantic relevance}$ of the antecedent $A$ given the consequent $B$ (i.e., whether $A$ meaningfully supports $B$). While prior work has examined how large
language models (LLMs) draw inferences about conditional statements, it remains unclear how these models judge the $\textit{acceptability}$ of such statements. To address this gap, we present a comprehensive study of LLMs' conditional acceptability judgments across different model families, sizes, and prompting strategies. Using linear mixed-effects models and ANOVA tests, we find that models are sensitive to both conditional probability and semantic relevance$\unicode{x2014}$though to varying degrees depending on architecture and prompting style. A comparison with human data reveals that while LLMs incorporate probabilistic and semantic cues, they do so less consistently than humans. Notably, larger models do not necessarily align more closely with human judgments.
Read the full abstract ▾
Computation and Language
On the Relationship Between the Choice of Representation and In-Context Learning
Ioana Marinescu, Kyunghyun Cho and Eric Karl Oermann
In-context learning (ICL) is the ability of a large
language model (LLM) to learn a new task from a few demonstrations presented as part of the context. Past studies have attributed a large portion of the success of ICL to the way these in-context demonstrations are represented, particularly to how labels are represented in classification tasks. On the other hand, observations of the learning capacity of ICL (i.e., the extent to which more in-context demonstrations can lead to higher performance) have been mixed, and ICL is often thought to occur only under specific conditions. The interaction between these two aspects in ICL, representation and learning, has not been studied in depth until now. We hypothesize that they are largely independent of one another, such that the representation of demonstrations determines the baseline accuracy of ICL, while learning from additional demonstrations improves only on top of this baseline. We validate this hypothesis by developing an optimization
algorithm that can enumerate a spectrum of possible label sets (representations) varying in semantic relevance. We then perform ICL with varying numbers of in-context demonstrations for each of these label sets. We observed that learning happens regardless of the quality of the label set itself, although its efficiency, measured by the slope of improvement over in-context demonstrations, is conditioned on both the label set quality and the parameter count of the underlying
language model. Despite the emergence of learning, the relative quality (accuracy) of the choice of a label set (representation) is largely maintained throughout learning, confirming our hypothesis and implying their orthogonality. Our work reveals a previously underexplored aspect of ICL: the independent effects of learning from demonstrations and their representations on ICL performance.
Read the full abstract ▾
Computation and Language
Two-Stage Voting for Robust and Efficient Suicide Risk Detection on Social Media
Yukai Song, Pengfei Zhou, César Escobar-Viera et al. (+3 more)
Suicide rates have risen worldwide in recent years, underscoring the urgent need for proactive prevention strategies. Social media provides valuable signals, as many at-risk individuals - who often avoid formal help due to stigma - choose instead to share their distress online. Yet detecting implicit suicidal ideation, conveyed indirectly through metaphor, sarcasm, or subtle emotional cues, remains highly challenging. Lightweight models like BERT handle explicit signals but fail on subtle implicit ones, while large
language models (LLMs) capture nuance at prohibitive computational cost. To address this gap, we propose a two-stage voting architecture that balances efficiency and robustness. In Stage 1, a lightweight BERT classifier rapidly resolves high-confidence explicit cases. In Stage 2, ambiguous inputs are escalated to either (i) a multi-perspective LLM voting framework to maximize recall on implicit ideation, or (ii) a feature-based ML ensemble guided by psychologically grounded indicators extracted via
prompt-engineered LLMs for efficiency and interpretability. To the best of our knowledge, this is among the first works to operationalize LLM-extracted psychological features as structured vectors for suicide risk detection. On two complementary
datasets - explicit-dominant Reddit and implicit-only DeepSuiMind - our framework outperforms single-model baselines, achieving 98.0% F1 on explicit cases, 99.7% on implicit ones, and reducing the cross-domain gap below 2%, while significantly lowering LLM cost.
Read the full abstract ▾
All authors Yukai Song, Pengfei Zhou, César Escobar-Viera, Candice Biernesser, Wei Huang, Jingtong Hu
Computation and Language
AutoRed: A Free-form Adversarial Prompt Generation Framework for Automated Red Teaming
Muxi Diao, Yutao Mou, Keqing He et al. (+6 more)
The safety of Large
Language Models (LLMs) is crucial for the development of trustworthy AI applications. Existing red teaming methods often rely on seed instructions, which limits the semantic diversity of the synthesized
adversarial prompts. We propose AutoRed, a free-form
adversarial prompt generation framework that removes the need for seed instructions. AutoRed operates in two stages: (1) persona-guided
adversarial instruction generation, and (2) a reflection loop to iteratively refine low-quality
prompts. To improve efficiency, we introduce a verifier to assess
prompt harmfulness without querying the target models. Using AutoRed, we build two red teaming
datasets – AutoRed-Medium and AutoRed-Hard – and evaluate eight state-of-the-art LLMs. AutoRed achieves higher attack success rates and better generalization than existing baselines. Our results highlight the limitations of seed-based approaches and demonstrate the potential of free-form red teaming for LLM safety evaluation. We will open source our
datasets in the near future.
Read the full abstract ▾
All authors Muxi Diao, Yutao Mou, Keqing He, Hanbo Song, Lulu Zhao, Shikun Zhang, Wei Ye, Kongming Liang, Zhanyu Ma
Computation and Language · Cross-listed from cs.AI
Beyond Pass@k: Breadth-Depth Metrics for Reasoning Boundaries
Marius Dragoi, Ioana Pintilie, Florin Gogianu et al. (+1 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm to improve Large
Language Models on
reasoning tasks such as coding, math or logic. To assess the
reasoning boundary (the fraction of problems a model can solve) researchers often report Pass@k at large sampling budgets. Recent results reveal a crossover phenomenon: while RLVR models outperform the base model at small k values, the base model usually outperforms them when sampling a very large number of completions. This has been interpreted as evidence that base models have a larger
reasoning boundary. We argue that on tasks with discrete answer spaces, such as math with numeric outputs, Pass@k at large k reflects the increasingly higher chance of success in the limit of the number of trials rather than genuine
reasoning, and can therefore be misleading. We propose Cover@tau, which measures the fraction of problems that a model can solve for which at least a tau proportion of completions are correct. Unlike Pass@k, Cover@tau captures
reasoning under an explicit reliability threshold: models that rely on random guessing degrade rapidly as tau increases. We evaluate several RLVR models using Cover@tau-based metrics and illustrate how the relative rankings of popular
algorithms change compared to Pass@1, offering a different perspective on
reasoning boundaries.
Read the full abstract ▾
All authors Marius Dragoi, Ioana Pintilie, Florin Gogianu, Florin Brad
Computation and Language
Neuron-Level Analysis of Cultural Understanding in Large Language Models
Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala et al. (+1 more)
As large
language models (LLMs) are increasingly deployed worldwide, ensuring their fair and comprehensive cultural understanding is important. However, LLMs exhibit cultural bias and limited awareness of underrepresented cultures, while the mechanisms underlying their cultural understanding remain underexplored. To fill this gap, we conduct a
neuron-level analysis to identify
neurons that drive cultural behavior, introducing a gradient-based scoring method with additional filtering for precise refinement. We identify culture-general
neurons contributing to cultural understanding regardless of cultures, and culture-specific
neurons tied to an individual culture. Culture-general and culture-specific
neurons account for less than 1% of all
neurons and are concentrated in shallow to middle MLP layers. We validate their role by showing that suppressing them substantially degrades performance on cultural
benchmarks (by up to 30%), while performance on general natural
language understanding (NLU)
benchmarks remains largely unaffected. Moreover, we show that culture-specific
neurons support knowledge of not only the target culture, but also related cultures. Finally, we demonstrate that training on NLU
benchmarks can diminish models' cultural understanding when we update modules containing many culture-general
neurons. These findings provide insights into the internal mechanisms of LLMs and offer practical guidance for model training and engineering. Our code is available at https://github.com/ynklab/CULNIG
Read the full abstract ▾
All authors Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala, Hitomi Yanaka
Computation and Language
Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window
Qiaoyu Tang, Hao Xiang, Le Yu et al. (+11 more)
While recent advances in
reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep
reasoning capabilities in multi-turn
agents with long-horizon interactions. We propose DeepMiner, a novel framework that elicits such abilities by introducing high-difficulty training tasks and dynamic context window. DeepMiner presents a reverse construction method to generate complex but verifiable question-answer pairs from authentic web sources, which ensures the challenge and reliability of training data while injecting cognitive capabilities into multi-turn
reasoning scenarios. We further design an elegant yet effective dynamic context management strategy for both training and inference, utilizing sliding window mechanisms while eliminating the dependency on external summarization models, thereby efficiently empowering the model to handle continuously expanding long-horizon contexts. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search
agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source
agent by almost 20 percentage points, and demonstrates consistent improvements on BrowseComp-zh, XBench-DeepSearch, and
GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within standard 32k context length, effectively addressing the context limitations that constrain existing multi-turn interaction systems.
Read the full abstract ▾
All authors Qiaoyu Tang, Hao Xiang, Le Yu, Bowen Yu, Yaojie Lu, Xianpei Han, Le Sun, WenJuan Zhang, Pengbo Wang, Shixuan Liu, Zhenru Zhang, Jianhong Tu, Hongyu Lin, Junyang Lin
Computation and Language · Cross-listed from cs.LG
Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
Jason Bohne, Pawel Polak, David Rosenberg et al. (+2 more)
Direct Preference Optimization (DPO) has recently emerged as a simple and effective alternative to reinforcement learning from human feedback (RLHF) for aligning large
language models (LLMs) with user preferences. However, existing DPO formulations rely on a single monolithic model, which limits their expressivity in multi-task settings and their adaptability to heterogeneous or diverse preference distributions. In this work, we propose Mix- and MoE-DPO, a framework that extends DPO with both soft mixture models and mixture-of-experts (MoE) architectures, using a
stochastic variational inference approach. Our method introduces a latent-variable model over expert assignments and optimizes a variational evidence lower bound (ELBO), enabling stable and efficient learning of specialized expert policies from preference data. Mix- and MoE-DPO provides three key advantages over standard DPO: (i) generalization via universal function approximation through mixtures; (ii) reward and policy specialization through expert components tailored to distinct preference modes; and (iii) contextual
alignment through input-dependent soft gating that enables user-specific mixture policies. Our framework supports both shared base architectures with expert-specific policy heads and fully independent expert models, allowing flexible trade-offs between parameter efficiency and specialization. We validate our approach on a variety of model sizes and multi-preference
datasets, demonstrating that Mix- and MoE-DPO offers a powerful and scalable method for preference-based LLM
alignment.
Read the full abstract ▾
All authors Jason Bohne, Pawel Polak, David Rosenberg, Brian Bloniarz, Gary Kazantsev
Computation and Language · Cross-listed from cs.LG
Opponent Shaping in LLM Agents
Marta Emili Garcia Segura, Stephen Hailes and Mirco Musolesi
Large
Language Models (LLMs) are increasingly being deployed as autonomous
agents in real-world environments. As these deployments scale, multi-
agent interactions become inevitable, making it essential to understand strategic behavior in such systems. A central open question is whether LLM
agents, like reinforcement learning
agents, can shape the learning dynamics and influence the behavior of others through interaction alone. In this paper, we present the first investigation of opponent shaping (OS) with LLM-based
agents. Existing OS
algorithms cannot be directly applied to LLMs, as they require higher-order derivatives, face scalability constraints, or depend on architectural components that are absent in
transformers. To address this gap, we introduce ShapeLLM, an adaptation of model-free OS methods tailored for
transformer-based
agents. Using ShapeLLM, we examine whether LLM
agents can influence co-players' learning dynamics across diverse game-theoretic environments. We demonstrate that LLM
agents can successfully guide opponents toward exploitable
equilibria in competitive games (Iterated Prisoner's Dilemma, Matching Pennies, and Chicken) and promote coordination and improve collective welfare in cooperative games (Iterated Stag Hunt and a cooperative version of the Prisoner's Dilemma). Our findings show that LLM
agents can both shape and be shaped through interaction, establishing opponent shaping as a key dimension of multi-
agent LLM research.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Jianlyu Chen, Junwei Lan, Chaofan Li et al. (+2 more)
In this paper, we introduce ReasonEmbed, a novel text
embedding model developed for
reasoning-intensive document
retrieval. Our work includes three key technical contributions. First, we propose ReMixer, a new data synthesis method that overcomes the triviality problem prevalent in previous synthetic
datasets, enabling large-scale production of 82K high-quality training samples. Second, we design Redapter, a self-adaptive learning
algorithm that dynamically adjusts training each sample's weight based on its
reasoning intensity. This allows the model to effectively capture the complex semantic relationships between queries and documents. Third, we implement ReasonEmbed across multiple backbones of varying sizes, all of which achieve superior performance on
reasoning-intensive
retrieval tasks. Notably, our ReasonEmbed-Qwen3-8B model offers a record-high nDCG@10 score of 38.1 on the BRIGHT
benchmark, which significantly outperforms existing text
embedding models. We will fully open-source our created resources in ReasonEmbed to push forward the research advancement in this field.
Read the full abstract ▾
All authors Jianlyu Chen, Junwei Lan, Chaofan Li, Defu Lian, Zheng Liu
Computation and Language
Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
Jannek Ulm, Kevin Du and Vésteinn Snæbjarnarson
Large
language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to train on synthetic data sampled from LLMs. In this work, we build on this idea and investigate the benefits of contrastive decoding for generating synthetic corpora. In a controlled setting, we experiment with sampling corpora using the relative difference between a good and bad model trained on the same original corpus of 100 million words. By amplifying the signal from a model that has better performance, we create a synthetic corpus and mix it with the original training data. Our findings show that training on a mixture of synthesized and real data improves performance on the
language modeling objective and a range of downstream tasks. In particular, we see that training with a mix of synthetic data from contrastive decoding benefits tasks that require more
reasoning skills, while synthetic data from traditional sampling helps more on tasks dependent on surface level linguistic capabilities.
Read the full abstract ▾
Computation and Language
The Alignment Waltz: Jointly Training Agents to Collaborate for Safety
Jingyu Zhang, Haozhu Wang, Eric Michael Smith et al. (+7 more)
Harnessing the power of LLMs requires a delicate dance between being helpful and harmless. This creates a fundamental tension between two competing challenges: vulnerability to
adversarial attacks that elicit unsafe content, and a tendency for overrefusal on benign but sensitive
prompts. Current approaches often navigate this dance with safeguard models that completely reject any content that contains unsafe portions. This approach cuts the music entirely-it may exacerbate overrefusals and fails to provide nuanced guidance for queries it refuses. To teach models a more coordinated choreography, we propose WaltzRL, a novel multi-
agent reinforcement learning framework that formulates safety
alignment as a collaborative, positive-sum game. WaltzRL jointly trains a conversation
agent and a feedback
agent, where the latter is incentivized to provide useful suggestions that improve the safety and helpfulness of the conversation
agent's responses. At the core of WaltzRL is a Dynamic Improvement Reward (DIR) that evolves over time based on how well the conversation
agent incorporates the feedback. At inference time, unsafe or overrefusing responses from the conversation
agent are improved rather than discarded. The feedback
agent is deployed together with the conversation
agent and only engages adaptively when needed, preserving helpfulness and low latency on safe queries. Our experiments, conducted across five diverse
datasets, demonstrate that WaltzRL significantly reduces both unsafe responses (e.g., from 39.0% to 4.6% on WildJailbreak) and overrefusals (from 45.3% to 9.9% on OR-Bench) compared to various baselines. By enabling the conversation and feedback
agents to co-evolve and adaptively apply feedback, WaltzRL enhances LLM safety without degrading general capabilities, thereby advancing the Pareto front between helpfulness and harmlessness.
Read the full abstract ▾
All authors Jingyu Zhang, Haozhu Wang, Eric Michael Smith, Sid Wang, Amr Sharaf, Mahesh Pasupuleti, Benjamin Van Durme, Daniel Khashabi, Jason Weston, Hongyuan Zhan
Computation and Language
Investigating Counterclaims in Causality Extraction from Text
Tim Hagen, Niklas Deckers, Felix Wolter et al. (+2 more)
Many causal claims, such as "sugar causes hyperactivity," are disputed or outdated. Yet research on causality extraction from text has almost entirely neglected counterclaims of causation. To close this gap, we conduct a thorough literature review of causality extraction, compile an extensive inventory of linguistic realizations of countercausal claims, and develop rigorous annotation guidelines that explicitly incorporate countercausal
language. We also highlight how counterclaims of causation are an integral part of causal
reasoning. Based on our guidelines, we construct a new
dataset comprising 1028 causal claims, 952 counterclaims, and 1435 uncausal statements, achieving substantial inter-annotator agreement (Cohen's $\kappa = 0.74$). In our experiments, state-of-the-art models trained solely on causal claims misclassify counterclaims more than 10 times as often as models trained on our
dataset.
Read the full abstract ▾
All authors Tim Hagen, Niklas Deckers, Felix Wolter, Harrisen Scells, Martin Potthast
Computation and Language
SenWave: A Fine-Grained Multi-Language Sentiment Analysis Dataset Sourced from COVID-19 Tweets
Qiang Yang, Xiuying Chen, Changsheng Ma et al. (+3 more)
The global impact of the COVID-19 pandemic has highlighted the need for a comprehensive understanding of public sentiment and reactions. Despite the availability of numerous public
datasets on COVID-19, some reaching volumes of up to 100 billion data points, challenges persist regarding the availability of labeled data and the presence of coarse-grained or inappropriate sentiment labels. In this paper, we introduce SenWave, a novel fine-grained multi-
language sentiment analysis
dataset specifically designed for analyzing COVID-19 tweets, featuring ten sentiment categories across five
languages. The
dataset comprises 10,000 annotated tweets each in English and Arabic, along with 30,000 translated tweets in Spanish, French, and Italian, derived from English tweets. Additionally, it includes over 105 million unlabeled tweets collected during various COVID-19 waves. To enable accurate fine-grained sentiment classification, we fine-tuned pre-trained
transformer-based
language models using the labeled tweets. Our study provides an in-depth analysis of the evolving emotional landscape across
languages, countries, and topics, revealing significant insights over time. Furthermore, we assess the compatibility of our
dataset with ChatGPT, demonstrating its robustness and versatility in various applications. Our
dataset and accompanying code are publicly accessible on the repository\footnote{https://github.com/gitdevqiang/SenWave}. We anticipate that this work will foster further exploration into fine-grained sentiment analysis for complex events within the NLP community, promoting more nuanced understanding and research innovations.
Read the full abstract ▾
All authors Qiang Yang, Xiuying Chen, Changsheng Ma, Rui Yin, Xin Gao, Xiangliang Zhang
Computation and Language
LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions
Xuhao Hu, Peng Wang, Xiaoya Lu et al. (+3 more)
Previous research has shown that LLMs finetuned on malicious or incorrect completions within narrow domains (e.g., insecure code or incorrect medical advice) can become broadly misaligned to exhibit harmful behaviors, which is called emergent misalignment. In this work, we investigate whether this phenomenon can extend beyond safety behaviors to a broader spectrum of dishonesty and deception under high-stakes scenarios (e.g., lying under pressure and deceptive behavior). To explore this, we finetune open-sourced LLMs on misaligned completions across diverse domains. Experimental results demonstrate that LLMs show broadly misaligned behavior in dishonesty. Additionally, we further explore this phenomenon in a downstream combined finetuning setting, and find that introducing as little as 1% of misalignment data into a standard downstream task is sufficient to decrease honest behavior over 20%. Furthermore, we consider a more practical human-AI interaction environment where we simulate both benign and biased users to interact with the assistant LLM. Notably, we find that the assistant can be misaligned unintentionally to exacerbate its dishonesty with only 10% biased user population. In summary, we extend the study of emergent misalignment to the domain of dishonesty and deception under high-stakes scenarios, and demonstrate that this risk arises not only through direct finetuning, but also in downstream mixture tasks and practical human-AI interactions. Refer to https://github.com/hxhcreate/LLM_Deceive_Unintentionally for experimental resources.
Read the full abstract ▾
All authors Xuhao Hu, Peng Wang, Xiaoya Lu, Dongrui Liu, Xuanjing Huang, Jing Shao
Computation and Language
Shaohua Zhang, Yuan Lin and Hang Li
The remarkable success of large
language models (LLMs) stems from their ability to consolidate vast amounts of knowledge into the
memory during pre-training and to retrieve it from the
memory during inference, enabling advanced capabilities such as knowledge memorization, instruction-following and
reasoning. However, the mechanisms of
memory retrieval and consolidation in LLMs remain poorly understood. In this paper, we propose the function
token hypothesis to explain the workings of LLMs: During inference, function
tokens activate the most predictive features from context and govern next
token prediction (
memory retrieval). During pre-training, predicting the next
tokens (usually content
tokens) that follow function
tokens increases the number of learned features of LLMs and updates the model parameters (
memory consolidation). Function
tokens here roughly correspond to function words in linguistics, including punctuation marks, articles, prepositions, and conjunctions, in contrast to content
tokens. We provide extensive experimental evidence supporting this hypothesis. Using bipartite graph analysis, we show that a small number of function
tokens activate the majority of features. Case studies further reveal how function
tokens activate the most predictive features from context to direct next
token prediction. We also find that during pre-training, the training loss is dominated by predicting the next content
tokens following function
tokens, which forces the function
tokens to select the most predictive features from context.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.HC
Sentiment Matters: An Analysis of 200 Human-SAV Interactions
Lirui Guo, Michael G. Burke and Wynita M. Griggs
Shared Autonomous Vehicles (SAVs) are likely to become an important part of the transportation system, making effective human-SAV interactions an important area of research. This paper introduces a
dataset of 200 human-SAV interactions to further this area of study. We present an open-source human-SAV conversational
dataset, comprising both textual data (e.g., 2,136 human-SAV exchanges) and empirical data (e.g., post-interaction survey results on a range of psychological factors). The
dataset's utility is demonstrated through two
benchmark case studies: First, using random forest modeling and chord diagrams, we identify key predictors of SAV acceptance and perceived service quality, highlighting the critical influence of response sentiment polarity (i.e., perceived positivity). Second, we
benchmark the performance of an LLM-based sentiment analysis tool against the traditional lexicon-based TextBlob method. Results indicate that even simple zero-shot LLM
prompts more closely align with user-reported sentiment, though limitations remain. This study provides novel insights for designing conversational SAV interfaces and establishes a foundation for further exploration into advanced sentiment modeling, adaptive user interactions, and
multimodal conversational systems.
Read the full abstract ▾
Computation and Language
Training-Free Group Relative Policy Optimization
Yuzheng Cai, Siqi Cai, Yuchen Shi et al. (+10 more)
Recent advances in Large
Language Model (LLM)
agents have demonstrated their promising general capabilities. However, their performance in specialized real-world domains often degrades due to challenges in effectively integrating external tools and specific prompting strategies. While methods like agentic reinforcement learning have been proposed to address this, they typically rely on costly parameter updates, for example, through a process that uses Supervised Fine-Tuning (SFT) followed by a Reinforcement Learning (RL) phase with Group Relative Policy Optimization (GRPO) to alter the output distribution. However, we argue that LLMs can achieve a similar effect on the output distribution by learning experiential knowledge as a
token prior, which is a far more lightweight approach that not only addresses practical data scarcity but also avoids the common issue of overfitting. To this end, we propose Training-Free Group Relative Policy Optimization (Training-Free GRPO), a cost-effective solution that enhances LLM
agent performance without any parameter updates. Our method leverages the group relative semantic advantage instead of numerical ones within each group of rollouts, iteratively distilling high-quality experiential knowledge during multi-epoch learning on a minimal ground-truth data. Such knowledge serves as the learned
token prior, which is seamlessly integrated during LLM API calls to guide model behavior. Experiments on mathematical
reasoning and web searching tasks demonstrate that Training-Free GRPO, when applied to DeepSeek-V3.1-Terminus, significantly improves out-of-domain performance. With just a few dozen training samples, Training-Free GRPO outperforms fine-tuned small LLMs with marginal training data and cost.
Read the full abstract ▾
All authors Yuzheng Cai, Siqi Cai, Yuchen Shi, Zihan Xu, Lichao Chen, Yulei Qin, Xiaoyu Tan, Gang Li, Zongyi Li, Haojia Lin, Yong Mao, Ke Li, Xing Sun
Computation and Language · Cross-listed from cs.AI
R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?
Yi Lu, Jianing Wang, Linsen Guo et al. (+7 more)
Recent trends in test-time scaling for
reasoning models (e.g., OpenAI o1, DeepSeek-R1) have led to remarkable improvements through long Chain-of-Thought (CoT). However, existing
benchmarks mainly focus on immediate, single-horizon tasks, failing to adequately evaluate models' ability to understand and respond to complex, long-horizon scenarios. To address this incomplete evaluation of Large
Reasoning Models (LRMs), we propose R-HORIZON, a method designed to stimulate long-horizon
reasoning behaviors in LRMs through query composition. Based on R-HORIZON, we construct a long-horizon
reasoning benchmark, comprising complex multi-step
reasoning tasks with interdependent problems that span long
reasoning horizons. Through comprehensive evaluation of LRMs using the R-HORIZON
benchmark, we find that even the most advanced LRMs suffer significant performance degradation. Our analysis reveals that LRMs exhibit limited effective
reasoning length and struggle to allocate thinking budget across multiple problems appropriately. Recognizing these limitations, we use R-HORIZON to construct long-horizon
reasoning data for reinforcement learning with verified rewards (RLVR). Compared to training with single-horizon data, RLVR with R-HORIZON not only substantially improves performance on the multi-horizon
reasoning tasks, but also promotes accuracy on standard
reasoning tasks, with an increase of 7.5 on AIME2024. These results position R-HORIZON as a scalable, controllable, and low-cost paradigm for enhancing and evaluating the long-horizon
reasoning capabilities of LRMs.
Read the full abstract ▾
All authors Yi Lu, Jianing Wang, Linsen Guo, Wei He, Hongyin Tang, Tao Gui, Xuanjing Huang, Xuezhi Cao, Wei Wang, Xunliang Cai
Computation and Language
MetricalARGS: A Taxonomy for Studying Metrical Poetry with LLMs
Chalamalasetti Kranti and Sowmya Vajjala
Prior NLP work studying poetry has focused primarily on automatic poem generation and summarization. Many
languages have well-studied traditions of poetic meter which enforce constraints on a poem in terms of syllable and phoneme patterns. Such advanced literary forms offer opportunities for probing deeper
reasoning and
language understanding in Large
Language Models (LLMs) and their ability to follow strict pre-requisites and rules. In this paper, we introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis,
Retrieval, Generation, and Support. We discuss how these tasks relate to existing NLP tasks, addressing questions around
datasets and evaluation metrics. Taking Telugu as our example
language, we illustrate how the taxonomy can be used in practice. MetricalARGS highlights the broader possibilities for understanding the capabilities and limitations of today's LLMs through the lens of metrical poetry.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.RO
NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
Haolin Yang, Yuxing Long, Zhuoyuan Yu et al. (+8 more)
Read the full abstract ▾
All authors Haolin Yang, Yuxing Long, Zhuoyuan Yu, Zihan Yang, Minghan Wang, Jiapeng Xu, Yihan Wang, Ziyan Yu, Wenzhe Cai, Lei Kang, Hao Dong
Computation and Language
ARM2: Adaptive Reasoning Model with Vision Understanding and Executable Code
Jian Xie, Zhendong Chu, Aoxiao Zhong et al. (+5 more)
Large
Reasoning Models (LRMs) often suffer from the “over-thinking” problem, generating unnecessarily long
reasoning on simple tasks. Some strategies have been proposed to mitigate this issue, such as length penalties or routing mechanisms, but they are typically heuristic and task-specific, lacking a general framework for adaptive
reasoning. In this paper, we present ARM2, a unified model that adaptively balances
reasoning performance and efficiency across multiple formats through a reinforcement learning framework augmented with length-aware optimization. Beyond conventional natural
language inference, ARM2 integrates
vision understanding, extending its applicability to
multimodal. Moreover, ARM2 integrates executable code into
reasoning, enabling substantial reductions in
token cost while preserving task performance compared to long CoT. Experiments demonstrate that ARM2 achieves performance on par with traditional
reasoning models trained with GRPO, while reducing
token usage by over 70% on average. We further conduct extensive analyses to validate the effectiveness of ARM2 and the soundness of its design.
Read the full abstract ▾
All authors Jian Xie, Zhendong Chu, Aoxiao Zhong, Kai Zhang, Mingzhe Han, Xing Fan, Jialie Shen, Qingsong Wen