Computation and Language · Cross-listed from cs.AI
On the Role of Temperature Sampling in Test-Time Scaling
Yuheng Wu, Azalia Mirhoseini and Thierry Tambe
Large
language models (LLMs) can improve
reasoning at inference time through test-time scaling (TTS), where multiple
reasoning traces are generated and the best one is selected. Prior work shows that increasing the number of samples K steadily improves accuracy. In this paper, we demonstrate that this trend does not hold indefinitely: at large K, further scaling yields no gains, and certain hard questions remain unsolved regardless of the number of traces. Interestingly, we find that different sampling temperatures solve different subsets of problems, implying that single-temperature scaling explores only part of a model's potential. We therefore propose scaling along the temperature dimension, which enlarges the
reasoning boundary of LLMs. Averaged over Qwen3 (0.6B, 1.7B, 4B, 8B) and five representative
reasoning benchmarks (AIME 2024/2025, MATH500, LiveCodeBench, Hi-ToM), temperature scaling yields an additional 7.3 points over single-temperature TTS. Temperature scaling also enables base models to reach performance comparable to reinforcement learning (RL)-trained counterparts, without additional post-training. We further provide a comprehensive analysis of this phenomenon and design a multi-temperature voting method that reduces the overhead of temperature scaling. Overall, our findings suggest that TTS is more powerful than previously thought, and that temperature scaling offers a simple and effective way to unlock the latent potential of base models.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
How Confident are Video Models? Empowering Video Models to Express their Uncertainty
Zhiting Mei, Ola Shorinwa and Anirudha Majumdar
Generative
video models demonstrate impressive text-to-
video capabilities, spurring widespread adoption in many real-world applications. However, like large
language models (LLMs),
video generation models tend to hallucinate, producing plausible
videos even when they are factually wrong. Although uncertainty quantification (UQ) of LLMs has been extensively studied in prior work, no UQ method for
video models exists, raising critical safety concerns. To our knowledge, this paper represents the first work towards quantifying the uncertainty of
video models. We present a framework for uncertainty quantification of generative
video models, consisting of: (i) a metric for evaluating the calibration of
video models based on robust rank correlation estimation with no stringent modeling assumptions; (ii) a black-box UQ method for
video models (termed S-QUBED), which leverages latent modeling to rigorously decompose predictive uncertainty into its aleatoric and epistemic components; and (iii) a UQ
dataset to facilitate benchmarking calibration in
video models. By conditioning the generation task in the latent space, we disentangle uncertainty arising due to vague task specifications from that arising from lack of knowledge. Through extensive experiments on
benchmark video datasets, we demonstrate that S-QUBED computes calibrated total uncertainty estimates that are negatively correlated with the task accuracy and effectively computes the aleatoric and epistemic constituents.
Read the full abstract ▾
Computation and Language
Transcribe, Translate, or Transliterate: An Investigation of Intermediate Representations in Spoken Language Models
Tolúlopé Ògúnrèmí, Christopher D. Manning, Dan Jurafsky et al. (+1 more)
Spoken
language models (SLMs) that integrate
speech with large
language models (LMs) rely on modality adapters (MAs) to map the output of
speech encoders to a representation that is understandable to the decoder LM. Yet we know very little about how these crucial MAs transform representations. Here we examine the MA output representation in three SLMs (SALMONN, Qwen2-Audio and Phi-4-
Multimodal-Instruct). By finding the nearest decoder LM
token to an MA representation, we uncover two strategies for MA representations. For models using a Whisper encoder, MAs appear to represent the meaning of the input using an English-based interlingua, allowing them to handle
languages unseen in instruction tuning. For models that don't, like Phi-4-
Multimodal-Instruct, MAs instead represent the phonetics of the input, but expressed with English words. We hypothesise that which arises depends on whether the
speech encoder is trained only for
speech recognition or also for translation.
Read the full abstract ▾
All authors Tolúlopé Ògúnrèmí, Christopher D. Manning, Dan Jurafsky, Karen Livescu
Computation and Language
Knowledge-Graph Based RAG System Evaluation Framework
Sicheng Dong, Vahid Zolfaghari, Nenad Petrovic et al. (+1 more)
Large
language models (LLMs) has become a significant research focus and is utilized in various fields, such as text generation and dialog systems. One of the most essential applications of LLM is
Retrieval Augmented Generation (RAG), which greatly enhances generated content's reliability and relevance. However, evaluating RAG systems remains a challenging task. Traditional evaluation metrics struggle to effectively capture the key features of modern LLM-generated content that often exhibits high fluency and naturalness. Inspired by the RAGAS tool, a well-known RAG evaluation framework, we extended this framework into a KG-based evaluation paradigm, enabling multi-hop
reasoning and semantic community clustering to derive more comprehensive scoring metrics. By incorporating these comprehensive evaluation criteria, we gain a deeper understanding of RAG systems and a more nuanced perspective on their performance. To validate the effectiveness of our approach, we compare its performance with RAGAS scores and construct a human-annotated subset to assess the correlation between human judgments and automated metrics. In addition, we conduct targeted experiments to demonstrate that our KG-based evaluation method is more sensitive to subtle semantic differences in generated outputs. Finally, we discuss the key challenges in evaluating RAG systems and highlight potential directions for future research.
Read the full abstract ▾
All authors Sicheng Dong, Vahid Zolfaghari, Nenad Petrovic, Alois Knoll
Computation and Language
Hierarchical Semantic Retrieval with Cobweb
Anant Gupta, Karthik Singaravadivelan and Zekun Wang
Neural document
retrieval often treats a corpus as a flat cloud of vectors scored at a single granularity, leaving corpus structure underused and explanations opaque. We use Cobweb–a hierarchy-aware framework–to organize sentence
embeddings into a prototype tree and rank documents via coarse-to-fine traversal. Internal nodes act as concept prototypes, providing multi-granular relevance signals and a transparent rationale through
retrieval paths. We instantiate two inference approaches: a generalized best-first search and a lightweight path-sum ranker. We evaluate our approaches on MS MARCO and QQP with encoder (e.g., BERT/T5) and decoder (GPT-2) representations. Our results show that our
retrieval approaches match the dot product search on strong encoder
embeddings while remaining robust when kNN degrades: with GPT-2 vectors, dot product performance collapses whereas our approaches still retrieve relevant results. Overall, our experiments suggest that Cobweb provides competitive effectiveness, improved robustness to
embedding quality, scalability, and interpretable
retrieval via hierarchical prototypes.
Read the full abstract ▾
Computation and Language
Unraveling Syntax: Language Modeling and the Substructure of Grammars
Laura Ying Schulz, Daniel Mitropolsky and Tomaso Poggio
While
language models achieve impressive results, their learning dynamics are far from understood. Many domains of interest – such as natural
language syntax, coding
languages, arithmetic – are captured by context-free grammars (CFGs). In this work, we extend prior work on neural
language modeling of CFGs in a novel direction: how
language modeling behaves with respect to CFG substructure, namely subgrammars. We define subgrammars, and prove a set of fundamental
theorems connecting
language modeling and subgrammars. We show that
language modeling loss recurses linearly over its top-level subgrammars; applied recursively, the loss decomposes into losses for "irreducible" subgrammars. Under additional assumptions, and empirically, parametrized models learn subgrammars in parallel, unlike children who first master simple substructures. We find that subgrammar pretraining can improve final performance, but only for tiny models relative to the grammar, while
alignment analyses show that pretraining consistently leads to internal representations that better reflect the grammar's substructure.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Beyond Imitation: Recovering Dense Rewards from Demonstrations
Jiangnan Li, Thuy-Trang Vu, Ehsan Abbasnejad et al. (+1 more)
Conventionally, supervised fine-tuning (SFT) is treated as a simple imitation learning process that only trains a policy to imitate expert behavior on demonstration
datasets. In this work, we challenge this view by establishing a fundamental equivalence between SFT and Inverse Reinforcement Learning. We prove that the SFT objective is a special case of Inverse Q-Learning, which implies that the SFT process does not just learn a policy, but also an implicit, dense,
token-level reward model that explains the expert demonstrations. We then show how to recover this dense reward signal directly from the SFT model by formulating a baseline-relative reward function. The availability of such a dense reward model offers numerous benefits, providing granular credit assignment for each
token generated. We demonstrate one key application by using these recovered rewards to further improve the policy with reinforcement learning. Our method, Dense-Path REINFORCE, consistently outperforms the original SFT models on instruction-following
benchmarks. This work reframes SFT not merely as policy imitation but as a powerful reward learning mechanism, opening new possibilities for leveraging expert demonstrations.
Read the full abstract ▾
All authors Jiangnan Li, Thuy-Trang Vu, Ehsan Abbasnejad, Gholamreza Haffari
Computation and Language · Cross-listed from cs.LG
Litespark Technical Report: High-Throughput, Energy-Efficient LLM Training Framework
Nii Osae Osae Dade and Moinul Hossain Rahat
Training Large
Language Models (LLMs) is plagued by long training times and massive energy consumption, with modern models requiring months of computation and gigawatt-hours of electricity. In light of these challenges,we introduce Litespark, a novel pre-training framework that addresses these inefficiencies through targeted optimizations to
transformer attention and MLP layers. Our approach combines architectural improvements with algorithmic enhancements to maximize Model FLOPs Utilization (MFU) while maintaining compatibility with standard
transformer implementations. Comprehensive benchmarking on 3B and 30B parameter Llama models using the SlimPajama-627B
dataset demonstrates substantial performance gains: 2x-6x training throughput improvement and $55\%-83$% energy consumption reduction across multi-node H200 GPU clusters. These optimizations are model- and hardware-agnostic, enabling broad applicability across
transformer architectures and extending to post-training phases including supervised fine-tuning and direct preference optimization.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.RO
SIMSplat: Language-Aligned 4D Gaussian Splatting for Driving Scenario Generation
Sung-Yeon Park, Adam Lee, Juanwu Lu et al. (+6 more)
Driving scene manipulation using real-world sensor data has emerged as a promising alternative to traditional driving simulators. Despite advances in
language control and neural scene representations, existing methods treat grounding, editing, and simulation as loosely connected stages, relying on heuristic object localization, manual guidance, and single-
agent validation, thereby constraining semantic expressiveness and hindering scalable, reactive scenario generation. We introduce SIMSplat, a driving scene editor built on scene-graph-based 4D Gaussian Splatting augmented with
language-aligned features. By
embedding appearance, motion, and location semantics directly into Gaussian scene-graph nodes, SIMSplat makes reconstructed scenes queryable through free-form natural
language, bridging
language understanding to object-level editing and multi-
agent simulation within a single framework. Building on this
language-grounded scene graph, SIMSplat supports diverse edits including fine-grained pedestrian manipulation, while a multi-
agent path refinement module propagates changes across all
agents to ensure reactive, physically plausible simulations. The pipeline further integrates with
Vision-
Language Models for automated scenario mining. Experiments show that SIMSplat more than doubles baseline grounding accuracy, achieves the highest task completion rate, and produces the lowest failure rates across diverse driving scenarios.
Read the full abstract ▾
All authors Sung-Yeon Park, Adam Lee, Juanwu Lu, Can Cui, Luyang Jiang, Rohit Gupta, Kyungtae Han, Ahmadreza Moradipari, Ziran Wang
Computation and Language
CLARITY: Clinical Assistant for Routing, Inference, and Triage
Vladimir Shaposhnikov, Aleksandr Nesterov, Ilia Kopanichuk et al. (+7 more)
We present CLARITY (Clinical Assistant for Routing, Inference and Triage), an AI-driven platform designed to facilitate patient-to-specialist routing, clinical consultations, and severity assessment of patient conditions. Its hybrid architecture combines a Finite State Machine (FSM) for structured dialogue flows with collaborative
agents that employ Large
Language Model (LLM) to analyze symptoms and prioritize referrals to appropriate specialists. Built on a modular microservices framework, CLARITY ensures safe, efficient, and robust performance, flexible and readily scalable to meet the demands of existing workflows and IT solutions in healthcare. We report integration of our clinical assistant into a large-scale national interhospital platform, with more than 55,000 content-rich user dialogues completed within the two months of deployment, 2,500 of which were expert-annotated for subsequent validation. The validation results show that CLARITY surpasses human-level performance in terms of the first-attempt routing precision, naturally requiring up to 3 times shorter duration of the consultation than with a human.
Read the full abstract ▾
All authors Vladimir Shaposhnikov, Aleksandr Nesterov, Ilia Kopanichuk, Ivan Bakulin, Egor Zhelvakov, Ruslan Abramov, Ekaterina Tsapieva, Iaroslav Bespalov, Dmitry V. Dylov, Ivan Oseledets
Computation and Language · Cross-listed from cs.LG
How to Train Your Advisor: Steering Black-Box LLMs with Advisor Models
Parth Asawa, Alan Zhu, Abigail O'Neill et al. (+3 more)
Frontier
language models are deployed as black-box services, where model weights cannot be modified and customization is limited to prompting. We introduce Advisor Models, a method to train small open-weight models to generate dynamic, per-instance natural
language advice that improves the capabilities of black-box frontier models. Advisor Models improve GPT-5.2's performance on RuleArena (Taxes) by 27.4%, reduce Gemini 3 Pro's steps taken in SWE
agent tasks by 24.6%, and outperform static
prompt optimizers in personalizing GPT-5 to user preferences (85-100% vs. 40-60%). We also find that advisors are transferable: an advisor trained with a low-cost student model still transfers improvements to a frontier model. Moreover, Advisor Models are robust: we observe no degradation on other
benchmarks than the pipeline is trained on. Our method shows how to perform parametric optimization for black-box frontier models in a practical and cost-effective way.
Read the full abstract ▾
All authors Parth Asawa, Alan Zhu, Abigail O'Neill, Matei Zaharia, Alexandros G. Dimakis, Joseph E. Gonzalez
Computation and Language
Words That Make Language Models Perceive
Sophie L. Wang, Phillip Isola and Brian Cheung
Large
language models (LLMs) trained purely on text ostensibly lack any direct perceptual experience, yet their internal representations are implicitly shaped by
multimodal regularities encoded in
language. We test the hypothesis that explicit sensory prompting can surface this latent structure, bringing a text-only LLM into closer representational
alignment with specialist
vision and audio encoders. When a sensory
prompt tells the model to 'see' or 'hear', it cues the model to resolve its next-
token predictions as if they were conditioned on latent visual or auditory evidence that is never actually supplied. Our findings reveal that lightweight
prompt engineering can reliably activate modality-appropriate representations in purely text-trained LLMs.
Read the full abstract ▾
Computation and Language
Drawing Conclusions from Draws: Rethinking Preference Semantics in Arena-Style LLM Evaluation
Raphael Tang, Crystina Zhang, Wenyan Li et al. (+3 more)
In arena-style evaluation of large
language models (LLMs), two LLMs respond to a user query, and the user chooses the winning response or deems the "battle" a draw, resulting in an adjustment to the ratings of both models. The prevailing approach for modeling these rating dynamics is to view battles as two-player game matches, as in chess, and apply the Elo rating system and its derivatives. In this paper, we critically examine this paradigm. Specifically, we question whether a draw genuinely means that the two models are equal and hence whether their ratings should be equalized. Instead, we
conjecture that draws are more indicative of query difficulty: if the query is too easy, then both models are more likely to succeed equally. On three real-world arena
datasets, we show that ignoring rating updates for draws yields a 1-3% relative increase in battle outcome prediction accuracy (which includes draws) for all four rating systems studied. Further analyses suggest that draws occur more for queries rated as very easy and those as highly objective, with risk ratios of 1.37 and 1.35, respectively. We recommend future rating systems to reconsider existing draw semantics and to account for query properties in rating updates.
Read the full abstract ▾
All authors Raphael Tang, Crystina Zhang, Wenyan Li, Carmen Lai, Pontus Stenetorp, Yao Lu
Computation and Language · Cross-listed from cs.LG
Interactive Training: Feedback-Driven Neural Network Optimization
Wentao Zhang, Yang Young Lu and Yuntian Deng
Traditional neural network training typically follows fixed, predefined optimization recipes, lacking the flexibility to dynamically respond to instabilities or emerging training issues. In this paper, we introduce Interactive Training, an open-source framework that enables real-time, feedback-driven intervention during neural network training by human experts or automated AI
agents. At its core, Interactive Training uses a control server to mediate communication between users or
agents and the ongoing training process, allowing users to dynamically adjust optimizer hyperparameters, training data, and model checkpoints. Through three case studies, we demonstrate that Interactive Training achieves superior training stability, reduced sensitivity to initial hyperparameters, and improved adaptability to evolving user needs, paving the way toward a future training paradigm where AI
agents autonomously monitor training logs, proactively resolve instabilities, and optimize training dynamics.
Read the full abstract ▾
Computation and Language
F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Data
Ziyin Zhang, Zihan Liao, Hang Yu et al. (+2 more)
We introduce F2LLM - Foundation to Feature Large
Language Models, a suite of state-of-the-art
embedding models in three sizes: 0.6B, 1.7B, and 4B. Unlike previous top-ranking
embedding models that require massive contrastive pretraining, sophisticated training pipelines, and costly synthetic training data, F2LLM is directly finetuned from foundation models on 6 million query-document-negative tuples curated from open-source, non-synthetic
datasets, striking a strong balance between training cost, model size, and
embedding performance. On the MTEB English leaderboard, F2LLM-4B ranks 2nd among models with approximately 4B parameters and 7th overall, while F2LLM-1.7B ranks 1st among models in the 1B-2B size range. To facilitate future research in the field, we release the models, training
dataset, and code, positioning F2LLM as a strong, reproducible, and budget-friendly baseline for future works.
Read the full abstract ▾
All authors Ziyin Zhang, Zihan Liao, Hang Yu, Peng Di, Rui Wang
Computation and Language
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens
Hala Sheta, Eric Huang, Shuyu Wu et al. (+9 more)
We introduce VLM-Lens, a toolkit designed to enable systematic benchmarking, analysis, and interpretation of
vision-
language models (VLMs) by supporting the extraction of intermediate outputs from any layer during the forward pass of open-source VLMs. VLM-Lens provides a unified, YAML-configurable interface that abstracts away model-specific complexities and supports user-friendly operation across diverse VLMs. It currently supports 16 state-of-the-art base VLMs and their over 30 variants, and is extensible to accommodate new models without changing the core logic. The toolkit integrates easily with various interpretability and analysis methods. We demonstrate its usage with two simple analytical experiments, revealing systematic differences in the hidden representations of VLMs across layers and target concepts. VLM-Lens is released as an open-sourced project to accelerate community efforts in understanding and improving VLMs.
Read the full abstract ▾
All authors Hala Sheta, Eric Huang, Shuyu Wu, Ilia Alenabi, Jiajun Hong, Ryker Lin, Ruoxi Ning, Daniel Wei, Jialin Yang, Jiawei Zhou, Ziqiao Ma, Freda Shi
Computation and Language · Cross-listed from cs.LG
Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks
Ruohao Guo, Afshin Oroojlooy, Roshan Sridhar et al. (+3 more)
Despite recent rapid progress in AI safety, current large
language models remain vulnerable to
adversarial attacks in multi-turn interaction settings, where attackers strategically adapt their
prompts across conversation turns and pose a more critical yet realistic challenge. Existing approaches that discover safety vulnerabilities either rely on manual red-teaming with human experts or employ automated methods using pre-defined templates and human-curated attack data, with most focusing on single-turn attacks. However, these methods did not explore the vast space of possible multi-turn attacks, failing to consider novel attack trajectories that emerge from complex dialogue dynamics and strategic conversation planning. This gap is particularly critical given recent findings that LLMs exhibit significantly higher vulnerability to multi-turn attacks compared to single-turn attacks. We propose DialTree, an on-policy reinforcement learning framework integrated with tree search that autonomously discovers diverse multi-turn attack strategies by treating the dialogue as a sequential decision-making problem, enabling systematic exploration without manually curated data. Through extensive experiments, our approach not only achieves more than 44.2% higher ASR across 12 target models compared to previous state-of-the-art approaches, but also effectively uncovers new attack strategies by learning optimal dialogue policies that maximize attack success across multiple turns.
Read the full abstract ▾
All authors Ruohao Guo, Afshin Oroojlooy, Roshan Sridhar, Miguel Ballesteros, Alan Ritter, Dan Roth
Computation and Language
How Well Does Mathematical Reasoning Transfer Across Languages in English-Centric LLMs?
Wen Yang, Junhong Wu, Chong Li et al. (+2 more)
Recent advances in Reinforcement Post-Training (RPT) have substantially improved the mathematical
reasoning capabilities of large
language models (LLMs), but it remains unclear how well such gains transfer across
languages. In this work, we study cross-lingual mathematical
reasoning transfer in English-centric LLMs under controlled
multilingual evaluation. We first evaluate a range of English-centric
reasoning models on
multilingual mathematical
reasoning benchmarks and find that transfer is highly variable across initial model choice, target
language, and training paradigm. Through controlled comparative experiments, we further find that stronger English-centric initialization does not necessarily lead to stronger relative transfer efficiency, even when it yields higher
multilingual reasoning accuracy. Finally, we show that the largest improvement comes from moving beyond English-only supervision: adding a single parallel
language yields a substantial gain, while further gains from additional
languages are smaller but consistent. Overall, our results highlight the need to evaluate
multilingual mathematical
reasoning directly, using both absolute performance and relative transfer-efficiency metrics.
Read the full abstract ▾
All authors Wen Yang, Junhong Wu, Chong Li, Chengqing Zong, Jiajun Zhang
Computation and Language
InfoMosaic-Bench: Evaluating Multi-Source Information Seeking in Tool-Augmented Agents
Yaxin Du, Yuanshuo Zhang, Xiyuan Yang et al. (+10 more)
Information seeking is a fundamental requirement for humans. However, existing LLM
agents rely heavily on open-web search, which exposes two fundamental weaknesses: online content is noisy and unreliable, and many real-world tasks require precise, domain-specific knowledge unavailable from the web. The emergence of the Model Context Protocol (MCP) now allows
agents to interface with thousands of specialized tools, seemingly resolving this limitation. Yet it remains unclear whether
agents can effectively leverage such tools – and more importantly, whether they can integrate them with general-purpose search to solve complex tasks. Therefore, we introduce InfoMosaic-Bench, the first
benchmark dedicated to multi-source information seeking in tool-augmented
agents. Covering six representative domains (medicine, finance, maps,
video, web, and multi-domain integration), InfoMosaic-Bench requires
agents to combine general-purpose search with domain-specific tools. Tasks are synthesized with InfoMosaic-Flow, a scalable pipeline that grounds task conditions in verified tool outputs, enforces cross-source dependencies, and filters out shortcut cases solvable by trivial lookup. This design guarantees both reliability and non-triviality. Experiments with 14 state-of-the-art LLM
agents reveal three findings: (i) web information alone is insufficient, with GPT-5 achieving only 38.2% accuracy and 67.5% pass rate; (ii) domain tools provide selective but inconsistent benefits, improving some domains while degrading others; and (iii) 22.4% of failures arise from incorrect tool usage or selection, highlighting that current LLMs still struggle with even basic tool handling.
Read the full abstract ▾
All authors Yaxin Du, Yuanshuo Zhang, Xiyuan Yang, Yifan Zhou, Cheng Wang, Gongyi Zou, Xianghe Pang, Wenhao Wang, Menglan Chen, Shuo Tang, Zhiyu Li, Feiyu Xiong, Siheng Chen
Computation and Language · Cross-listed from cs.AI
RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
Yuxiao Qu, Anikait Singh, Yoonho Lee et al. (+4 more)
Reasoning requires going beyond pattern matching or memorization of solutions to identify and implement "algorithmic procedures" that can be used to deduce answers to hard problems. Doing so requires realizing the most relevant primitives, intermediate results, or shared procedures, and building upon them. While RL post-training on long chains of thought ultimately aims to uncover this kind of algorithmic behavior, most
reasoning traces learned by large models fail to consistently capture or reuse procedures, instead drifting into verbose and degenerate exploration. To address more effective
reasoning, we introduce
reasoning abstractions: concise natural
language descriptions of procedural and factual knowledge that guide the model toward learning successful
reasoning. We train models to be capable of proposing multiple abstractions given a problem, followed by RL that incentivizes building a solution while using the information provided by these abstractions. This results in a two-player RL training paradigm, abbreviated as RLAD, that jointly trains an abstraction generator and a solution generator. This setup effectively enables structured exploration, decouples learning signals of abstraction proposal and solution generation, and improves generalization to harder problems. We also show that allocating more test-time compute to generating abstractions is more beneficial for performance than generating more solutions at large test budgets, illustrating the role of abstractions in guiding meaningful exploration.
Read the full abstract ▾
All authors Yuxiao Qu, Anikait Singh, Yoonho Lee, Amrith Setlur, Ruslan Salakhutdinov, Chelsea Finn, Aviral Kumar
Computation and Language · Cross-listed from cs.AI
Scaling Agents for Computer Use
Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee et al. (+3 more)
Computer-use
agents (CUAs) hold promise for automating everyday digital tasks, but their performance on long-horizon, complex problems remains unreliable. Single-rollout execution is brittle, with small errors compounding over time and leading to high variance in outcomes. While prior work has attempted to scale within a single rollout, such approaches have yielded limited gains. Scaling over multiple rollouts offers a more promising alternative but doing so effectively is challenging due to the difficulty of evaluating and selecting among long-horizon
agent behaviors. We introduce Behavior Judge (BJudge), which addresses this challenge by representing
agent executions as behavior narratives and comparing candidate behaviors at this level, substantially improving robustness and success rates. Using multiple rollouts, BJudge establishes a new state of the art (SoTA) in OSWorld at 72.6%, significantly outperforming prior methods and surpassing human-level performance at 72.36%, with comprehensive ablations validating key design choices. We further demonstrate strong generalization results to different operating systems on WindowsAgentArena and AndroidWorld. Crucially, our results highlight the strong effectiveness of scaling CUAs, when you do it right: effective scaling requires structured trajectory understanding and selection, and BJudge provides a practical framework to achieve this.
Read the full abstract ▾
All authors Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee, Jiachen Yang, Ang Li, Xin Eric Wang
Computation and Language
Explore Briefly, Then Decide: Mitigating LLM Overthinking via Cumulative Entropy Regulation
Yi Bin, Tianyi Jiang, Yujuan Ding et al. (+5 more)
Large
Language Models (LLMs) have demonstrated remarkable
reasoning abilities on complex problems using long Chain-of-Thought (CoT)
reasoning. However, they often suffer from overthinking, meaning generating unnecessarily lengthy
reasoning steps for simpler problems. This issue may degrade the efficiency of the models and make them difficult to adapt the
reasoning depth to the complexity of problems. To address this, we introduce a novel metric
Token Entropy Cumulative Average (TECA), which measures the extent of exploration throughout the
reasoning process. We further propose a novel
reasoning paradigm named "Explore Briefly, Then Decide", with an associated Cumulative
Entropy Regulation (CER) mechanism. This paradigm leverages TECA to help the model dynamically determine the optimal point to conclude its thought process and provide a final answer, thus achieving efficient
reasoning. Experimental results across diverse mathematical
benchmarks show that our approach substantially mitigates overthinking without sacrificing problem solving ability. With our thinking paradigm, the average response length decreases by up to 71% on simpler
datasets, demonstrating the effectiveness of our method in creating a more efficient and adaptive
reasoning process.
Read the full abstract ▾
All authors Yi Bin, Tianyi Jiang, Yujuan Ding, Kainian Zhu, Fei Ma, Jingkuan Song, Yang Yang, Heng Tao Shen
Computation and Language · Cross-listed from cs.LG
ExGRPO: Learning to Reason from Experience
Runzhe Zhan, Yafu Li, Zhi Wang et al. (+5 more)
Reinforcement learning from verifiable rewards (RLVR) is an emerging paradigm for improving the
reasoning ability of large
language models. However, standard on-policy training discards rollout experiences after a single update, leading to computational inefficiency and instability. While prior work on RL has highlighted the benefits of reusing past experience, the role of experience characteristics in shaping learning dynamics of large
reasoning models remains underexplored. In this paper, we are the first to investigate what makes a
reasoning experience valuable and identify rollout correctness and
entropy as effective indicators of experience value. Based on these insights, we propose ExGRPO (Experiential Group Relative Policy Optimization), a framework that organizes and prioritizes valuable experiences, and employs a mixed-policy objective to balance exploration with experience exploitation. Experiments on five backbone models (1.5B-8B parameters) show that ExGRPO consistently improves
reasoning performance on mathematical/general
benchmarks, with an average gain of +3.5/7.6 points over on-policy RLVR. Moreover, ExGRPO stabilizes training on both stronger and weaker models where on-policy methods fail. These results highlight principled experience management as a key ingredient for efficient and scalable RLVR.
Read the full abstract ▾
All authors Runzhe Zhan, Yafu Li, Zhi Wang, Xiaoye Qu, Dongrui Liu, Jing Shao, Derek F. Wong, Yu Cheng
Computation and Language
AccurateRAG: A Framework for Building Accurate Retrieval-Augmented Question-Answering Applications
Linh The Nguyen, Chi Tran, Dung Ngoc Nguyen et al. (+3 more)
We introduce AccurateRAG – a novel framework for constructing high-performance question-answering applications based on
retrieval-augmented generation (RAG). Our framework offers a pipeline for development efficiency with tools for raw
dataset processing, fine-tuning data generation, text
embedding & LLM fine-tuning, output evaluation, and building RAG systems locally. Experimental results show that our framework outperforms previous strong baselines and obtains new state-of-the-art question-answering performance on
benchmark datasets.
Read the full abstract ▾
All authors Linh The Nguyen, Chi Tran, Dung Ngoc Nguyen, Van-Cuong Pham, Hoang Ngo, Dat Quoc Nguyen
Computation and Language · Cross-listed from cs.IR
Study on LLMs for Promptagator-Style Dense Retriever Training
Daniel Gwon, Nour Jedidi and Jimmy Lin
Promptagator demonstrated that Large
Language Models (LLMs) with few-shot
prompts can be used as task-specific query generators for fine-tuning domain-specialized dense
retrieval models. However, the original Promptagator approach relied on proprietary and large-scale LLMs which users may not have access to or may be prohibited from using with sensitive data. In this work, we study the impact of open-source LLMs at accessible scales ($\leq$14B parameters) as an alternative. Our results demonstrate that open-source LLMs as small as 3B parameters can serve as effective Promptagator-style query generators. We hope our work will inform practitioners with reliable alternatives for synthetic data generation and give insights to maximize fine-tuning results for domain-specific applications.
Read the full abstract ▾
Computation and Language
Enhanced Arabic-language cyberbullying detection: deep embedding and transformer (BERT) approaches
Ebtesam Jaber Aljohani and Wael M. S. Yafoo
Recent technological advances in smartphones and communications, including the growth of such online platforms as massive social media networks such as X (formerly known as Twitter) endangers young people and their emotional well-being by exposing them to cyberbullying, taunting, and bullying content. Most proposed approaches for automatically detecting cyberbullying have been developed around the English
language, and methods for detecting Arabic-
language cyberbullying are scarce. Methods for detecting Arabic-
language cyberbullying are especially scarce. This paper aims to enhance the effectiveness of methods for detecting cyberbullying in Arabic-
language content. We assembled a
dataset of 10,662 X posts, pre-processed the data, and used the kappa tool to verify and enhance the quality of our annotations. We conducted four experiments to test numerous deep learning models for automatically detecting Arabic-
language cyberbullying. We first tested a long short-term
memory (LSTM) model and a bidirectional long short-term
memory (Bi-LSTM) model with several experimental word
embeddings. We also tested the LSTM and Bi-LSTM models with a novel pre-trained bidirectional encoder from representations (BERT) and then tested them on a different experimental models BERT again. LSTM-BERT and Bi-LSTM-BERT demonstrated a 97% accuracy. Bi-LSTM with FastText
embedding word performed even better, achieving 98% accuracy. As a result, the outcomes are generalize
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
Phuc Minh Nguyen, Chinh D. La, Duy M. H. Nguyen et al. (+3 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a key method for improving Large
Language Models'
reasoning capabilities, yet recent evidence suggests it may paradoxically shrink the
reasoning boundary rather than expand it. This paper investigates the shrinkage issue of RLVR by analyzing its learning dynamics and reveals two critical phenomena that explain this failure. First, we expose negative interference in RLVR, where learning to solve certain training problems actively reduces the likelihood of correct solutions for others, leading to the decline of Pass@$k$ performance, or the probability of generating a correct solution within $k$ attempts. Second, we uncover the winner-take-all phenomenon: RLVR disproportionately reinforces problems with high likelihood, correct solutions, under the base model, while suppressing other initially low-likelihood ones. Through extensive theoretical and empirical analysis on multiple mathematical
reasoning benchmarks, we show that this effect arises from the inherent on-policy sampling in standard RL objectives, causing the model to converge toward narrow solution strategies. Based on these insights, we propose a simple yet effective data curation
algorithm that focuses RLVR learning on low-likelihood problems, achieving notable improvement in Pass@$k$ performance. Our code is available at https://github.com/mail-research/SELF-llm-interference.
Read the full abstract ▾
All authors Phuc Minh Nguyen, Chinh D. La, Duy M. H. Nguyen, Nitesh V. Chawla, Binh T. Nguyen, Khoa D. Doan
Computation and Language
More Than One Teacher: Adaptive Multi-Guidance Policy Optimization for Diverse Exploration
Xiaoyang Yuan, Yujuan Ding, Yi Bin et al. (+5 more)
Reinforcement Learning with Verifiable Rewards (RLVR) is a promising paradigm for enhancing the
reasoning ability in Large
Language Models (LLMs). However, prevailing methods primarily rely on self-exploration or a single off-policy teacher to elicit long chain-of-thought (LongCoT)
reasoning, which may introduce intrinsic model biases and restrict exploration, ultimately limiting
reasoning diversity and performance. Drawing inspiration from multi-teacher strategies in knowledge
distillation, we introduce Adaptive Multi-Guidance Policy Optimization (AMPO), a novel framework that adaptively leverages guidance from multiple proficient teacher models, but only when the on-policy model fails to generate correct solutions. This "guidance-on-demand" approach expands exploration while preserving the value of self-discovery. Moreover, AMPO incorporates a comprehension-based selection mechanism, prompting the student to learn from the
reasoning paths that it is most likely to comprehend, thus balancing broad exploration with effective exploitation. Extensive experiments show AMPO substantially outperforms a strong baseline (GRPO), with a 4.3% improvement on mathematical
reasoning tasks and 12.2% on out-of-distribution tasks, while significantly boosting Pass@k performance and enabling more diverse exploration. Notably, using four peer-sized teachers, our method achieves comparable results to approaches that leverage a single, more powerful teacher (e.g., DeepSeek-R1) with more data. These results demonstrate a more efficient and scalable path to superior
reasoning and generalizability. Our code is available at https://github.com/SII-Enigma/AMPO.
Read the full abstract ▾
All authors Xiaoyang Yuan, Yujuan Ding, Yi Bin, Wenqi Shao, Jinyu Cai, Jingkuan Song, Yang Yang, Heng Tao Shen
Computation and Language · Cross-listed from cs.LG
StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?
Yanxu Chen, Zijun Yao, Yantao Liu et al. (+5 more)
Large
language models (LLMs) demonstrate strong potential as autonomous
agents, with promising capabilities in
reasoning, tool use, and sequential decision-making. While prior
benchmarks have evaluated LLM
agents in various domains, the financial domain remains underexplored, despite its significant economic value and complex
reasoning requirements. Most existing financial
benchmarks focus on static question-answering, failing to capture the dynamics of real-
market trading. To address this gap, we introduce STOCKBENCH, a contamination-free
benchmark designed to evaluate LLM
agents in realistic, multi-month stock trading environments.
Agents receive daily
market signals – including prices, fundamentals, and news – and make sequential buy, sell, or hold decisions. Performance is measured using financial metrics such as cumulative return, maximum drawdown, and the Sortino ratio, capturing both profitability and risk management. We evaluate a wide range of state-of-the-art proprietary and open-source LLMs. Surprisingly, most models struggle to outperform the simple buy-and-hold baseline, while some models demonstrate the potential to achieve higher returns and stronger risk management. These findings highlight both the challenges and opportunities of LLM-based trading
agents, showing that strong performance on static financial question-answering do not necessarily translate into effective trading behavior. We release STOCKBENCH as an open-source
benchmark to enable future research on LLM-driven financial
agents.
Read the full abstract ▾
All authors Yanxu Chen, Zijun Yao, Yantao Liu, Amy Xin, Jin Ye, Jianing Yu, Lei Hou, Juanzi Li
Computation and Language
Say One Thing, Do Another? Diagnosing Reasoning-Execution Gaps in VLM-Powered Mobile-Use Agents
Lingzhong Dong, Ziqi Zhou, Shuaibo Yang et al. (+6 more)
Mobile-use
agents powered by
vision-
language models (VLMs) have shown great potential in interpreting natural
language instructions and generating corresponding actions based on mobile graphical user interface. Recent studies suggest that incorporating chain-of-thought (CoT)
reasoning tends to improve the execution accuracy. However, existing evaluations emphasize execution accuracy while neglecting whether CoT
reasoning aligns with ground-truth actions. This oversight fails to assess potential
reasoning-execution gaps, which in turn foster over-trust: users relying on seemingly plausible CoTs may unknowingly authorize harmful actions, potentially resulting in financial loss or trust crisis. In this work, we introduce a new evaluation framework to diagnose
reasoning-execution gaps. At its core lies Ground-Truth
Alignment (GTA), which measures whether the action implied by a CoT matches the ground-truth action. By combining GTA with the standard Exact Match (EM) metric, we jointly assess both the
reasoning accuracy and execution accuracy. This joint perspective reveals two types of
reasoning-execution gaps: (i) Execution Gap (EG), where the
reasoning correctly identifies the correct action but execution fails, and (ii)
Reasoning Gap (RG), where execution succeeds but
reasoning process conflicts with the actual execution. Experimental results across a wide range of mobile interaction tasks reveal that
reasoning-execution gaps are prevalent, with execution gaps occurring more frequently than
reasoning gaps. Moreover, while scaling up model size reduces the overall gap, sizable execution gaps persist even in the largest models. Further analysis shows that our framework reliably reflects systematic EG/RG patterns in state-of-the-art models. These findings offer concrete diagnostics and support the development of more trustworthy mobile-use
agents.
Read the full abstract ▾
All authors Lingzhong Dong, Ziqi Zhou, Shuaibo Yang, Haiyue Sheng, Pengzhou Cheng, Zongru Wu, Zheng Wu, Gongshen Liu, Zhuosheng Zhang