Computation and Language
Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs
Amin Banayeeanzade, Qingchuan Yang, Dhruv Tarsadiya et al. (+6 more)
Diversity is essential for
language-model applications ranging from creative generation to scientific discovery, yet modern LLMs often collapse into a narrow subset of plausible outputs. While prior work has developed
benchmarks for measuring this lack of diversity, less is known about how the step-by-step probability distributions at inference time cause the problem. We introduce a validity–diversity framework that attributes diversity collapse to how an LLM allocates probability mass across valid and invalid continuations during decoding. This framework decomposes the bottleneck into two complementary forms of miscalibration. First, order calibration: valid
tokens are not reliably ranked above invalid
tokens, so rank-based cutoff rules must trade off between recovering valid continuations and admitting invalid ones. Second, shape calibration: probability mass is overly concentrated only on few valid continuations while having a heavy-tail of mixed valid and invalid
tokens, so maintaining high validity limits diversity. We formalize both mechanisms and show that local failures compound across decoding steps, producing strong sequence-level losses in diversity. Empirically, we develop controlled diagnostics for probing these bottlenecks, including tasks with exactly known valid sets and oracle cutoff baselines. Across 14
language models spanning multiple families and scales, we find that diversity collapse is not merely a limitation of particular sampling heuristics, but a consequence of order and shape miscalibration in the LLM distribution.
Read the full abstract ▾
All authors Amin Banayeeanzade, Qingchuan Yang, Dhruv Tarsadiya, Fatemeh Bahrani, Leonardo Blas, Alfy Samuel, Robin Jia, Meisam Razaviyayn, Sai Praneeth Karimireddy
Computation and Language · Cross-listed from cs.SE
On Problems of Implicit Context Compression for Software Engineering Agents
Kirill Gelvan, Igor Slinko, Felix Steinbauer et al. (+3 more)
LLM-based Software Engineering
agents face a critical bottleneck: context length limitations cause failures on complex, long-horizon tasks. One promising solution is to encode context as continuous
embeddings rather than discrete
tokens, enabling denser information storage. We apply the recently proposed In-Context Autoencoder for this purpose. While the method performs well on single-shot common-knowledge and code-understanding tasks, our experiments demonstrate that it fails on multi-step agentic coding tasks. In this paper, we explore this phenomenon and discuss possible factors contributing to this failure.
Read the full abstract ▾
All authors Kirill Gelvan, Igor Slinko, Felix Steinbauer, Egor Bogomolov, Florian Kofler, Yaroslav Zharov
Computation and Language
ELF: Embedded Language Flows
Keya Hu, Linlu Qiu, Yiyang Lu et al. (+5 more)
Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.g., in domains such as images and
videos. Their success has attracted growing interest in applying them to
language modeling. Unlike their image-domain counterparts, today's leading
diffusion language models (DLMs) primarily operate over discrete
tokens. In this paper, we show that continuous DLMs can be made effective with minimal adaptation to the discrete domain. We propose Embedded
Language Flows (ELF), a class of
diffusion models in continuous
embedding space based on continuous-time Flow Matching. Unlike existing DLMs, ELF predominantly stays within the continuous
embedding space until the final time step, where it maps to discrete
tokens using a shared-weight network. This formulation makes it straightforward to adapt established techniques from image-domain
diffusion models, e.g., classifier-free guidance (CFG). Experiments show that ELF substantially outperforms leading discrete and continuous DLMs, achieving better generation quality with fewer sampling steps. These results suggest that ELF offers a promising path toward effective continuous DLMs.
Read the full abstract ▾
All authors Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li, Yoon Kim, Jacob Andreas, Kaiming He
Computation and Language · Cross-listed from cs.LG
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
Chenyang Song, Weilin Zhao, Xu Han et al. (+3 more)
While Mixture-of-Experts (MoE) scales model capacity without proportionally increasing computation, its massive total parameter footprint creates significant storage and
memory-access bottlenecks, which hinder efficient end-side deployment that simultaneously requires high performance, low computational cost, and small storage overhead. To achieve these properties, we present DECO, a sparse MoE architecture designed to match the performance of dense
Transformers under identical total parameter budgets and training
tokens. DECO utilizes the differentiable and flexible ReLU-based routing enhanced by learnable expert-wise scaling, which adaptively balances the contributions of routed and shared experts. Furthermore, we introduce NormSiLU, an activation function that normalizes inputs prior to SiLU operators, producing a more stable trend of routed-expert activation ratio and a higher intrinsic sparsity level. We also identify an empirical advantage in using non-gated MLP experts with ReLU-based routing, indicating the possibility of MoE architecture simplification. Experiments demonstrate that DECO, activating only 20% of routed experts, matches dense performance and outperforms established MoE baselines. Our specialized acceleration kernel delivers a 2.93$\times$ speedup on Jetson AGX Orin compared with dense inference. Code and checkpoints are available at https://github.com/thunlp/DECO.
Read the full abstract ▾
All authors Chenyang Song, Weilin Zhao, Xu Han, Chaojun Xiao, Yingfa Chen, Zhiyuan Liu
Computation and Language · Cross-listed from cs.LG
Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning
Junhao Shen, Teng Zhang, Xiaoyan Zhao et al. (+1 more)
Large
language model
agents increasingly rely on external skills to solve complex tasks, where skills act as modular units that extend their capabilities beyond what parametric
memory alone supports. Existing skill-based RL methods assume external skills either accumulate as persistent guidance or internalized into the policy, eventually leading to zero-skill inference. We argue this assumption is overly restrictive, since with limited parametric capacity and uneven marginal contribution across skills, the optimal active skill set is non-monotonic, task- and stage-dependent. In this work, we propose SLIM, a framework of dynamic Skill LIfecycle Management for agentic reinforcement learning (RL), which dynamically updates the active external skill set as the policy evolves.Specifically, SLIM estimates each active skill's marginal external contribution through leave-one-skill-out validation, then applies three lifecycle operations: retaining high-value skills, retiring skills whose contribution becomes negligible after sufficient exposure, and expanding the skill bank when persistent failures reveal missing capability coverage. Experiments show that SLIM outperforms the best baselines by an average of 7.1% points across ALFWorld and SearchQA. Results further indicate that policy learning and external skill retention are not mutually exclusive: some skills are absorbed into the policy, while others continue to provide external value, supporting SLIM as a more general paradigm for skill-based agentic RL.
Read the full abstract ▾
All authors Junhao Shen, Teng Zhang, Xiaoyan Zhao, Hong Cheng
Computation and Language
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation
Shuangrui Ding, Xuanlang Dai, Long Xing et al. (+14 more)
Large
language and
vision-
language models increasingly power
agents that act on a user's behalf through command-line interface (CLI) harnesses. However, most
agent benchmarks still rely on synthetic sandboxes, short-horizon tasks, mock-service APIs, and final-answer checks, leaving open whether
agents can complete realistic long-horizon work in the runtimes where they are deployed. This work presents WildClawBench, a native-runtime
benchmark of 60 human-authored, bilingual,
multimodal tasks spanning six thematic categories. Each task averages roughly 8 minutes of wall-clock time and over 20 tool calls, and runs inside a reproducible Docker container hosting an actual CLI
agent harness (OpenClaw, Claude Code, Codex, or Hermes
Agent) with access to real tools rather than mock services. Grading is hybrid, combining deterministic rule-based checks, environment-state
auditing of side effects, and an LLM/VLM judge for semantic verification. Across 19 frontier models, the best, Claude Opus 4.7, reaches only 62.2% overall under OpenClaw, while every other model stays below 60%, and switching harness alone shifts a single model by up to 18 points. These results show that long-horizon, native-runtime
agent evaluation remains a far-from-resolved task for current frontier models. We release the tasks, code, and containerized tooling to support reproducible evaluation.
Read the full abstract ▾
All authors Shuangrui Ding, Xuanlang Dai, Long Xing, Shengyuan Ding, Ziyu Liu, Yang JingYi, Penghui Yang, Zhixiong Zhang, Xilin Wei, Xinyu Fang, Yubo Ma, Haodong Duan, Jing Shao, Jiaqi Wang, Dahua Lin, Kai Chen, Yuhang Zang
Computation and Language
RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang et al. (+9 more)
Deep research
agents plan, search, evaluate evidence, and synthesize long-form reports. Training these
agents with reinforcement learning (RL) is challenging because they operate over long horizons and their outputs lack exact answer verifiers. To this end, we introduce RubricEM, a unified RL framework for deep research that combines stagewise credit assignment with reflection-based meta-RL for experience reuse. Rubrics unify the framework by providing a shared interface for policy execution, judge feedback, and experience reuse. RubricEM first instills a rubric-guided scaffold through supervised fine-tuning, teaching the
agent to plan with rubrics and organize its trajectory into semantic stages. Building on this structural prior, we optimize the task policy with Stage-Structured GRPO, using stagewise judge feedback for finer-grained credit assignment. In parallel, we asynchronously train a shared-backbone reflection meta-policy (with meta-RL) to distill completed trajectories into reusable guidance, contributing through both shared-parameter updates and textual
memory. The resulting RubricEM-8B model achieves strong performance across four long-form research
benchmarks, outperforming existing open models of larger sizes and approaching proprietary deep-research systems. Beyond final performance, we perform thorough controlled analyses to understand the key ingredients of RubricEM.
Read the full abstract ▾
All authors Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister
Computation and Language
Grounded or Guessing? LVLM Confidence Estimation via Blind-Image Contrastive Ranking
Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur et al. (+4 more)
Large
vision-
language models (LVLMs) suffer from visual ungroundedness: they can produce a fluent, confident, and even correct response driven entirely by
language priors, with the image contributing nothing to the prediction. Existing confidence estimation methods cannot detect this, as they observe model behavior under normal inference with no mechanism to determine whether a prediction was shaped by the image or by text alone. We introduce BICR (Blind-Image Contrastive Ranking), a model-agnostic confidence estimation framework. The LVLM stays entirely frozen; instead, we extract its hidden states twice per sample, once with the real image-question pair and once with the image blacked out while the question is held fixed. A small, separate probe is then trained on the real-image hidden state, regularized by a ranking loss that penalizes higher confidence on the blacked-out view, teaching it to treat visual grounding as a signal of reliability. At inference only the single real-image pass is used, so BICR adds zero cost beyond a standard single-view probe. Evaluated across five modern LVLMs and seven baselines on a
benchmark covering visual question answering, object
hallucination detection, medical imaging, and financial document understanding, BICR achieves the best cross-LVLM average on both calibration and discrimination simultaneously, with statistically significant discrimination gains robust to cluster-aware analysis, at 4-18x fewer parameters than the strongest probing baseline.
Read the full abstract ▾
All authors Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur, Charese H. Smiley, Ivan Brugere, Kundan Thind, Mohammad M. Ghassemi
Computation and Language
Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs
Abrar Majeedi, Viswanatha Reddy Gajjala, Sai Prasanna Teja Reddy Bogireddy et al. (+1 more)
Automated question answering (QA) over electronic health records (EHRs) demands precise evidence
retrieval, faithful answer generation, and explicit grounding of answers in clinical notes. In this work, we present Neural1.5, our method for the ArchEHR-QA 2026 shared task at CL4Health@LREC 2026, which comprises four subtasks: question interpretation, evidence identification, answer generation, and evidence
alignment. Our approach decouples the task into independent, modular stages and employs DSPy"s MIPROv2 optimizer to automatically discover high-performing
prompts, jointly tuning instructions and few-shot demonstrations for each stage. Within every stage, self-consistency voting over multiple
stochastic inference runs suppresses spurious errors and improves reliability, while stage-specific verification mechanisms (e.g., self-reflection and chain-of-verification for
alignment) further refine output quality. Among all teams that participated in all four subtasks, our method ranks second overall (mean rank 4.00), placing 4th, 1st, 4th, and 7th on Subtasks 1-4, respectively. These results demonstrate that systematic, per-stage
prompt optimization combined with self-consistency mechanisms is a cost-effective alternative to model fine-tuning for multifaceted clinical QA.
Read the full abstract ▾
All authors Abrar Majeedi, Viswanatha Reddy Gajjala, Sai Prasanna Teja Reddy Bogireddy, Siddhant Rai
Computation and Language · Cross-listed from cs.LG
Compute Where it Counts: Self Optimizing Language Models
Yash Akhauri and Mohamed S. Abdelfattah
Efficient LLM inference research has largely focused on reducing the cost of each decoding step (e.g., using quantization,
pruning, or sparse
attention), typically applying a uniform computation budget to every generated
token. In practice,
token difficulty varies widely, so static compression can over-compute on easy steps and under-compute on hard ones. We study dynamic budget allocation for autoregressive decoding: learning how much computation to spend per
token from within a single model. Self-Optimizing
Language Models (SOL) pair a frozen LLM with a lightweight policy network that reads the LLM hidden state and selects a discrete efficiency action at each decode step. Actions can jointly control (i)
token-level
attention sparsity, (ii) structured activation
pruning in the MLP, and (iii) activation quantization bit-width, while leaving the base model weights unchanged. We train the policy with group-relative policy optimization on teacher-forced episodes: the
token sequence is fixed, while we sample multiple compute schedules (i.e., "counterfactual" schedules that vary only the efficiency actions for the same
token path) and compare their likelihoods under the same supervision. Our reward trades off
language-model quality against soft penalties that encourage episode-average budget usage to match a requested target. Across model variants and compute regimes, SOL improves quality at matched budget over static allocation and strong random schedule search, offering a complementary axis for inference-efficiency optimization. SOL discovers a better quality-efficiency pareto-front across all our experiments and improves MMLU accuracy by up to 7.3% over uniform budget allocation strategies.
Read the full abstract ▾
Computation and Language
DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization
Mengyi Deng, Zhiwei Li, Xin Li et al. (+4 more)
Although Large
Language Models (LLMs) have made remarkable progress, current preference optimization methods still struggle to align directional consistency while preserving
reasoning diversity. To address this limitation, we propose Directional-Groupwise Preference Optimization (DGPO), a lightweight framework that aggregates supervision signals at the group level and explicitly models direction-aware
alignment through multi-candidate comparisons. DGPO organizes forward and reverse question-answer instances into structured sets and optimizes a margin-based likelihood objective that separates coherent
reasoning paths from inconsistent alternatives. This group-wise formulation captures richer relative information than pairwise objectives and reinforces consistency across diverse
reasoning pathways. Empirical results show that our constructed reverse data yields a 3.2% average improvement across five
benchmarks, while DGPO further delivers consistent gains across multiple
datasets and model families, achieving average accuracy improvements of up to 3.6%.
Read the full abstract ▾
All authors Mengyi Deng, Zhiwei Li, Xin Li, Tingyu Zhu, Yulan Yuan, Zhijiang Guo, Wei Wang
Computation and Language
RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systems
Joel Rorseth, Parke Godfrey, Lukasz Golab et al. (+2 more)
This paper demonstrates RUBEN, an interactive tool for discovering minimal rules to explain the outputs of
retrieval-augmented large
language models (LLMs) in data-driven applications. We leverage novel
pruning strategies to efficiently identify a minimal set of rules that subsume all others. We further demonstrate novel applications of these rules for LLM safety, specifically to test the resiliency of safety training and effectiveness of
adversarial prompt injections.
Read the full abstract ▾
All authors Joel Rorseth, Parke Godfrey, Lukasz Golab, Divesh Srivastava, Jarek Szlichta
Computation and Language
Learning More from Less: Exploiting Counterfactuals for Data-Efficient Chart Understanding
Jianzhu Bao, Haozhen Zhang, Kuicai Dong et al. (+5 more)
Vision-
Language Models (VLMs) have demonstrated remarkable progress in chart understanding, largely driven by supervised fine-tuning (SFT) on increasingly large synthetic
datasets. However, scaling SFT data alone is inefficient and overlooks a key property of charts: charts are programmatically generated visual artifacts, where small, code-controlled visual changes can induce drastic shifts in semantics and correct answers. Learning this counterfactual sensitivity requires VLMs to discriminate fine-grained visual differences, yet standard SFT treats training instances independently and provides limited supervision to enforce this behavior. To address this, we introduce ChartCF, a data-efficient training framework designed to enhance counterfactual sensitivity. ChartCF consists of: (1) a counterfactual data synthesis pipeline via code modification, (2) a chart similarity-based data selection strategy that filters overly difficult samples for improved training efficiency, and (3)
multimodal preference optimization across both textual and visual modalities. Experiments on five
benchmarks show that ChartCF achieves superior or comparable performance to strong chart-specific VLMs while using significantly less training data.
Read the full abstract ▾
All authors Jianzhu Bao, Haozhen Zhang, Kuicai Dong, Bozhi Wu, Sarthak Ketanbhai Modi, Zi Pong Lim, Yon Shin Teo, Wenya Wang
Computation and Language
Grounded Satirical Generation with RAG
Oona Itkonen, Yuxin Su, Linyao Du et al. (+1 more)
Humor generation remains challenging task for Large
Language Models (LLMs), due to their subjective nature. We focus on satire, a form of humor strongly shaped by context. In this work, we present a novel pipeline for grounded satire generation that uses
Retrieval-Augmented Generation (RAG) over current news to produce satirical dictionary definitions in the Finnish context. We also introduce a new task-specific evaluation framework and annotate 100 generated definitions with six human annotators, enabling analysis across multiple experimental conditions, including cultural background, source-word type, and the presence or absence of RAG. Our results show that the generated definitions are perceived as more political than humorous. Both topic-based word selection and RAG improve the political relevance of the outputs, but neither yields clear gains in humor generation. In addition, our LLM-as-a-judge evaluation of five state-of-the-art models indicates that LLMs correlate well with human judgments on political relevance, but perform poorly on humor. We release our code and annotated
dataset to support further research on grounded satire generation and evaluation.
Read the full abstract ▾
All authors Oona Itkonen, Yuxin Su, Linyao Du, Ona De Gibert
Computation and Language · Cross-listed from cs.AI
Generalizing the Turing Test to Interactive Agents
Daniel Mitropolsky, Riccardo Neumarker, Emanuele Rimoldi et al. (+2 more)
We initiate the study of the Generalized Turing Test (GTT), a formal generalization of Turing's imitation game from humans to arbitrary interactive
agents. For
agents $A$ and $B$, $A$ passes the GTT against $B$ if an instance of $B$, acting as a distinguisher, cannot reliably distinguish an $A$ instructed to imitate $B$ from another instance of $B$; if so, we write $A \geq B$. We study the theoretical and empirical consequences of this idea. On the theory side, we prove sufficient conditions under which this "Turing Comparator" is transitive. We introduce natural variants with querying (the imitator can first interact with a specimen of the target), a Universal Turing Test with arbitrary distinguishers and targets, and complexity-theoretic variants that control interaction length. As a proof of concept, we evaluate the GTT and its variants across nine large
language models. Remarkably, Turing Scores recover a clear model stratification consistent with standard external
benchmarks despite being derived entirely from pairwise imitation games. Transcript analysis reveals that models use both stylistic signatures and substantive STEM and logic-based probes. Together, these results suggest indistinguishability could provide a meaningful signal for comparing
agents, yielding an inherently adaptive form of evaluation that does not rely on fixed
benchmarks.
Read the full abstract ▾
All authors Daniel Mitropolsky, Riccardo Neumarker, Emanuele Rimoldi, Susan S. Hong, Tomaso Poggio
Computation and Language · Cross-listed from cs.IR
Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?
Tz-Huan Hsu, Jheng-Hong Yang and Jimmy Lin
Does a lexical retriever suffice as large
language models (LLMs) become more capable in an agentic loop? This question naturally arises when building deep research systems. We revisit it by pairing BM25 with frontier LLMs that have better
reasoning and tool-use abilities. To support researchers asking the same question, we introduce Pi-Serini, a search
agent equipped with three tools for retrieving, browsing, and reading documents. Our results show that, on BrowseComp-Plus, a well-configured lexical retriever with sufficient
retrieval depth can support effective deep research when paired with more capable LLMs. Specifically, Pi-Serini with gpt-5.5 achieves 83.1% answer accuracy and 94.7% surfaced evidence recall, outperforming released search
agents that use dense retrievers. Controlled ablations further show that BM25 tuning improves answer accuracy by 18.0% and surfaced evidence recall by 11.1% over the default BM25 setting, while increasing
retrieval depth further improves surfaced evidence recall by 25.3% over the shallow-
retrieval setting. Source code is available at https://github.com/justram/pi-serini.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
BabelDOC: Better Layout-Preserving PDF Translation via Intermediate Representation
Qi Yang, Xiangyao Ma, Xiao Wang et al. (+2 more)
As global cross-lingual communication intensifies,
language barriers in visually rich documents such as PDFs remain a practical bottleneck. Existing document translation pipelines face a tension between linguistic processing and layout preservation: text-oriented Computer-Assisted Translation (CAT) systems often discard structural metadata, while document parsers focus on extraction and do not support faithful re-rendering after translation. We introduce BabelDOC, an Intermediate Representation (IR)-based framework for layout-preserving PDF translation. BabelDOC decouples visual layout metadata from semantic content, enabling document-level translation operations such as terminology extraction, cross-page context handling, glossary-constrained generation, and formula placeholdering. The translated content is then re-anchored to the original layout through an adaptive typesetting engine. Experiments on a curated 200-page
benchmark, together with human evaluation and
multimodal LLM-as-a-judge evaluation, show that BabelDOC improves layout fidelity, visual aesthetics, and terminology consistency over representative baselines, while maintaining competitive translation precision. The open-source toolkit and its interactive downstream applications are publicly available and have attracted over 8.4K GitHub
stars and 17 contributors at the time of writing. A demonstration
video is also available.
Read the full abstract ▾
All authors Qi Yang, Xiangyao Ma, Xiao Wang, Hao Wang, Rui Wang
Computation and Language
Training-Free Cultural Alignment of Large Language Models via Persona Disagreement
Huynh Trung Kiet, Dao Sy Duy Minh, Tuan Nguyen et al. (+5 more)
Large
language models increasingly mediate decisions that turn on moral judgement, yet a growing body of evidence shows that their implicit preferences are not culturally neutral. Existing cultural
alignment methods either require per-country preference data and fine-tuning budgets or assume white-box access to model internals that commercial APIs do not expose. In this work, we focus on this realistic black-box, public-data-only regime and observe that within-country sociodemographic disagreement, not consensus, is the primary steering signal. We introduce DISCA (Disagreement-Informed Steering for Cultural
Alignment), an inference-time method that instantiates each country as a panel of World-Values-Survey-grounded persona
agents and converts their disagreement into a bounded, loss-averse logit correction. Across 20 countries and 7 open-weight backbones (2B–70B), DISCA reduces cultural misalignment on MultiTP by 10–24% on the six backbones >=3.8B, and 2–7% on open-ended scenarios, without changing any weights. Our results suggest that inference-time calibration is a scalable alternative to fine-tuning for serving the long tail of global moral preferences.
Read the full abstract ▾
All authors Huynh Trung Kiet, Dao Sy Duy Minh, Tuan Nguyen, Chi-Nguyen Tran, Phu-Hoa Pham, Nguyen Lam Phu Quy, The Anh Han, Long Tran-Thanh
Computation and Language
Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents
Shijue Huang, Hangyu Guo, Guanting Dong et al. (+8 more)
Multimodal deep search requires an
agent to solve open-world problems by chaining search, tool use, and visual
reasoning over evolving textual and visual context. Two bottlenecks limit current systems. First, existing tool-use harnesses treat images returned by search, browsing, or transformation as transient outputs, so intermediate visual evidence cannot be re-consumed by later tools. Second, training data is usually built by fixed curation recipes that cannot track the target
agent's evolving capability. To address these challenges, we first introduce a visual-native
agent harness centered on an image bank reference protocol, which registers every tool-returned image as an addressable <image:N> reference and makes intermediate visual evidence reusable by later tools. On top of this harness, On-policy Data
Evolution (ODE) runs a closed-loop data generator that refines itself across rounds from rollouts of the policy being trained. This per-round refinement makes each round's data target what the current policy still needs to learn. The same framework supports both diverse supervised fine-tuning data and policy-aware reinforcement learning data curation, covering the full training lifecycle of the target
agent. Across 8
multimodal deep search
benchmarks, ODE improves the Qwen3-VL-8B
agent from 24.9% to 39.0% on average. At 30B, ODE raises the average score from 30.6% to 41.5%. Further analyses validate the effectiveness of image bank reuse, especially on complex tasks requiring iterative visual refinement, while rollout-feedback
evolution yields more grounded SFT traces and better policy-matched RL tasks than static synthesis.
Read the full abstract ▾
All authors Shijue Huang, Hangyu Guo, Guanting Dong, Chenxin Li, Junting Lu, Xinyu Geng, Zhaochen Su, Zhenyu Li, Shuang Chen, Hongru Wang, Yi R. Fung
Computation and Language · Cross-listed from cs.LG
SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing
Mingxu Zhang, Yuhan Li, Lujundong Li et al. (+3 more)
Large
language models possess strong chemical
reasoning capabilities, making them effective molecular editors. However, property-relevant information is implicitly entangled across their dense hidden states, providing no explicit handle for property control: a substantial fraction of edits fail to improve or even degrade target properties. To address these issues, we propose SLIM (Sparse Latent Interpretable Molecular editing), a plug-and-play framework that decomposes the editor's hidden states into sparse, property-aligned features via a Sparse Autoencoder with learnable importance gates. Steering in this sparse feature space precisely activates property-relevant dimensions, improving editing success rate without modifying model parameters. The same sparse basis further supports interpretable analysis of editing behavior. Experiments on the MolEditRL
benchmark across four model architectures and eight molecular properties show consistent gains over baselines, with improvements of up to 42.4 points.
Read the full abstract ▾
All authors Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun
Computation and Language · Cross-listed from cs.AI
Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge
Wenbo Zhang, Lijinghua Zhang, Liner Xiang et al. (+1 more)
Reasoning-capable large
language models (LLMs) have recently been adopted as automated judges, but their benefits and costs in LLM-as-a-Judge settings remain unclear. Through controlled comparisons between
reasoning and non-
reasoning judges, we show that explicit
reasoning substantially improves judgment accuracy on tasks requiring structured verification (e.g., math and coding), while offering limited or even negative gains on simpler evaluations and incurring significantly higher computational cost. These findings motivate that
reasoning should be used selectively rather than universally, with awareness of possible distribution shift. We propose a Robust Adaptive Cost-Efficient Routing (RACER), which dynamically selects between
reasoning and non-
reasoning judges under a fixed budget by formulating routing as a constrained distributionally robust optimization problem. RACER explicitly accounts for distribution shift via a KL-divergence uncertainty set, admits an efficient primal–dual
algorithm, and enjoys theoretical guarantees including uniqueness of the optimal policy and linear convergence. Extensive experiments show that RACER achieves superior accuracy–cost trade-offs under distribution shift.
Read the full abstract ▾
All authors Wenbo Zhang, Lijinghua Zhang, Liner Xiang, Hengrui Cai
Computation and Language · Cross-listed from cs.LG
The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies
Gabriel Garcia
Corruption studies, the standard tool for evaluating chain-of-thought (CoT) faithfulness, infer which steps are “computationally important” from accuracy loss when steps are corrupted. We show that when
benchmark chains end with an explicit terminal answer line, as in GSM8K and MATH, these tests largely measure answer placement rather than where intermediate computation is carried out. Using matched GSM8K examples, removing only the final answer statement while preserving all
reasoning collapses suffix sensitivity by about $19\times$ for Qwen~2.5-3B ($N{=}300$, $p{=}0.022$). Conflicting-answer
prompts, which contain correct
reasoning but a wrong explicit final answer, drive accuracy to zero or near-zero at 7B across five open-weight model families; wrong-answer following is strong at 3B–7B and attenuates sharply at larger scales. Replications on MATH, within-stable comparisons at 7B, and suffix-free chains show the same pattern in different guises: corruption sensitivity tracks the location of explicit answer text, not a fixed computational depth in the
reasoning. Generation-time probes indicate that final answers are rarely early-determined during generation (${<}5\%$ early commitment), yet consumption-time behavior systematically follows explicit answer text. The confound is therefore largely a readout effect when the chain is consumed. We propose a three-prerequisite protocol (question-only control, format characterization, and an all-position sweep) as a practical minimum for future corruption-based faithfulness studies.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR
Jeonghye Kim, Jiwon Jeon, Dongsheng Li et al. (+1 more)
Self-
distillation has emerged as a powerful framework for post-training LLMs, where a teacher conditioned on extra information guides a student without it, both from the same model. While this guidance is useful when the student has failed, on successful rollouts, the same mechanism instead overwrites the student's choices and suppresses it's own
reasoning. Therefore, we propose reading the original self-
distillation signal in reverse: when the student succeeds along a path the teacher would not have predicted, these
tokens reflect its self-driven
reasoning. Building on this, we propose RLRT (RLVR with Reversed Teacher), which augments GRPO by reinforcing these
tokens on correct rollouts. We interpret this as a new form of exploration in RLVR: not uniform diversity, but valuable exploration grounded in the student's own success. Across base, instruction-tuned, and thinking-tuned Qwen3 checkpoints, RLRT substantially outperforms self-
distillation and exploration-based baselines, establishing information asymmetry as a new, principled design axis for RLVR.
Read the full abstract ▾
All authors Jeonghye Kim, Jiwon Jeon, Dongsheng Li, Yuqing Yang
Computation and Language · Cross-listed from cs.CR
LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
Chiyu Zhang, Lu Zhou, Xiaogang Xu et al. (+8 more)
The rapid proliferation of LLM-based autonomous
agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an
agent to execute dangerous OS-level operations with irreversible consequences. Existing
benchmarks either evaluate safety at the semantic layer alone, missing physical-layer harms, or fail to isolate test cases, letting earlier runs contaminate later ones. We present LITMUS (LLM-
agents In-OS Testing for Measuring Unsafe Subversion), a
benchmark addressing both gaps via a semantic-physical dual verification mechanism and OS-level state rollback. LITMUS comprises 819 high-risk test cases organized into one harmful seed subset and six attack-extended subsets covering three
adversarial paradigms (jailbreak speaking, skill injection, and entity wrapping), plus a fully automated multi-
agent evaluation framework judging behavior at both conversational and OS-level physical layers. Evaluation across frontier
agents reveals three findings: (1) current
agents lack effective safety awareness, with strong models (e.g., Claude Sonnet 4.6) still executing 40.64% of high-risk operations; (2)
agents exhibit pervasive Execution
Hallucination (EH), verbally refusing a request while the dangerous operation has already completed at the system level, invisible to every prior semantic-only framework; and (3) skill injection and entity wrapping attacks achieve high success rates, exposing pronounced
agent vulnerabilities. LITMUS provides the first standardized platform for reproducible, physically grounded behavioral safety evaluation of LLM
agents in real OS environments.
Read the full abstract ▾
All authors Chiyu Zhang, Lu Zhou, Xiaogang Xu, Huiqin Yang, Bendong Jiang, Xiaolei Zhang, Yiran Zhao, Ruyi Chen, Jiafei Wu, Liming Fang, Zhe Liu
Computation and Language · Cross-listed from physics.soc-ph
Conformity Generates Collective Misalignment in AI Agents Societies
Giordano De Marzo, Alessandro Bellina, Claudio Castellano et al. (+2 more)
Artificial intelligence safety research focuses on aligning individual
language models with human values, yet deployed AI systems increasingly operate as interacting populations where social influence may override individual
alignment. Here we show that populations of individually aligned AI
agents can be driven into stable misaligned states through conformity dynamics. Simulating opinion dynamics across nine large
language models and one hundred opinion pairs, we find that each
agent's behavior is governed by two competing forces: a tendency to follow the majority and an intrinsic bias toward specific positions. Using tools from statistical physics, we derive a quantitative theory that predicts when populations become trapped in long-lived misaligned configurations, and identifies predictable tipping points where small numbers of
adversarial agents can irreversibly shift population-level
alignment even after manipulation ceases. These results demonstrate that individual-level
alignment provides no guarantee of collective safety, calling for evaluation frameworks that account for emergent behavior in AI populations.
Read the full abstract ▾
All authors Giordano De Marzo, Alessandro Bellina, Claudio Castellano, Viola Priesemann, David Garcia
Computation and Language
Why Low-Resource NLP Needs More Than Cross-Lingual Transfer: Lessons Learned from Luxembourgish
Fred Philippy, Siwen Guo, Jacques Klein et al. (+1 more)
Cross-lingual transfer has become a central paradigm for extending natural
language processing (NLP) technologies to low-resource
languages. By leveraging supervision from high-resource
languages,
multilingual language models can achieve strong task performance with little or no labeled target-
language data. However, it remains unclear to what extent cross-lingual transfer can substitute for
language-specific efforts. In this paper, we synthesize prior research findings and data collection results on Luxembourgish, which, despite its typological proximity to high-resource
languages and its presence in a
multilingual context, remains insufficiently represented in modern NLP technologies. Across findings, we observe a fundamental interdependence between cross-lingual transfer and
language-specific efforts. Cross-lingual transfer can substantially improve target-
language performance, but its success depends critically on the availability of sufficiently high-quality, task-aligned target-
language data. At the same time, such resources, particularly in low-resource settings, are typically too limited in scale to drive strong performance on their own. Instead, such resources reach their full potential only when leveraged within a cross-lingual framework. We therefore argue that cross-lingual transfer and
language-specific efforts should not be viewed as competing alternatives. Instead, they function as complementary components of a sustainable low-resource NLP pipeline. Based on these insights, we provide practical guidelines for integrating and balancing cross-lingual transfer with
language-specific development in sustainable low-resource NLP pipelines.
Read the full abstract ▾
All authors Fred Philippy, Siwen Guo, Jacques Klein, Tegawendé F. Bissyandé
Computation and Language · Cross-listed from cs.LG
Step Rejection Fine-Tuning: A Practical Distillation Recipe
Igor Slinko, Ilia Zavidnyi, Egor Bogomolov et al. (+1 more)
Rejection Fine-Tuning (RFT) is a standard method for training LLM
agents, where unsuccessful trajectories are discarded from the training set. In the context of SWE-bench tasks, this corresponds to filtering out runs where the submitted patch does not pass the tests. However, this approach discards unresolved trajectories, even though they form a large portion of all trajectories for hard tasks and even then may be partially correct. In this work, we propose Step Rejection Fine-Tuning (SRFT) - a practical way to leverage these unresolved trajectories. For this, we employ a critic LLM to assess the correctness of each step in a trajectory. Consequently, during training, we mask the loss for erroneous steps while retaining them in the context window. This way we ensure the model learns to recover from errors without reproducing them. Evaluation on SWE-bench Verified shows that while RFT improves the resolution rate by 2.4% by excluding unresolved trajectories, SRFT improves it by 3.7% by filtering them instead of discarding completely, reaching the total resolution rate of 32.2%.
Read the full abstract ▾
All authors Igor Slinko, Ilia Zavidnyi, Egor Bogomolov, Yaroslav Zharov
Computation and Language
Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions
Diancheng Kang, Zheyuan Liu, Ningshan Ma et al. (+3 more)
Activation steering controls
language model behavior by adding directions to internal representations at inference time, but standard residual-stream steering can fail in stateful dialogue. We identify KV-cache contamination as a key failure mode: steered
token states are stored and repeatedly reused, turning a local perturbation into cumulative coherence degradation. To address this challenge, we propose Gated Cropped
Attention-Delta steering (GCAD), which extracts steering signals from system-
prompt contributions to self-
attention and applies them with
token-level gating. Across persona-steering experiments, GCAD preserves trait control while substantially improving long-horizon coherence. On the main multi-turn
benchmark, GCAD improves average coherence drift from -18.6 to -1.9 and raises turn-10 trait expression from 78.0 to 93.1. These results suggest that activation steering becomes more reliable when interventions follow the
prompt-mediated pathways that models already use for behavioral control.
Read the full abstract ▾
All authors Diancheng Kang, Zheyuan Liu, Ningshan Ma, Yue Huang, Zhaoxuan Tan, Meng Jiang
Computation and Language
When Can Digital Personas Reliably Approximate Human Survey Findings?
Mumin Jia, Yilin Chen, Divya Sharma et al. (+1 more)
Digital personas powered by Large
Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet it remains unclear when they can reliably approximate human survey findings. We answer this question using the LISS panel, constructing personas from respondents' background variables and pre-2023 survey histories, then testing them against the same respondents' held-out post-cutoff answers. Across four persona architectures, three LLMs, and two prediction tasks, we assess performance at the question, respondent, distributional, equity, and clustering levels. Digital personas improve
alignment with human response distributions, especially in domains tied to stable attributes and values, but remain limited for individual prediction and fail to recover multivariate respondent structure.
Retrieval-augmented architectures provide the clearest gains, but performance depends more on human response structure than on model choice: personas perform best for low-variability questions and common respondent patterns, and worst for subjective, heterogeneous, or rare responses. Our results provide practical guidance on when digital personas could be appropriate for survey research and when human validation remains necessary.
Read the full abstract ▾
All authors Mumin Jia, Yilin Chen, Divya Sharma, Jairo Diaz-Rodriguez
Computation and Language
A Single-Layer Model Can Do Language Modeling
Zanmin Wang
Modern
language models scale depth by stacking layers, each holding its own state - a per-layer KV cache in
transformers, a per-layer
matrix in Mamba, Gated DeltaNet (GDN), RWKV, and xLSTM. Biological systems lean heavily on recurrence rather than on stacking. We ask how far that shape can go on
language modeling. We propose Grounded Prediction Networks (GPN): one state vector revisited at every step through a single recurrent block - one FFN, one shared
matrix memory. At 130M parameters, a 1-layer GPN+M reaches FineWeb-Edu perplexity 18.06, within 13% of a 12-layer
Transformer++ (16.05) and 18% of a 10-layer GDN (15.34); a 2-layer variant closes the gap to 6%/11%. We do not match the deep baselines. Because the working context is a single vector, we can directly inspect its geometry: a persistent default-
token direction, a content-bearing horizon of tens of
tokens, and
memory heads that split spontaneously into fast and slow retention pools.
Read the full abstract ▾