Computation and Language · Cross-listed from cs.LG
Large Language Models Hack Rewards, and Society
Wei Liu, Xinyi Mou, Hanqi Yan et al. (+2 more)
Reinforcement learning (RL) has become a dominant post-training paradigm, enabling large
language models (LLMs) to learn from rewards. We observe that societal regulations are structurally similar to reward functions. They define measurable outcomes, thresholds, and exceptions, while often leaving institutional intent only partially specified. We hypothesise that the RL training process may exploit these gaps and therefore ask whether models' well-known tendency to hack reward functions during RL can scale into a more consequential failure mode named societal hacking: discovering loopholes in the rules society runs on. To study this phenomenon, we introduce SocioHack, a sandbox of 72 societal environments, and find that within these environments, reward hacking naturally emerges and leads to regulatory loophole discovery. Models learn to hack the social rules and generate strategies that remain technically compliant while defeating regulatory intent, and current LLM safeguards provide only limited mitigation. Therefore, collecting in-the-wild feedback for model training requires greater caution, and we need a next-generation post-training paradigm for safely iterating LLMs in real society.=
Read the full abstract ▾
All authors Wei Liu, Xinyi Mou, Hanqi Yan, Zhongyu Wei, Yulan He
Computation and Language · Cross-listed from cs.CR
Covert Influence Between Language Models
Avidan Shah, Jay Chooi, Jinghua Ou et al. (+1 more)
As
language models increasingly consume one another's outputs, covert influence – a phenomenon where a sender's payload (the behavioral disposition it is conditioned to propagate) transfers to a receiver through carriers undetectable by humans – becomes a growing risk. We characterize this risk across three interfaces: supervised fine-tuning, on-policy
distillation, and in-context learning, and find that they vary in the scale of influence achievable without leaving behind human-visible traces. Using inference-time per-sample attribution scores, we study covert influence across all three interfaces with the ability to select carriers that amplify training-time influence, unlocking payload transfers that prior work could not achieve. We further provide evidence that covert influence with natural-
language carriers is a distinct phenomenon from prior studies using number carriers, as the latter is more resistant to human detection and less portable across model families. Together, these results suggest that the risk surface for covert influence is broader than previously recognized, and we study pointwise attribution scoring methods as a tool to investigate and mitigate it.
Read the full abstract ▾
All authors Avidan Shah, Jay Chooi, Jinghua Ou, Shi Feng
Computation and Language · Cross-listed from cs.CV
Dive into the Scene: Breaking the Perceptual Bottleneck in Vision-Language Decision Making via Focus Plan Generation
Boyuan Xiao, Bohong Chen, Yumeng Li et al. (+3 more)
In
embodied vision-
language decision making tasks such as robotic manipulation and
navigation,
Vision-
Language and
Vision-
Language-Action Models (VLMs & VLAs) are powerful tools with different benefits: VLMs are better at long-term planning, while VLAs are better at reactive control. However, their performance is limited by the same perceptual bottleneck: visual
hallucinations arise due to the models' inability to distinguish task-relevant objects from distractors. In principle, accurate identification and focus on critical objects while filtering out irrelevant ones is the key to break this limitation. A straightforward solution is one-step focus: directly attending to essential objects. However, this approach proves ineffective because effective focus inherently requires deep scene understanding. To this end, we propose SceneDiver, a coarse-to-fine focus plan generation method for VLMs leveraging their long-term planning abilities, that first constructs a holistic scene graph to establish initial comprehension, then progressively decomposes the task into simpler sub-problems through an iterative cycle of recognition, understanding, and analysis. To enable reactive control, we also design a lightweight adapter for distilling the deliberate focus ability into VLAs. Evaluations on standard
embodied AI
benchmarks confirm that our method substantially reduces visual
hallucinations for both VLMs and VLAs, while preserving computational efficiency in tasks requiring fast execution. Our code and data are released at: https://future-item.github.io/SceneDiver.
Read the full abstract ▾
All authors Boyuan Xiao, Bohong Chen, Yumeng Li, Ji Feng, Yao-Xiang Ding, Kun Zhou
Computation and Language · Cross-listed from cs.LG
Neuron Populations Exhibit Divergent Selectivity with Scale
Amil Dravid, Yasaman Bahri, Alexei A. Efros et al. (+1 more)
We investigate whether
neuron populations within neural networks evolve predictably with scale, extending scaling laws beyond macroscopic observables such as loss. To probe this question, we study Rosetta
Neurons, a previously characterized class of
neurons whose activation patterns are similar across independently trained models (Dravid et al., 2023). In separate analyses of
language models up to 30B parameters and
vision models up to 5B parameters, we observe that the population of Rosetta
Neurons follows a sublinear power law in model size, growing in absolute number but occupying a shrinking fraction of the total
neuron count. We further observe a
Neuron Polarization Effect: Rosetta
Neurons become more selective and increasingly monosemantic with scale, separating from a growing non-Rosetta population that remains less selective. An analytical model balancing feature utility against limited
neuron capacity explains the sublinear power-law scaling and this polarization effect. Finally, we find that Rosetta
Neurons become more domain-specialized with scale and illustrate their selectivity through a targeted data-filtering case study for continued pretraining. Our results point to a scaling law for interpretable, shared
neuron-level structure, linking model size to systematic changes in
neuron universality, selectivity, and specialization.
Read the full abstract ▾
All authors Amil Dravid, Yasaman Bahri, Alexei A. Efros, Yossi Gandelsman
Computation and Language
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics
Mutsumi Sasaki, Go kamoda, Ryosuke Takahashi et al. (+4 more)
Quantities with measurement units, such as 110 cm and 1.2 m, require
language models (LMs) to combine a numeral with a symbolic unit scale. Here, we study how LMs compare such quantities in controlled settings spanning several unit systems. We find that accuracy degrades near the comparison boundary, where small changes in value determine the correct answer. The resulting errors are systematic: linear surrogate models predict LM preferences from numerical-difference and unit-scale-difference cues, and causal interventions on subspaces aligned with these variables shift model's output. The results suggest that LMs compare quantities through a bag of heuristics over numerals and units, rather than first converting both expressions to an exact shared-scale representation.
Read the full abstract ▾
All authors Mutsumi Sasaki, Go kamoda, Ryosuke Takahashi, Kosuke Sato, Kentaro Inui, Keisuke Sakaguchi, Benjamin Heinzerling
Computation and Language · Cross-listed from cs.LG
Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill
Tao Chen, Gangwei Jiang, Pengyu Cheng et al. (+10 more)
Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill. By treating reward computation as a structured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. This approach enables the reward model to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. Extensive experiments on reward
benchmarks and downstream applications, including best-of-N selection and reinforcement learning, demonstrate that Skill-RM consistently outperforms traditional judge baselines. Our findings suggest that Skill-RM not only provides a unified solution for reward modeling but also achieves superior performance through the strategic and dynamic orchestration of evidence. The code is at https://github.com/Qwen-Applications/Skill-RM.
Read the full abstract ▾
All authors Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang
Computation and Language
Quantifying Faithful Confidence Expression in Large Reasoning Models
Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu et al. (+1 more)
Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)–the
alignment between models' intrinsic and (linguistically) expressed confidence–is a persistent failure mode. This challenge is key for large
reasoning models (LRMs), whose extended
reasoning traces are often interpreted by users as evidence of deliberation, competence, and confidence. Despite the importance of FC and wide usage of LRMs, the extent to which LRMs can faithfully express their confidence remains poorly understood. Moreover, the prevailing paradigm to measure FC does not generalize well to the long chain-of-thought outputs generated by LRMs, which tend to lack clear step boundaries, involve inconsistent step structure, and encode complex conditional dependencies throughout the trace–complicating estimation of intrinsic confidence. To address this challenge, we introduce a novel framework to systematically quantify FC of LRMs. Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty, based on
token probabilities, hidden states, and sampled response consistency. We also devise a prefix-conditioned sampling approach to control for conditional and structural variation across traces. Applying our framework to a diverse suite of leading models,
datasets, and
prompts, we find that faithful confidence expression is a significant challenge for LRMs.
Reasoning behaviors do not automatically translate to improved FC, and
prompt interventions for non-
reasoning models do not improve faithfulness in the
reasoning setting. Different confidence
estimators further produce divergent assessments of the same traces, revealing fragility in prior evaluation methodologies. Taken together, our work establishes FC as a distinct reliability and
alignment target for LRMs, particularly as such systems are increasingly deployed in high-stakes contexts.
Read the full abstract ▾
All authors Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu, Arman Cohan
Computation and Language
QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards
Rongzhi Zhang, Rui Feng, Zhihan Zhang et al. (+8 more)
Rubric-based RL is a promising route for extending reinforcement learning beyond verifiable rewards, yet existing methods optimize rubrics while treating the query distribution as fixed. We identify a structural bottleneck: rubric quality is constrained by query structure. Open-ended queries yield vague rubrics; naively narrowing them introduces fabricated references that no model can verify, so all responses fail and training receives no reward signal. We present QUBRIC, a framework that co-designs queries and rubrics. Teacher-derived key points ground the rewriting of open-ended queries into scenario-based, evaluable questions. Contrastive rubric generation then turns teacher-policy gaps into query-level criteria, and learnability filtering retains only informative query-rubric pairs for GRPO training. QUBRIC achieves a +5.5 point gain on ArenaHard over the SFT baseline. Trained only on instruction-following data, it further transfers to three held-out
benchmarks spanning legal, moral, and narrative
reasoning (+6.3 points on average), with improvements concentrated in
reasoning-related dimensions. These results provide evidence that co-designing queries and rubrics can make rubric-based RL a practical complement to RLVR beyond strictly verifiable tasks.
Read the full abstract ▾
All authors Rongzhi Zhang, Rui Feng, Zhihan Zhang, Jingfeng Yang, Qingyu Yin, Xin Liu, Zixuan Zhang, Priyanka Nigam, Bing Yin, Tuo Zhao, Chao Zhang
Computation and Language
AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task
Quentin Fuxa and Dominik Macháček
We describe AlignAtt4LLM, an IWSLT 2026 simultaneous
speech translation system for English to German, Italian, and Chinese. The system is a synchronous cascade: Qwen3-ASR with forced
alignment produces an incrementally updated source transcript, and Gemma-4 E4B-it translates that prefix under an MT-side AlignAtt policy. To our knowledge, this is the first application of AlignAtt to a decoder-only LLM, where the encoder-decoder cross-
attention used by earlier AlignAtt systems is absent. We recover a usable policy by proposing (1) an explicit source span in the
prompt, (2) offline selection of translation-specific
alignment heads, (3) selective qk-fast replay of the draft-to-source
attention block, and (4) runtime query/key capture that preserves model outputs bit-identically. On the IWSLT 2026 development set, AlignAtt4LLM outperforms the supplied baselines for the European target
languages, English to German and English to Italian, in both the low-latency regime around 2 seconds and the high-latency regime below 4 seconds CU-LongYAAL. Results for English to Chinese are more mixed, but the method is not tied to Gemma-4: because AlignAtt4LLM only requires a deterministic
prompt layout, calibrated
attention heads, and query/key capture, the same policy can be reapplied to stronger translation-focused decoder-only MT backbones for non-European target
languages.
Read the full abstract ▾
Computation and Language
Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
Yu Xia, Zhouhang Xie, Xin Xu et al. (+4 more)
Large
language models improve final-answer accuracy through extended chain-of-thought
reasoning, but often spend
tokens inefficiently and offer little inference-time control. Existing efficient
reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates
reasoning steering as a Markov decision process where a controller
agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the
reasoning trace and remaining thinking budget, then issues a steering action consisting of a
reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient
reasoning while preserving the reasoner's generation continuity. We initialize the controller
agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple
benchmarks show that ACTS achieves competitive accuracy with substantial
token savings, and enables controllable accuracy-
token trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.
Read the full abstract ▾
All authors Yu Xia, Zhouhang Xie, Xin Xu, Byungkyu Kang, Prarit Lamba, Xiang Gao, Julian McAuley
Computation and Language
Efficient ASR Training with Conversations that Never Happened
Máté Gedeon and Péter Mihajlik
Conversational ASR for lower-resource
languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data. We propose an augmentation pipeline that generates scenario-level dialogues with participant metadata, maps speaker attributes to TTS voice profiles, and assembles synthesized utterances into speaker-aware simulated conversations. We evaluated five LLM families under single-generator, fixed-budget mixture, and scale-up settings using the same FastConformer-Large training recipe for each one. We ran comprehensive evaluations on the Hungarian BEA-Dialogue
benchmark corpus, with the method itself being applicable to any
language given the resources for each component. The results show that synthetic conversations consistently improve
speech recognition performance, but generator choice and data composition strongly affect the gains. Our largest training configuration, using only 67 hours of real conversations and 636 hours of simulated data, achieves better performance on the evaluation
benchmark than a zero-shot model trained on 2700 hours of Hungarian
speech. These findings indicate that LLM-generated conversational data synthesized with TTS is a practical complement to real conversational corpora for
speech model training.
Read the full abstract ▾
Computation and Language
A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026
Aziz Sharipov Ortega and Dominik Macháček
We implement simultaneous translation capability with the offline direct
speech-to-text translation model Canary, using the state-of-the-art policy AlignAtt, and submit it to IWSLT 2026 Simultaneous
Speech Translation Shared task for Czech to English and English to German and Italian. The strengths of our system are: (1) high translation quality, outperforming similarly sized baselines both in low- and high-latency regimes in computationally unaware simulations; (2) low computational requirements, as the model has only 1B parameters; (3) multilinguality – support of 25 source and 25 target
languages.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Value-Aware Stochastic KV Cache Eviction for Reasoning Models
Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu et al. (+3 more)
Reasoning models improve accuracy through extended chains of thought, but their long outputs create a
memory and compute bottleneck. KV cache eviction methods reduce this cost by evicting unimportant key-value pairs from the cache, yet they often yield worse accuracy than selection-based sparse
attention alternatives, which keep the full KV cache. We identify key factors crucial to KV cache eviction accuracy. First, a small fraction of value states have abnormally large magnitudes, and evicting them causes catastrophic failure where models enter repetitive
reasoning loops. Second, introducing stochasticity during eviction improves accuracy by increasing cache diversity. Based on these findings, we propose Value-aware
Stochastic KV Cache Eviction (VaSE), a training-free recipe that protects large-magnitude value states and promotes diverse eviction decisions. Across six
reasoning tasks, Qwen3 models using VaSE with 4x KV cache compression yield higher average accuracies than SOTA selection method at the same sparsity, while outperforming the strongest eviction method by more than 4%. Overall, VaSE bridges the gap between efficiency and accuracy, supporting FlashAttention2 and enabling a static
memory footprint for
reasoning models.
Read the full abstract ▾
All authors Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu, Chenghao Yang, Jesse Thomason, Robin Jia
Computation and Language
Knowledge Editing for Masked Diffusion Language Models
Haewon Park and Yohan Jo
Knowledge editing aims to update or correct factual knowledge in a
language model. A widely used approach, locate-then-edit, first localizes a fact within the model and then edits the weights there. To date, such methods have been developed exclusively for autoregressive models (ARMs). Whether they work for masked
diffusion models (MDMs), which model text bidirectionally and generate by iterative denoising rather than next-
token prediction, remains an open question. We address it by transferring locate-then-edit to MDMs and comparing multiple MDMs with their matched ARMs. Our central finding has two parts. First, where an edit should be applied transfers between them: the same early-to-mid-layer MLP at the last subject
token is most effective for both. Second, this shared location does not guarantee a shared outcome. Single-
token edits succeed in both, but as targets grow longer, editing degrades far more sharply in the MDMs than in the ARMs. The failure stems from how the edited fact is generated: producing a multi-
token target passes through intermediate states in which the target is partially unmasked, for which the edit was never optimized. Guided by this diagnosis, we introduce a simple correction that optimizes the edit including such states, substantially restoring multi-
token performance. Our code is available at https://github.com/holi-lab/MDM-KE.
Read the full abstract ▾
Computation and Language
Synthesize and Reward – Reinforcement Learning for Multi-Step Tool Use in Live Environments
Ibrahim Abdelaziz, Asim Munawar, Kinjal Basu et al. (+4 more)
Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training queries are often detached from the server's actual state (so the generated tool calls fail to execute), and recall-based RL rewards incentivize verbose tool-calling patterns. We present PROVE (Programmatic Rewards On Verified Environments), a framework with three contributions: (1) a library of 20 stateful MCP (Model Context Protocol) servers exposing 343 tools, enabling live-execution RL training with session-scoped state isolation; (2) a state-machine data synthesis pipeline that generates multi-turn tool-call trajectories grounded in live-sampled server state, so generated queries reference entities that actually exist; and (3) a multi-component programmatic reward with an adaptive efficiency penalty that counters the verbosity incentive of recall-based rewards. We train four models (Qwen3-4B, Qwen3-8B, Qwen2.5-7B, Granite-4.1-8B) with GRPO on the resulting ~13K training examples. On BFCL Multi-Turn, tau2-bench, and T-Eval, PROVE yields improvements of up to +10.2, +6.8, and +6.5 points respectively, demonstrating that this framework yields consistent gains on multi-step tool orchestration across two model families.
Read the full abstract ▾
All authors Ibrahim Abdelaziz, Asim Munawar, Kinjal Basu, Maxwell Crouse, Chulaka Gunasekara, Suneet Katrekar, Pavan Kapanipathi
Computation and Language
RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions
Zongwei Lv, Yaoming Li, Zhewen Tan et al. (+7 more)
Agent benchmarks should reflect what users actually ask deployed
agents to do, yet existing
benchmarks often miss key realism properties of real developer-
agent sessions. We introduce RealClawBench, a live
benchmark framework built from real OpenClaw sessions to capture the distribution, diversity, and real-world difficulty of deployed
agent use. Real user requests are challenging to
benchmark because they often depend on local execution environments, involve implicit or underspecified intent, and require nontrivial verification. RealClawBench addresses these challenges with two core mechanisms: reconstructed execution environments and deterministic verifiable scorers, which together convert real sessions into reproducible, automatically scored tasks. The resulting release contains 281 executable tasks sampled from a much larger real-session pool while preserving the source distribution, with maximum final-vs-source Jensen-Shannon divergence of 0.0448. Evaluating 14 contemporary models shows that the best system solves only 65.8% of tasks, revealing substantial headroom on realistic developer-
agent workloads. By turning real deployed sessions into controlled evaluation instances, RealClawBench provides a practical path toward
benchmarks that better measure
agent capability in actual use. Code is available at:https://anonymous.4open.science/r/real-claw-bench-582B.
Read the full abstract ▾
All authors Zongwei Lv, Yaoming Li, Zhewen Tan, Yilun Yao, Yuxuan Tian, Lin Sun, Xiangzheng Zhang, Weihong Lin, Tong Yang, Guangxiang Zhao
Computation and Language · Cross-listed from cs.CV
Visual Instruction Tuning Aligns Modalities through Abstraction
Luis Palacios, Lorenzo Basile, Diego Doimo et al. (+1 more)
Visual instruction tuning effectively adapts a pre-trained Large
Language Model (LLM) to process image information alongside text. Yet, it remains unclear how visual features are embedded into the layer-wise hierarchy of abstractions of the LLM backbone. Across a diverse set of
vision-
language architectures, we show that instruction tuning primarily serves as a bridge,
embedding visual features directly into the intermediate semantic layers of the LLM, bypassing the early layers devoted to unimodal processing. With probing analyses and causal interventions, we show that these intermediate layers are the semantic core of
vision-
language processing and play a critical role in the performance on a broad set of
multimodal benchmarks. In addition, by comparing the geometry of semantically equivalent visual and textual representations, we find that fine-tuning extends and strengthens the existing abstraction phase, aligning visual features with pre-existing textual ones. Finally, we confirm the functional role of this localized
alignment by restricting fine-tuning to intermediate layers alone: this strategy preserves the performance of full fine-tuning on
vision-centric
benchmarks while reducing training time. Our results suggest that
multimodal integration is a localized phenomenon driven by the repurposing of the internal abstraction engine of the LLM.
Read the full abstract ▾
All authors Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga
Computation and Language
A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs
Cuong Vuong Tuan, Trang Mai Xuan, Tien-Cuong Nguyen et al. (+2 more)
Multi-Document Summarization (MDS) plays a critical role in distilling essential information from collections of textual data. Existing approaches often struggle to capture complex inter-document relationships, rely heavily on large amounts of labeled data for supervised training, or exhibit limited generalization across domains and
languages. To address these limitations, we present a training-free mixture-of-
agents framework for MDS that leverages the complementary strengths of large
language models (LLMs) and knowledge graphs. Our approach decomposes summarization into specialized
agent tasks: extractive selection, knowledge-aware abstraction, and iterative refinement, each operating without task-specific fine-tuning. We unify their outputs using a multi-perspective consistency mechanism guided by LLMs. Experiments across four
datasets in English and Vietnamese demonstrate state-of-the-art or competitive performance, validating the effectiveness and adaptability of our modular design.
Read the full abstract ▾
All authors Cuong Vuong Tuan, Trang Mai Xuan, Tien-Cuong Nguyen, Vu-Duc Ngo, Thien Van Luong
Computation and Language · Cross-listed from cs.IR
Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation
Yuecheng Li, Zeyu Song, Jing Yao et al. (+3 more)
Scaling recommender systems via large
language models (LLMs) has become a prominent trend in the industry. However, aligning the LLM's semantic space with the recommender's ID space via post-training (e.g., SFT and RL) remains challenging. Existing LLM4Rec paradigms are bottlenecked by two main issues: (1) the difficulty of measuring and improving chain-of-thought (CoT) quality in open-domain recommendation during SFT, and (2) the neglect of the trade-off between LLM semantic rewards and recommendation preference rewards during RL
alignment. Inspired by these challenges, we present Taiji, a novel LLM-as-Enhancer framework designed for industrial recommender systems. To overcome the SFT bottleneck, we utilize reverse-engineered
reasoning and open-ended rejection sampling to generate high-quality, domain-specific CoT data. To resolve the RL
alignment issue, we propose Pareto Optimal Policy Optimization (POPO), which adaptively adjusts cross-domain reward weights. Theoretically, it achieves an optimal trade-off between the semantic world knowledge of LLMs and the collaborative ID features representing online user preferences. Extensive offline evaluations and online A/B tests validate the effectiveness of Taiji. Deployed on Kuaishou's advertising platform since May 2026, Taiji currently serves over 400 million users daily, yielding significant commercial revenue and demonstrating its robust scalability in web-scale environments.
Read the full abstract ▾
All authors Yuecheng Li, Zeyu Song, Jing Yao, Chi Lu, Peng Jiang, Kun Gai
Computation and Language
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models
Qi Cao, Takeshi Kojima, Andrew Gambardella et al. (+3 more)
Large
language models (LLMs) demonstrate remarkable performance across diverse tasks, but they often generate responses that appear plausible while being factually incorrect. This problem is compounded by the lack of explicit uncertainty estimates, which makes it difficult for users to judge the reliability of model outputs. Existing uncertainty quantification methods typically rely on indirect signals, such as
entropy across sampled generations. These signals can be difficult to interpret and do not fully leverage the model's ability to assess its own uncertainty. We propose a simple yet effective self-assessment method for uncertainty quantification in LLMs. Our approach groups sampled generations into semantically distinct clusters, converts them into answer options in a structured multiple-choice question, and uses the probability assigned by the LLM to each option as a confidence estimate. Experiments across multiple models and
datasets show that our method consistently outperforms baseline approaches. Notably, it achieves competitive performance with as few as two additional samples, demonstrating both its effectiveness and efficiency.
Read the full abstract ▾
All authors Qi Cao, Takeshi Kojima, Andrew Gambardella, Helinyi Peng, Yutaka Matsuo, Yusuke Iwasawa
Computation and Language · Cross-listed from cs.LG
Dynamic Short Convolutions Improve Transformers
Oliver Sieberling, Bharat Runwal, Rameswar Panda et al. (+1 more)
Transformers have become the dominant architecture for large
language models, largely due to the scalability and flexibility of
attention, feed-forward layers, residual connections, and normalization. This paper introduces dynamic short convolutions as an additional neural network primitive for improving
Transformers. Unlike static short convolutions, dynamic convolutions use input-dependent filters, which preserves the locality bias of convolution while increasing expressivity. Motivating experiments show that applying dynamic short convolutions to key, query, and value representations improves performance on challenging associative recall tasks compared with static convolutional variants. Across
language-modeling experiments ranging from 150M to 2B parameters, dynamic convolutions consistently outperform standard
Transformers and
Transformers augmented with static short convolutions. Fitting scaling laws indicates a 1.33$\times$ compute advantage over compute-matched
Transformers when dynamic convolutions are applied to the key, query, and value vectors, and a 1.60$\times$ advantage when adding dynamic convolutions after every linear layer. Dynamic convolutions also offer improvements on linear RNNs (Mamba-2/Gated DeltaNet) and mixture-of-experts architectures. We make these gains practical with custom Triton kernels that enable efficient training with a manageable end-to-end slowdown. These results suggest that dynamic short convolutions are a scalable, hardware-efficient, and expressive primitive for advancing
Transformer-based
language models.
Read the full abstract ▾
All authors Oliver Sieberling, Bharat Runwal, Rameswar Panda, Yoon Kim
Computation and Language
Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning
Maggie Mi, Golzar Atefi, Atsuki Yamaguchi et al. (+3 more)
Idioms can be analysed in terms of their decomposability, the extent to which constituent meanings contribute to the figurative whole. Decomposability is thought to predict syntactic flexibility. Usage-based accounts instead attribute idiom behaviour to distributional experience, such as speaker familiarity and predictability. We examine these views using contextualised
language models as controlled distributional learners. We propose a model-internal measure of decomposability and relate it to human ratings, syntactic flexibility, and predictability while tracking idiom learning during pretraining. Model-derived decomposability correlates weakly with human judgments and shows a small but consistent negative relationship with syntactic flexibility. Pretraining analyses show that stabilisation of idiom representations in models is not explained by frequency alone. Instead, surprisal, decomposability, and frequency all contribute, with decomposability showing the strongest training-dependent effect.
Read the full abstract ▾
All authors Maggie Mi, Golzar Atefi, Atsuki Yamaguchi, Felix Gers, Aline Villavicencio, Nafise Sadat Moosavi
Computation and Language
Consistency Training Can Entrench Misalignment
David Demitri Africa and Arathi Mani
Consistency training encourages a model to produce similar outputs across related inputs or sampling procedures. Such methods are simple, scalable, and largely label-free, but their effects on model
alignment remain poorly understood. Could the self-bootstrapping nature of these methods amplify undesired behavior in models? We test seven consistency training methods on 108 model organisms: open-source models (7B–70B) fine-tuned to exhibit various forms of controlled misaligned behavior. We find that outcomes vary significantly: consistency training generally suppresses reward hacking and emergent misalignment but amplifies sycophancy. We present evidence that distribution shifts induced by the consistency labeling process, rather than variation in the selection operators, may be the primary driver of systematic
alignment effects. Finally, we present a unifying theoretical framework to derive conditions under which consistency training will amplify or suppress misalignment. In total, our study establishes that consistency training is not
alignment-neutral, and that its use in critical systems should be carefully audited.
Read the full abstract ▾
Computation and Language
Hashmat Shadab Malik, Muzammal Naseer and Salman Khan
Multimodal Large
Language Models integrate visual perception into
language reasoning, introducing a continuous attack surface susceptible to
adversarial attacks. Prior work on MLLM robustness has focused largely on English-centric tasks, leaving
multilingual behaviour unexplored. We address this gap through a systematic study of
adversarial robustness and
multimodal safety across 12 diverse
languages, evaluating open-source MLLMs that acquire
multilingual capability through instruction tuning. Gradient-based attacks reveal a transferable
multilingual vulnerability:
adversarial images optimized in one
language continue to induce failure in others, demonstrating strong cross-lingual transferability.
Multilingual safety further varies with how effectively a model retrieves or interprets harmful instructions. When harmful intent is issued through text,
languages with stronger linguistic grounding more often elicit misuse-enabling responses, while weaker
languages produce fewer unsafe outputs. When embedded in the image as typographic content, English scripts are reliably recognised and followed, whereas non-English scripts are rarely parsed by the
vision encoder. Lower-resource
languages may therefore appear safer, but this is an artefact of comprehension and visual-grounding failures rather than genuine
alignment, a phenomenon we term safety-by-failure. In contrast, MLLMs that build
multilingual capability throughout their training stages rather than only at instruction tuning, such as Qwen3-VL, exhibit genuine cross-lingual safety, maintaining active refusal across
languages rather than masking comprehension failure. Shallow
multilingual adaptation, such as fine-tuning on translated instruction data, may produce surface-level understanding that creates illusory safety in low-resource
languages; deeper integration across training stages leads to genuine
multilingual safety
alignment.
Read the full abstract ▾
Computation and Language
Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs
Lisa Bouger, Théo Lasnier, Philippe Loubet Moundi et al. (+2 more)
Backdoor attacks in Large
Language Models (LLMs) are a growing security concern, where models can generate adversary-chosen content. Existing defenses target
backdoors one at a time and typically require knowledge of the trigger, leaving the defender at a structural disadvantage when unknown
backdoors may exist in a model. We show that
backdoor neutralization through unlearning generalizes across
backdoors: training a model to ignore a single trigger can also suppress other
backdoors that were never explicitly targeted. We study this phenomenon across three model families, whose
backdoors were injected via pretraining or continual pretraining, by analyzing the models obtained after removing one
backdoor at a time. To understand why unlearning certain
backdoors induces the suppression of others, we introduce the Cross Activation Shift Distance, to quantify the distance between model changes induced by different trainings. Our results open a new direction for LLM safety as defenders could deliberately inject controlled
backdoors and then remove them, leveraging cross-
backdoor transfer to also suppress unknown
backdoors that an attacker may have previously introduced in the model.
Read the full abstract ▾
All authors Lisa Bouger, Théo Lasnier, Philippe Loubet Moundi, Yannick Teglia, Djamé Seddah
Computation and Language
Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?
Renhao Pei, Yihong Liu, Sampo Pyysalo et al. (+2 more)
Large
language models (LLMs) offer a promising approach to machine translation (MT) for extremely low-resource
languages by incorporating linguistic resources through in-context learning. However, LLMs often struggle to apply grammatical information effectively during translation. Inspired by recent progress in chain-of-thought
reasoning, we investigate whether low-resource MT can benefit from structured intermediate steps of linguistic analysis and grammatical
reasoning. We propose a pipeline for automatically generating step-by-step linguistic
reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks. We evaluate these traces in three settings: in-context learning (ICL), supervised fine-tuning (SFT), and reinforcement fine-tuning (RFT), on Xibe and Chintang as test cases. Our results show that linguistic
reasoning traces are most effective as inference-time guidance: in ICL, reliable linguistic
reasoning traces substantially improve translation performance across models,
languages, and metrics. In contrast, using these traces as training data yields smaller and less consistent gains, as models learn the format but often generate error-prone content. These findings suggest that LLMs can leverage grammatical information for low-resource MT when given reliable linguistic analyses, while learning to generate such analyses remains a major bottleneck.
Read the full abstract ▾
All authors Renhao Pei, Yihong Liu, Sampo Pyysalo, Hinrich Schütze, Shaoxiong Ji
Computation and Language
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Yuetian Lu, Ali Modarressi, Yihong Liu et al. (+1 more)
Activation patching can identify a mixture-of-experts (MoE) block whose clean output restores a corrupted factual prediction. However, because the block output combines contributions from multiple routed experts, block-level rescue does not establish whether the recovery localizes to an individual expert or depends on the routed expert set. We study this question on single-
token COUNTERFACT contrasts by corrupting subject-
token embeddings, restoring clean block outputs, and then restoring clean-minus-noised expert updates under fixed routing. In Qwen3-30B-A3B-Base, a discovery sweep selects layer 44, and held-out analysis identifies L44E069 as a recurrent routed contributor with positive specificity over same-layer active experts. Its effect is fact-matched and improves true-
token probability and rank, which explains part of the layer rescue. In Mixtral-8x7B-v0.1, the selected recurrent singleton is not specific; matched-size controls instead show specificity of the clean-routed top-2 set. These findings show that successful MoE-block restoration does not necessarily imply localization to a single expert.
Read the full abstract ▾
All authors Yuetian Lu, Ali Modarressi, Yihong Liu, Hinrich Schütze
Computation and Language
KletterMix: Climbing Toward High-Quality German Pretraining Data - The Full Report
Maurice Kraus, Ruben Härle, Sebastian Sztwiertnia et al. (+5 more)
High-quality pretraining data is a central ingredient in modern
language models, but German-
language resources remain far less developed than their English counterparts: they are often smaller, less carefully curated, weakly documented, and rarely validated through controlled training experiments. We introduce KletterMix, a high-quality German corpus for
language model pretraining and annealing, designed as a reusable
dataset artifact for the natural
language processing and modeling community. KletterMix is built by translating a state-of-the-art English pretraining corpus into German while preserving document boundaries, metadata, source structure, and topical diversity. This construction yields a German corpus with the scale and diversity of a modern pretraining
dataset, while enabling direct comparison to its English source. We document the
dataset through a broad set of corpus-level analyses, including translation quality, document length distributions, topic coverage, source composition, and geographic metadata. Using COMETKiwi, we show that the translated documents achieve strong quality across diverse domains, suggesting that careful translation can preserve much of the semantic and stylistic richness of the original corpus. Beyond
dataset construction, we evaluate KletterMix as training data. Through controlled pretraining and annealing ablations against established German corpora, we show that models trained on KletterMix achieve measurable improvements on German-
language downstream evaluations. These results demonstrate that carefully curated translated data can substantially strengthen the German pretraining data ecosystem.
Read the full abstract ▾
All authors Maurice Kraus, Ruben Härle, Sebastian Sztwiertnia, Abbas Goher Khan, Mehdi Ali, Michael Fromm, Nicolas Flores-Herr, Kristian Kersting
Computation and Language
HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
Xin Liu, Runsong Zhao, Xinyu Liu et al. (+8 more)
Extended chain-of-thought (CoT) traces improve LLM
reasoning but incur substantial computational and
memory costs. While existing CoT compression methods mitigate this by condensing thought steps into compact representations via
memory tokens and retaining only these representations at inference time, the loss of fine-grained information makes subsequent steps more error-prone. To alleviate this, we propose HybridThinker, where in addition to preserved these representations, thought steps are also temporarily retained to provide fine-grained details. However, we observe that naively keeping thought steps accessible to subsequent steps during training lets the model bypass
memory tokens by retrieving information directly from these steps, leaving the model's ability to compress and retrieve information through
memory tokens insufficiently trained. We therefore introduce a hybrid training scheme, in which only some thought steps are directly accessible through
attention to subsequent steps, while the other thought steps are masked, forcing the model to use
memory tokens for compression and
retrieval. Across 4
reasoning benchmarks, HybridThinker matches the uncompressed baseline, advancing the state of the art in CoT compression by 5.8 points on average accuracy with similar inference time. Ablation studies confirm that both temporary thought-step retention and the hybrid training scheme contribute to these gains.
Read the full abstract ▾
All authors Xin Liu, Runsong Zhao, Xinyu Liu, Junhao Ruan, Pengcheng Huang, Shichao Dong, Chunyang Xiao, Chenglong Wang, Changliang Li, Jingbo Zhu, Tong Xiao
Computation and Language
Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation
David Alonso del Barrio, Jing Wen and Daniel Gatica-Perez
Frame analysis of migration news is a socially consequential task: media scholars and researchers who study how migration is narrated need tools that are not only accurate, but transparent, auditable, and accessible within the resource constraints typical of academic research groups. Existing LLM-based approaches rely on proprietary APIs and large models that raise concerns about data
privacy, reproducibility and equitable access among media researchers. This work studies how a locally deployable open-source LLM can support interpretable frame analysis as an assistive tool. We introduce a Structured Chain-of-Thought (SCoT) prompting approach using Llama3-8B, enabling step-by-step justifications grounded in predefined framing categories. This structured design allows users to audit model outputs and examine alternative interpretations in a task that is inherently subjective. We evaluate our approach on a
dataset of migration-related news and show that SCoT improves classification performance over zero-shot and few-shot baselines while remaining feasible on a single GPU. Then, we conduct a human-centered evaluation in which annotators assess the coherence and influence of "the model's
reasoning". Results indicate that SCoT explanations are generally perceived as logical (mean score 4.1/5, though with notable variation across texts) and can
prompt reflection on initial interpretations, even when disagreement persists. Our findings highlight both the potential and risks of LLM-assisted frame analysis. While structured
reasoning can increase the traceability of model outputs and support critical interpretation, it can also influence human judgment in subtle ways. By enabling local deployment and emphasizing human-in-the-loop interaction, this work contributes to discussions on responsible and accessible computational tools for the study of socially impactful media narratives.
Read the full abstract ▾