Computation and Language
Chronos: Learning Temporal Dynamics of Reasoning Chains for Test-Time Scaling
Kai Zhang, Jiayi Liao, Chengpeng Li et al. (+3 more)
Test-Time Scaling (TTS) has emerged as an effective paradigm for improving the
reasoning performance of large
language models (LLMs). However, existing methods – most notably majority voting and heuristic
token-level scoring – treat
reasoning traces or
tokens equally, thereby being susceptible to substantial variations in trajectory quality and localized logical failures. In this work, we introduce Chronos, a lightweight and plug-and-play chronological
reasoning scorer that models each trajectory as a time series. Specifically, Chronos learns to capture trajectory features of
token probabilities, assigns quality scores accordingly, and employs a weighted voting mechanism. Extensive evaluations on both in-domain and out-of-domain
benchmarks demonstrate that Chronos consistently delivers substantial gains across a variety of models, with negligible computational overhead. Notably, Chronos@128 achieves relative improvements of 34.21% over Pass@1 and 22.70% over Maj@128 on HMMT25 using Qwen3-4B-Thinking-2507, highlighting its effectiveness.
Read the full abstract ▾
All authors Kai Zhang, Jiayi Liao, Chengpeng Li, Ziyuan Xie, Sihang Li, Xiang Wang
Computation and Language
ASTER: Agentic Scaling with Tool-integrated Extended Reasoning
Xuqin Zhang, Quan He, Zhenrui Zheng et al. (+3 more)
Reinforcement learning (RL) has emerged as a dominant paradigm for eliciting long-horizon
reasoning in Large
Language Models (LLMs). However, scaling Tool-Integrated
Reasoning (TIR) via RL remains challenging due to interaction collapse: a pathological state where models fail to sustain multi-turn tool usage, instead degenerating into heavy internal
reasoning with only trivial, post-hoc code verification. We systematically study three questions: (i) how cold-start SFT induces an agentic, tool-using behavioral prior, (ii) how the interaction density of cold-start trajectories shapes exploration and downstream RL outcomes, and (iii) how the RL interaction budget affects learning dynamics and generalization under varying inference-time budgets. We then introduce ASTER (Agentic Scaling with Tool-integrated Extended
Reasoning), a framework that circumvents this collapse through a targeted cold-start strategy prioritizing interaction-dense trajectories. We find that a small expert cold-start set of just 4K interaction-dense trajectories yields the strongest downstream performance, establishing a robust prior that enables superior exploration during extended RL training. Extensive evaluations demonstrate that ASTER-4B achieves state-of-the-art results on competitive mathematical
benchmarks, reaching 90.0% on AIME 2025, surpassing leading frontier open-source models, including DeepSeek-V3.2-Exp.
Read the full abstract ▾
All authors Xuqin Zhang, Quan He, Zhenrui Zheng, Zongzhang Zhang, Xu He, Dong Li
Computation and Language
Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
Zizhuo Fu, Wenxuan Zeng, Runsheng Wang et al. (+1 more)
Large
Language Models (LLMs) often assign disproportionate
attention to the first
token, a phenomenon known as the
attention sink. Several recent approaches aim to address this issue, including Sink
Attention in GPT-OSS and Gated
Attention in Qwen3-Next. However, a comprehensive analysis of the relationship among these
attention mechanisms is lacking. In this work, we provide both theoretical and empirical evidence demonstrating that the sink in Vanilla
Attention and Sink
Attention naturally construct a Mixture-of-Experts (MoE) mechanism within
attention layers. This insight explains the head collapse phenomenon observed in prior work, where only a fixed subset of
attention heads contributes to generation. To mitigate head collapse, we propose a sink-aware training
algorithm with an auxiliary load balancing loss designed for
attention layers. Extensive experiments show that our method achieves effective head load balancing and improves model performance across Vanilla
Attention, Sink
Attention, and Gated
Attention. We hope this study offers a new perspective on
attention mechanisms and encourages further exploration of the inherent MoE structure within
attention layers.
Read the full abstract ▾
All authors Zizhuo Fu, Wenxuan Zeng, Runsheng Wang, Meng Li
Computation and Language
Bridging Lexical Ambiguity and Vision: A Mini Review on Visual Word Sense Disambiguation
Shashini Nilukshi and Deshan Sumanathilaka
This paper offers a mini review of Visual Word Sense Disambiguation (VWSD), which is a
multimodal extension of traditional Word Sense Disambiguation (WSD). VWSD helps tackle lexical ambiguity in
vision-
language tasks. While conventional WSD depends only on text and lexical resources, VWSD uses visual cues to find the right meaning of ambiguous words with minimal text input. The review looks at developments from early
multimodal fusion methods to new frameworks that use contrastive models like CLIP,
diffusion-based text-to-image generation, and large
language model (LLM) support. Studies from 2016 to 2025 are examined to show the growth of VWSD through feature-based, graph-based, and contrastive
embedding techniques. It focuses on
prompt engineering, fine-tuning, and adapting to multiple
languages. Quantitative results show that CLIP-based fine-tuned models and LLM-enhanced VWSD systems consistently perform better than zero-shot baselines, achieving gains of up to 6-8% in Mean Reciprocal Rank (MRR). However, challenges still exist, such as limitations in context, model bias toward common meanings, a lack of
multilingual datasets, and the need for better evaluation frameworks. The analysis highlights the growing overlap of CLIP
alignment,
diffusion generation, and LLM
reasoning as the future path for strong, context-aware, and
multilingual disambiguation systems.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
ASP-Bench: From Natural Language to Logic Programs
Stefan Szeider
Automating the translation of natural-
language specifications into logic programs is a challenging task that affects neurosymbolic engineering. We present ASP-Bench, a
benchmark comprising 128 natural
language problem instances, 64 base problems with easy and hard variants. It evaluates systems that translate natural-
language problems into Answer Set Programs (ASPs), a prominent form of logic programming. It provides systematic coverage of ASP features, including choice rules, aggregates, and optimization. Each problem includes reference validators that check whether solutions satisfy the problem specification. We characterize problems along seven largely independent
reasoning aspects (optimization, temporal
reasoning, default logic, resource allocation, recursion, spatial
reasoning, and quantitative complexity), providing a multidimensional view of modeling difficulty. We test the
benchmark using an agentic approach based on the ReAct (Reason and Act) framework, which achieves full saturation, demonstrating that feedback-driven iterative refinement with solver feedback provides a reliable and robust approach for modeling natural
language in ASP. Our analysis across multiple
agent runs enables us to gain insights into what determines a problem's modeling hardness.
Read the full abstract ▾
Computation and Language
EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech
Besher Hassan, Ibrahim Alsarraj, Musaab Hasan et al. (+3 more)
This work presents EmoAra, an end-to-end
emotion-preserving pipeline for cross-lingual spoken communication, motivated by banking customer service where emotional context affects service quality. EmoAra integrates
Speech Emotion Recognition, Automatic
Speech Recognition, Machine Translation, and Text-to-
Speech to process English
speech and deliver an Arabic spoken output while retaining emotional nuance. The system uses a CNN-based
emotion classifier, Whisper for English transcription, a fine-tuned MarianMT model for English-to-Arabic translation, and MMS-TTS-Ara for Arabic
speech synthesis. Experiments report an F1-score of 94% for
emotion classification, translation performance of BLEU 56 and BERTScore F1 88.7%, and an average human evaluation score of 81% on banking-domain translations. The implementation and resources are available at the accompanying GitHub repository.
Read the full abstract ▾
All authors Besher Hassan, Ibrahim Alsarraj, Musaab Hasan, Yousef Melhim, Shahem Fadi, Shahem Sultan
Computation and Language
PedagoSense: A Pedology Grounded LLM System for Pedagogical Strategy Detection and Contextual Response Generation in Learning Dialogues
Shahem Sultan, Shahem Fadi, Yousef Melhim et al. (+2 more)
This paper addresses the challenge of improving interaction quality in dialogue based learning by detecting and recommending effective pedagogical strategies in tutor student conversations. We introduce PedagoSense, a pedology grounded system that combines a two stage strategy classifier with large
language model generation. The system first detects whether a pedagogical strategy is present using a binary classifier, then performs fine grained classification to identify the specific strategy. In parallel, it recommends an appropriate strategy from the dialogue context and uses an LLM to generate a response aligned with that strategy. We evaluate on human annotated tutor student dialogues, augmented with additional non pedagogical conversations for the binary task. Results show high performance for pedagogical strategy detection and consistent gains when using data augmentation, while analysis highlights where fine grained classes remain challenging. Overall, PedagoSense bridges pedagogical theory and practical LLM based response generation for more adaptive educational technologies.
Read the full abstract ▾
All authors Shahem Sultan, Shahem Fadi, Yousef Melhim, Ibrahim Alsarraj, Besher Hassan
Computation and Language
Typologically-Informed Candidate Reranking for LLM-based Translation into Low-Resource Languages
Nipuna Abeykoon, Ashen Weerathunga, Pubudu Wijesinghe et al. (+1 more)
Large
language models trained predominantly on high-resource
languages exhibit systematic biases toward dominant typological patterns, leading to structural non-conformance when translating into typologically divergent low-resource
languages. We present a framework that leverages linguistic typology to improve translation quality without parallel training data or model retraining. The framework consists of two components: the Universal Metalinguistic Framework (UMF), which represents
languages as structured profiles across 16 typological dimensions with divergence-weighted scoring, and the Computational Engine, which operates through linguistic disambiguation during generation and typological compliance scoring during selection. Evaluation across nine
language pairs demonstrates intervention rates strongly correlating with typological distance from English. In experiments on 341 English sentences each having different morphological and syntactic phenomena, the framework shows an intervention precision of 48.16% for conservatively treated
languages, 28.15% for morphologically dense
languages, and 86.26% for structurally profiled
languages. The framework requires no parallel training data and operates with any LLM capable of producing multiple candidate outputs, enabling practical deployment for under-resourced
languages.
Read the full abstract ▾
All authors Nipuna Abeykoon, Ashen Weerathunga, Pubudu Wijesinghe, Parameswari Krishnamurthy
Computation and Language
The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models
Reem I. Masoud, Chen Feng, Shunta Asano et al. (+3 more)
The global deployment of large
language models (LLMs) has raised concerns about cultural misalignment, yet the linguistic properties of fine-tuning
datasets used for cultural adaptation remain poorly understood. We adopt a
dataset-centric view of cultural
alignment and investigate which properties of post-training data are associated with cultural performance, whether they can guide data selection before fine-tuning, and how their effects vary across
languages and model families. We compute lightweight linguistic, semantic, and structural metrics for Arabic, Chinese, and Japanese
datasets and apply principal component analysis (PCA) separately within each
language. The resulting components form broadly interpretable axes: PC1 is generally dominated by semantic structure, PC2 captures diversity and lexical variation, and PC3 reflects more
language-specific organization. We fine-tune LLaMA, Mistral, and DeepSeek models and evaluate them on
benchmarks of cultural knowledge, values, and norms. Although the PCA-derived
dataset descriptors are associated with downstream performance, the strongest relationships vary across models,
benchmarks, and
languages, indicating that no single component serves as a universal predictor. Controlled, size-matched subset interventions further show that PCA-guided selection can improve cultural performance when the relevant component and direction are validated against random sampling. PC3 provides the strongest signal for Arabic, while High-PC1 is most effective for Japanese, particularly for LLaMA. Chinese results are weaker and more model-specific and remain exploratory because of smaller subset sizes. Overall, our findings show that lightweight
dataset descriptors can support pre-training data diagnostics, but effective cultural adaptation requires
language- and architecture-aware selection rather than a universal linguistic criterion.
Read the full abstract ▾
All authors Reem I. Masoud, Chen Feng, Shunta Asano, Saied Alshahrani, Philip Colin Treleaven, Miguel R. D. Rodrigues
Computation and Language
Don't Judge a Book by its Cover: Testing LLMs' Robustness Under Logical Obfuscation
Abhilekh Borah, Shubhra Ghosh, Kedar Joshi et al. (+2 more)
Tasks such as solving arithmetic equations, evaluating truth tables, and completing syllogisms are handled well by large
language models (LLMs) in their standard form, but they often fail when the same problems are posed in logically equivalent yet obfuscated formats. To study this vulnerability, we introduce Logifus, a structure-preserving logical obfuscation framework, and, utilizing this, we present LogiQAte, a first-of-its-kind diagnostic
benchmark with 1,108 questions across four
reasoning tasks: (i) Obfus FOL (first-order logic entailment under equivalence-preserving rewrites), (ii) Obfus Blood Relation (family-graph entailment under indirect relational chains), (iii) Obfus Number Series (pattern induction under symbolic substitutions), and (iv) Obfus Direction Sense (
navigation reasoning under altered directions and reference frames). Across all the tasks, evaluating six state-of-the-art models, we find that obfuscation severely degrades zero-shot performance, with performance dropping on average by 47% for GPT-4o, 27% for GPT-5, and 22% for
reasoning model, o4-mini. Our findings reveal that current LLMs parse questions without deep understanding, highlighting the urgency of building models that genuinely comprehend and preserve meaning beyond surface form.
Read the full abstract ▾
All authors Abhilekh Borah, Shubhra Ghosh, Kedar Joshi, Aditya Kumar Guru, Kripabandhu Ghosh
Computation and Language
Long-range Modeling and Processing of Multimodal Event Sequences
Jichu Li, Yilun Zhong, Zhiting Li et al. (+2 more)
Temporal point processes (TPPs) have emerged as powerful tools for modeling asynchronous event sequences. While recent advances have extended TPPs to handle textual information, existing approaches are limited in their ability to generate rich,
multimodal content and reason about event dynamics. A key challenge is that incorporating
multimodal data dramatically increases sequence length, hindering the ability of
attention-based models to generate coherent, long-form textual descriptions that require long-range understanding. In this paper, we propose a novel framework that extends LLM-based TPPs to the visual modality, positioning text generation as a core capability alongside time and type prediction. Our approach addresses the long-context problem through an adaptive sequence compression mechanism based on temporal similarity, which reduces sequence length while preserving essential patterns. We employ a two-stage paradigm of pre-training on compressed sequences followed by supervised fine-tuning for downstream tasks. Extensive experiments, including on the challenging DanmakuTPP-QA
benchmark, demonstrate that our method outperforms state-of-the-art baselines in both predictive accuracy and the quality of its generated textual analyses.
Read the full abstract ▾
All authors Jichu Li, Yilun Zhong, Zhiting Li, Feng Zhou, Quyu Kong
Computation and Language · Cross-listed from cs.LG
MarkovScale: Towards Optimal Sequential Scaling at Inference Time
Youkang Wang, Jian Wang, Rubing Chen et al. (+3 more)
Sequential scaling is a prominent inference-time scaling paradigm, yet its performance improvements are typically modest and not well understood, largely due to the prevalence of heuristic, non-principled approaches that obscure clear optimality bounds. To address this, we propose a principled framework that models sequential scaling as a two-state Markov process. This approach reveals the underlying properties of sequential scaling and yields closed-form solutions for essential aspects, such as the specific conditions under which accuracy is improved and the theoretical upper, neutral, and lower performance bounds. Leveraging this formulation, we develop MarkovScale, a practical system that applies these optimality criteria to achieve a theoretically grounded balance between accuracy and efficiency. Comprehensive experiments across 3 backbone LLMs, 5
benchmarks, and over 20 configurations show that MarkovScale consistently outperforms state-of-the-art parallel and sequential scaling methods, representing a significant step toward optimal and resource-efficient inference in LLMs. The source code will be open upon acceptance at https://open-upon-acceptance.
Read the full abstract ▾
All authors Youkang Wang, Jian Wang, Rubing Chen, Tianyi Zeng, Xiao-Yong Wei, Qing Li
Computation and Language
Tendem: A Hybrid AI+Human Platform
Konstantin Chernyshev, Ekaterina Artemova, Viacheslav Zhukov et al. (+8 more)
Tendem is a hybrid system where AI handles structured, repeatable work and Human Experts step in when the models fail or to verify results. Each result undergoes a comprehensive quality review before delivery to the Client. To assess Tendem's performance, we conducted a series of in-house evaluations on 94 real-world tasks, comparing it with AI-only
agents and human-only workflows carried out by Upwork freelancers. The results show that Tendem consistently delivers higher-quality outputs with faster turnaround times. At the same time, its operational costs remain comparable to human-only execution. On third-party agentic
benchmarks, Tendem's AI
Agent (operating autonomously, without human involvement) performs near state-of-the-art on web browsing and tool-use tasks while demonstrating strong results in frontier domain knowledge and
reasoning.
Read the full abstract ▾
All authors Konstantin Chernyshev, Ekaterina Artemova, Viacheslav Zhukov, Maksim Nerush, Mariia Fedorova, Iryna Repik, Olga Shapovalova, Aleksey Sukhorosov, Vladimir Dobrovolskii, Natalia Mikhailova, Sergei Tilga
Computation and Language
Logic-Oriented Retriever Enhancement via Contrastive Learning
Wenxuan Zhang, Yuan-Hao Jiang, Changyong Qi et al. (+2 more)
Large
language models (LLMs) struggle in knowledge-intensive tasks, as retrievers often overfit to surface similarity and fail on queries involving complex logical relations. The capacity for logical analysis is inherent in model representations but remains underutilized in standard training. LORE (Logic ORiented Retriever Enhancement) introduces fine-grained contrastive learning to activate this latent capacity, guiding
embeddings toward evidence aligned with logical structure rather than shallow similarity. LORE requires no external upervision, resources, or pre-
retrieval analysis, remains index-compatible, and consistently improves
retrieval utility and downstream generation while maintaining efficiency. The
datasets and code are publicly available at https://github.com/mazehart/Lore-RAG.
Read the full abstract ▾
All authors Wenxuan Zhang, Yuan-Hao Jiang, Changyong Qi, Rui Jia, Yonghe Wu
Computation and Language
What If We Allocate Test-Time Compute Adaptively?
Ahsan Bilal, Ahmed Mohsin, Muhammad Umer et al. (+4 more)
Test-time compute scaling allocates inference computation uniformly, uses fixed sampling strategies, and applies verification only for reranking. In contrast, we propose a verifier-guided adaptive framework treating
reasoning as iterative trajectory generation and selection. For each problem, the
agent runs multiple inference iterations. In each iteration, it optionally produces a high-level plan, selects a set of
reasoning tools and a compute strategy together with an exploration parameter, and then generates a candidate
reasoning trajectory. A process reward model (PRM) serves as a unified control signal: within each iteration, step-level PRM scores are aggregated to guide
pruning and expansion during generation, and across iterations, aggregated trajectory rewards are used to select the final response. Across
datasets, our dynamic, PRM-guided approach consistently outperforms direct test-time scaling, yielding large gains on MATH-500 and several-fold improvements on harder
benchmarks such as AIME24 and AMO-Bench. We characterize efficiency using theoretical FLOPs and a compute intensity metric penalizing wasted generation and tool overhead, demonstrating that verification-guided allocation concentrates computation on high-utility
reasoning paths.
Read the full abstract ▾
All authors Ahsan Bilal, Ahmed Mohsin, Muhammad Umer, Ali Subhan, Hassan Rizwan, Ayesha Mohsin, Dean Hougen
Computation and Language
From Utterance to Vividity: Training Expressive Subtitle Translation LLM via Adaptive Local Preference Optimization
Chaoqun Cui, Shijing Wang, Liangbin Huang et al. (+4 more)
The rapid development of Large
Language Models (LLMs) has significantly enhanced the general capabilities of machine translation. However, as application scenarios become more complex, the limitations of LLMs in vertical domain translations are gradually becoming apparent. In this study, we focus on how to construct translation LLMs that meet the needs of domain customization. We take visual media subtitle translation as our topic and explore how to train expressive and vivid translation LLMs. We investigated the situations of subtitle translation and other domains of literal and liberal translation, verifying the reliability of LLM as reward model and evaluator for translation. Additionally, to train an expressive translation LLM, we constructed and released a multidirectional subtitle parallel corpus
dataset and proposed the Adaptive Local Preference Optimization (ALPO) method to address fine-grained preference
alignment. Experimental results demonstrate that ALPO achieves outstanding performance in multidimensional evaluation of translation quality.
Read the full abstract ▾
All authors Chaoqun Cui, Shijing Wang, Liangbin Huang, Qingqing Gu, Zhaolong Huang, Xiao Zeng, Wenji Mao
Computation and Language
Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Ruihan Jin, Pengpeng Shao, Zhengqi Wen et al. (+5 more)
Knowledge
distillation has emerged as a pivotal technique for transferring knowledge from stronger large
language models (LLMs) to smaller, more efficient models. However, traditional
distillation approaches face challenges related to knowledge conflicts and high resource demands, particularly when leveraging multiple teacher models. In this paper, we introduce the concept of Knowledge Purification, which consolidates the rationales from multiple teacher LLMs into a single rationale, thereby mitigating conflicts and enhancing efficiency. To investigate the effectiveness of knowledge purification, we further propose five purification methods from various perspectives. Our experiments demonstrate that these methods not only improve the performance of the distilled model but also effectively alleviate knowledge conflicts. Moreover, router-based methods exhibit robust generalization capabilities, underscoring the potential of innovative purification techniques in optimizing multi-teacher
distillation and facilitating the practical deployment of powerful yet lightweight models.
Read the full abstract ▾
All authors Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Mingkuan Feng, Shuo Yang, Chu Yuan Zhang, Jianhua Tao
Computation and Language
Personality Expression Across Contexts: Linguistic and Behavioral Variation in LLM Agents
Bin Han, Deuksin Kwon and Jonathan Gratch
Large
Language Models (LLMs) can be conditioned with explicit personality
prompts, yet their behavioral realization often varies depending on context. This study examines how identical personality
prompts lead to distinct linguistic, behavioral, and emotional outcomes across four conversational settings: ice-breaking, negotiation, group decision, and empathy tasks. Results show that contextual cues systematically influence both personality expression and emotional tone, suggesting that the same traits are expressed differently depending on social and affective demands. This raises an important question for LLM-based dialogue
agents: whether such variations reflect inconsistency or context-sensitive adaptation akin to human behavior. Viewed through the lens of Whole Trait Theory, these findings highlight that LLMs exhibit context-sensitive rather than fixed personality expression, adapting flexibly to social interaction goals and affective conditions.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learning
Dylan Zhang, Yufeng Xu, Haojin Wang et al. (+2 more)
Post-training of
reasoning LLMs is a holistic process that typically consists of an offline SFT stage followed by an online reinforcement learning (RL) stage. However, SFT is often optimized in isolation to maximize SFT performance alone. We show that, after identical RL training, models initialized from stronger SFT checkpoints can significantly underperform those initialized from weaker ones. We attribute this to a mismatch typical in current SFT-RL pipelines: the distribution that generates the offline SFT data can differ substantially from the policy optimized during online RL, which learns from its own rollouts. We propose PEAR (Policy Evaluation-inspired
Algorithm for Offline Learning Loss Re-weighting), an SFT-stage method that corrects this mismatch and better prepares the model for RL. PEAR uses importance sampling to reweight the SFT loss, with three variants operating at the
token, block, and sequence levels. It can be used to augment standard SFT objectives and incurs little additional training overhead once probabilities for the offline data are collected. We conduct controlled experiments on verifiable
reasoning games and mathematical
reasoning tasks on Qwen 2.5 and 3 and DeepSeek-distilled models. PEAR consistently improves post-RL performance over canonical SFT, with pass at 8 gains up to a 14.6 percent on AIME2025. Our results suggest that PEAR is an effective step toward more holistic LLM post-training by designing and evaluating SFT with downstream RL in mind rather than in isolation.
Read the full abstract ▾
All authors Dylan Zhang, Yufeng Xu, Haojin Wang, Qingzhi Chen, Hao Peng
Computation and Language · Cross-listed from cs.AI
Stepwise Intrinsic Rewards for Reasoning in Large Language Models
Xiangwei Wang, Wei Wang, Ken Chen et al. (+3 more)
Reinforcement learning (RL) has become a widely used paradigm for improving the
reasoning abilities of large
language models (LLMs) and
Vision-
language models (VLMs). Sparse binary outcome rewards, however, score only final correctness and cannot identify which intermediate steps contributed to it; in
multimodal tasks, they may also reward answers driven by linguistic priors rather than visual evidence. Process reward models (PRMs) densify supervision but usually require process annotations, auxiliary models, or inference-time search. In this paper, we introduce Stepwise Marginal Information Gain (MIG), an intrinsic process reward computed from the policy itself. MIG measures how each structured
reasoning prefix changes the length-normalized, teacher-forced log-likelihood of the reference answer. A monotonic historical watermark rewards only new likelihood maxima, avoiding duplicate credit after sub-record detours. We combine this signal with outcome and format rewards and a gated self-
distillation objective that retains only structurally valid and correct trajectories. For VLMs, a real-versus-blank likelihood gate down-weights rewards when answers remain predictable without the image. Across eight task-specific
benchmarks, the full method exceeds outcome-only GRPO in every single-run comparison. In broad-data transfer, it improves average accuracy by up to 4.8 points over binary-reward training and gains 12.6 points on MathVerse. At 7B, it exceeds an external PRM-BoN@16 baseline by 12.9 points on
vision-
language transfer without inference-time reranking. These results support policy-derived stepwise credit as an annotation-free alternative to explicit process reward modeling.
Read the full abstract ▾
All authors Xiangwei Wang, Wei Wang, Ken Chen, Nanduni Nimalsiri, Sachith Seneviratne, Saman Halgamuge
Computation and Language · Cross-listed from cs.AI
Dongyang Fan, Sebastien Delsad, Nicolas Flammarion et al. (+1 more)
Large
language models (LLMs) still produce plausible-sounding but ungrounded factual claims, a problem that worsens in multi-turn dialogue as context grows and early errors cascade. We introduce $\textbf{HalluHard}$, a challenging multi-turn
hallucination benchmark with 950 seed questions spanning four high-stakes domains: legal cases, research questions, medical guidelines, and coding. We operationalize groundedness by requiring inline citations for factual assertions. To support reliable evaluation in open-ended settings, we propose a judging pipeline that iteratively retrieves evidence via web search. It can fetch, filter, and parse full-text sources (including PDFs) to assess whether cited material actually supports the generated content. Across a diverse set of frontier proprietary and open-weight models,
hallucinations remain substantial even with web search ($\approx 30\%$ for the strongest configuration, Opus-4.5 with web search), with content-grounding errors persisting at high rates. Finally, we show that
hallucination behavior is shaped by model capacity, turn position, effective
reasoning, and the type of knowledge required.
Read the full abstract ▾
All authors Dongyang Fan, Sebastien Delsad, Nicolas Flammarion, Maksym Andriushchenko
Computation and Language
Bias in the Ear of the Listener: Assessing Sensitivity in Audio Language Models Across Linguistic, Demographic, and Positional Variations
Sheng-Lun Wei, Yu-Ling Liao, Yen-Hua Chang et al. (+2 more)
This work presents the first systematic investigation of
speech bias in
multilingual MLLMs. We construct and release the BiasInEar
dataset, a
speech-augmented
benchmark based on Global MMLU Lite, spanning English, Chinese, and Korean, balanced by gender and accent, and totaling 70.8 hours ($\approx$4,249 minutes) of
speech with 11,200 questions. Using four complementary metrics (accuracy,
entropy, APES, and Fleiss' $\kappa$), we evaluate nine representative models under linguistic (
language and accent), demographic (gender), and structural (option order) perturbations. Our findings reveal that MLLMs are relatively robust to demographic factors but highly sensitive to
language and option order, suggesting that
speech can amplify existing structural biases. Moreover, architectural design and
reasoning strategy substantially affect robustness across
languages. Overall, this study establishes a unified framework for assessing
fairness and robustness in
speech-integrated LLMs, bridging the gap between text- and
speech-based evaluation. The resources can be found at https://github.com/ntunlplab/BiasInEar.
Read the full abstract ▾
All authors Sheng-Lun Wei, Yu-Ling Liao, Yen-Hua Chang, Hen-Hsen Huang, Hsin-Hsi Chen
Computation and Language
Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident
Conrad Borchers, Jill-Jênn Vie and Roger Azevedo
Large
language models (LLMs) are increasingly embedded in AI-based tutoring systems. Can they faithfully model novice
reasoning and metacognitive judgments? Existing evaluations emphasize problem-solving accuracy, overlooking the fragmented and imperfect
reasoning that characterizes human learning. We evaluate LLMs as novices using 630 think-aloud utterances from multi-step chemistry tutoring problems with problem-solving logs of student hint use, attempts, and problem context. We compare LLM-generated
reasoning to human learner utterances under minimal and extended contextual prompting, and assess the models' ability to predict step-level learner success. Although GPT-4.1 generates fluent and contextually appropriate continuations, its
reasoning is systematically over-coherent, verbose, and less variable than human think-alouds. These effects intensify with a richer problem-solving context during prompting. Learner performance was consistently overestimated. These findings highlight epistemic limitations of simulating learning with LLMs. We attribute these limitations to LLM training data, including expert-like solutions devoid of expressions of affect and working
memory constraints during problem solving. Our evaluation framework can guide future design of adaptive systems that more faithfully support novice learning and self-regulation using generative artificial intelligence.
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages
Yang Xiao, Eun-Jung Holden and Ting Dang
Recent
speech foundation models excel at
multilingual automatic
speech recognition (ASR) for high-resource
languages, but adapting them to low-resource
languages remains challenging due to data scarcity and efficiency constraints. Full-model fine-tuning is computationally expensive and prone to overfitting, while parameter-efficient methods like LoRA apply adaptation uniformly across layers, overlooking internal representations thus compromising effectiveness and efficiency. We analyze
multilingual ASR models and reveal a U-shaped adaptability pattern: early and late layers are
language-specific and require more adaptation, while intermediate layers retain shared semantics and need less. Building on this observation, we propose DAMA, a Depth-Aware Model Adaptation framework that allocates adaptation capacity according to each layer's role. DAMA also introduces Singular Value Decomposition (SVD)-based initialization to constrain adaptation and preserve the U-shaped pattern, as well as a frozen middle-layer basis for further efficiency. Evaluated on 18 low-resource
languages across two
benchmark datasets, DAMA matches or surpasses state-of-the-art accuracy with 80% fewer trainable parameters, achieves a 29% error reduction under extreme data scarcity, and significantly improves
memory, training time, and computational efficiency over baselines. These results highlight the benefits of structure-aware adaptation for efficient, scalable
multilingual ASR.
Read the full abstract ▾
Computation and Language
Distilling Token-Trained Models into Byte-Level Models
Zishuo Bao, Jiaqi Leng, Junxiong Wang et al. (+2 more)
Byte
Language Models (BLMs) have emerged as a promising direction for scaling
language models beyond tokenization. However, existing BLMs typically require training from scratch on trillions of bytes, making them prohibitively expensive. In this paper, we propose an efficient
distillation recipe that converts existing
token-trained LLMs into BLMs while retaining comparable capabilities. Our recipe follows a two-stage curriculum: (1) Progressive Knowledge
Distillation, which aligns byte-level representations with the
embeddings of the
token-trained teacher model; and (2) Byte-Level Supervised Fine-Tuning, which enables end-to-end generation entirely in the byte space. We validate our approach across multiple model families, including Llama, Qwen, and OLMo, and demonstrate that the distilled BLMs retain most of the teacher models' performance using only approximately 125B bytes.
Read the full abstract ▾
All authors Zishuo Bao, Jiaqi Leng, Junxiong Wang, Bowen Peng, Yucheng Lu
Computation and Language
Reliable Use of Lemmas via Eligibility Reasoning and Section$-$Aware Reinforcement Learning
Zhikun Xu, Xiaodong Yu, Ben Zhou et al. (+6 more)
Recent large
language models (LLMs) perform strongly on mathematical
benchmarks yet often misapply lemmas, importing conclusions without validating assumptions. We formalize lemma$-$judging as a structured prediction task: given a statement and a candidate lemma, the model must output a precondition check and a conclusion$-$utility check, from which a usefulness decision is derived. We present RULES, which encodes this specification via a two$-$section output and trains with reinforcement learning plus section$-$aware loss masking to assign penalty to the section responsible for errors. Training and evaluation draw on diverse natural
language and formal proof corpora; robustness is assessed with a held$-$out perturbation suite; and end$-$to$-$end evaluation spans competition$-$style, perturbation$-$aligned, and
theorem$-$based problems across various LLMs. Results show consistent in$-$domain gains over both a vanilla model and a single$-$label RL baseline, larger improvements on applicability$-$breaking perturbations, and parity or modest gains on end$-$to$-$end tasks; ablations indicate that the two$-$section outputs and section$-$aware reinforcement are both necessary for robustness.
Read the full abstract ▾
All authors Zhikun Xu, Xiaodong Yu, Ben Zhou, Jiang Liu, Jialian Wu, Ze Wang, Ximeng Sun, Hao Chen, Zicheng Liu
Computation and Language · Cross-listed from cs.AI
Error Taxonomy-Guided Prompt Optimization
Mayank Singh, Vikas Yadav and Eduardo Blanco
Automatic
Prompt Optimization (APO) is a powerful approach for extracting performance from large
language models without modifying their weights. Many existing methods rely on trial-and-error, testing different
prompts or in-context examples until a good configuration emerges, often consuming substantial compute. Recently, natural
language feedback derived from execution logs has shown promise as a way to identify how
prompts can be improved. However, most prior approaches operate in a bottom-up manner, iteratively adjusting the
prompt based on feedback from individual problems, which can cause them to lose the global perspective. In this work, we propose Error Taxonomy-Guided
Prompt Optimization (ETGPO), a
prompt optimization
algorithm that adopts a top-down approach. ETGPO focuses on the global failure landscape by collecting model errors, categorizing them into a taxonomy, and augmenting the
prompt with guidance targeting the most frequent failure modes. Across multiple
benchmarks spanning mathematics, question answering, and logical
reasoning, ETGPO achieves accuracy that is comparable to or better than state-of-the-art methods, while requiring roughly one third of the optimization-phase
token usage and evaluation budget.
Read the full abstract ▾
Computation and Language
DeALOG: Decentralized Multi-Agents Log-Mediated Reasoning Framework
Abhijit Chakraborty, Ashish Raj Shekhar, Shiven Agarwal et al. (+1 more)
Complex question answering across text, tables and images requires integrating diverse information sources. A framework supporting specialized processing with coordination and interpretability is needed. We introduce DeALOG, a decentralized multi-
agent framework for
multimodal question answering. It uses specialized
agents: Table, Context, Visual, Summarizing and Verification, that communicate through a shared natural-
language log as persistent
memory. This log-based approach enables collaborative error detection and verification without central control, improving robustness. Evaluations on FinQA, TAT-QA, CRT-QA, WikiTableQuestions, FeTaQA, and MultiModalQA show competitive performance. Analysis confirms the importance of the shared log,
agent specialization, and verification for accuracy. DeALOG, provides a scalable approach through modular components using natural-
language communication.
Read the full abstract ▾
All authors Abhijit Chakraborty, Ashish Raj Shekhar, Shiven Agarwal, Vivek Gupta
Computation and Language
Sparse Reward Subsystem in Large Language Models
Guowei Xu, Mert Yuksekgonul and James Zou
Recent studies show that LLM hidden states encode reward-related information, such as answer correctness and model confidence. However, existing approaches typically fit black-box probes on the full hidden states, offering little insight into how this information is structured across
neurons. In this paper, we show that reward-related information is concentrated in a sparse subset of
neurons. Using simple probing, we identify two types of
neurons: value
neurons, whose activations predict state value, and dopamine
neurons, whose activations encode step-level temporal difference (TD) errors. Together, these
neurons form a sparse reward subsystem within LLM hidden states. These names are drawn by analogy with neuroscience, where value
neurons and dopamine
neurons in the biological reward subsystem also encode value and reward prediction errors, respectively. We demonstrate that value
neurons are robust and transferable across diverse
datasets and models, and provide causal evidence that they encode reward-related information. Finally, we show applications of the reward subsystem: value
neurons serve as effective predictors of model confidence, and dopamine
neurons can function as a process reward model (PRM) to guide inference-time search. Our code is available at https://github.com/zou-group/reward_subsystem.
Read the full abstract ▾
Computation and Language
DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning
Batuhan K. Karaman, Aditya Rawal, Suhaila Shakiah et al. (+4 more)
Reinforcement learning with verifiable rewards has emerged as a promising paradigm for enhancing the
reasoning capabilities of large
language models particularly in mathematics. Current approaches in this domain present a clear trade-off: PPO-style methods (e.g., GRPO/DAPO) offer training stability but exhibit slow learning trajectories due to their trust-region constraints on policy updates, while REINFORCE-style approaches (e.g., CISPO) demonstrate improved learning efficiency but suffer from performance instability as they clip importance sampling weights while still permitting non-zero gradients outside the trust-region. To address these limitations, we introduce DISPO, a simple yet effective REINFORCE-style
algorithm that decouples the up-clipping and down-clipping of importance sampling weights for correct and incorrect responses, yielding four controllable policy update regimes. Through targeted ablations, we uncover how each regime impacts training: for correct responses, weights >1 increase the average
token entropy (i.e., exploration) while weights <1 decrease it (i.e.,
distillation) – both beneficial but causing gradual performance degradation when excessive. For incorrect responses, overly restrictive clipping triggers sudden performance collapse through repetitive outputs (when weights >1) or vanishing response lengths (when weights <1). By separately tuning these four clipping parameters, DISPO maintains the exploration-
distillation balance while preventing catastrophic failures, achieving 61.04% on AIME'24 (vs. 55.42% CISPO and 50.21% DAPO) with similar gains across various
benchmarks and models.
Read the full abstract ▾
All authors Batuhan K. Karaman, Aditya Rawal, Suhaila Shakiah, Mohammad Ghavamzadeh, Mingyi Hong, Arijit Biswas, Ruida Zhou