Computation and Language
VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination
Chunyu Liu, Zhengyang Fan, Kaisen Yang et al. (+1 more)
MDLMs generate text by denoising a preallocated masked response canvas, making response-length modeling central to instruction tuning. Existing MDLMs often inherit the autoregressive convention of using repeated [EOS]
tokens for padding during instruction tuning, giving [EOS] a dual role as both a semantic terminator and a padding
token. We show that this dual role is a root cause of [EOS] overflow under large-block decoding. To decouple these roles, we propose VoidPadding, which introduces [VOID] for padding and reserves [EOS] for termination. During inference, the learned [EOS] signal enables early stopping, while the learned [VOID] signal guides adaptive response canvas expansion. On Dream-7B-Instruct, VoidPadding improves the block-size-averaged four-task mean across mathematical
reasoning and code generation
benchmarks by \(+17.84\) points over the original model and \(+6.95\) points over RainbowPadding, while reducing decoding NFE by 55.7% on average. Code is available at https://github.com/Haru-LCY/VoidPadding.
Read the full abstract ▾
All authors Chunyu Liu, Zhengyang Fan, Kaisen Yang, Alex Lamb
Computation and Language
Fine-tuning LLMs for Passive Depression Severity Estimation from AI Mental Health Dialogue
Olivier Tieleman, Ziyi Zhu, Ting Su et al. (+3 more)
Depression is the leading cause of disability worldwide, and early detection of symptom change is essential for timely intervention. Validated instruments such as the Patient Health Questionnaire-9 (PHQ-9) support symptom monitoring at scale, but real-world completion rates are low, introducing response bias and systematic missingness. Passive approaches that infer severity from routinely generated data could close this gap. We address this by predicting PHQ-9 total scores directly from transcripts of conversations between users and an AI mental health application, requiring only conversation text and no additional clinical data. We fine-tune a Qwen3.5-27B backbone with a regression head, augment 3,111 ground-truth labels with pseudolabels generated by a
reasoning model (Claude Opus) and iteratively trained intermediate models, for a combined
dataset of 6,283 users. On a held-out test set of 842 users, our best model achieves MAE = 2.6, RMSE = 4.0, Pearson r = 0.80, and AUC = 0.91 at the PHQ-9 >= 10 clinical threshold. We also find AUC > 0.87 at every severity threshold from PHQ-9 >= 3 to PHQ-9 >= 24, demonstrating that the model captures depression severity across the full clinical spectrum. This work opens the door to passive, continuous symptom monitoring in AI mental health platforms, without requiring users to complete self-report measures.
Read the full abstract ▾
All authors Olivier Tieleman, Ziyi Zhu, Ting Su, Samuel J. Bell, Thomas D. Hull, Caitlin A. Stamatis
Computation and Language
Learning task-specific subspaces via interventional post-training of speech foundation models
Jack Cox and Jon Barker
Speech foundation models, pre-trained on large corpora of unlabelled
speech data, produce general-purpose representations which are useful across tasks. However, these representations encode information about salient
speech variables in a distributed manner, while downstream
speech tasks rely on only some of this variability. In this work, we propose a post-training refinement approach using interventional contrastive learning. By leveraging an interventional
dataset and multi-part contrastive loss, we learn a transformation from the entangled representation space of
speech foundation models into separate content and speaker subspaces. We evaluate the learnt representations on speaker verification and keyword spotting tasks, showing improved out-of-domain speaker verification performance and evidence that speaker and content information are separated across the learned subspaces.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Non-negative Elastic Net Decoding for Information Retrieval
Koki Okajima, Yasutoshi Ida, Tsukasa Yoshida et al. (+1 more)
Dense
retrieval has become the dominant paradigm in information
retrieval, in which each document is scored against a query by the inner product of their vector
embeddings, and the top-$k$ documents by score are retrieved for this query. However, since each document's score depends solely on the
embedding of the query and itself, the
retrieval process is oblivious to the content of the entire corpus. Therefore, dense
retrieval cannot avoid selecting semantically similar documents from the corpus, which may result in a non-diverse, redundant set of retrieved documents. To this end, we approach
retrieval as a joint decoding problem, in which documents are selected as a set with regard to the context of the rest of the corpus. To achieve this, we propose Non-Negative elastic Net (NNN) decoding, which selects documents whose
embeddings jointly reconstruct the query
embedding as a sparse non-negative linear combination. Our main theoretical result establishes a strict separation between dense
retrieval and NNN decoding. For any corpus, every query correctly handled by dense
retrieval is also handled by NNN decoding, while on corpora containing correlated documents, NNN decoding additionally handles queries that dense
retrieval cannot. Experimental results indicate that applying NNN decoding to frozen
embeddings trained for inner-product scoring yields consistent improvements across several
benchmarks. Moreover, we introduce an end-to-end training procedure which optimizes the
embeddings for NNN decoding, producing significant performance gains surpassing in all metrics and
benchmarks compared to dense
retrieval. Our work establishes a new paradigm for leveraging dense
embeddings in information
retrieval, beyond the standard practice of inner-product scoring.
Read the full abstract ▾
All authors Koki Okajima, Yasutoshi Ida, Tsukasa Yoshida, Yasuaki Nakamura
Computation and Language
ChLogic: Evaluating Robustness of Logical Reasoning in Chinese Expressions
Peixian Zhou, Yuxu Chen, Chaorui Zhang et al. (+3 more)
Large
language models perform increasingly well on standardized logical
reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English–Chinese aligned
benchmark that tests whether models preserve logical
reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the
benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15
language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English–Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect
multilingual logical
reasoning. Overall, ChLogic provides a useful stress test for the robustness of
multilingual reasoning.
Read the full abstract ▾
All authors Peixian Zhou, Yuxu Chen, Chaorui Zhang, Wei Han, Bo Bai, Xueyan Niu
Computation and Language
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Zihao Wei, Wenjie Shi, Liang Pang et al. (+8 more)
Long-form chain-of-thought
reasoning can improve LLM performance on complex tasks, but models often continue generating unnecessary
reasoning after a correct answer has emerged. We refer to this behavior as overthinking. We study this phenomenon from the perspective of GRPO-style reinforcement learning (RL) post-training, framing it as a training-time credit-assignment problem rather than merely a decoding-time stopping problem. In rollouts sampled at the onset of GRPO training, we observe that successful trajectories can exhibit a slightly higher degree of overthinking than unsuccessful trajectories for the same
prompts. This early imbalance provides a starting point for an undesirable feedback loop: because GRPO assigns sequence-level credit, it cannot distinguish the solution-reaching prefix from the unnecessary continuation that lengthens a successful trajectory. Both receive positive update signal, allowing the initial imbalance to grow into more severe overthinking during training. To address this issue, we introduce Dynamic Rollout Editing (DRE), a training-time intervention for successful trajectories that continue thinking after answer emergence. DRE preserves the accepted verified prefix, edits the remaining thinking, and prefers the edited trajectory within the same RL group, weakening the preference signal for unnecessary thinking without penalizing the
reasoning needed to reach the answer. Experiments across diverse tasks show the effectiveness of DRE.
Read the full abstract ▾
All authors Zihao Wei, Wenjie Shi, Liang Pang, Jingcheng Deng, Shicheng Xu, Shasha Guo, Zenghao Duan, Jiahao Liu, Jingang Wang, Huawei Shen, Xueqi Cheng
Computation and Language
GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?
Tongxu Luo, Rongsheng Wang, Jiaxi Bi et al. (+22 more)
Game generation is an emerging application of coding
agents, requiring models to transform natural-
language specifications into playable interactive systems. Unlike traditional coding tasks, game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay. We formalize end-to-end game generation as the problem of producing a complete game artifact that realizes a specification through observable player-game interaction in a target environment. We argue that evaluating this setting requires three desiderata: Engine Grounding, Artifact Completeness, and Interactive Verification. We propose an interaction-grounded evaluation framework that assesses executable gameplay through replayed demonstrations and rubric-guided
multimodal judging. We instantiate this framework as GameCraft-Bench, a
benchmark comprising 140 Godot tasks across 15 game families. Evaluations of frontier coding
agents show that end-to-end game generation remains highly challenging: the strongest
agent achieves only 41.46%, and most
agents score below 40%. Further analysis reveals that while
agents often implement recognizable mechanics, they struggle to deliver complete games with sufficient content, functional visual feedback, and coherent presentation. See https://tongxuluo.github.io/gamecraft-bench-website for demos, code, and data.
Read the full abstract ▾
All authors Tongxu Luo, Rongsheng Wang, Jiaxi Bi, Chenming Xu, Zhengyang Tang, Jianlong Chen, Juhao Liang, Ke Ji, Shuqi Guo, Yuhao Du, Fan Bu, Wenyu Du, Xiaotong Zhang, Kyle Li, Shaobo Wang, Linfeng Zhang, Yuxuan Liu, Xin Lai, Chenxin Li, Yiduo Guo, Zhexin Zhang, Xinyuan Wang, Tianyi Bai, Ziniu Li, Benyou Wang
Computation and Language
Environment-Grounded Automated Prompt Optimization for LLM Game Agents
Rean Clive Fernandes, Lukas Fehring, Theresa Eimer et al. (+2 more)
LLM
agents in interactive environments are highly sensitive to their
prompts, yet
prompt engineering remains a manual, task-specific process. We introduce an automated
prompt optimization framework for LLM
agents that decomposes the observation-to-action pipeline into a goal-conditioned descriptor
agent and an action selection
agent, and iteratively refines each module's
prompt through an LLM-driven evolutionary loop guided by environment returns. We propose a behavior analyzer to attribute episode outcomes to specific
prompt components, and a mutator to propose targeted revisions to the
prompt, before validating them through environment rollouts. We evaluate on all five BabyAI tasks in the BALROG
benchmark, comparing our pipeline against BALROG's RobustCoTAgent under both plain and guided
prompt initializations. Optimization improves performance consistently across tasks and conditions, without requiring updates to the model weights. On PutNext, a multi-step coordination task where the RobustCoTAgent achieves 0% success, our framework reaches up to 72.5% success rate using the same underlying LLM with optimized
prompts. These results suggest that a multi-
agent framework, combined with automatic
prompt optimization, enhances LLMs without the need for fine-tuning or extensive human supervision.
Read the full abstract ▾
All authors Rean Clive Fernandes, Lukas Fehring, Theresa Eimer, Marius Lindauer, Matthias Feurer
Computation and Language
Perceptual compensation for tonal context in self-supervised speech models
James Kirby, Ioana Krehan and Michele Gubian
This study examines the extent to which the wav2vec2.0 architecture exhibits evidence of compensation for phonological context. We conducted a pseudo-replication of a perceptional compensation experiment on Mandarin Chinese tones, and compared the
embedding similarities and probing classifier outputs between a purely self-supervised pre-trained model and a model fine-tuned for Mandarin ASR. No evidence of compensation was found in the
embedding similarities of the purely pre-trained model. Probing classifiers showed some evidence of compensation in addition to the expected layer-wise improvements in categorization, but failed to replicate human performance on isolated test syllables. Our findings contrast with previous reports of sensitivity to phonological structure emerging through pre-training alone, and suggest that supervised objectives may be necessary to encourage the abstraction of at least some types of phonological regularities.
Read the full abstract ▾
Computation and Language
When Multiple Scripts Matter: Evaluating ASR in Clinical Settings
Jean Seo, Minkyu Kim, Jeonguk Lee et al. (+3 more)
Automatic
speech recognition (ASR) in non-English clinical settings is challenged by multiscript variability, where the same term may appear in multiple valid orthographic forms. Conventional string-matching evaluation metrics often underestimate ASR performance by treating orthographic variants as errors. To address this issue, we introduce MultiClin, a clinical ASR
benchmark designed to evaluate robustness to multiscript variability. Experiments across diverse ASR models show that multiscript-aware evaluation provides a fairer assessment of recognition quality than conventional single-reference evaluation. We further investigate the impact of script consistency during training and find that inconsistent script mappings increase orthographic uncertainty and hinder model convergence, with a balanced 50% mapping ratio producing the highest
entropy. In contrast, script unification consistently yields the best ASR performance. Our
dataset and code are publicly available at: https://github.com/aitrics-ronaldo/Interspeech_MultiClin.
Read the full abstract ▾
All authors Jean Seo, Minkyu Kim, Jeonguk Lee, Jisoo Jung, Wooseok Han, Eunho Yang
Computation and Language
Improving low-resource ASR using bilingual fine-tuning with language identification: a cross-linguistic evaluation
Reihaneh Amooie, Yun Hao, Wietse de Vries et al. (+3 more)
This study explores how bilingual fine-tuning affects automatic
speech recognition (ASR) in low-resource
languages. We evaluate this method across nine linguistically and geographically diverse
language pairs, covering a range of
language families and writing systems. To distinguish the two
languages, during training, we pre-pend each input text with a
language identification
token. At inference, the model jointly predicts both the
language and transcription from the
speech input alone. As texts for which the
language is incorrectly determined show low ASR performance, we also conduct a follow-up experiment in which the
language identification
token is provided both during training and inference. Our results show that bilingual fine-tuning can be beneficial when
language identification accuracy is high, and that in cases where
language identification performance is low, including the
language identification
token at inference helps to improve ASR performance.
Read the full abstract ▾
All authors Reihaneh Amooie, Yun Hao, Wietse de Vries, Jelske Dijkstra, Matt Coler, Martijn Wieling
Computation and Language · Cross-listed from cs.SE
A Framework for Evaluating Agentic Skills at Scale
Maksim Shaposhnikov, Nicolas Fortuin, Simon Stipcich et al. (+3 more)
Agent skills – structured, reusable knowledge artifacts that augment LLM
agent capabilities – have been rapidly adopted in industry, yet their cross-domain impact and use across commercial and open-source models remain under-studied, and no reusable methodology exists for evaluating an individual skill. In this work, we present an evaluation framework that lets a skill author construct realistic tasks to rigorously assess the aspects of a skill that matter most to them, and that estimates skill utility by solving those tasks. Further, we apply our evaluation approach at scale to 500 real-world skills, generating 1,000 tasks derived from the skills' content, along with instruction-following and goal-completion scoring rubrics. Using these metrics, we evaluate how 19
agent-model configurations, both proprietary and open-source, perform on the tasks. Our results show that models vary widely in how closely they adhere to the instructions encoded in skills, leading to substantial differences in their performance gains. Furthermore, we show that access to a skill significantly changes model behavior compared to the no-skill setup, providing an essential mechanism for encoding opinionated workflows into LLM
agents. We release our evaluation
dataset to support future work on
agent skills.
Read the full abstract ▾
All authors Maksim Shaposhnikov, Nicolas Fortuin, Simon Stipcich, Maria I. Gorinova, Amy Heineike, Rob Willoughby
Computation and Language · Cross-listed from cs.CR
Beyond Native Success: Auditing Deployment-Interface Exposure of CLIP Backdoors
Kunlan Xiang, Haomiao Yang and Wenbo Jiang
Contrastive
Language-Image Pre-training models are widely reused across downstream interfaces, including feature extraction,
retrieval, reranking, and selection. Existing CLIP
backdoor, however, usually validate attacks on a small attack-native task, leaving unclear whether the same poisoned checkpoint remains exposed, weakens, or becomes not applicable when reused through other interfaces. We introduce DIFE, a Deployment-Interface Footprint Evaluation framework that audits backdoored CLIP checkpoints across deployment interfaces. DIFE makes various evaluations comparable by specifying each interface's component readout, trigger channel, target event, reference condition, and metric. DIFE also introduces effective-footprint diagnosis to identify the reusable CLIP component or component combination that carries exposure and explains where risk transfers.
Auditing reproduced CLIP
backdoors with DIFE reveals a structured landscape: native success is not a checkpoint-level risk certificate, exposure follows component footprints, text-side poisoning does not yield textual-encoder control, and some coupled attacks remain mechanism-bound. This audit reveals a import gapin existing CLIP
backdoors: a textual encoder that itself becomes a reusable carrier of
adversarial behavior. We therefore introduce BadTextTower to fill this gap. BadTextTower produces strong text-conditioned
retrieval, reranking, and selection exposure while leaving visual-only reuse nearly clean.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering
Maria I. Gorinova, Macey Baker, Amy Heineike et al. (+3 more)
Coding
agents have become a major mode of software engineering, but the
benchmarks we use to compare them were designed in a pre-
agent era: they collapse model, harness, and environment into a single end-to-end score, typically computed against one reference solution, with no component-level signal for iteration. We argue that current coding
benchmarks are misaligned with agentic software engineering. A coding
agent in practice is not a model: it is a system harness – a composite of models, harnesses, contexts, environments, and feedback signals, any one of which can move the
benchmark score by margins comparable to those between adjacent model generations. We discuss three symptoms: (i)
benchmark scores conflate the model with the rest of the harness; (ii) grading against a single reference solution penalises equally valid alternatives; and (iii) the absence of signal at the level of individual harness components makes the end-to-end system score difficult to iterate on.
Read the full abstract ▾
All authors Maria I. Gorinova, Macey Baker, Amy Heineike, Maksim Shaposhnikov, Rob Willoughby, Dru Knox
Computation and Language
The Slop Paradox: How Synthetic Standardization Erodes Clinical Uncertainty and Cross-Modal Alignment in AI-Rewritten Radiology Reports
Samar Ansari
AI-assisted clinical documentation tools increasingly summarize, standardize, and reformat radiology reports using large
language models (LLMs). We present a controlled measurement of the resulting information degradation. Using 450 chest X-ray reports from the Indiana University
dataset, we generate synthetic versions via three realistic LLM rewriting tasks: EHR summarization, standardized rewriting, and teaching case preparation. We measure entity erosion (via medical NER), hedging collapse (loss of clinical uncertainty
language), and cross-modal
alignment degradation (via BiomedCLIP image-text similarity). Our central finding is a dissociation between information loss and cross-modal fidelity. EHR summarization is the most destructive at the content level, eroding 51.4% of clinical entities and 43.7% of hedging
language, yet it preserves image-text
alignment almost entirely (a 2.5% drop). The two tasks meant to produce cleaner training data, standardized rewriting and teaching case preparation, do the reverse: they preserve more entities (26.8% and 29.3% eroded) but cause 14.9-16.5%
alignment drops, six to seven times those of EHR summarization. We term this the slop paradox: rewriting that makes clinical text look cleaner for
multimodal training is precisely what pulls it away from the image. Contrary to our pre-specified hypothesis, rare pathologies were not preferentially degraded: across nine rare-versus-common comparisons, no difference survived multiple-comparison correction, and nominal differences ran in the opposite direction (common > rare), so contamination is invisible to condition-specific monitoring. The dominant determinant of degradation is the type of AI rewriting task, not the clinical content. These findings bear on
multimodal medical AI
dataset construction and the governance of AI-assisted clinical documentation.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.HC
Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars
Pascal Riachi, Sofie Kamber, Stella Brogna et al. (+2 more)
Training psychotherapists in evidence-based interventions such as Acceptance and Commitment Therapy (ACT) requires repeated practice with meaningful feedback, yet opportunities for safe, standardized training are limited by ethical, logistical, and resource constraints. We introduce a system designed to support ACT-oriented psychotherapy training through spoken dialogue with an
embodied virtual patient. The system uses large
language models to simulate patient behavior conditioned on profiles derived from real therapy sessions and configurable clinical scenarios, while a separate automated evaluator provides turn-by-turn feedback on therapist responses based on established ACT fidelity criteria. Rather than aiming to replace supervision, the system is intended to support deliberate practice by enabling experimentation, reflection, and immediate feedback in low-risk settings. Expert evaluation with practicing psychologists confirmed high realism in patient behavior and demonstrated that immediate turn-by-turn ACT feedback increased therapists' awareness of intervention choices and enabled effective experimentation with alternative responses. Quantitative evaluation across 49 therapy transcripts identified GPT-4o-mini as the optimal feedback model, achieving the lowest mean absolute error (MAE = 6.12) in replicating human supervisor ACT fidelity ratings with statistically significant agreement. This work demonstrates the potential of fidelity-aware simulated patients as a scalable complement to psychotherapy training.
Read the full abstract ▾
All authors Pascal Riachi, Sofie Kamber, Stella Brogna, Andrew Gloster, Rafael Wampfler
Computation and Language · Cross-listed from cs.CV
Vision-language models for chest radiography do not always need the image
Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams et al. (+4 more)
Vision-
language models that answer questions about chest radiographs are evaluated by their accuracy on labels derived from radiology reports. High
benchmark accuracy is often interpreted as evidence that the model uses the image. A model that answers from the finding named in the question can score as well as a model that uses the radiograph. Keeping the question fixed, we audit eight open-weight systems by swapping in another patient's radiograph with the same or the opposite label, occluding the radiologist-marked region or an equal region elsewhere, and removing the radiograph or replacing it with noise or a photograph. On 2,548 yes-or-no questions from MIMIC-CXR, one
multimodal model answers Yes regardless of the image, another
multimodal model changes its answers without following the label, and four systems use the image but keep about half of their correct answers when the radiograph is swapped for an opposite-label radiograph. A medical model that receives only the question text scores 55.3% on the pooled questions, higher than two
multimodal systems. It scores 91.8% where every finding is present, and answering Yes to every question scores 100% there. Where the image is necessary, the best
multimodal system exceeds this model by 10.4% in balanced accuracy. The categories are unchanged on CheXpert. Confidence is not higher when a correct answer depends on the marked region. In a reader study with three radiologists, the two radiologists who read a balanced set of 200 cases score 86.0% and 82.0%, and the systems score 50.0% to 73.0%. Accuracy does not establish image use, but an intervention on the image can test it.
Read the full abstract ▾
All authors Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams, Tri-Thien Nguyen, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
Computation and Language · Cross-listed from cs.AI
EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent
Zeyao Du, Tong Li and Haibo Zhang
As LLM-based shopping
agents enter production, existing
benchmarks fail to capture how a shopper's requirements arrive: stated implicitly in the query, recorded in a profile, or revealed only when the right question is asked.
Benchmarks that expose full intent upfront and grade only the final choice can neither pose this long-horizon challenge nor explain which requirement an
agent missed. To address this gap, we introduce EComAgentBench, a
benchmark of 662 tasks grounded in real Amazon products and reviews. Each task scatters these requirements across a visible query, a tool-gated profile, and scripted clarification; an
agent must uncover hidden intent, verify candidates against attributes and review evidence, and commit to a single product within 100 tool calls. Moreover, typed, source-tagged rubrics grade every task, attributing each failure to a requirement and its source. Construction is automated yet reliable, with every answer fixed in code before any text is generated and every sample validated. Our evaluation of seven models reveals that even the strongest attains only 57.1% overall accuracy, and rubric satisfaction degrades from visible to hidden sources. Overall, we believe EComAgentBench will serve as a reproducible foundation for moving shopping
agents from single-query search toward dependable assistance over long horizons.
Read the full abstract ▾
Computation and Language
LLMs Infer Cultural Context but Fail to Apply It When Responding
Yisong Miao, Jian Zhu and Vered Shwartz
Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others. We investigate whether this affects models' ability to generate culturally adapted responses by evaluating their use of local measurement units based on the user's perceived cultural background. We introduce Cultural and Pragmatic Response Inference (CAPRI), a
dataset of conversations with varying levels of cultural cues. Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information to adapt their answers to the relevant cultural conventions, unless explicitly prompted to perform the tasks sequentially. We further evaluate adaptation to the interpretation of time and quantity expressions, two subjective
language grounding dimensions that are affected by culture. We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model's country of origin. Overall, CAPRI provides a resource for future research aimed at narrowing the gap between cultural knowledge and culturally adaptive
language generation.
Read the full abstract ▾
Computation and Language
SuCo: Sufficiency-guided Continuous Adaptive Reasoning
Jiahao Wang, Bingyu Liang, Chenhao Hu et al. (+5 more)
Despite remarkable performance on complex tasks, Large
Reasoning Models (LRMs) often generate excessively long Chain-of-Thoughts (CoT), inflating computational costs even for simple queries. Existing efforts to mitigate this inefficiency typically rely on discrete
reasoning modes or fixed budget tiers, lacking a principled criterion of when
reasoning is sufficient. In this work, we introduce Minimal Sufficient CoT (MSC), defined as the shortest prefix of a CoT trajectory which is adequate for producing the correct answer. We empirically show that MSC not only reduces
reasoning tokens, but also improves accuracy across difficulty levels. Building on MSC, we propose Sufficiency-guided Continuous Adaptive
Reasoning (SuCo), a two-stage training framework for autonomous
reasoning control along a continuous spectrum. In stage 1, MSC-Aligned Fine-Tuning (MFT) constructs MSC data using problem-adaptive sufficiency thresholds that naturally scale with question difficulty, then fine-tunes the model to internalize concise yet sufficient
reasoning patterns. In stage 2, Sufficiency-Aware Policy Optimization (SAPO) further optimizes the model through reinforcement learning with dynamic complexity tracking and sufficiency-aware rewards that penalize both over- and under-thinking. Extensive experiments across mathematics, code, and science
benchmarks show that SuCo consistently achieves improvements in both accuracy and
reasoning efficiency.
Read the full abstract ▾
All authors Jiahao Wang, Bingyu Liang, Chenhao Hu, Longhui Zhang, Xuebo Liu, Min zhang, Jing Li, Xuelong Li
Computation and Language
Bridging Functional Correctness and Runtime Efficiency Gaps in LLM-Based Code Translation
Longhui Zhang, Jiahao Wang, Chenhao Hu et al. (+3 more)
While large
language models (LLMs) have greatly advanced the functional correctness of automated code translation systems, the runtime efficiency of translated programs has received comparatively little
attention. With the waning of Moore's law, runtime efficiency has become increasingly important for program quality, alongside functional correctness. Our preliminary study reveals that LLM-translated programs often run slower than human-written ones, and this issue cannot be remedied through
prompt engineering alone. Therefore, our work proposes SwiftTrans, a code translation framework comprising two key stages: (1) Multi-Perspective Exploration, where MpTranslator leverages parallel in-context learning (ICL) to generate diverse translation candidates; and (2) Difference-Aware Selection, where DiffSelector identifies the optimal candidate by explicitly comparing differences between translations. We further introduce Hierarchical Guidance for MpTranslator and Ordinal Guidance for DiffSelector, enabling LLMs to better adapt to these two core components. To support the evaluation of runtime efficiency in translated programs, we extend existing
benchmarks, CodeNet and F2SBench, and introduce a new
benchmark, SwiftBench. Experimental results across all three
benchmarks show that SwiftTrans achieves consistent improvements in both correctness and runtime efficiency.
Read the full abstract ▾
All authors Longhui Zhang, Jiahao Wang, Chenhao Hu, Bingyu Liang, Jing Li, Min Zhang
Computation and Language
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
Chao Chen, Chengzu Li, Zhiwei Li et al. (+2 more)
Reinforcement learning pipelines for Large
Language Model (LLM) training often rely on manually redesigned environments between stages, requiring practitioners to heuristically infer which configuration will best improve the current policy. To automate this process, we propose the LLM-as-Environment-Engineer framework in which the current policy model analyzes failure trajectories together with contextual information and proposes modifications to the next-stage training environment configuration. We also introduce MAPF-FrozenLake, a controllable testbed whose generator exposes multi-dimensional environment configurations, making it suitable for studying and benchmarking environment redesign. On this testbed, we condition the environment engineer on structured summaries of policy behavior, failure cases, and environment statistics, from which it produces the configuration for the next training stage. With Qwen3-4B as the backbone, our framework achieves the strongest aggregate performance on our
benchmarks, outperforming larger proprietary LLMs (e.g., GPT, Gemini) and fixed-environment training baselines. We further analyze which forms of context are most effective, finding that successful environment updates rely on failure evidence and preserve configurations that already work. Interestingly, the current RL checkpoint serves as a better environment engineer than the original base model, suggesting that policy learning improves the model's ability to diagnose its remaining weaknesses.
Read the full abstract ▾
All authors Chao Chen, Chengzu Li, Zhiwei Li, Yinhong Liu, Zhijiang Guo
Computation and Language · Cross-listed from cs.LG
EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
Zhitong Wang, Songze Li, Hao Peng et al. (+4 more)
Reinforcement learning (RL) has emerged as a powerful paradigm for training Large
Language Models (LLMs) as
agents. However, conventional RL methods for long-horizon agentic tasks often struggle with sparse outcome rewards. Intuitively, this overlooks the rich environment dynamics information contained in rollout interaction trajectories. We argue that the interaction experience inherently serves as an implicit supervision signal, reveals the underlying transition mechanisms of the environment, and enables the
agent to construct a more accurate internal model of the environment.. Therefore, in this work, we investigate how to leverage this additional signal to improve policy learning. Specifically, we propose EnvRL, a framework that incorporates environment dynamics learning into agentic RL via two auxiliary objectives: state prediction and inverse dynamics. By jointly optimizing with the primary RL objective, we encourage the
agent to internalize environment dynamics from its own interaction experience. Extensive experiments on two long-horizon agentic
benchmarks demonstrate that EnvRL achieves significant improvements on success-rates over RL-only baselines, e.g., when trained with GRPO, lifting Qwen-2.5-1.5B-Instruct from 72.8% to 77.4% on ALFWorld, and from 56.8% to 67.0% on WebShop.
Read the full abstract ▾
All authors Zhitong Wang, Songze Li, Hao Peng, Shuzheng Si, Yi Wang, Maosong Sun, Juanzi Li
Computation and Language · Cross-listed from cs.CV
MambaCount: Efficient Text-guided Open-vocabulary Object Counting with Spatial Sparse State Space Duality Block
Hao-Yuan Ma, Li Zhang, Minjie Qiang et al. (+1 more)
Text-guided Open-vocabulary Object Counting (TOOC) aims to estimate the number of objects described by text
prompts, which is particularly challenging in dense scenes with large scale variations. Existing TOOC approaches predominantly rely on
Transformers, whose quadratic complexity with respect to image resolution limits their scalability. Mamba offers a promising alternative due to its linear complexity. However, previous Mamba-based methods have two main limitations. On the one hand, the inherent causal formulation of Mamba constrains the bidirectional spatial dependency modeling required by non-causal
vision tasks. On the other hand, existing Mamba-based
vision models often overlook the unconstrained high
entropy in the spatial
token responses, which can weaken local details and high-frequency cues. To address these limitations, we propose MambaCount, an efficient framework built on the Spatial Sparse State Space Duality (S^4D) block. Specifically, we analyze and reconstruct the decay dynamics of hidden states in Mamba to alleviate the dependency constraints introduced by causal modeling. Moreover, we introduce a Spatial
Token Selection (STS) sub-block to reduce the unconstrained high
entropy in spatial
token responses within Mamba. In addition, we design Multi-Granularity Prototypes (MGP) to identify object-like regions at different semantic levels, improving cross-modal
alignment and interpretability. Extensive experiments on FSC-147 demonstrate that MambaCount achieves state-of-the-art performance among methods without secondary querying, obtaining a test MAE of 12.23, while retaining linear complexity.
Read the full abstract ▾
All authors Hao-Yuan Ma, Li Zhang, Minjie Qiang, Jie Gao
Computation and Language · Cross-listed from cs.AI
Beyond Domains: Reusing Web Skills via Transferable Interaction Patterns
Shiqi He, Yue Cui, Feijie Wu et al. (+5 more)
Large
language model (LLM) web
agents are usually deployed as tool callers: each turn, the model reads a fresh page observation and emits one structured tool action. When every action is a low-level primitive, horizons grow quickly and so do policy-facing LLM completions, dominating latency and cost on
benchmarks such as Mind2Web and WebArena. Recent systems therefore wrap repeated interaction fragments as web skills: callable tools built from successful trajectories or induced programs, so one call can replace several primitives. However, prior skill libraries are still triggered mainly by instruction similarity or coarse site metadata, which yields low skill reuse on held-out sites and leaves much of the potential step and
token reduction on the table. We present SkillMigrator, an
agent that learns reusable web skills and transfers them across sites by matching layout structure rather than specific element references. Each induced skill is stored as a transferable interaction pattern (TIP): the skill paired with a structural sketch of the snapshot at induction time. At test time, SkillMigrator retrieves TIPs by layout similarity and grounds their references on the live page. The rest of the stack is standard: accessibility-snapshot observations with stable references, and fixed tool calling over primitives plus skill invocations. Compared with the state-of-the-art approaches, SkillMigrator reduces the average LLM-action count on successful trajectories by 8-10% across both WebArena and Mind2Web at matched success rate.
Read the full abstract ▾
All authors Shiqi He, Yue Cui, Feijie Wu, Xinyu Ma, Jiaheng Lu, Yaliang Li, Bolin Ding, Mosharaf Chowdhury
Computation and Language
Prompt Perturbation for Reliable LLM Evaluation over Comparison Graphs
Dong Huang, Jianbo Sun and Pengkun Yang
Evaluating large
language models (LLMs) is important for understanding their capabilities, comparing competing systems, and supporting the deployment of reliable models in practice. For open-ended tasks, pairwise evaluation has become a popular paradigm, in which two responses to the same
prompt are compared and the resulting judgments are aggregated into an overall ranking. A central challenge of this paradigm is intransitivity: the induced comparison outcomes may fail to support any coherent global ranking. For example, one may observe cyclic preferences such as $A \succ B \succ C \succ A$, or inconsistencies involving ties such as $A \equiv B\equiv C\neq A$. Such contradictions make the resulting leaderboard unstable and challenging to interpret. In this paper, we propose a
prompt perturbation framework for improving the consistency of pairwise LLM evaluation. Our approach generates perturbed variants of each
prompt, uses the resulting comparison graphs to identify and filter out structurally inconsistent comparison patterns, and then applies standard ranking methods to the filtered comparisons. A key feature of the proposed framework is that graph-level structural consistency is incorporated explicitly into the evaluation pipeline before ranking aggregation. This provides a simple and principled way to reduce cyclic inconsistencies and improve the reliability of LLM rankings.
Read the full abstract ▾
Computation and Language
OPD-Evolver: Cultivating Holistic Agent Evolver via On-Policy Distillation
Guibin Zhang, Xun Xu, Yanwei Yue et al. (+4 more)
Memory has become a standard substrate for self-evolving
agents, yet retaining experience is not the same as learning how to evolve through it. Existing
memory agents can store trajectories, retrieve reflections, or accumulate skills, but often lack the holistic competence to select useful experience, act on it, write reusable knowledge, and maintain a growing repository. We introduce OPD-Evolver, a slow-fast co-
evolution framework that cultivates such an
agent evolver through on-policy self-
distillation. In the fast loop, OPD-Evolver interacts with a four-level
memory hierarchy to read, use, write, and maintain experience for rapid test-time
evolution. In the slow loop, outcome-calibrated
memory attribution and privileged hindsight distill these four abilities into the deployable policy. Across multi-domain
benchmarks, OPD-Evolver surpasses
memory systems such as ReasoningBank by up to 11.5%, and training-based methods such as Skill0 by ~5.8%. Further analysis shows that OPD-Evolver internalizes high-value experience and
memory management, enabling OPD-Evolver-9B to challenge
giant counterparts such as Qwen3.5-397B-A17B and Step-3.5-Flash, pointing beyond
memory-augmented
agents toward genuinely qualified
agent evolvers.
Read the full abstract ▾
All authors Guibin Zhang, Xun Xu, Yanwei Yue, Zikun Su, Wangchunshu Zhou, Xiaobin Hu, Shuicheng Yan
Computation and Language
The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer
Rui Wen, Lu Sun, Jiayang Liu et al. (+3 more)
Compressing large
language models reduces
memory use and inference cost, but it can also create failures that standard
benchmarks miss. A pruned model may still perform well on multiple-choice evaluations, yet fail to answer the same question in open generation. We ask what
pruning changes: does it erase the correct answer, or does it make the answer harder to produce as the top output? We study this question with
multilingual question answering, tracking the same questions before and after
pruning. We find a
benchmark illusion. Under high-sparsity
pruning, especially Wanda, models often fail in greedy open generation while still selecting the correct answer under multiple-choice scoring. In these recognition-only errors, the answer is usually not gone, but demoted: it often reappears with beam search, sampling, or one in-context example. Overall, multiple-choice
benchmarks can overstate the usability of compressed LLMs, creating an evaluation blind spot. Compressed models should be tested on what they can produce, not only on what they can recognize.
Read the full abstract ▾
All authors Rui Wen, Lu Sun, Jiayang Liu, Zesheng Xu, Tianshuo Cong, Zheng Li
Computation and Language · Cross-listed from cs.LG
LLM Features Can Hurt GNNs: Concatenation Interference on Homophilous Graph Benchmarks
Zhongyuan Wang and Pratyusha Vemuri
Adding LLM-generated node features to graph neural networks (GNNs) is widely reported to improve accuracy on standard
benchmarks. We document a contrasting observation: when LLM features are introduced through pure input concatenation (rather than joint training,
distillation, or
prompt-conditioning), they can systematically degrade accuracy on the same homophilous
benchmarks where end-to-end LLM pipelines succeed. With an MLP backbone on the Planetoid public split and bag-of-words original features, concatenating SBERT-encoded GPT-4o-mini TAPE features reduces PubMed test accuracy by -17.0 +/- 0.3 pp and Cora by -4.3 +/- 0.6 pp (CiteSeer -0.6 +/- 0.8 pp, within seed noise). The drop attenuates as we relax each condition (GCN / GCNII / GAT backbones, random splits, smaller encoders) and reverses on medium-homophily WikiCS (+4.4 pp) and ogbn-arxiv (+11.7 pp). To predict when concatenation helps versus hurts, we report a simple measure of LLM-alone discriminability, Delta_sig. Across 9
datasets Delta_sig correlates with the concatenation cost more strongly than homophily at point estimate (r^2 = 0.38 vs. 0.06; N=9, bootstrap CIs overlap). The bootstrap-best change-point is tau = 13.8 pp, and the rule "Delta_sig <= tau predicts non-positive concat cost" classifies 7/9
datasets correctly; since 60% of bootstrap samples place tau in [5, 30] pp, we treat Delta_sig as an interpretive lens rather than a precision filter. A dimension-controlled ablation on PubMed places the LLM-feature drop between same-source PCA (-2.3 pp) and same-dim Gaussian noise (-37.3 pp), ruling out dimensionality and weight-decay artifacts. Nine PubMed configurations fit a power law |Delta_concat| proportional to (sqrt(d_l/n))^1.31 with r^2 = 0.97; the low-Delta_sig, small-n corner is exactly where the headline -17 pp PubMed deficit appears.
Read the full abstract ▾
Computation and Language
Evaluating Large Language Models Abilities for Addressee, Turn-change, and Next Speaker Prediction in Meetings
Ryo Fukuda, Takatomo Kano, Siddhant Arora et al. (+7 more)
We investigate turn-taking in
multimodal multi-party conversations using large
language models (LLMs). We construct an evaluation framework for three tasks: addressee detection, turn-change prediction, and next speaker prediction. We compare supervised models trained for these tasks, text-based LLMs,
multimodal LLMs (MM-LLMs), and human subjects. Experiments on the AMI corpus showed that LLMs outperformed supervised models and humans in next speaker prediction, despite not being trained on the target domain and without access to audio or visual information. An MM-LLM performed better than text-based LLMs on addressee detection and turn-change prediction but remained below human performance, indicating difficulty leveraging raw audio-visual signals. Ablation analyses revealed that conversational context was critical, particularly for next speaker prediction. We observed that human and LLM prediction patterns were similar, and intervals with frequent turn changes were difficult for both.
Read the full abstract ▾
All authors Ryo Fukuda, Takatomo Kano, Siddhant Arora, Marc Delcroix, Naohiro Tawara, Atsunori Ogawa, Yuya Chiba, Atsushi Ando, William Chen, Shinji Watanabe