Computation and Language · Cross-listed from cs.AI
VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct
Haoling Li, Kai Zheng, Jie Wu et al. (+4 more)
Scaling reinforcement learning for visual mathematical
reasoning requires more than generating harder questions: as data volume grows, the reward labels themselves must remain reliable. Yet existing data pipelines scale supervision while trusting the labeller, and policy-side methods assume the underlying answers are already correct. We instead treat scaling as a verifiable data-construction problem and decouple two axes before any policy update:
prompt difficulty, expanded by route-specific
evolution operators, and answer reliability, enforced by offline hypothesis-test falsification. We instantiate this as VeriEvol, an iterative framework with two extensible components: a type-aware
evolution module that rewrites low-difficulty image-question seeds into harder, image-grounded
prompts; and HTV-
Agent, a verifier that accepts an answer only after multi-source counter-evidence has failed to refute it. The resulting verified data scales in volume, extends by adding
evolution routes or verifier channels, and plugs directly into existing GRPO-style RL recipes. On a five-
benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73; then, with backbone, SFT initialization, and GRPO recipe held fixed, VeriEvol adds a cumulative +3.88 over an un-evolved RL baseline, of which +1.82 comes from evolved
prompts and +2.06 from the HTV-
Agent verifier. We release the
prompts, data, models, code, and the full verifier trace of every sample, so that downstream work can scale and audit the pipeline rather than only inspect its outputs.
Read the full abstract ▾
All authors Haoling Li, Kai Zheng, Jie Wu, Can Xu, Qingfeng Sun, Han Hu, Yujiu Yang
Computation and Language
Tianjian Li, Jingyu Zhang, William Jurayj et al. (+5 more)
Long
agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a
token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SelfCompact, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time elements: (i) a compaction tool the model invokes to summarize the accumulated context, and (ii) a lightweight rubric specifying when to fire (a sub-task has resolved, or the trajectory is converging) and when to suppress (mid-derivation, or when stuck). Both are needed. The tool alone is unevenly used across open-weight models, often invoked at unhelpful moments or not at all; the rubric alone cannot act. Together, they elicit effective adaptive compaction without any fine-tuning or external supervision. We present empirical results on six
benchmarks (competitive math and agentic search) and seven models. Our results show that SelfCompact matches or exceeds fixed-interval summarization at a fraction of the
token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5-9 points on agentic search at 30-70% lower per-question cost. Our results expose a meta-cognitive gap: although unprompted models cannot reliably tell when their own context is rotting, a lightweight rubric closes this gap, reframing when to compact as a capability that scaffolds can supply without training.
Read the full abstract ▾
All authors Tianjian Li, Jingyu Zhang, William Jurayj, Xi Wang, Chuanyang Jin, Mehrdad Farajtabar, Eric Nalisnick, Daniel Khashabi
Computation and Language
War in the Abstract: The Rise and Consequences of Militarized Language in Scientific Communication
Sovesh Mohapatra, David Lydon-Staley and Dani S. Bassett
Scientists do not, by profession, wage war. Yet warfare's vocabulary consistently appears in their abstracts. To quantify the extent to which warfare's vocabulary pervades scientific abstracts, we analyze 21.4 million papers (2010-2025; OpenAlex, PubMed). We additionally run a within-subject war-framing experiment ($N = 801$; 32{,}040 trials) designed to provide causal insight into the effects of militaristic
language on persuasion. Between 2010 and 2025, the presence of militaristic terms in scientific abstracts rose 48% in OpenAlex and 32% in PubMed, with the rise accelerating sharply after 2019 (cross-database $r = 0.96$, $p < 10^{-8}$). The prevalence of militaristic
language is conflict-aligned at both country and annual scales (Uppsala Conflict Data Program; $r = 0.77$-$0.84$), with the abstracts from the Global South displaying the fastest rise in militaristic
language. Among disciplines, social sciences leads in level of such
language while engineering and computer science lead in growth. The COVID and post-2022 large-
language-model eras also accelerated the rise and narrowed the
language gap between native-English-speaking and non-English-speaking countries. In our follow-up experiment, we found that war framing reduced credibility (mean shift $-0.176$ Likert units, 95% CI $[-0.21, -0.14]$; $d_z = -0.28$, $p < 10^{-20}$), funding willingness ($d_z = -0.12$) and policy support ($d_z = -0.08$), with a trend-level increase in sense of urgency ($d_z = +0.07$). Collectively, findings reveal that while scientific abstracts drift toward warfare, the use of militaristic
language may erode credibility, funding willingness, and policy support.
Read the full abstract ▾
Computation and Language
TriggerBench: Investigating Prospective Memory for Large Language Models
Tianhua Zhang, Xinjiang Wang, Qianxi Zhang et al. (+6 more)
While Large
Language Models (LLMs) are increasingly deployed in long interactions, existing evaluations focus predominantly on retrospective
memory (RM) via explicit queries. Prospective
memory (PM), the critical ability to spontaneously recall and act on latent constraints without direct
prompts, remains largely unevaluated. We introduce TriggerBench, a comprehensive PM
benchmark spanning five dimensions across both daily assistants and professional workflows. TriggerBench pairs scenarios with matched RM controls, contrastive positive/negative variants, and overloaded triggers, enabling fine-grained measurement of proactive recall, false-alarm rate, and attentional robustness under a single protocol. Our evaluation yields three key findings. (i) PM shows a precision-recall trade-off and attentional fragility. Though enhanced
reasoning significantly improves proactive recall, models may overfit to an "always-remind" heuristic. Furthermore, PM accuracy degrades substantially under implicit constraints or triggers overloaded by concurrent user requests, indicating that robust PM remains an open challenge. (ii) PM is notably harder than RM: on identical contexts, RM near-saturates up to 100K
tokens, while PM decays sharply as context length scales. (iii) PM may serve as a behavioral probe of spare
reasoning capacity. Pairing PM scenarios with AIME-2025 math problems reveals that successful trajectories yield higher PM accuracy than failed ones at the same context length, showing PM tracks spare
reasoning budget that
token count obscures. Project page: https://github.com/KristenZHANG/TriggerBench-Official.
Read the full abstract ▾
All authors Tianhua Zhang, Xinjiang Wang, Qianxi Zhang, Qi Chen, Kun Li, Yaoqi Chen, Dingdong Wang, Helen Meng, Yan Lu
Computation and Language
UnBias-Plus: Detect, Explain, and Rewrite Bias
Ahmed Y. Radwan, Ahmed ElKady, Sindhuja Chaduvula et al. (+3 more)
Bias in natural
language remains a persistent challenge in both human-written and AI-generated content, affecting domains such as journalism, education, and AI research. Most existing detection methods identify only the presence of bias, with limited support for granular detection, interpretable explanations, neutral rewriting, and openly available trained models. We present UnBias-Plus, an open-source toolkit unifying (1) segment-level multi-class bias classification, (2) biased span localization, (3) neutral text rewriting, and (4)
reasoning for each decision. Available via Python, CLI, REST API, and web interfaces, UnBias-Plus supports accessible bias analysis. The toolkit, source code, models,
datasets, and documentation are publicly available.
Read the full abstract ▾
All authors Ahmed Y. Radwan, Ahmed ElKady, Sindhuja Chaduvula, Mohamed Hafez, Amrit Krishnan, Shaina Raza
Computation and Language
ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Models
Jun Zhang, Jiasheng Zheng, Boxi Cao et al. (+5 more)
The emergence of Large
Reasoning Models has introduced exceptionally long Chain-of-Thought traces, creating a transparency burden where critical logic is often buried under massive procedural text. To address this, we present ReasoningLens, an open-source framework designed for the hierarchical visualization and diagnostic
auditing of complex
reasoning chains. ReasoningLens addresses information necropsy by: (1) structuring traces into interactive hierarchies that separate high-level strategy from low-level execution; (2) leveraging an agentic auditor for automated error detection and tool-augmented verification; and (3) synthesizing systemic
reasoning profiles to reveal model-specific blind spots. By transforming unstructured walls of text into actionable insights, ReasoningLens provides a modular foundation for interpreting, debugging, and optimizing the next generation of
reasoning-centric AI.
Read the full abstract ▾
All authors Jun Zhang, Jiasheng Zheng, Boxi Cao, Yaojie Lu, Hongyu Lin, Jia Zheng, Xianpei Han, Le Sun
Computation and Language
Do LLM Embedding Spaces Recover Expert Structure?
Yixuan Zhu, Zhenke Duan and Fanghen Li
Pretrained text
embeddings are increasingly used as representational maps, yet high category separability does not imply that their geometry recovers expert-defined structure. We study this problem in mental-health-related
language, where symptom relations provide an external reference and online communities introduce strong domain, affective, stylistic, and discourse confounds. Using 28 Reddit communities, we compare pretrained and supervised fine-tuned Qwen3
embedding spaces at two scales (0.6B and 4B). We construct category prototypes, evaluate their representational dissimilarity
matrices against an expert symptom
matrix with representational similarity analysis, and complement this global test with prototype-based typicality and multi-baseline confound controls. Pretrained
embeddings show measurable
alignment with expert structure within the mental-health subset; fine-tuning strengthens this
alignment most at the finest category level; and larger scale improves both zero-shot
alignment and supervision-induced gains. Residual
alignment remains substantial after controlling for VAD, LIWC, lexical style, and topic-distribution structure. These results suggest that LLM
embeddings can recover expert-relevant category geometry, but this recovery is level-dependent and should be tested against explicit confounds rather than inferred from classification alone.
Read the full abstract ▾
Computation and Language
Self-Stigma Is Not a Monolith, but Generic Empathy Is: Persona-Conditioned LLM Support for People Who Use Drugs
Layla Bouzoubaa and Rezvaneh Rezapour
Self-stigma predicts treatment avoidance and disengagement among people who use drugs (PWUD), yet conversational systems aiming to provide support typically treat self-stigma expression as a uniform signal. We present a three-phase, proof-of-concept study of a persona-aware approach to LLM support. Latent Profile Analysis (LPA) on indicator-level features from 1,174 self-stigma expressors on Reddit yields a four-persona typology validated against held-out behavioral and linguistic features. Sequential
Bayesian and recurrent neural classifiers recover these personas from limited posting histories, substantially outperforming batch and few-shot LLM baselines (macro-F1 = 0.74 at 30 posts). Evaluation by eight clinical experts across three contemporary LLMs revealed a misalignment: persona-matched responses successfully achieved targeted behavioral shifts, yet raters holistically preferred the generic empathy of the persona-neutral baseline. Our findings suggest that holistic empathy judgments and clinically-aligned response design can pull in opposite directions, and that evaluating LLM-based stigma support requires rubrics capable of decomposing the two.
Read the full abstract ▾
Computation and Language
Energy-Based Transformers as Predictors of Reading Difficulty
Jakub Dotlacil and Ece Takmaz
Transformer language models have become established tools for modeling human sentence processing, with measures such as surprisal and
attention entropy serving as effective predictors of reading difficulty that together capture complementary aspects of processing load. Here, we explore a related class of
transformer models: energy-based
transformers, which provide a principled formal link to associative
memory models, bringing processing research into direct contact with the broader literature on Hopfield networks and dense associative
memory. To our knowledge, this is the first exploration of an energy-based
transformer measure in computational psycholinguistics. Across reading-time corpora (Natural Stories, UCL eye-tracking, UCL self-paced reading), the energy measure is a robust predictor of reading times, providing significant fit beyond surprisal and
entropy in all three. In a controlled experiment on relative clause processing, energy at a single layer captures the well-known object/subject asymmetry. We find evidence that it subsumes effects attributable to both
attention entropy and surprisal, suggesting that energy may serve as a single unified predictor where multiple complementary measures have previously been required.
Read the full abstract ▾
Computation and Language
Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts
Arthur Wuhrmann, Gaetan Stein, Daniel Brunner et al. (+1 more)
While the wider applicability of LLMs in the legal field is currently debated due to their reliability and the
gravity of any errors, narrow uses with well-understood and mitigated risks have emerged. Notably the Swiss Federal Supreme Court uses small on-premises models for tentative translations and short-passage summarization across the four official
languages. However, such usage is challenging in the context of Criminal Law. Since rulings and cases employees work on routinely can contain detailed descriptions of violent and sexual offenses, their legitimate work is compromised by refusals and disclaimers due to the activation of model guardrails (over-
alignment). To measure this phenomenon, we introduce TF-RefusalBench, a
multilingual benchmark for criminal-law translation and summarization derived from public Swiss Supreme Court rulings. TF-RefusalBench contains 5,200 total
prompts across French, German, Italian, and English, corresponding to common task
prompts and passages likely to trigger refusal. We then use TF-RefusalBench to show that over-
alignment is a multifaceted phenomenon, influenced by the model and the
prompt and text
languages being processed, and that its impact cannot be evaluated solely from an over-refusal perspective, given the disclaimer's impact on task faithfulness. Finally, we evaluate approaches to enable on-premises LLMs for Criminal Law Tasks, demonstrating that while prompting can be effective, abliteration (refusal directions ablation) eliminates refusal with minimal impact on task performance.
Read the full abstract ▾
All authors Arthur Wuhrmann, Gaetan Stein, Daniel Brunner, Andrei Kucharavy
Computation and Language
WaveDetect: Robust Framework for Machine-Generated Text Detection via Wavelet Transform
Zhichen Liu, Kaitong Qin, Linhan He et al. (+1 more)
As Large
Language Models asymptotically approach human-level fluency in natural
language generation, solely relying on surface-level semantic artifacts for detecting LLM-generated texts has become increasingly precarious. Existing detectors often falter when facing three critical challenges:
adversarial perturbations, cross-domain shifts, and the rapid temporal
evolution of the foundation model. To address these issues, we propose \wavedetect, a novel framework that reformulates text detection as a signal processing task within the time-frequency domain. Unlike previous methods that analyze static
token probability distributions, \wavedetect models the generated output as a probability signal, upon which a differentiable Continuous Wavelet Transform is applied to convert them into learnable spectral representations. This process reveals the intrinsic “spectral fingerprints” in machine-generated texts–patterns that remain invisible in time domain. Comprehensive evaluations on three well-curated
datasets (RAID, EvoBench, and Domain-Shift) show that our method achieves a new state-of-the-art. It not only achieves superior accuracy but also exhibits remarkable robustness against sophisticated attacks, generalization across out-of-distribution topics and unseen evolving LLMs. Our results validate the efficacy of spectral analysis as a promising paradigm for LLM-generated texts detection.
Read the full abstract ▾
All authors Zhichen Liu, Kaitong Qin, Linhan He, Yang Xu
Computation and Language
Tmax: A simple recipe for terminal agents
Hamish Ivison, Junjie Oscar Yin, Rulin Shao et al. (+3 more)
Terminal-using
agents have quickly become the most popular downstream application of
language models (LMs). Despite their prevalence, relatively little academic work has examined RL-based training of these models, likely due to difficult
benchmarks, a lack of data, and a lack of simple baseline recipes. We present Tmax, the strongest open RL recipe for terminal
agents to date, bringing open data recipes closer to the frontier. While simple, our recipe achieves 27% on Terminal-Bench 2.0 with only 9B parameters, outperforming much larger models from prior work. Concretely, we generate data using a novel taxonomy, combining difficulty control, personas, and verifier diversification, which allows us to cheaply generate large amounts of terminal environments for RL and SFT training. We open-source our terminal
dataset, which is over 2.5x larger than previously released terminal-
agent datasets. We then train open-weight models using RL with our data, using a simple, outcome-only recipe. We release our data, models, and code as a strong baseline for future open academic work on terminal
agents at https://github.com/hamishivi/tmax.
Read the full abstract ▾
All authors Hamish Ivison, Junjie Oscar Yin, Rulin Shao, Teng Xiao, Nathan Lambert, Hannaneh Hajishirzi
Computation and Language · Cross-listed from cs.CY
Uncertainty-based Debiasing and Unlearning for Decontamination
Guangzhi Sun, Xiao Zhan and Mark Gales
Benchmark-based evaluation is the dominant paradigm for assessing large
language model (LLM) capabilities, yet data contamination inflates reported performance and undermines fair comparison. Existing decontamination methods are evaluated solely through aggregate accuracy, which can obscure substantial differences in per-sample model behaviour, and many require access to an uncontaminated model. In this paper, we propose a sample-level evaluation framework for decontamination that complements accuracy-based assessment with distributional distance metrics, measuring how closely a decontaminated model recovers the output distribution of an uncontaminated model on each sample. Building on this framework, we introduce Uncertainty-Based Decontamination (UBD), a family of methods that leverage deep ensembles of the contaminated model to estimate per-sample memorization without requiring a uncontaminated model or knowledge of which samples are contaminated. UBD estimates a per-sample correction scalar from ensemble uncertainty, which is used to construct a debiased target distribution that suppresses the inflated probability mass on correct answers induced by contamination. This target is then used either as a post-hoc output correction (debiasing) or as a soft training signal for parameter update (unlearning). Experiments on MMLU-Pro and MATH-MCQA across multiple LLM backbones demonstrate that UBD produces per-sample output distributions substantially closer to those of an uncontaminated model than paraphrasing or choice-permutation baselines, while preserving model performance on uncontaminated data.
Read the full abstract ▾
Computation and Language
The Anatomy of the CTC Oracle Gap: Acoustic Exhaustion and Linguistic Recovery
Ivan Novosad
We study the limits of CTC-internal scoring for N-best hypothesis selection and locate the information bottleneck separating acoustic confidence from linguistic plausibility. Eleven CTC-internal and acoustic-feature scoring strategies produce no statistically significant WER improvement over greedy decoding on LibriSpeech dev-other at G=16 (all p > 0.05). The exhaustion is systematic: CTC's Spearman $\rho$ between hypothesis score and per-utterance WER degrades from -0.574 at G=4 to -0.270 at G=128, a 53% loss driven by blank-path proliferation. This establishes that the discriminative capacity of CTC-internal representations is saturated: no recombination of acoustic signals can close the oracle gap. Confirming that the bottleneck is linguistic, not acoustic, external linguistic information introduced via MBR decoding breaks through it. MBR-CER decoding with a RoBERTa pseudo-log-likelihood (PLL) posterior ($\tau$=10, G=128) achieves 5.42% WER on held-out LibriSpeech test-other (greedy 5.96%, $\Delta$=-0.535 pp, p<0.0001, 9.0% relative). RoBERTa PLL $\rho$ degrades only 21% over the same range, retaining discriminating power where CTC loses it. Applied without retuning across two Zipformer architectures, three domains (LibriSpeech, TED-LIUM 3, VoxPopuli), and four MUSAN noise levels, the recipe gives significant gains in 11 of 13 conditions. On the training side, standard MWER training via the CTC forward-backward
algorithm implements Rao-Blackwellized REINFORCE at the output projection (variance about 3x below Viterbi). Yet sequence-level fine-tuning fails at near-converged checkpoints: all four MWER configurations on CR-CTC collapse (+6.18 to +8.90 pp WER), as a training oracle gap of 0.007 pp provides no usable reward signal.
Read the full abstract ▾
Computation and Language
On the Effect of Segmentation Width and Cluster Size on Speech Resynthesis and Continuation in Generative Spoken Language Models
Shunsuke Kando, Wataru Nakata, Shinnosuke Takamichi et al. (+1 more)
Generative Spoken
Language Modeling (GSLM) enables text-free
speech modeling by training
language models (LMs) using discrete
speech representations instead of textual transcription. In this paper, we investigate the performance of GSLM on
speech synthesis and continuation using discrete
speech representations with varying bitrates. We segment
speech representations with fixed widths and train K-means models in multiple cluster sizes, resulting in various bitrate settings. We demonstrate that intelligible and natural
speech can be synthesized at lower bitrate settings than the baseline. Furthermore,
speech continuation quality remains stable at lower bitrates across multiple metrics, suggesting that the conventional GSLM setting may be redundant for effective
speech generation. Although LLM-based metrics show higher correlation with human subjective score than conventional metrics, it remains low, highlighting the need for more stable automatic evaluation methods.
Read the full abstract ▾
All authors Shunsuke Kando, Wataru Nakata, Shinnosuke Takamichi, Yusuke Miyao
Computation and Language
Towards Root Memory: Benchmarking and Enhancing Implicit Logical Memory Retrieval for Personalized LLMs
Hongxun Ding, Xiang Yu, Chengbing Wang et al. (+4 more)
Memory systems are essential for personalized Large
Language Models (LLMs). However,
retrieval within these systems primarily relies on semantic similarity, potentially missing logically critical
memories with limited semantic overlap. We refer to this failure as semantic deflection, which remains insufficiently evaluated by existing
benchmarks. To address this gap, we construct IMLogic, a
benchmark for implicit logical
memory retrieval in long dialogues. Each question is designed around the divergence between semantic and logical relevance, with diagnostic response options that probe semantic deflection,
hallucination, and lack of personalization. To help systems utilize associations that are difficult to discover through semantic matching, we introduce root
memory, a structured representation that distills reusable, response-guiding logic from user histories through LLM
reasoning and grounds it in supporting evidence. We further develop RootMem, a plug-and-play framework that constructs and routes root
memories to complement semantic
retrieval. Extensive experiments reveal persistent challenges across
retrieval methods and
memory systems, including substantial preference-driven errors even under full-
memory access. RootMem consistently improves performance, demonstrating the potential of structured personalized logic while leaving considerable challenges unresolved. Our
benchmark and code are available at https://github.com/ThisIsCosine/IMLogic.
Read the full abstract ▾
All authors Hongxun Ding, Xiang Yu, Chengbing Wang, Jianfei Xiao, Keqin Bao, Wenjie Wang, Xiangnan He
Computation and Language
Scaling LLM Knowledge Boundaries via Distribution-Optimized Synthesis
Songze Li, Yarong Lan, Zhongpu Bo et al. (+16 more)
Knowledge injection via synthetic data is crucial for enhancing Large
Language Models (LLMs). However, current synthesis methods simply stop at preset
token counts or fixed data ratios, lacking awareness of knowledge distribution. This results in some domains being sparse while others are redundant, limiting LLM knowledge boundaries. We revisit knowledge injection from a distribution perspective and hypothesize that an optimal knowledge distribution exists to maximize knowledge boundary expansion. We propose KDoS (Knowledge Distribution-optimized Synthesis), a framework that introduces knowledge density to drive synthesis through a three-stage feedback mechanism, shifting from blind generation to distribution-optimized synthesis. We construct Wikipedia-based synthetic data with varying knowledge distributions and conduct experiments on models from 0.6B to 16B (Qwen, Ling, LLaMA) and data scales from 1B to 5B
tokens. Our key findings are: (1) an optimal knowledge distribution consistently maximizes boundary expansion; (2) this distribution is stable across backbones and scales; (3) KDoS outperforms baselines across six knowledge
benchmarks. Our work offers a new perspective and practical framework for synthetic data-driven knowledge injection.
Read the full abstract ▾
All authors Songze Li, Yarong Lan, Zhongpu Bo, Zhaoyang Wang, Zhiqiang Liu, Yuan Yuan, Chengtao Gan, Menghao Qian, Enpei Niu, Xiaoke Guo, Yuanxiang Liu, Zhaoyan Gong, Xiangjin Hu, Liangyurui Liu, Jingdian Lu, Lei Liang, Jun Zhou, Huajun Chen, Wen Zhang
Computation and Language
Judgment-Grounded Expansion for Peer Review Generation
Sheng Lu, Lizhen Qu and Iryna Gurevych
Automatic review generation is a promising direction for accelerating scientific progress. While most work adopts an end-to-end setup, its fully automated nature makes it less suitable for settings that demand accountability. To better balance automation and accountability, we formalize judgment-grounded expansion, a human-AI collaboration mode where a reviewer provides an evaluative claim and the system expands it into review comment candidate(s). We model it as a structured generate-check-refine process and conduct a user study to collect human-model interaction data. We study two practical challenges for judgment-grounded expansion: scalable evaluation and candidate set curation. We develop methods to simulate the process for large-scale evaluation, and show that conformal prediction is well suited to balancing candidate set size and target coverage. Our work establishes judgment-grounded expansion as a concrete task and provides empirical and methodological foundations for the design of future collaborative review generation systems.
Read the full abstract ▾
Computation and Language
MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversations
Elena Sofia Ruzzetti, Cornelius Emde, Sangdoo Yun et al. (+2 more)
LLM
agents are increasingly deployed in multi-party environments, handling sensitive personal data on behalf of individual users, for instance in group chats. When such an
agent discloses private information, it reaches every group member at once. This risk is structurally harder to control than in one-to-one settings, as every piece of private information must be appropriate for every recipient in the group. Yet all existing contextual
privacy benchmarks consider only single-interlocutor settings, leaving multi-party
privacy risks unmeasured. We introduce MuPPET (Multi-Party
Privacy Exposure Testing), a
benchmark for contextual
privacy in multi-party conversations. Our experiments show that models leak substantially more in multi-party settings than one-to-one evaluations suggest. Frontier models are vulnerable, and smaller open-weights models, often preferred for local deployment with sensitive data, even more so. Existing contextual
privacy defences offer only partial protection, degrade utility, and do not resolve the underlying party-tracking problem.
Read the full abstract ▾
All authors Elena Sofia Ruzzetti, Cornelius Emde, Sangdoo Yun, Seong Joon Oh, Martin Gubri
Computation and Language · Cross-listed from cs.CV
CFPO: Counterfactual Policy Optimization for Multimodal Reasoning
Zhangyuan Yu, Wanran Sun, Guangjing Yang et al. (+2 more)
Large
Vision-
Language Models (LVLMs) have demonstrated remarkable capabilities in
multimodal reasoning. However, prevailing reinforcement learning (RL) paradigms lack explicit counterfactual enhancement and causal learning mechanisms. This fundamental deficiency results in severe grounding failures, manifesting as a tendency to ignore visual evidence in favor of
language priors or exhibiting
hallucination drift during long chain-of-thought
reasoning. To address this root cause, we propose CounterFactual Policy Optimization (CFPO), a novel framework that enforces causal consistency between visual perception and textual
reasoning. CFPO introduces a cross-modal counterfactual enhancement mechanism, which regularizes the policy by maximizing the discrepancy between the model's predictions and those from a counterfactual state where critical visual cues are suppressed. This approach seamlessly integrates with standard
algorithms like GRPO and DAPO without requiring external reward models or additional supervision. Extensive experiments demonstrate that CFPO significantly improves
reasoning fidelity, achieving consistent gains of 3.17%-6.25% over standard RL baselines and 1.32%-2.13% over the state-of-the-art perception-aware method (PAPO). Code is available at https://github.com/Raven-July/CFPO.
Read the full abstract ▾
All authors Zhangyuan Yu, Wanran Sun, Guangjing Yang, Xiaohu Wu, Qicheng Lao
Computation and Language
When Does Intrinsic Self-Correction Help? A Task-Sensitive Analysis
Elroy Stav, Dvir Berlowitz, Maayan Orner et al. (+1 more)
Intrinsic self-correction (SC) aims to improve large
language model outputs by prompting a model to revisit its own initial answer without external feedback. Recent studies have questioned the reliability of this approach, showing that models often struggle to judge whether their initial responses are correct. In this work, we take a task-sensitive view of SC. Rather than asking whether it works in general, we examine settings where SC may operate through different mechanisms: verifying explicit constraints, revisiting a complex
reasoning process, or providing a second opinion over competing strategies in word-game tasks. Across multiple
benchmarks and models, we find that SC can yield consistent performance gains when the underlying task structure facilitates these modes of revision. These results suggest that SC is best understood as a task-dependent inference-time strategy whose usefulness depends on the role the revision stage can play in a given task, rather than as a uniformly reliable method for improving initial model outputs.
Read the full abstract ▾
All authors Elroy Stav, Dvir Berlowitz, Maayan Orner, Sarit Kraus
Computation and Language · Cross-listed from cs.LG
Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory
Zewen Liu
Large
Language Model (LLM)
agents increasingly rely on
memory systems to maintain long-term coherence. Recent work shows that
agent memories degrade during continuous consolidation. However, existing research assumes
memories are derived from unbiased experiences. In this work, we identify and formalize a novel phenomenon:
Memory Contagion – the cross-temporal propagation of evaluator bias through
agent memory. We show that when
agents are trained or guided by biased evaluators, their experiences become biased; when these trajectories are stored and consolidated into
memory, the bias propagates to future
agents retrieving from the same
memory store, even when consolidation is perfect (oracle). Across two bias types (length preference, authority bias) and four experimental phases, we demonstrate: (1)
Memory Contagion occurs for length bias even with perfect consolidation on older models (Gamma_A = 13.18, DeepSeek V4-Chat), while newer models (V4-Pro, Claude) are immune, proving both that biased input is a sufficient cause and that contagion is model-generation-dependent; (2) authority bias fails to propagate in all 15 controlled multi-seed experiments (Gamma_A = 0.00), revealing that not all evaluator biases can cross temporal boundaries through current
memory architectures; (3) No observed safe threshold: length bias propagation is detected at contamination rates as low as p=0.2. Our findings expose a critical but contingent vulnerability in current
agent memory designs and provide formal tools for measuring cross-temporal bias propagation.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?
Anmol Goel and Iryna Gurevych
Computer-use
agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-application access is useful, but it also creates a
privacy risk that has been largely overlooked: when an
agent works in one context, it can pull in information from another that is inappropriate in that context. Hence, we introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios. We target three common failure modes in CUAs: visual co-location, where the
agent pulls in prohibited items that sit next to the task target in the UI; task-ambiguity overshare, where the
agent dumps dense personal state in response to an under-specified
prompt; and recipient misalignment, where the
agent sends content to an addressee for whom it is inappropriate. We evaluate 15 frontier
agents and find a surprisingly high failure rate: 11 of 15 leak on more than 50% of scenarios, with an average leakage of 67.9%, and the same failures persist when
agents act end-to-end in the environment to complete the task. We release AgentCIBench to encourage the development of safer computer-use
agents and position contextual disclosure testing as a pre-deployment safety check.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
DART: Draft-Agreement Routing for Training-Free Adaptive Thinking Budgets in Hybrid Reasoning Models
Jungseob Lee, Seongtae Hong, Seungjun Lee et al. (+7 more)
Hybrid
reasoning models can answer directly or spend extra
tokens on extended thinking. A practical router should choose between these modes for each query, so easy problems avoid unnecessary
reasoning and hard problems receive enough budget to finish the answer. Existing routers move in this direction, but they typically require labeled training data or fix thinking budgets up front, ignoring answer-level evidence from the model itself. We introduce DART, a training-free routing framework that samples two cheap no-think drafts, accepts direct answering when the drafts agree, and predicts a thinking budget from draft
entropy when they disagree. Across the main comparisons, DART preserves or improves always-thinking accuracy in most settings while reducing thinking-
token use. Accuracy improves by up to +9.0 points on Olympiad-level math and by up to +22.5 points on code under execution-based equivalence, while thinking-
token use drops by 32-73%. The Stage~1 signal extends across model scales (0.6B–32B), model families, and API-only hosted settings, with no labeled data and no gradient updates required. Our code is available at https://github.com/js-lee-AI/DART.
Read the full abstract ▾
All authors Jungseob Lee, Seongtae Hong, Seungjun Lee, Jaehyung Seo, Junyoung Son, Sugyeong Eo, Chanjun Park, Hyeongju Park, Hyeonseok Moon, Heuiseok Lim
Computation and Language · Cross-listed from cs.SD
Synthesizing the Lombard Effect: Multi-Level Control of Speech Clarity and Vocal Effort in TTS
Seymanur Akti and Alexander Waibel
Humans tend to speak louder and clearer in challenging environments, such as noisy conditions or when addressing hearingimpaired listeners, which is called Lombard effect. To simulate this behavior in
speech synthesis systems, we introduce a flow-matching based text-to-
speech (TTS) model trained with vocal effort and articulation pseudo-labels. The proposed model achieves continuous and disentangled control of vocal effort and articulation, while also enabling word-level emphasis for clarifying specific segments of an utterance. Experimental results show that these control mechanisms effectively improve clarityrelated acoustic features. Furthermore,
speech-in-noise experiments demonstrate that our model successfully simulates the intelligibility gains of human clear
speech in noisy conditions.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
The Language Blind Spot: How Query Language and Brand Recognition Tier Shape AI-Constructed Brand Reputation Across Twelve European Languages
Dmitrij Żatuchin
Large
language models (LLMs) increasingly mediate how people form impressions of organisations, yet most monitoring is done in English, assuming an English query returns a representative picture. We measure how far that holds. We queried three grounded LLMs (GPT-5.4, Gemini 3.1 Pro, Perplexity Sonar Pro) about 66 brands from eleven Northern, Baltic, and Central European
markets, in twelve
languages across four families (Germanic, Uralic, Baltic, Slavic), generating 35,640 responses.
Multilingual embeddings (BGE-M3) allow cross-
language comparison without translation. Three results emerge. First, AI-constructed reputation is
language-bound: mean cross-
language cosine similarity is 0.825, same-family responses are more similar than cross-family (0.844 vs 0.820; d = 0.31), and sentiment varies by
language (F = 268.5, eta^2 = 0.077), with Uralic and Baltic
languages most positive and Germanic, including English, most critical; clustering recovers the Slavic and Baltic families (cophenetic 0.915). Second, query
language shifts which brands are recommended far more than how they are described: moving from an English query to a brand's home
language raises recommendation share by 0.80 for local champions but only 0.15 for global multinationals (t = -8.84, p < 0.001), with no comparable reversal in sentiment. An English-only audit therefore understates a local champion's AI visibility. Third, response stability varies more with model choice than with
language (eta^2_model = 0.32 vs eta^2_
language = 0.01, on a five-iteration replication over a 20-brand subset). These results indicate that English-only AI reputation monitoring leaves a measurable
language blind spot, concentrated in the visibility of locally headquartered brands.
Read the full abstract ▾
Computation and Language
Same question, different history: language, national identity, and credit in large language models
William Guey, Pierrick Bougault, Wei Zhang et al. (+2 more)
Who invented the
radio, Russia's Alexander Popov or Italy's Guglielmo Marconi? Was the telephone the achievement of Bell in the United States or Meucci in Italy? Does printing belong to China's Bi Sheng or Germany's Gutenberg? The answer depends not only on historical record but also on
language and perspective. We analyse eleven widely used large
language models across 21 disputed inventions and discoveries, evaluated in twelve
languages and 75,896 responses. While models generally acknowledge that credit is contested, query
language systematically affects which claimant is surfaced. Lower-status claimants are more likely to appear when questions are asked in their associated
language, whereas dominant Anglophone figures remain stable across
languages. These patterns persist after controlling for response length, model differences, historical prominence, and levels of national commemoration.
Language thus acts as a switch that activates different national versions of the same history, producing systematically different national
memories from the same question. We interpret this as evidence that large
language models function as distributed systems of cultural
memory, where
language conditions which histories become visible, contributing to a computational form of banal nationalism.
Read the full abstract ▾
All authors William Guey, Pierrick Bougault, Wei Zhang, Vitor D. de Moura, José O. Gomes
Computation and Language · Cross-listed from cs.CV
Koshur Pixel: a large-scale synthetic ocr dataset for kashmiri
Haq Nawaz Malik, Faizan Iqbal and Nahfid Nissar
Optical Character Recognition (OCR) for low-resource
languages is often constrained by the lack of annotated training data and the complexity of script-specific rendering. Kashmiri, written primarily in the Perso-Arabic Nastaliq script, presents additional challenges due to contextual glyph shaping, dense ligatures, and orthographic variability. We introduce Koshur Pixel, the first large-scale synthetic OCR
dataset for Kashmiri, comprising 613,078 image-text pairs generated from the KS-PRET-5M corpus using the SynthOCR-Gen framework. The
dataset spans multiple fonts and textual granularities, ranging from individual words to full-page documents, and incorporates more than 25 augmentation strategies that emulate real-world document degradations. Koshur Pixel provides a scalable and cost-effective alternative to manual annotation, establishing a foundational resource for training OCR systems, digitizing Kashmiri textual heritage, and advancing
language technologies for a severely under-resourced
language.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Managing Procedural Memory in LLM Agents: Control, Adaptation, and Evaluation
Julia Belikova, Rauf Parchiev, Evgeny Egorov et al. (+4 more)
Procedural
memory is increasingly used to improve LLM
agents on recurring workplace tasks, yet its ability to produce reusable skills remains poorly understood. We introduce AFTER, a
benchmark of 382 realistic enterprise tasks spanning six professional roles and 22 procedural skills, designed to evaluate how skills transfer across tasks, roles, and model backbones. The
benchmark includes controlled evaluation settings for local improvement, cross-task transfer, cross-role transfer, and cross-model generalization. Experiments show that procedural
memory delivers consistent gains in industrial workflows: a single refinement round improves aggregate performance by 3.7-6.7 points, while skills evolved from diverse multi-model execution traces achieve 73.1% cross-model test accuracy, outperforming all single-model trace sources. We further find that some skills generalize broadly across tasks and models, whereas others become specialized to role-specific workflows and lose effectiveness under transfer. These results provide practical guidance for building, evaluating, and deploying procedural
memory systems in production
agent platforms.
Read the full abstract ▾
All authors Julia Belikova, Rauf Parchiev, Evgeny Egorov, Grigorii Davydenko, Gleb Gusev, Andrey Savchenko, Maksim Makarenko
Computation and Language
PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation
Jiaqiang Wu, Zhouan Zhu and Shangfei Wang
Large
language models have demonstrated significant capabilities in generating diverse and context-aware responses for empathetic dialogue. However, their computational demands severely limit their deployment in resource-constrained environments. While knowledge
distillation offers a promising compression solution, it often fails to transfer the nuanced understanding essential for empathy, as it overlooks the implicit contextual cues that guide human connection. To bridge this gap, we propose a privileged information-enhanced knowledge
distillation method for empathetic dialogue generation (PRIDE). Our method leverages privileged information, such as expert psychological annotations or future event summaries, which is available exclusively during training but unavailable at inference time. This allows us to transfer the teacher model's empathetic
reasoning to smaller models without relying on extra inputs during deployment. Specifically, PRIDE has three key components: (1) An empathy-
reasoning prompt that guides the teacher to explicitly decompose the empathetic process into understanding feelings and analyzing situations step-by-step; (2) A multi-source
attention mechanism that directs the student to effectively integrate privileged information; (3) A dual-
alignment loss that combines reversed Kullback-Leibler divergence and maximum mean discrepancy to ensure robust knowledge transfer at both logit and feature levels. Experiments on multi-modal and text-only
datasets demonstrate that our method achieves competitive performance, and in some cases matches or even surpasses larger teacher models in terms of accuracy and semantic relevance.
Read the full abstract ▾