Computation and Language
ReadingMachine: A Computational Methodology for Structured Corpus Reading and Large-Scale Synthesis
James Morrissey
ReadingMachine is a computational methodology for structured corpus reading that uses large
language models to perform bounded reading operations over entire document collections. Rather than relying on
retrieval or recursive summarization, the approach decomposes analysis into inspectable stages including insight extraction, semantic clustering, theme generation, and iterative omission detection. By delaying irreversible compression and explicitly tracking intermediate representations, the method prioritizes coverage, traceability, and preservation of disagreement across large corpora. The system is demonstrated on a heterogeneous corpus of 152 industrial policy documents, producing more than 17,500 extracted insights and a structured thematic map. ReadingMachine is released as an open-source experimental framework for large-scale qualitative synthesis and corpus analysis.
Read the full abstract ▾
Computation and Language · Cross-listed from quant-ph
Benchmarking Quantum Algorithmic Resilience for CVaR Portfolio Optimization: The Expressibility-Coherence Trade-off
Prashik N. Somkuwar, K. Srinivasan and G. Raghavan
Quantum combinatorial optimization offers theoretical advantages for complex financial modeling, but physical implementation on Noisy Intermediate Scale
Quantum (NISQ) devices is severely constrained by hardware
topology. This study presents a hardware benchmarking analysis between a Hardware Efficient Variational
Quantum Neural Network (HE-VQNN) and the Warm Start
Quantum Approximate Optimization
Algorithm (WS-QAOA) for a hybrid Mean Variance and Conditional Value at Risk (CVaR) portfolio objective. By implementing a novel classical
quantum hybrid proxy
matrix to bypass the CVaR auxiliary
qubit bottleneck, we map up to 16 assets from the NIFTY 50 index onto an IBM heavy hex processor. We systematically quantify algorithmic resilience to the "SWAP tax" incurred during routing. Empirical results reveal a critical operational trade-off: WS-QAOA provides exact theoretical mapping but suffers catastrophic hardware decoherence due to exponential nonlocal gate overhead. Conversely, HE-VQNN preserves hardware coherence but lacks the mathematical expressibility to capture dense tail risk asset correlations. This study exposes the limitations of dense financial optimization on current architectures forces an nonviable choice between algorithmic inexpressibility and hardware decoherence. This is indicative of a deeper limitation as to what can and cannot be done with NISQ computers lacking in all-to-all connectivity.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Why Limit the Residual Stream to Layers and Not Tokens? Persistent Memory for Continuous Latent Reasoning
Mujtaba Farhan and Maheep Chaudhary
Large
language models (LLMs) have demonstrated remarkable
reasoning abilities on mathematical and multi-hop planning tasks. The CoCoNuT (Chain of Continuous Thought) paradigm~\cite{hao2024coconut} extends this by enabling models to reason in latent space, exploring multiple
reasoning paths simultaneously rather than committing to a single chain early on. However, we identify a limitation we term the concept bottleneck. At each
reasoning pass, intermediate hidden states are overwritten, causing the model to lose critical facts computed in earlier steps as
reasoning depth increases. We observe this empirically. On HotpotQA, vanilla CoCoNuT (10.4% EM) fails to improve over the CoT baseline (11.0% EM), and performance degrades with curriculum depth on GSM8K. To address this, we propose AGCLR (Adaptive Gated Continuous Latent
Reasoning), which augments CoCoNuT with a Gated Concept Stream. A persistent residual
memory maintained across all
reasoning passes, controlled by three learned gates: a write gate that commits intermediate facts to
memory, a read gate that retrieves relevant prior states, and a forget gate that prunes irrelevant context. Evaluated on GSM8K, HotpotQA, and ProsQA using GPT-2 as our base model, AGCLR achieves consistent improvements across all types of
datasets. With the performance gap compounding as curriculum depth increases, directly resolving the concept bottleneck. Code available at https://anonymous.4open.science/r/JJJJ/README.md
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
How Much Dense Attention is Necessary? Oracle-Guided Sparse Prefill for Full/GQA Layers in Hybrid Long-Context Models
Hongxing Wang, Harenome Razanajato, Zhen Zhang et al. (+2 more)
Long-context prefill remains expensive because full/GQA layers still score the historical sequence, even in hybrid models with local, sparse, linear, or recurrent components. We study how much dense
attention is needed to preserve task-level behavior under explicit support granularity and top-k budgets. We introduce an
attention-mass top-k oracle for existing GQA checkpoints: for each layer and query position, it computes dense
attention, selects head-averaged
token support, and recomputes
attention only on that support. The oracle is a diagnostic reference, not a deployable accelerator, and separates sparse-budget feasibility from indexer error and runtime realization effects. On Qwen-family
retrieval-heavy evaluations, the longest per-query oracle rows stay within 1 point of dense, and a Qwen3.5-9B RULER-style sweep from 4K to 100K stays within 0.48 points. Guided by the oracle, we derive a head-collapsed auxiliary indexer trained by KL
distillation from dense
attention-mass distributions while keeping the backbone frozen. With separately distilled Qwen3.5-0.8B and Qwen3.5-9B indexers, the reported 16K/32K validation macro gaps are +2.04 and +1.13 points, treated as quality preservation rather than improvement; fused selection-block-shared support can introduce a larger realization gap. Preliminary single-card TTFT measurements show distilled-indexer sparse serving speedups of 1.71x for Qwen3.5-0.8B on NPU and 1.93x for Qwen3.5-9B on GPU against its dense FlashAttention-2 baseline. Additional random-init stress rows reach 3.44x, indicating sparse-runtime headroom but not validated output quality. This first release separates oracle feasibility, distilled-indexer quality, and runtime headroom, leaving a fully matched quality-latency frontier to future work.
Read the full abstract ▾
All authors Hongxing Wang, Harenome Razanajato, Zhen Zhang, Yujie Yuan, Hongsheng Liu
Computation and Language · Cross-listed from cs.IR
TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation
Ziheng Chen, Jiali Cheng, Zezhong Fan et al. (+4 more)
Generative recommendation formulates next-item prediction as autoregressive generation over semantic ID (SID) sequences derived from users' historical interactions, making modern recommender systems structurally similar to large
language models (LLMs). As
privacy and safety concerns grow, these systems increasingly require concept unlearning to remove sensitive or harmful concepts associated with items. However, existing LLM unlearning methods cannot be directly applied to generative recommendation. Unlike word
tokens with explicit semantics, SIDs are abstract identifiers that are often shared by both forget and retain items, leading to severe conflicts between concept removal and recommendation utility preservation. To address this challenge, we propose TRACER, an end-to-end concept unlearning framework based on
token reassignment. Rather than directly suppressing shared SIDs, TRACER reassigns concept-related items to alternative
tokens that better facilitate forgetting while minimizing side effects on retained items. We further introduce a coherence regularizer to preserve semantic consistency among retain items during unlearning. Experiments on real-world recommendation
datasets demonstrate that TRACER effectively removes target concepts while substantially better preserving recommendation utility than existing unlearning baselines.
Read the full abstract ▾
All authors Ziheng Chen, Jiali Cheng, Zezhong Fan, Hadi Amiri, Diyuan Wu, Gabriele Tolomei, Yang Zhang
Computation and Language
How reliable are LLMs when it comes to playing dice?
Luca Avena, Gianmarco Bet and Bernardo Busoni
We investigate the probabilistic
reasoning capabilities of large
language models through a controlled benchmarking study on discrete probability problems. We constructed two
datasets, respectively a set of standard exercises and a set of counterintuitive exercises, designed to trigger heuristic
reasoning, and evaluated 8 state-of-the-art models, each tested with and without Chain-of-Thought prompting. Models achieve an average accuracy of 0.96 on standard problems but only 0.59 on counterintuitive ones. We further provide empirical evidence of
token bias: performance drops by over 20% when canonical formulations are replaced by disguised variants.
Embedding misleading suggestions in the
prompt reduces performance by up to 34%, with no model proving immune. Taken together, the reported findings suggest that current LLMs are not yet genuine probabilistic reasoners, despite their success in advanced mathematical problems.
Read the full abstract ▾
Computation and Language
Agentopia: Long-Term Life Simulation and Learning in Agent Societies
Xintao Wang, Sirui Zheng, Hongqiu Wu et al. (+10 more)
Humans learn from social life. Simulating this process with LLM-powered
agents represents a promising research direction, raising a natural question: whether LLMs can learn from such simulated social experience to better understand and replicate human behavior. However, prior
agent society simulations typically operate at the scale of days, limiting the depth of social interactions and long-term growth. In this paper, we study long-term life simulation and LLM learning in
agent societies, with two goals: (1) investigating social behaviors that emerge from life-long simulation, and (2) developing anthropomorphic capabilities in LLMs, particularly intelligence in social life, through years of simulated social experience. Specifically, we present Agentopia, a comprehensive framework for long-term life simulation in multi-
agent societies, where 100
agents autonomously pursue personal growth, develop social relationships, and fulfill their needs and goals over 10 simulated years. We define life reward to mirror human well-being, and leverage this reward to train LLMs via rejection sampling. Extensive experiments show that
agents exhibit rich emergent social behaviors. Furthermore, life reward training effectively enhances the underlying LLM, which leads to improved
agent well-being in simulation, and generalizes to downstream role-playing
benchmarks with +15.6% improvement.
Read the full abstract ▾
All authors Xintao Wang, Sirui Zheng, Hongqiu Wu, Weiyuan Li, Jen-tse Huang, Minghao Zhu, Can Zu, Qi Deng, Jiawei Wang, Qianyu He, Heng Wang, Xiaojian Wu, Yunzhe Tao
Computation and Language · Cross-listed from cs.CV
MemDreamer: Decoupling Perception and Reasoning for Long Video Understanding via Hierarchical Graph Memory and Agentic Retrieval Mechanism
Cong Chen, Guo Gan, Kaixiang Ji et al. (+7 more)
Current
Vision-
Language Models struggle with hours-long
videos because processing full-length visual sequences induces prohibitive
token explosion and
attention dilution. To overcome this, we introduce MemDreamer to decouple perception and
reasoning, shifting long-
video understanding into an agentic exploration process. As a plug-and-play framework, it incrementally streams
videos to construct a Hierarchical Graph
Memory, a top-down three-tier architecture for semantic abstraction, anchored by a foundational graph capturing spatiotemporal and causal relations. During inference, the
reasoning model employs agentic tool-augmented
retrieval, navigating hierarchies, searching nodes, and traversing logical edges via an Observation-Reason-Action loop. Experiments show MemDreamer achieves SOTA results across four mainstream
benchmarks, narrowing the gap with human experts to only 3.7 points. It constrains the
reasoning context window to merely 2% of full-context ingestion while delivering a 12.5 point absolute accuracy gain. Furthermore, statistical analysis uncovers a strong positive linear correlation between an VLM's performance on logic
reasoning and long-
video understanding
benchmarks, establishing agentic capability scaling as a new paradigm for
multimodal comprehension.
Read the full abstract ▾
All authors Cong Chen, Guo Gan, Kaixiang Ji, ZhaoYang Zhang, Zhen Yang, Guangming Yao, Hao Chen, Jingdong Chen, Yi Yuan, Chunhua Shen
Computation and Language
Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings
Songhao Wu, Zhongxin Chen, Yuxuan Liu et al. (+3 more)
Large
language models exhibit impressive zero-shot capabilities across a wide range of downstream tasks. However, they struggle to function as off-the-shelf
embedding models, leading to suboptimal performance on massive text
embedding benchmarks. In this paper, we identify a potential cause underlying this deficiency. Our motivation stems from an unexpected observation: text
embeddings tend to align with frequent but uninformative
tokens when projected onto the vocabulary space. We argue that this excessive expression of high-frequency
tokens suppresses the model's ability to capture nuanced semantics. To address this, we introduce EmbedFilter, a simple linear transformation designed to refine text
embeddings derived from LLMs directly. Specifically, we uncover that the unembedding
matrix within LLMs encodes a latent space that is actively writing these frequent
tokens into
embedding space. By filtering out this subspace, EmbedFilter suppress the influence of high-frequency
tokens, thereby enhancing semantic representations. As a compelling byproduct, this enables an inherent dimensionality reduction, lowering index storage and speedup
retrieval while fully preserving the refined
embedding quality. Our experiments across multiple LLM backbones demonstrate that LLMs equipped with EmbedFilter achieve superior zero-shot downstream performance even with significantly reduced
embedding dimensions. We hope our findings provide deeper insights into the mechanisms of LLM-based representations and inspire more principled designs to improve text
embeddings training. Our code is available at https://github.com/CentreChen/EmbFilter.
Read the full abstract ▾
All authors Songhao Wu, Zhongxin Chen, Yuxuan Liu, Heng Cui, Cong Li, Rui Yan
Computation and Language
Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification
Sercan Karakaş and Yusuf Şimşek
Turkish idiomatic light verb constructions (LVCs) are challenging for multiword expression processing because they often share the same surface form as fully literal verb-object combinations while functioning as a single, partially idiomatic predicate. We frame Turkish LVC detection as a binary classification task (literal meaning vs. idiomatic meaning) and evaluate on a manually created controlled set (N=147) with matched negatives: out-of-domain random sentences and in-domain literal controls (NLVC), alongside LVC positives. We compare a supervised Turkish encoder baseline (BERTurk with a classifier head) to three instruction-tuned LLMs from different families under zero-shot, one-shot, and few-shot prompting, and analyze how demonstrations shift error profiles. In zero-shot, LLMs perform well on negatives but show very low LVC recall. One-shot prompting sharply improves LVC detection but can induce strong, model-specific biases, leading models to overpredict or underpredict LVCs. A richer few-shot
prompt improves calibration and yields robust overall performance for GPT-OSS-20B and Qwen 2.5-14B. Overall, the results highlight substantial
prompt sensitivity in Turkish metalinguistic classification: the supervised baseline remains competitive, while prompted LLMs can match or exceed it on LVCs with carefully constructed demonstrations.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment
Sweta Mahajan, Sukrut Rao, Jiahao Xie et al. (+2 more)
Vision-
language models such as CLIP are highly useful for diverse tasks due to their shared image-text
embedding space. Despite this, the image and text
embeddings are often poorly aligned, affecting downstream performance. Recent work has hypothesized that this can be attributed to an information imbalance: images contain more information than their captions describe. In this work, we propose TEVI, a framework that uses captions as a signal for what to retain from image
embeddings. Specifically, we use sparse autoencoders to disentangle image
embeddings and train a masking module to selectively reconstruct the
embedding based on a given caption. In a controlled setup with synthetic captions, we show that TEVI is effective at preserving caption-described attributes while discarding others. We find that this extends to CLIP models trained on natural images, where TEVI learns to mask meaningfully and allows
retrieval based on conditioning. Finally, we use TEVI to achieve improved
retrieval performance across coarse-grained and fine-grained
benchmarks. Code available at https://github.com/neuroexplicit-saar/TEVI.
Read the full abstract ▾
All authors Sweta Mahajan, Sukrut Rao, Jiahao Xie, Alexander Koller, Bernt Schiele
Computation and Language
Sycophantic Praise: Evaluating Excessive Praise in Language Models
Daniel Vennemeyer, Phan Anh Duong, Meryl Ye et al. (+2 more)
Sycophancy in
language models is typically studied as excessive agreement or validation, while explicit praise and flattery have received comparatively little
attention. We argue that sycophantic praise is a distinct
alignment problem that cannot be reliably measured using current methods. We introduce a parameterized framework that measures whether praise is excessive relative to contribution quality and expected user ability. We show that our framework substantially outperforms generic LLM judges in agreement with human annotations, and that sycophantic praise occurs far more frequently in social and interpretive domains than in objective
reasoning settings. Together, these findings position praise calibration as a distinct
alignment challenge.
Read the full abstract ▾
All authors Daniel Vennemeyer, Phan Anh Duong, Meryl Ye, Ruihong Huang, Tianyu Jiang
Computation and Language · Cross-listed from cs.CV
The Lipreading Gap: Do VSR Models Perceive Visual Speech Like Human Lipreaders?
Rishabh Jain and Naomi Harte
Visual
speech recognition (VSR) models now surpass human lipreaders on
benchmarks, but do such gains establish human-like visual
speech perception? To explore this, we compare three VSR systems with human baselines on the MaFI word-level lipreading
dataset using word, character, phoneme, and viseme-level metrics. Although models achieve higher overall accuracy, they succeed and fail on different words than humans. A text-only n-gram baseline given only a few initial phonemes rivals human lipreading. VSR word-level errors are consistently better explained by training word frequency than by the visual informativeness of words. Viseme accuracies, confusion
matrices and human-model correlations further show that models gain most on visemes humans find hardest, and show much weaker dependence on visual clarity. Our work demonstrates that VSR systems rely primarily on
language cues from training data rather than visual perception, failing to bind visual features into meaningful words.
Read the full abstract ▾
Computation and Language
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
Yang Zhang, Xiao Fei, Amr Mohamed et al. (+6 more)
Large
language models are increasingly used to answer culturally grounded questions across
languages, yet it remains unclear whether local cultural knowledge is better accessed through English or the local
language. Existing evaluations face two key limitations: many rely on parallel template-based questions that may not reflect how cultural knowledge naturally appears, and raw accuracy conflates general
language proficiency with
language-conditioned knowledge access. We address these issues with a controlled framework built on real-world cultural questions collected from regional
benchmarks and local sources. By crossing question type (culture-agnostic vs. culture-specific) with query
language (English vs. local
language), and estimating ability with a shared 1PL item response theory model, we separate proficiency from localized knowledge access. Across 13 locales and roughly 80 models, we find a consistent English advantage on culture-agnostic questions, indicating stronger English proficiency. However, after accounting for this proficiency gap, local
languages show a positive knowledge-access advantage in nearly all locale-model settings. This advantage is often masked in raw accuracy but becomes more visible for frontier, regionally aligned, or
language-adapted models. Our results suggest that weaker local-
language performance does not necessarily imply weaker cultural knowledge; rather, local cultural knowledge may be more accessible through the local
language but hidden by limited
language proficiency.
Read the full abstract ▾
All authors Yang Zhang, Xiao Fei, Amr Mohamed, Sarah Almeida Carneiro, Mersin Konomi, Mingmeng Geng, Ahmed Asaad, Guokan Shang, Michalis Vazirgiannis
Computation and Language
M$^3$Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions
Zhengjun Huang, Wenxuan Liu, Zhoujin Tian et al. (+6 more)
Language agents are increasingly deployed over accumulating
multimodal information, yet existing
benchmarks assume a human-human form with sparse visuals and straightforward content, evaluating neither
reasoning over authentic
multimodal file interaction nor the interpretation of concealed user information. We therefore introduce M$^3$Exam, a query-centric
multimodal conversational
memory benchmark built on realistic user-
agent interaction, with multi-dimensional evaluation spanning cross-modal grounding and implicit information inference. Benchmarking MLLMs and
memory systems reveals persistent gaps in cross-modal grounding, cross session
reasoning, and the efficiency cost of accumulating
multimodal context. We further propose M$^3$Proctor, a
multimodal memory method that detects query modality bias and consumes raw visual sources only on demand, improving accuracy by 13% while cutting index-construction time and retrieved
tokens by over 70%.
Read the full abstract ▾
All authors Zhengjun Huang, Wenxuan Liu, Zhoujin Tian, Wei Chen, Junle Chen, Yuqian Wu, Fangyuan Zhang, Qintian Guo, Xiaofang Zhou
Computation and Language · Cross-listed from cs.LG
Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests
Thanawat Lodkaew, Johannes Ackermann, Soichiro Nishimori et al. (+3 more)
A growing failure mode in
agent evaluation and training is that models can achieve high evaluation scores by exploiting shortcuts instead of solving the intended task, producing deceptive performance. This makes evaluation scores unreliable as measures of true task-solving ability. We propose CapCode, a framework for constructing coding
datasets with randomized tests whose best achievable non-cheating performance is deliberately capped below one. This capped-performance design gives evaluation scores a clearer interpretation: scores substantially above the cap are implausible and therefore provide evidence of cheating. To prevent cheating, we propose CapReward, a reward design based on the CapCode principle to discourage optimization beyond the cap. Experiments across multiple
datasets show that CapCode detects cheating while preserving performance ranking of models, and CapReward reduces cheating behavior, yielding models that better follow the intended task specification.
Read the full abstract ▾
All authors Thanawat Lodkaew, Johannes Ackermann, Soichiro Nishimori, Nontawat Charoenphakdee, Masashi Sugiyama, Takashi Ishida
Computation and Language · Cross-listed from cs.SD
DirectAudioEdit: Inversion-Free Text-Guided Audio Editing via Diffusion Prediction Contrast
Zhengkun Ge, Xiaoqian Liu, Haoran Zhang et al. (+5 more)
Text-guided audio editing aims to modify the
language-specified acoustic content while preserving edit-irrelevant source components. Existing training-free methods typically rely on inversion-based editing. While inversion-free editing is appealing as it decreases computational overhead and reconstruction errors, it remains largely unexplored for audio editing. The key challenge is to construct a source-to-target editing path through
diffusion denoising dynamics. In this paper, we introduce DirectAudioEdit, the first attempt to develop a training-free and inversion-free method for audio editing. Experiments on music and event-level
benchmarks across two backbones show that DirectAudioEdit reduces macro-averaged FAD and KL by 15.9% and 15.8% compared with DDPM inversion, while achieving up to 64.5% editing speedup.
Read the full abstract ▾
All authors Zhengkun Ge, Xiaoqian Liu, Haoran Zhang, Yuan Ge, Junxiang Zhang, Zhengtao Yu, Jingbo Zhu, Tong Xiao
Computation and Language
LLM-Guided Evolution for Medical Decision Pipelines
Ivan Sviridov, Artem Oskin, Ivan Panin et al. (+4 more)
Adapting large
language models (LLMs) to clinical workflows often requires costly fine-tuning or manual
prompt and pipeline engineering. We study LLM-guided MAP-Elites
evolution as an inference-time alternative for discovering medical decision strategies and provide an implementation repository at https://github.com/univanxx/llm_guided_evo_medical. We formulate urgency triage, interactive consultation, and medical image classification as evolutionary searches over executable artifacts optimized by task-specific fitness functions. Across all three settings,
evolution improves over manually designed baselines under practical constraints. In triage, evolved programs increase Semigran accuracy from $77.3\%$ to $87.1\%$ and emergency recall from $0.60$ to $0.97$, while improving safety-weighted held-out MIMIC-ESI performance. In interactive consultation, evolved policies improve the accuracy–cost frontier across Llama-3, Qwen-3.5, and Gemma-4 and transfer to held-out iCRAFTMD. In PneumoniaMNIST,
prompt-only
evolution improves frozen MedGemma VLMs while preserving strict JSON outputs. Qualitative analysis shows that the gains come from interpretable program-level mechanisms, calibrated triage boundaries, targeted evidence acquisition, selective commitment, and finding-oriented visual decision rules, rather than superficial
prompt rewording alone.
Read the full abstract ▾
All authors Ivan Sviridov, Artem Oskin, Ivan Panin, Iaroslav Bespalov, Dmitry Dylov, Ivan Oseledets, Aleksandr Nesterov
Computation and Language
SV-Detect: AI-generated Text Detection with Steering Vectors
Mikhail Vishnyakov and Tatiana Gaintseva
Detecting AI-generated text is especially difficult under distribution shift, such as transfer across domains, source models, and editing attacks. We propose an AI-generated text detector based on steering vectors extracted from the hidden representations of a frozen
language model. At each layer, we construct a direction that separates human-written from AI-generated text, and represent each input by its layer-wise
alignment with these directions. A lightweight classifier trained on these projection features yields the final detection score. Our method achieves strong performance both in-distribution and under distribution shift, including across domains, source models, and machine-editing transformations such as polishing and rewriting. Interpretation analyses show that the learned directions align with recognizable stylistic cues while capturing substantial additional signal beyond surface features. These results position AI-generated text detection as a representation-space probing problem and show that steering vectors provide a simple and effective solution.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
Iosif Tsangko, Andreas Triantafyllopoulos and Björn W. Schuller
Instruction-following audio
language models (ALMs) can be augmented with explicit acoustic cues, yet it remains unclear whether such cues are used in a grounded way when the raw audio is already available. We study this question in
speech emotion recognition (SER) by deriving six interpretable acoustic concept
tokens from the standardised eGeMAPS paralinguistic feature set. These
tokens summarise energy, pitch, dynamics, brightness, formants, and voice quality, and are appended to the textual
prompt while the audio input is kept unchanged. Across the widely used FAU-Aibo and IEMOCAP
benchmarks, aligned
tokens improve unweighted average recall (UAR), whereas shuffled, conflicting, or corrupted
tokens reduce performance relative to aligned
tokens and shift confusions toward neutral. Importantly, predictions do not collapse under strong
token perturbations, suggesting that the models are sensitive to the symbolic cue channel but remain partly anchored to the audio signal. We argue that
token-only interventions provide a practical way to probe audio-grounded cue use, robustness, and interpretability in ALM-based affective computing.
Read the full abstract ▾
Computation and Language
Phun-Bench: Evaluating LLMs on Phonological Understanding in Chinese
Xing Yue, Yongliang Shen and Weiming Lu
Language is a vehicle for thought, intricately tied to sounds, symbols, and meaning. However, most large
language model (LLM) research focuses on meaning (semantics) and symbols (spelling) while largely overlooking sounds. Existing
benchmarks on LLMs' phonological abilities are either solvable through rote memorization or intertwined with other abilities, making them inadequate to measure LLMs' genuine ability in phonological understanding. Here, we present Phun-Bench, a purpose-built Chinese
benchmark with diverse tasks and settings across three dimensions (Homophony, Rhyme, and Phonetic Similarity), designed to systematically evaluate LLMs' phonological understanding. Our results show that while LLMs excel at recalling correct pronunciations, they generally struggle to leverage phonological knowledge in the flexible and intuitive way that human speakers do. Moreover, through detailed analyses, we propose a hypothesis regarding the underlying mechanism of LLMs' phonological understanding and "perception", highlighting an underexplored frontier for future research.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
SWE-Explore: Benchmarking How Coding Agents Explore Repositories
Shaoqiu Zhang, Yuhang Wang, Jialiang Liang et al. (+8 more)
Repository-level coding
benchmarks such as SWE-bench have driven a rapid surge in the capabilities of coding
agents. Yet they usually treat coding tasks as a holistic, binary prediction problem (e.g., resolved or unresolved), neglecting fine-grained
agent capabilities such as repository understanding, context
retrieval, code localization, and bug diagnosis. In this paper, we introduce SWE-Explore, a
benchmark that isolates the evaluation of repository exploration, a critical capability of coding
agents. Given a repository and an issue, SWE-Explore asks an explorer to return a ranked list of relevant code regions under a fixed line budget. SWE-Explore covers 848 issues across 10 programming
languages and 203 open-source repositories. For each instance, we derive line-level ground truth from independent
agent trajectories that successfully solved the same issue, distilling the specific code regions their solution paths actually consulted. We evaluate exploration along coverage, ranking, and context-efficiency dimensions, showing that these metrics strongly track downstream repair behavior. Across a broad set of
retrieval methods, general coding
agents, and specialized localizers, we find that agentic explorers form a clear tier above classical
retrieval. While file-level localization is already strong for modern methods, line-level coverage and efficient ranking remain the key axes differentiating state-of-the-art explorers.
Read the full abstract ▾
All authors Shaoqiu Zhang, Yuhang Wang, Jialiang Liang, Yuling Shi, Wenhao Zeng, Maoquan Wang, Shilin He, Ningyuan Xu, Siyu Ye, Kai Cai, Xiaodong Gu
Computation and Language
KIT's Submission to Cross-Lingual Voice Cloning in IWSLT 2026
Seymanur Akti and Alexander Waibel
Cross-lingual voice cloning aims to generate
speech in a target
language while preserving speaker identity from a source-
language reference. This task is central to
speech translation and is the focus of the IWSLT 2026 Cross-Lingual Voice Cloning track. A key challenge is maintaining intelligibility and naturalness in the presence of accent variation and domain-specific vocabulary. We build on a
multilingual text-to-
speech model, FishAudio-S2-Pro, and introduce
language tag prompting to improve
language control and reduce accent leakage. We further apply reinforcement learning (RL) fine-tuning for task adaptation and observe improvements in intelligibility. Finally, we propose a reference-conditioned lexical matching method that improves pronunciation of domain-specific terms when lexical overlap is present. Results show that
language prompting provides the largest gains, while lexical matching yields consistent improvements on matched subsets.
Read the full abstract ▾
Computation and Language
When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations
Mahdi Alkaeed
Large
Language Models (LLMs) are increasingly used in healthcare for tasks such as clinical question answering, diagnosis support, and report summarization. Despite their promise, these models remain highly sensitive to subtle
prompt perturbations, both lexical and syntactic, posing serious risks in safety-critical clinical applications. In this study, we conduct a systematic sensitivity analysis to evaluate the robustness of both general-purpose (e.g., GPT-3.5, Llama3) and medical-specific LLMs (e.g., ClinicalBERT, BioLlama3, BioBERT) using the MedMCQA
benchmark. We categorize perturbations into natural and
adversarial types and examine their effect on model consistency, accuracy, and reliability in clinical
reasoning tasks. Our findings reveal that medical LLMs are not intrinsically safe. Even minor variations in phrasing can alter clinical advice, and targeted
adversarial prompts can provoke harmful outputs. In high-stakes settings like healthcare, such unpredictability is unacceptable-models that change diagnoses due to reworded inputs or hallucinate medications when slightly rephrased cannot be reliably trusted by clinicians. While models tend to show resilience to simple lexical substitutions or paraphrasing, they often break down under syntactic reordering or misleading contextual cues. This fragility is evident across both general-purpose and domain-specific LLMs. Notably,
adversarial manipulations can lead to clinically dangerous outputs, such as recommending incorrect dosages or omitting critical findings.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
MMAE: A Massive Multitask Audio Editing Benchmark
Ziyang Ma, Ruiqi Yan, Ruiyang Xu et al. (+35 more)
We introduce MMAE, a Massive Multitask Audio Editing
benchmark, serving as the first comprehensive evaluation testbed designed for general-purpose instruction-based audio editing. Spurred by the shift toward intelligent creation, interactive editing has rapidly expanded from visual domains, pioneered by models like Nano-banana 2 for images and Gemini-Omni for
video, into audio. However, the current evaluation infrastructure lags severely, remaining highly fragmented and restricted to specific subdomains or basic operations. Unlike existing
benchmarks that are limited in scope, MMAE extends to a broad spectrum of real-world scenarios, encompassing 7 distinct audio modalities, including sound,
speech, music, and their mixtures. Furthermore, we establish a comprehensive taxonomy spanning 6 levels of task complexity, from basic modifications to multi-hop
reasoning and multi-round editing, 2 levels of granularity, and 8 distinct operation types. Meticulously curated through human-
agent collaboration, MMAE comprises 2,000 high-fidelity samples paired with a pioneering rubric-based evaluation framework. By decomposing free-form tasks into 17,741 verifiable criteria, this robust rubric-based paradigm enables a precise, multi-dimensional assessment of both instruction following and context consistency. Our extensive evaluation of leading models reveals that current systems remain far from achieving reliable edits. Strikingly, the Exact Match Rate (EMR) consistently falls below 5% and plummets to an absolute 0% in complex, mixed-modality tasks, exposing critical bottlenecks in precise execution and structural robustness. We hope MMAE will serve as a catalyst for future advances in the intelligent creation community, providing a clear diagnostic roadmap and establishing a standardized, long-lasting evaluation paradigm for next-generation audio editing systems.
Read the full abstract ▾
All authors Ziyang Ma, Ruiqi Yan, Ruiyang Xu, Jie Fang, Zhikang Niu, Yi-Wen Chao, Wenming Tu, Tianrui Wang, Auden, Qi Chen, Wenxi Chen, Jiaying Chi, Yanru Huo, Zixuan Jiang, Xiquan Li, Yalin Li, Junxi Liu, Minghao Liu, Binghao Qiang, Yijia Shan, Zheshu Song, Tian Tan, Zixiang Wang, Zeyu Xie, Zhifei Xie, Xiaoyu Xing, Qixiang Xu, Chen Yang, Guanrou Yang, Shan Yang, Yifan Yang, Steve Yves, Haotian Zhang, Haina Zhu, Kai Yu, Liefeng Bo, Eng-Siong Chng, Xie Chen
Computation and Language · Cross-listed from cs.LG
DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarios
Tongzhou Yu, Mingjia Li, Hong Qian et al. (+6 more)
Human creativity has emerged as a critical competency in the era of large
language models. Assessing creativity in complex, open-ended environments is a grand challenge in data mining, currently hindered by a reliance on standardized simple tasks and the scarcity of fine-grained expert data. As an ecologically valid assessment context, debate reflects multiple dimensions of creativity, encompassing both divergent thinking and convergent thinking. Moreover, debate is a data-rich domain, with a large volume of publicly accessible materials. Current mainstream automated scoring methods are poorly suited to complex settings such as debate, and therefore still rely on costly human evaluation. To this end, this paper proposes DEFINED, a data-efficient computational framework for fine-grained creativity assessment in debate scenarios. DEFINED operationalizes debate creativity through a hierarchical eight-dimensional metric system, implemented via a pre-trained autoregressive
language model with a hierarchical scoring head that supports both fine-grained and coarse-grained evaluation. Statements and their associated expert scores were obtained from authentic debate competitions, and a constrained data augmentation strategy was employed to address the elite bias inherent in the original data. DEFINED adopts a mixed-granularity training strategy enabling robust learning from limited fine-grained supervision annotated by trained graduate experts. To rigorously validate ecological validity beyond synthetic
benchmarks, we incorporate an empirical study with debate-naive participants, utilizing these authentic data to serve as a qualitative case study for mid-to-low proficiency populations. Across our evaluation protocol, our scoring model achieves accurate and stable scoring, outperforming
prompt-based large
language model evaluators and existing debate scoring methods.
Read the full abstract ▾
All authors Tongzhou Yu, Mingjia Li, Hong Qian, Wenkai Wang, Zongbao Zhang, Yaoyu Jiang, Xiangfeng Wang, Aimin Zhou, Jiajun Guo
Computation and Language
Adversarial Creation and Detection of AI-Generated Social Bot Content
Mykola Trokhymovych, Ricardo Baeza-Yates, Alessandro Flammini et al. (+2 more)
The convergence of large
language models and social bots allows malicious actors to manipulate the information ecosystem by generating human-like content at scale. Existing models for detecting AI-generated content often fail in the wild, primarily due to the lack of ground-truth data. We address this gap through an
adversarial methodology that models the impersonation of real social media users by malicious actors. Using this methodology, we curate a
multilingual, cross-platform
dataset of paired human and AI-generated messages. Training on such
adversarial data yields accurate detection of AI-generated text. Our approach significantly outperforms existing models for content-based bot detection in real-world, out-of-distribution data.
Read the full abstract ▾
All authors Mykola Trokhymovych, Ricardo Baeza-Yates, Alessandro Flammini, Diego Saez-Trumper, Filippo Menczer
Computation and Language · Cross-listed from cs.IR
HKVM-RAG: Key-Value-Separated Hypergraph Evidence Organization for Multi-Hop RAG
Mingyu Zhang and Ying Ma
Multi-hop RAG poses a data-engineering problem beyond passage matching: under fixed
retrieval budgets, a system must organize retrieved text into evidence units that expose answer chains. Dense retrievers score passages independently, while graph-based
memories make associations explicit but often rely on pairwise or entity-centered keys that fragment multi-hop evidence. We present HKVM-RAG, a key-value-separated evidence-organization layer. It assembles answer-path hyperedges from cached passage-level LLM evidence tuples and uses them as
retrieval keys, while retaining passage text as answer values. To isolate key-space design, our fixed-substrate protocol holds the tuple cache, candidate passages, reader, and evaluation budget constant across pairwise graph and hypergraph variants. Weighted hypergraph key-value
retrieval improves over KG-PPR by +3.426 F1 on 2WikiMultiHopQA and +3.592 F1 on MuSiQue; HotpotQA shows that higher structured support coverage need not yield standalone answer-F1 gains. We therefore study WHG-KV as an evidence-control signal rather than a dense-
retrieval replacement. Oracle and train-to-dev analyses identify support selection as repairable, and a dense-aware controller combines frozen ColBERTv2 and HKVM rank/score features using out-of-fold HKVM predictions. It reaches 88.846, 65.073, and 85.810 F1 on the three
benchmarks, improving over ColBERTv2 by +11.084, +6.763, and +5.966 F1. Source-level ablations show that matched non-WHG structured signals do not match the WHG-KV gains. These results provide bounded evidence that key-value-separated hypergraph organization can serve as a reusable evidence-control mechanism for multi-hop RAG.
Read the full abstract ▾
Computation and Language
From Correctness to Utility: Gain-Based Prefix Evaluation for LLM Reasoning
Yuhang Zhou, Yixin Cao and Guangnan Ye
Reasoning prefixes shape the future trajectory of LLM problem solving, yet existing process reward models usually evaluate them through local step correctness. We argue that correctness is a useful but indirect proxy for the effect we ultimately care about: whether a prefix increases the probability of successful completion. We define this effect as prefix gain, the solve-rate improvement induced by conditioning lightweight student model group on a prefix, and use it to train a Prefix Utility Model (PUM) with a simple pairwise ranking objective. PUM learns outcome-grounded prefix utility and can score both complete trajectories and partial
reasoning prefixes. Across Best-of-$N$ selection, beam search, and reinforcement learning on mathematical
reasoning, PUM provides a strong prefix-level supervision signal, especially when candidate pools are large, search budgets increase, or rule-based rewards are sparse. We release all data, models, and code at https://zhiqix.github.io/pum-project-page.
Read the full abstract ▾
Computation and Language
Geometry of Semantic Space: Comparative Study of Discrete and Continuous Models
Gabriel Bounias and Sabine Ploux
This work examines the semantic geometry underlying NLP models. We compare supervised vector
embeddings, such as CamemBERT, with lexical co-occurrence graphs that encode semantic relations more directly. While
transformer-based
embeddings achieve strong performance, their induced geometries often display unsatisfactory distributions. In contrast, graph-based models reveal a clearer and more human-readable organization of meaning. We have implemented a methodology that allows us to perform a comparative analysis either based on the structure of the graphs or based on the
topology of the
embeddings induced by these two approaches. The results of the comparison – applied to the French "Great National Debate" corpus a collection of citizen contributions to the public debate – show a similar local
topology but a very different overall structure and
topology. Theses findings suggest complementary perspectives between deep supervised models and graph-based models, considering a new pathway to guide neural architectures toward more stable and interpretable convergence with graphs structures.
Read the full abstract ▾